Only add to cache if log stream is created

[jefchien]

Same change as #1566

Description of the issue

If there are multiple instances of the CloudWatch agent that are trying to create the same log group, there are cases where streams within those log groups will not be created because CreateLogGroup returns ResourceAlreadyExists.

amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target.go

Line 72 in af960d7

m.logger.Debugf("creating group fail due to : %v", err)

Description of changes

Resource already exists shouldn't be considered a failure to create the resource. Only add the target to the cache if the log stream was created or already exists.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Added more unit test cases to cover the issue. Added checks for cache sizes after each test.

=== RUN   TestTargetManager/CreateLogGroupAndStream
=== RUN   TestTargetManager/CreateLogGroupAndStream/GroupAlreadyExists
=== RUN   TestTargetManager/CreateLogGroupAndStream/RetryStreamFail
=== RUN   TestTargetManager/CreateLogGroupAndStream/RetryStreamAlreadyExists
    --- PASS: TestTargetManager/CreateLogGroupAndStream (0.00s)
    --- PASS: TestTargetManager/CreateLogGroupAndStream/GroupAlreadyExists (0.00s)
    --- PASS: TestTargetManager/CreateLogGroupAndStream/RetryStreamFail (0.00s)
    --- PASS: TestTargetManager/CreateLogGroupAndStream/RetryStreamAlreadyExists (0.00s)

Added tests run before changes

=== RUN   TestTargetManager/CreateLogGroupAndStream
    --- PASS: TestTargetManager/CreateLogGroupAndStream (0.00s)
=== RUN   TestTargetManager/CreateLogGroupAndStream/GroupAlreadyExists
    target_test.go:68: FAIL:	CreateLogStream(string)
        		at: [/Volumes/workplace/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target_test.go:62]
    target_test.go:68: FAIL: 2 out of 3 expectation(s) were met.
        	The code you are testing needs to make 1 more call(s).
        	at: [/Volumes/workplace/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target_test.go:68]
    --- FAIL: TestTargetManager/CreateLogGroupAndStream/GroupAlreadyExists (0.00s)

=== RUN   TestTargetManager/CreateLogGroupAndStream/RetryStreamFail
    target_test.go:84: 
        	Error Trace:	/Volumes/workplace/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target_test.go:84
        	Error:      	An error is expected but got nil.
        	Test:       	TestTargetManager/CreateLogGroupAndStream/RetryStreamFail
    target_test.go:85: FAIL:	CreateLogStream(string)
        		at: [/Volumes/workplace/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target_test.go:79]
    target_test.go:85: FAIL: 2 out of 3 expectation(s) were met.
        	The code you are testing needs to make 1 more call(s).
        	at: [/Volumes/workplace/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target_test.go:85]
    target_test.go:86: 
        	Error Trace:	/Volumes/workplace/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target_test.go:334
        	            				/Volumes/workplace/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher/target_test.go:86
        	Error:      	"map[{G1 S1  0}:{}]" should have 0 item(s), but has 1
        	Test:       	TestTargetManager/CreateLogGroupAndStream/RetryStreamFail
    --- FAIL: TestTargetManager/CreateLogGroupAndStream/RetryStreamFail (0.00s)

=== RUN   TestTargetManager/CreateLogGroupAndStream/RetryStreamAlreadyExists
    --- PASS: TestTargetManager/CreateLogGroupAndStream/RetryStreamAlreadyExists (0.00s)

Requirements

Before commit the code, please do the following steps.

1. Run make fmt and make fmt-sh
2. Run make lint