provider: Add -count 1 to testacc Makefile target to prevent issues with Go test caching #10245
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ith Go test caching Go 1.10+ will automatically cache successful test results if it detects no local changes: https://golang.org/doc/go1.10#test When running acceptance testing to attempt to find eventual consistency issues by repeatedly running the same tests, its possible this Go test caching will inadvertently make it hard to detect inconsistent remote results. The testing output does not make it obvious for individual tests (at least in Go 1.13.0) that the results are actually cached except for the final line which mentions `(cached)`: ```console $ time TF_LOG_PATH=/tmp/${AWS_PROFILE}.txt TF_LOG=debug TF_ACC=1 go test ./aws -v -timeout 120m -parallel 20 -run='TestAccAWSWafWebAcl_Rules' === RUN TestAccAWSWafWebAcl_Rules === PAUSE TestAccAWSWafWebAcl_Rules === CONT TestAccAWSWafWebAcl_Rules --- PASS: TestAccAWSWafWebAcl_Rules (35.42s) PASS ok github.com/terraform-providers/terraform-provider-aws/aws (cached) TF_LOG_PATH=/tmp/${AWS_PROFILE}.txt TF_LOG=debug TF_ACC=1 go test ./aws -v 10.87s user 3.64s system 267% cpu 5.419 total ``` Output from acceptance testing: ```console $ make testacc TEST=./aws TESTARGS=-run='TestAccAWSProvider_' ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSProvider_ -timeout 120m === RUN TestAccAWSProvider_Endpoints === PAUSE TestAccAWSProvider_Endpoints === RUN TestAccAWSProvider_Endpoints_Deprecated === PAUSE TestAccAWSProvider_Endpoints_Deprecated === RUN TestAccAWSProvider_Region_AwsChina === PAUSE TestAccAWSProvider_Region_AwsChina === RUN TestAccAWSProvider_Region_AwsCommercial === PAUSE TestAccAWSProvider_Region_AwsCommercial === RUN TestAccAWSProvider_Region_AwsGovCloudUs === PAUSE TestAccAWSProvider_Region_AwsGovCloudUs === CONT TestAccAWSProvider_Endpoints === CONT TestAccAWSProvider_Region_AwsChina === CONT TestAccAWSProvider_Endpoints_Deprecated === CONT TestAccAWSProvider_Region_AwsCommercial === CONT TestAccAWSProvider_Region_AwsGovCloudUs --- PASS: TestAccAWSProvider_Region_AwsCommercial (2.77s) --- PASS: TestAccAWSProvider_Region_AwsChina (2.78s) --- PASS: TestAccAWSProvider_Region_AwsGovCloudUs (2.91s) --- PASS: TestAccAWSProvider_Endpoints_Deprecated (2.98s) --- PASS: TestAccAWSProvider_Endpoints (3.01s) PASS ok github.com/terraform-providers/terraform-provider-aws/aws 4.065s ```
bflad
added
tests
|
This has been released in version 2.30.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks! |
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks! |
ghost
locked and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Community Note
Release note for CHANGELOG:
Go 1.10+ will automatically cache successful test results if it detects no local changes: https://golang.org/doc/go1.10#test
When running acceptance testing to attempt to find eventual consistency issues by repeatedly running the same tests, its possible this Go test caching will inadvertently make it hard to detect inconsistent remote results. The testing output does not make it obvious for individual tests (at least in Go 1.13.0) that the results are actually cached except for the final line which mentions
(cached):Output from acceptance testing: