Parallelize WorkSpaces Directory tests

[gdavison]

Description

WorkSpaces Directories require an IAM Role named workspaces_DefaultRole. If the role is created and destroyed as part of each test, they have to be completely serialized, as done in #11505. If the test suite grows, this will slow down execution, so we should try to make the tests parallel. This will require moving the IAM Role to an external dependency which the tests can assume has been defined.

[Tensho]

Is there any example of the test data with external dependencies? I supposed to think each test case should be self-sufficient and independent.

If I specify Test instead of ParallelTest then it means it will be run serialized out of the box and there is no need for the extra serialization code, isn't it?

[Tensho]

@gdavison Please could you provide some insight here? 🙏

[gdavison]

It's because the IAM role for the WorkSpaces directories has to be workspaces_DefaultRole. So if we run the tests in parallel, the first test will create the IAM role, but the other tests will fail. Ideally, we could create the IAM role once, run all the tests, then tear down the IAM role, but our framework doesn't currently allow that.

[Tensho]

@gdavison Yeah, I understand the issue reason and why we need to synchronize tests. What I can't understand, why does tests execution should be synchronized with an imperative code instead of Test declaration? I'd expect simply changing ParallelTest to Test should solve the problem until Terraform test framework provides better solution to handle shared resources.

[bflad]

@Tensho the maintainers use a TeamCity CI environment for running the acceptance testing (both daily and ad-hoc when not running it locally). The TeamCity runner being used splits each test into a separate Go process for parallelization (this was written well before go test -parallel and resource.ParallelTest()) and to capture test panics without affecting other in-progress tests. Using resource.Test() covers the local case, however does not fix our usage of test splitting. Since the imperative code only has one capital-T test, its serialized for both local and our TeamCity environment.

[Tensho]

@bflad Thank you for the thorough explanation 🙇 Now it's clear for me the code is for CI compatibility reason.

[ghost]

This has been released in version 2.64.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!