Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Semgrep rule to warn on calling 'd.SetId()' in a resource create or delete function #18376

Conversation

ewbankkit
Copy link
Contributor

@ewbankkit ewbankkit commented Mar 24, 2021

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Relates #12796.
Relates #18382.
Relates #18390.

Warns if d.SetId("") is called in a resource's Create function or d.SetId(...) is called in a resource's Delete function.

% make testacc TESTS=TestAccCloudFormationStack_basic PKG=cloudformation
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/cloudformation/... -v -count 1 -parallel 20 -run='TestAccCloudFormationStack_basic'  -timeout 180m
=== RUN   TestAccCloudFormationStack_basic
=== PAUSE TestAccCloudFormationStack_basic
=== CONT  TestAccCloudFormationStack_basic
--- PASS: TestAccCloudFormationStack_basic (58.15s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/cloudformation	62.549s
% make testacc TESTS=TestAccEC2Fleet_basic PKG=ec2                      
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/ec2/... -v -count 1 -parallel 20 -run='TestAccEC2Fleet_basic'  -timeout 180m
=== RUN   TestAccEC2Fleet_basic
=== PAUSE TestAccEC2Fleet_basic
=== CONT  TestAccEC2Fleet_basic
--- PASS: TestAccEC2Fleet_basic (58.07s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/ec2	65.831s
% make testacc TESTS=TestAccEC2SpotFleetRequest_basic PKG=ec2
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/ec2/... -v -count 1 -parallel 20 -run='TestAccEC2SpotFleetRequest_basic'  -timeout 180m
=== RUN   TestAccEC2SpotFleetRequest_basic
=== PAUSE TestAccEC2SpotFleetRequest_basic
=== CONT  TestAccEC2SpotFleetRequest_basic
--- PASS: TestAccEC2SpotFleetRequest_basic (142.54s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/ec2	145.979s
% make testacc TESTS=TestAccNeptuneEventSubscription_basic PKG=neptune
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/neptune/... -v -count 1 -parallel 20 -run='TestAccNeptuneEventSubscription_basic'  -timeout 180m
=== RUN   TestAccNeptuneEventSubscription_basic
=== PAUSE TestAccNeptuneEventSubscription_basic
=== CONT  TestAccNeptuneEventSubscription_basic
--- PASS: TestAccNeptuneEventSubscription_basic (130.70s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/neptune	135.190s
% make testacc TESTS=TestAccRedshiftSnapshotScheduleAssociation_basic PKG=redshift
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/redshift/... -v -count 1 -parallel 20 -run='TestAccRedshiftSnapshotScheduleAssociation_basic'  -timeout 180m
=== RUN   TestAccRedshiftSnapshotScheduleAssociation_basic
=== PAUSE TestAccRedshiftSnapshotScheduleAssociation_basic
=== CONT  TestAccRedshiftSnapshotScheduleAssociation_basic
--- PASS: TestAccRedshiftSnapshotScheduleAssociation_basic (260.14s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/redshift	265.418s
% make testacc TESTS=TestAccSNSTopicSubscription_basic PKG=sns            
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/sns/... -v -count 1 -parallel 20 -run='TestAccSNSTopicSubscription_basic'  -timeout 180m
=== RUN   TestAccSNSTopicSubscription_basic
=== PAUSE TestAccSNSTopicSubscription_basic
=== CONT  TestAccSNSTopicSubscription_basic
--- PASS: TestAccSNSTopicSubscription_basic (101.15s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/sns	104.894s
% make testacc TESTS=TestAccWAFRegionalRegexMatchSet_serial/basic PKG=wafregional
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/wafregional/... -v -count 1 -parallel 20 -run='TestAccWAFRegionalRegexMatchSet_serial/basic'  -timeout 180m
=== RUN   TestAccWAFRegionalRegexMatchSet_serial
=== RUN   TestAccWAFRegionalRegexMatchSet_serial/basic
--- PASS: TestAccWAFRegionalRegexMatchSet_serial (25.20s)
    --- PASS: TestAccWAFRegionalRegexMatchSet_serial/basic (25.19s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/wafregional	28.857s

@ewbankkit ewbankkit requested a review from a team as a code owner March 24, 2021 16:10
@ghost ghost added the size/XS Managed by automation to categorize the size of a PR. label Mar 24, 2021
@ewbankkit ewbankkit added linter Pertains to changes to or issues with the various linters. technical-debt Addresses areas of the codebase that need refactoring or redesign. labels Mar 24, 2021
@ewbankkit ewbankkit changed the title Add Semgrep rule to warn on calling 'd.SetId()' in a resource create function Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create function Mar 24, 2021
@ewbankkit

This comment has been minimized.

@ewbankkit ewbankkit changed the title Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create function [WIP] Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create function Mar 24, 2021
@ewbankkit ewbankkit changed the title [WIP] Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create function [WIP] Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create, update or delete function Mar 24, 2021
@ewbankkit ewbankkit force-pushed the td-add-semgrep-rule-calling-SetId-with-empty-string-in-resource-create branch from 7b716b4 to f37739f Compare March 24, 2021 20:52
@ghost ghost added size/XXL Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. provider Pertains to the provider itself, rather than any interaction with AWS. service/cloudfront Issues and PRs that pertain to the cloudfront service. service/ecs Issues and PRs that pertain to the ecs service. service/elasticache Issues and PRs that pertain to the elasticache service. service/iam Issues and PRs that pertain to the iam service. service/lightsail Issues and PRs that pertain to the lightsail service. service/pinpoint Issues and PRs that pertain to the pinpoint service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. and removed size/XS Managed by automation to categorize the size of a PR. labels Mar 26, 2021
@github-actions
Copy link

Thank you for your contribution! 🚀

Please note that the CHANGELOG.md file contents are handled by the maintainers during merge. This is to prevent pull request merge conflicts, especially for contributions which may not be merged immediately. Please see the Contributing Guide for additional pull request review items.

Remove any changes to the CHANGELOG.md file and commit them in this pull request to prevent delays with reviewing and potentially merging this pull request.

@ewbankkit ewbankkit force-pushed the td-add-semgrep-rule-calling-SetId-with-empty-string-in-resource-create branch from 6971d75 to 2a423b4 Compare April 26, 2021 17:05
@ghost ghost added size/XS Managed by automation to categorize the size of a PR. and removed size/XXL Managed by automation to categorize the size of a PR. labels Apr 26, 2021
@ewbankkit ewbankkit removed provider Pertains to the provider itself, rather than any interaction with AWS. service/cloudfront Issues and PRs that pertain to the cloudfront service. service/ecs Issues and PRs that pertain to the ecs service. labels Apr 26, 2021
@ewbankkit
Copy link
Contributor Author

Consider an additional rule for calling d.Set(...) in an update or delete function:

  - id: calling-Set-in-resource-update-or-delete
    languages: [go]
    message: Do not call `d.Set(...)` inside a resource update or delete function
    paths:
      include:
        - aws/
    patterns:
      - pattern: |
          func $FUNC(...) {
            ...
            d.Set(...)
          }
      - metavariable-regex:
          metavariable: "$FUNC"
          regex: "^resourceAws\\w*(Update|Delete|Disable)$"
    severity: WARNING

@ewbankkit ewbankkit force-pushed the td-add-semgrep-rule-calling-SetId-with-empty-string-in-resource-create branch from cb4d598 to d3be9c3 Compare October 6, 2021 20:50
@github-actions github-actions bot removed the tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. label Oct 6, 2021
@ewbankkit
Copy link
Contributor Author

aws/resource_aws_cloudformation_stack.go
severity:warning rule:calling-SetId-with-empty-string-in-resource-create: Do not call `d.SetId("")` inside a resource create function
123:func resourceAwsCloudFormationStackCreate(d *schema.ResourceData, meta interface{}) error {
124:	conn := meta.(*AWSClient).cfconn
125:	defaultTagsConfig := meta.(*AWSClient).DefaultTagsConfig
126:	tags := defaultTagsConfig.MergeTags(keyvaluetags.New(d.Get("tags").(map[string]interface{})))
127:
128:	requestToken := resource.UniqueId()
129:	input := cloudformation.CreateStackInput{
130:		StackName:          aws.String(d.Get("name").(string)),
131:		ClientRequestToken: aws.String(requestToken),
132:	}
-------- [hid 70 additional lines, adjust with --max-lines-per-finding] --------

aws/resource_aws_ec2_fleet.go
severity:warning rule:calling-SetId-with-empty-string-in-resource-create: Do not call `d.SetId("")` inside a resource create function
341:func resourceAwsEc2FleetCreate(d *schema.ResourceData, meta interface{}) error {
342:	conn := meta.(*AWSClient).ec2conn
343:	defaultTagsConfig := meta.(*AWSClient).DefaultTagsConfig
344:	tags := defaultTagsConfig.MergeTags(keyvaluetags.New(d.Get("tags").(map[string]interface{})))
345:
346:	input := &ec2.CreateFleetInput{
347:		ExcessCapacityTerminationPolicy:  aws.String(d.Get("excess_capacity_termination_policy").(string)),
348:		LaunchTemplateConfigs:            expandEc2FleetLaunchTemplateConfigRequests(d.Get("launch_template_config").([]interface{})),
349:		OnDemandOptions:                  expandEc2OnDemandOptionsRequest(d.Get("on_demand_options").([]interface{})),
350:		ReplaceUnhealthyInstances:        aws.Bool(d.Get("replace_unhealthy_instances").(bool)),
-------- [hid 50 additional lines, adjust with --max-lines-per-finding] --------

aws/resource_aws_spot_fleet_request.go
severity:warning rule:calling-SetId-with-empty-string-in-resource-create: Do not call `d.SetId("")` inside a resource create function
924:func resourceAwsSpotFleetRequestCreate(d *schema.ResourceData, meta interface{}) error {
925:	// http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RequestSpotFleet.html
926:	conn := meta.(*AWSClient).ec2conn
927:	defaultTagsConfig := meta.(*AWSClient).DefaultTagsConfig
928:	tags := defaultTagsConfig.MergeTags(keyvaluetags.New(d.Get("tags").(map[string]interface{})))
929:
930:	_, launchSpecificationOk := d.GetOk("launch_specification")
931:	_, launchTemplateConfigsOk := d.GetOk("launch_template_config")
932:
933:	// http://docs.aws.amazon.com/sdk-for-go/api/service/ec2.html#type-SpotFleetRequestConfigData
------- [hid 187 additional lines, adjust with --max-lines-per-finding] --------

The aws_cloudformation_stack finding looks like it could be related to #107 and #6800.

@zhelding
Copy link
Contributor

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

@ewbankkit ewbankkit force-pushed the td-add-semgrep-rule-calling-SetId-with-empty-string-in-resource-create branch from d3be9c3 to e87a781 Compare November 16, 2021 21:51
@ewbankkit ewbankkit force-pushed the td-add-semgrep-rule-calling-SetId-with-empty-string-in-resource-create branch from e87a781 to f9b3a15 Compare January 18, 2022 13:18
@ewbankkit ewbankkit changed the title [WIP] Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create, update or delete function Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create or delete function Jan 18, 2022
@github-actions github-actions bot added service/cloudformation Issues and PRs that pertain to the cloudformation service. service/ec2 Issues and PRs that pertain to the ec2 service. service/neptune Issues and PRs that pertain to the neptune service. service/redshift Issues and PRs that pertain to the redshift service. service/sns Issues and PRs that pertain to the sns service. service/waf Issues and PRs that pertain to the waf service. size/M Managed by automation to categorize the size of a PR. and removed size/S Managed by automation to categorize the size of a PR. labels Jan 18, 2022
@ewbankkit
Copy link
Contributor Author

To keep this PR's scope smaller open a separate PR for the case of calling d.SetId("") in resource Update.
Often these are due to calling resource Update immediately after Create rather than resource Read.

@ewbankkit ewbankkit changed the title Add Semgrep rule to warn on calling 'd.SetId("")' in a resource create or delete function Add Semgrep rule to warn on calling 'd.SetId()' in a resource create or delete function Jan 18, 2022
@ewbankkit ewbankkit merged commit 972e99f into hashicorp:main Jan 18, 2022
@ewbankkit ewbankkit deleted the td-add-semgrep-rule-calling-SetId-with-empty-string-in-resource-create branch January 18, 2022 14:10
@github-actions github-actions bot added this to the v3.73.0 milestone Jan 18, 2022
@github-actions
Copy link

This functionality has been released in v3.73.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
linter Pertains to changes to or issues with the various linters. service/cloudformation Issues and PRs that pertain to the cloudformation service. service/ec2 Issues and PRs that pertain to the ec2 service. service/neptune Issues and PRs that pertain to the neptune service. service/redshift Issues and PRs that pertain to the redshift service. service/sns Issues and PRs that pertain to the sns service. service/waf Issues and PRs that pertain to the waf service. size/M Managed by automation to categorize the size of a PR. technical-debt Addresses areas of the codebase that need refactoring or redesign.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants