r/aws_quicksight_data_set: Allow physical_table_map to be optional

[g-dx]

Description

Currently, the schema for the aws_quicksight_data_set resource marks the physical_table_map field as Required.

However, the AWS API makes clear that while required, the map it represents may contain zero entries:

PhysicalTableMap

    Declares the physical tables that are available in the underlying data sources.

    Type: String to PhysicalTable object map

    Map Entries: Minimum number of 0 items. Maximum number of 32 items.

    Key Length Constraints: Minimum length of 1. Maximum length of 64.

    Key Pattern: [0-9a-zA-Z-]*

    Required: Yes

Source

The present schema definition of Required therefore makes it impossible to specify an empty map.

This PR fixes this by updating the physical_table_map field to be Optional and ensures that an empty map is passed to the AWS SDK in the event that no physical_table_map blocks have been specified.

Relations

Fixes #31863
Related To: #30349

References

• https://docs.aws.amazon.com/quicksight/latest/APIReference/API_CreateDataSet.html#API_CreateDataSet_RequestSyntax

Output from Acceptance Testing

$ make testacc PKG=quicksight TESTS=TestAccQuickSightDataSet
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/quicksight/... -v -count 1 -parallel 20 -run='TestAccQuickSightDataSet'  -timeout 180m
?   	github.com/hashicorp/terraform-provider-aws/internal/service/quicksight/schema	[no test files]
=== RUN   TestAccQuickSightDataSetDataSource_basic
=== PAUSE TestAccQuickSightDataSetDataSource_basic
=== RUN   TestAccQuickSightDataSet_basic
=== PAUSE TestAccQuickSightDataSet_basic
=== RUN   TestAccQuickSightDataSet_disappears
=== PAUSE TestAccQuickSightDataSet_disappears
=== RUN   TestAccQuickSightDataSet_columnGroups
=== PAUSE TestAccQuickSightDataSet_columnGroups
=== RUN   TestAccQuickSightDataSet_columnLevelPermissionRules
=== PAUSE TestAccQuickSightDataSet_columnLevelPermissionRules
=== RUN   TestAccQuickSightDataSet_dataSetUsageConfiguration
=== PAUSE TestAccQuickSightDataSet_dataSetUsageConfiguration
=== RUN   TestAccQuickSightDataSet_fieldFolders
=== PAUSE TestAccQuickSightDataSet_fieldFolders
=== RUN   TestAccQuickSightDataSet_logicalTableMap
=== PAUSE TestAccQuickSightDataSet_logicalTableMap
=== RUN   TestAccQuickSightDataSet_permissions
=== PAUSE TestAccQuickSightDataSet_permissions
=== RUN   TestAccQuickSightDataSet_rowLevelPermissionTagConfiguration
=== PAUSE TestAccQuickSightDataSet_rowLevelPermissionTagConfiguration
=== RUN   TestAccQuickSightDataSet_refreshProperties
    data_set_test.go:373: Environment variable QUICKSIGHT_ATHENA_TESTING_ENABLED is not set
--- SKIP: TestAccQuickSightDataSet_refreshProperties (0.00s)
=== RUN   TestAccQuickSightDataSet_tags
=== PAUSE TestAccQuickSightDataSet_tags
=== RUN   TestAccQuickSightDataSet_noPhysicalTableMap
=== PAUSE TestAccQuickSightDataSet_noPhysicalTableMap
=== CONT  TestAccQuickSightDataSetDataSource_basic
=== CONT  TestAccQuickSightDataSet_fieldFolders
=== CONT  TestAccQuickSightDataSet_noPhysicalTableMap
=== CONT  TestAccQuickSightDataSet_tags
=== CONT  TestAccQuickSightDataSet_rowLevelPermissionTagConfiguration
=== CONT  TestAccQuickSightDataSet_columnGroups
=== CONT  TestAccQuickSightDataSet_permissions
=== CONT  TestAccQuickSightDataSet_logicalTableMap
=== CONT  TestAccQuickSightDataSet_basic
=== CONT  TestAccQuickSightDataSet_disappears
=== CONT  TestAccQuickSightDataSet_columnLevelPermissionRules
=== CONT  TestAccQuickSightDataSet_dataSetUsageConfiguration
--- PASS: TestAccQuickSightDataSet_disappears (44.39s)
--- PASS: TestAccQuickSightDataSetDataSource_basic (49.79s)
--- PASS: TestAccQuickSightDataSet_rowLevelPermissionTagConfiguration (54.89s)
--- PASS: TestAccQuickSightDataSet_columnGroups (54.89s)
--- PASS: TestAccQuickSightDataSet_basic (55.04s)
--- PASS: TestAccQuickSightDataSet_fieldFolders (55.82s)
--- PASS: TestAccQuickSightDataSet_dataSetUsageConfiguration (55.87s)
--- PASS: TestAccQuickSightDataSet_columnLevelPermissionRules (56.42s)
--- PASS: TestAccQuickSightDataSet_noPhysicalTableMap (56.95s)
--- PASS: TestAccQuickSightDataSet_logicalTableMap (82.24s)
--- PASS: TestAccQuickSightDataSet_tags (95.20s)
--- PASS: TestAccQuickSightDataSet_permissions (106.93s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/quicksight	107.055s
...

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

• Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
• Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

[jar-b]

Hey @g-dx - would it be possible to add a test case covering the desired configuration from the original issue? I've tried a few variants, but may be missing the intent.

---

1. physical_map with only the ID set (attempting to match original issue)

resource "aws_quicksight_data_set" "test" {
  data_set_id = %[1]q
  name        = %[2]q
  import_mode = "SPICE"

  physical_table_map {
    physical_table_map_id = %[1]q
  }
}

resource "aws_quicksight_data_set" "test" {
  data_set_id = %[1]q
  name        = %[2]q
  import_mode = "SPICE"

  physical_table_map {
    physical_table_map_id = %[1]q
  }

  logical_table_map {
    logical_table_map_id = %[1]q
    alias                = "Group1"
    source {
      physical_table_id = %[1]q
    }
  }
}

Error (in both cases):

        Error: creating QuickSight Data Set: InvalidParameterValueException: Invalid PhysicalTableMap
        {
          RespMetadata: {
            StatusCode: 400,
            RequestID: "7b63831d-e481-42e7-b098-2d0123793d50"
          },
          Message_: "Invalid PhysicalTableMap"
        }

2. Entirely omitted physical_map

resource "aws_quicksight_data_set" "test" {
  data_set_id = %[1]q
  name        = %[2]q
  import_mode = "SPICE"
}

Error:

        Error: creating QuickSight Data Set: InvalidParameterValueException: LogicalTableMap can only be empty when there is exactly one physicalTable
        {
          RespMetadata: {
            StatusCode: 400,
            RequestID: "4436d501-3746-4c61-9d4a-968b6ce7a346"
          },
          Message_: "LogicalTableMap can only be empty when there is exactly one physicalTable"
        }

I also tried creating a "parent" data set with a physical table and leaving the child with only a logical table map reference, but the physical table ID references don't appear to work across data sets.

[g-dx]

Hey, @jar-b of course I can add a test.

More generally the intent of this PR is that it allows data sets which are composed of other datasets, via a join instruction. In these cases there is no physical table map.

Roughly like this:

resource "aws_quicksight_data_set" "one" {
 // ...
}

resource "aws_quicksight_data_set" "two" {
 // ...
}

resource "aws_quicksight_data_set" "joined" {

  logical_table_map {
    logical_table_map_id = "one"
    source {
      data_set_arn = aws_quicksight_data_set.one.arn
    }
  }

  logical_table_map {
    logical_table_map_id = "two"
    source {
      data_set_arn = aws_quicksight_data_set.two.arn
    }
  }

  logical_table_map {
    logical_table_map_id = "result"

    source {
      join_instruction {
        left_operand  = "one"
        right_operand = "two"
        type          = "INNER"
        on_clause     = "{col1} = {col2}"
      }
    }
  }
}

[jar-b]

Ah, thanks for the clarification. I should have been using source.data_set_arn from a separate data set instead of source.physical_table_id. I'll await a functional test from someone that knows what they're doing :). Thanks!

[g-dx]

Hey @jar-b I've updated the PR with a test which creates a data set without a physical table map. Let me know if that's all good or there's anything else which should be added.

Thanks again for your help!

[jar-b]

LGTM 🎉

$ make testacc PKG=quicksight TESTS=TestAccQuickSightDataSet_
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/quicksight/... -v -count 1 -parallel 20 -run='TestAccQuickSightDataSet_'  -timeout 180m
?       github.com/hashicorp/terraform-provider-aws/internal/service/quicksight/schema  [no test files]

--- PASS: TestAccQuickSightDataSet_disappears (40.21s)
--- PASS: TestAccQuickSightDataSet_columnLevelPermissionRules (43.61s)
--- PASS: TestAccQuickSightDataSet_rowLevelPermissionTagConfiguration (44.26s)
--- PASS: TestAccQuickSightDataSet_fieldFolders (45.44s)
--- PASS: TestAccQuickSightDataSet_basic (45.47s)
--- PASS: TestAccQuickSightDataSet_columnGroups (45.92s)
--- PASS: TestAccQuickSightDataSet_dataSetUsageConfiguration (45.92s)
--- PASS: TestAccQuickSightDataSet_noPhysicalTableMap (48.54s)
--- PASS: TestAccQuickSightDataSet_logicalTableMap (70.08s)
--- PASS: TestAccQuickSightDataSet_tags (88.90s)
--- PASS: TestAccQuickSightDataSet_permissions (93.98s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/quicksight 97.212s

[jar-b]

Thanks for your contribution, @g-dx! 👍

[github-actions]

This functionality has been released in v5.4.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.