Wrong policy action when create an AwsCustomResource #4533
Comments
jagu-sayan
added
bug
SomayaB
added
needs-reproduction
|
The problem seems to originate in the SDK, the prefix used comes from this file: https://github.com/aws/aws-sdk-js/blob/master/apis/metadata.json In there the prefix for SES is defined as |
|
Thanks for reporting this! We are looking into it. Someone will update when there is movement. |
eladb
added
@aws-cdk/custom-resources
|
This issue is present for some more APIs of node js sdk. We are trying to create inventory configuration on s3 using AwsCustomResource with policy statements fromSdkCalls. Same is the case for s3:CreateJob (IAM action) vs s3control:CreateJob (sdk method). I guess this is dumb to just use the sdk method. There should be a sort of mapping between the two or sdk methods should exactly map with the actions. |
|
Is there an open ticket in the AWS Javascript SDK GitHub repository for this issue? I don't see one linked above. I just happened to need to Verify both an SES domain and SES Email Identity, so I thought I would use the handy dandy example for const verifyDomainIdentity = new AwsCustomResource(this, 'VerifyDomainIdentity', {
onCreate: {
service: 'SES',
action: 'verifyDomainIdentity',
parameters: {
Domain: 'example.com'
},
physicalResourceId: PhysicalResourceId.fromResponse('VerificationToken') // Use the token returned by the call as physical id
},
policy: AwsCustomResourcePolicy.fromSdkCalls({resources: AwsCustomResourcePolicy.ANY_RESOURCE})
});
new route53.TxtRecord(this, 'SESVerificationRecord', {
zone,
recordName: `_amazonses.example.com`,
values: [verifyDomainIdentity.getResponseField('VerificationToken')]
});It's funny because the example in the docs for Turns out the generated SDK metadata file is completely wrong for SES. It generates the following IAM Policy: AWS679f53fac002430cb0da5b7982bd2287ServiceRoleDefaultPolicyD28E1A5E:
Type: AWS::IAM::Policy
Properties:
PolicyDocument:
Statement:
- Action: email:VerifyDomainIdentity
Effect: Allow
Resource: "*"
- Action: email:VerifyEmailIdentity
Effect: Allow
Resource: "*"
- Action: email:DeleteIdentity
Effect: Allow
Resource: "*"
Version: "2012-10-17"
PolicyName: AWS679f53fac002430cb0da5b7982bd2287ServiceRoleDefaultPolicyD28E1A5E
Roles:
- Ref: AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2That should obviously be If the AWS JS SDK is going to continue to be wrong, and it's too difficult to get the appropriate fixes applied to the AWS JS SDK (which admittedly is a total beast and has a zillion moving parts), then perhaps the AWS CDK needs some sort of "custom override" mapping for known issues (like SES, and @vipingoel's example)? Also, this is probably something which should have more tests for scenarios like this. |
|
I figured it out. Houston, we have a problem. 🚀 💥 The referenced metadata.json file from the AWS SDK isn't supposed to be used for IAM policies. It's used for AWS SDK API Client code generation, AWS API Model file references, and the distribution (build-time) processes. Firstly, it's important to know exactly where in the AWS JS SDK this file is used: aws-sdk-js/dist-tools/service-collector.js
The Lets use SES as an example, since that's why I'm going down this rabbit hole. The "ses": {
"prefix": "email",
"name": "SES",
"cors": true
},Lets break this down, line by line:
{
"version": "2.0",
"metadata": {
"apiVersion": "2010-12-01",
"endpointPrefix": "email",
"protocol": "query",
"serviceAbbreviation": "Amazon SES",
"serviceFullName": "Amazon Simple Email Service",
"serviceId": "SES",
"signatureVersion": "v4",
"signingName": "ses",
"uid": "email-2010-12-01",
"xmlNamespace": "http://ses.amazonaws.com/doc/2010-12-01/"
},This is most obvious when looking at a raw HTTP GET / POST example, from the SES Documentation: POST / HTTP/1.1
Host: email.us-west-2.amazonaws.com
Content-Type: application/x-www-form-urlencoded
Date: Tue, 25 May 2010 21:20:27 +0000
Content-Length: 174
Action=SendRawEmail
&Destinations.member.1=allan%40example.com
&RawMessage.Data=RnJvbTp1c2VyQGV4YW1wbGUuY29tDQpTdWJqZWN0OiBUZXN0DQoNCk1lc3 ...
Unfortunately, I don't have a magic recommendation, fix or obvious alternative to suggest. But one thing is for certain - this metadata never going to change in the AWS JavaScript SDK into a form that the |
|
As a refresher, here's the problem function: /**
* Transform SDK service/action to IAM action using metadata from aws-sdk module.
* Example: CloudWatchLogs with putRetentionPolicy => logs:PutRetentionPolicy
*
* TODO: is this mapping correct for all services?
*/
function awsSdkToIamAction(service: string, action: string): string {
const srv = service.toLowerCase();
const iamService = (awsSdkMetadata[srv] && awsSdkMetadata[srv].prefix) || srv;
const iamAction = action.charAt(0).toUpperCase() + action.slice(1);
return `${iamService}:${iamAction}`;
}
I loaded the metadata file into a node REPL and produced a side-by-side diff of the Metadata Keys ( In practical terms, this means that if the prefix exists (the value on the right) the | **Metadata Keys** | **Metadata Prefixes** |
|-----------------------------------------------------------------|
| acmpca | acm-pca |
| applicationautoscaling | application-autoscaling |
| applicationinsights | application-insights |
| augmentedairuntime | sagemaker-a2i-runtime |
| autoscalingplans | autoscaling-plans |
| cloudwatch | monitoring |
| cloudwatchevents | events |
| cloudwatchlogs | logs |
| codegurureviewer | codeguru-reviewer |
| codestarconnections | codestar-connections |
| codestarnotifications | codestar-notifications |
| cognitoidentity | cognito-identity |
| cognitoidentityserviceprovider | cognito-idp |
| cognitosync | cognito-sync |
| computeoptimizer | compute-optimizer |
| configservice | config |
| costexplorer | ce |
| directoryservice | ds |
| dynamodbstreams | streams.dynamodb |
| ec2instanceconnect | ec2-instance-connect |
| efs | elasticfilesystem |
| elasticinference | elastic-inference |
| elb | elasticloadbalancing |
| elbv2 | elasticloadbalancingv2 |
| emr | elasticmapreduce |
| forecastqueryservice | forecastquery |
| forecastservice | forecast |
| iot1clickdevicesservice | iot1click-devices |
| iot1clickprojects | iot1click-projects |
| iotdata | iot-data |
| ioteventsdata | iotevents-data |
| iotjobsdataplane | iot-jobs-data |
| kinesisvideoarchivedmedia | kinesis-video-archived-media |
| kinesisvideomedia | kinesis-video-media |
| kinesisvideosignalingchannels | kinesis-video-signaling |
| lexmodelbuildingservice | lex-models |
| lexruntime | runtime.lex |
| licensemanager | license-manager |
| marketplacecatalog | marketplace-catalog |
| marketplaceentitlementservice | entitlement.marketplace |
| marketplacemetering | meteringmarketplace |
| mediapackagevod | mediapackage-vod |
| mediastoredata | mediastore-data |
| migrationhub | AWSMigrationHub |
| migrationhubconfig | migrationhub-config |
| mturk | mturk-requester |
| personalizeevents | personalize-events |
| personalizeruntime | personalize-runtime |
| pinpointemail | pinpoint-email |
| pinpointsmsvoice | sms-voice |
| qldbsession | qldb-session |
| rdsdataservice | rds-data |
| resourcegroups | resource-groups |
| sagemakerruntime | runtime.sagemaker |
| serverlessapplicationrepository | serverlessrepo |
| servicequotas | service-quotas |
| ses | email |
| simpledb | sdb |
| ssooidc | sso-oidc |
| stepfunctions | states |
| transcribeservice | transcribe |
| wafregional | waf-regional |62 lines. Here's the node.js script used to produce the above output: // metadata-keys-prefixes.js
const {execSync} = require('child_process');
const {writeFileSync} = require('fs');
const metadata = require('aws-sdk/apis/metadata.json');
const keys = Object.keys(metadata).sort();
const prefixKeys = keys.map(k => metadata[k].prefix ?? k);
writeFileSync('./metadata-keys.txt', keys.join('\n'), {encoding: 'utf8'});
writeFileSync('./metadata-prefixes.txt', prefixKeys.join('\n'), {encoding: 'utf8'});
try {
execSync('diff metadata-keys.txt metadata-prefixes.txt -dty -W 80 --suppress-common-lines > metadata-keys-prefixes.diff', {encoding: 'utf8'});
} catch (error) {
// diff will exit 1 which makes execSync throw a fit
} |
|
Perhaps we can use the JavaScript-encoded IAM Policy reference from the IAM Policy Generator instead or in combination to the AWS JS SDK API Metadata? https://awspolicygen.s3.amazonaws.com/js/policies.js 'Amazon SES': {
'StringPrefix': 'ses',
'Actions': ['CloneReceiptRuleSet', 'CreateConfigurationSet', 'CreateConfigurationSetEventDestination', 'CreateConfigurationSetTrackingOptions', 'CreateCustomVerificationEmailTemplate', 'CreateReceiptFilter', 'CreateReceiptRule', 'CreateReceiptRuleSet', 'CreateTemplate', 'DeleteConfigurationSet', 'DeleteConfigurationSetEventDestination', 'DeleteConfigurationSetTrackingOptions', 'DeleteCustomVerificationEmailTemplate', 'DeleteIdentity', 'DeleteIdentityPolicy', 'DeleteReceiptFilter', 'DeleteReceiptRule', 'DeleteReceiptRuleSet', 'DeleteTemplate', 'DeleteVerifiedEmailAddress', 'DescribeActiveReceiptRuleSet', 'DescribeConfigurationSet', 'DescribeReceiptRule', 'DescribeReceiptRuleSet', 'GetAccountSendingEnabled', 'GetCustomVerificationEmailTemplate', 'GetIdentityDkimAttributes', 'GetIdentityMailFromDomainAttributes', 'GetIdentityNotificationAttributes', 'GetIdentityPolicies', 'GetIdentityVerificationAttributes', 'GetSendQuota', 'GetSendStatistics', 'GetTemplate', 'ListConfigurationSets', 'ListCustomVerificationEmailTemplates', 'ListIdentities', 'ListIdentityPolicies', 'ListReceiptFilters', 'ListReceiptRuleSets', 'ListTemplates', 'ListVerifiedEmailAddresses', 'PutIdentityPolicy', 'ReorderReceiptRuleSet', 'SendBounce', 'SendBulkTemplatedEmail', 'SendCustomVerificationEmail', 'SendEmail', 'SendRawEmail', 'SendTemplatedEmail', 'SetActiveReceiptRuleSet', 'SetIdentityDkimEnabled', 'SetIdentityFeedbackForwardingEnabled', 'SetIdentityHeadersInNotificationsEnabled', 'SetIdentityMailFromDomain', 'SetIdentityNotificationTopic', 'SetReceiptRulePosition', 'TestRenderTemplate', 'UpdateAccountSendingEnabled', 'UpdateConfigurationSetEventDestination', 'UpdateConfigurationSetReputationMetricsEnabled', 'UpdateConfigurationSetSendingEnabled', 'UpdateConfigurationSetTrackingOptions', 'UpdateCustomVerificationEmailTemplate', 'UpdateReceiptRule', 'UpdateTemplate', 'VerifyDomainDkim', 'VerifyDomainIdentity', 'VerifyEmailAddress', 'VerifyEmailIdentity'],
'ARNFormat': 'arn:aws:ses:<region>:<account_ID>:<arn_type>/<resource_id>',
'ARNRegex': '^arn:aws:ses:.+:[0-9]+:.+',
'conditionKeys': ['ses:FeedbackAddress', 'ses:FromAddress', 'ses:FromDisplayName', 'ses:Recipients'],
'HasResource': true,
},
// ...
'Amazon S3': {
'StringPrefix': 's3',
'Actions': ['AbortMultipartUpload', 'BypassGovernanceRetention', 'CreateAccessPoint', 'CreateBucket', 'CreateJob', 'DeleteAccessPoint', 'DeleteAccessPointPolicy', 'DeleteBucket', 'DeleteBucketPolicy', 'DeleteBucketWebsite', 'DeleteJobTagging', 'DeleteObject', 'DeleteObjectTagging', 'DeleteObjectVersion', 'DeleteObjectVersionTagging', 'DescribeJob', 'GetAccelerateConfiguration', 'GetAccessPoint', 'GetAccessPointPolicy', 'GetAccessPointPolicyStatus', 'GetAccountPublicAccessBlock', 'GetAnalyticsConfiguration', 'GetBucketAcl', 'GetBucketCORS', 'GetBucketLocation', 'GetBucketLogging', 'GetBucketNotification', 'GetBucketObjectLockConfiguration', 'GetBucketPolicy', 'GetBucketPolicyStatus', 'GetBucketPublicAccessBlock', 'GetBucketRequestPayment', 'GetBucketTagging', 'GetBucketVersioning', 'GetBucketWebsite', 'GetEncryptionConfiguration', 'GetInventoryConfiguration', 'GetJobTagging', 'GetLifecycleConfiguration', 'GetMetricsConfiguration', 'GetObject', 'GetObjectAcl', 'GetObjectLegalHold', 'GetObjectRetention', 'GetObjectTagging', 'GetObjectTorrent', 'GetObjectVersion', 'GetObjectVersionAcl', 'GetObjectVersionForReplication', 'GetObjectVersionTagging', 'GetObjectVersionTorrent', 'GetReplicationConfiguration', 'HeadBucket', 'ListAccessPoints', 'ListAllMyBuckets', 'ListBucket', 'ListBucketMultipartUploads', 'ListBucketVersions', 'ListJobs', 'ListMultipartUploadParts', 'ObjectOwnerOverrideToBucketOwner', 'PutAccelerateConfiguration', 'PutAccessPointPolicy', 'PutAccountPublicAccessBlock', 'PutAnalyticsConfiguration', 'PutBucketAcl', 'PutBucketCORS', 'PutBucketLogging', 'PutBucketNotification', 'PutBucketObjectLockConfiguration', 'PutBucketPolicy', 'PutBucketPublicAccessBlock', 'PutBucketRequestPayment', 'PutBucketTagging', 'PutBucketVersioning', 'PutBucketWebsite', 'PutEncryptionConfiguration', 'PutInventoryConfiguration', 'PutJobTagging', 'PutLifecycleConfiguration', 'PutMetricsConfiguration', 'PutObject', 'PutObjectAcl', 'PutObjectLegalHold', 'PutObjectRetention', 'PutObjectTagging', 'PutObjectVersionAcl', 'PutObjectVersionTagging', 'PutReplicationConfiguration', 'ReplicateDelete', 'ReplicateObject', 'ReplicateTags', 'RestoreObject', 'UpdateJobPriority', 'UpdateJobStatus'],
'ARNFormat': 'arn:aws:s3:::<bucket_name>/<key_name>',
'ARNRegex': '^arn:aws:s3:::.+',
'conditionKeys': ['aws:RequestTag/${TagKey}', 'aws:ResourceTag/${TagKey}', 'aws:TagKeys', 's3:AccessPointNetworkOrigin', 's3:DataAccessPointAccount', 's3:DataAccessPointArn', 's3:ExistingJobOperation', 's3:ExistingJobPriority', 's3:ExistingObjectTag/<key>', 's3:JobSuspendedCause', 's3:LocationConstraint', 's3:RequestJobOperation', 's3:RequestJobPriority', 's3:RequestObjectTag/<key>', 's3:RequestObjectTagKeys', 's3:VersionId', 's3:authType', 's3:delimiter', 's3:locationconstraint', 's3:max-keys', 's3:object-lock-legal-hold', 's3:object-lock-mode', 's3:object-lock-remaining-retention-days', 's3:object-lock-retain-until-date', 's3:prefix', 's3:signatureAge', 's3:signatureversion', 's3:versionid', 's3:x-amz-acl', 's3:x-amz-content-sha256', 's3:x-amz-copy-source', 's3:x-amz-grant-full-control', 's3:x-amz-grant-read', 's3:x-amz-grant-read-acp', 's3:x-amz-grant-write', 's3:x-amz-grant-write-acp', 's3:x-amz-metadata-directive', 's3:x-amz-server-side-encryption', 's3:x-amz-server-side-encryption-aws-kms-key-id', 's3:x-amz-storage-class', 's3:x-amz-website-redirect-location'],
'HasResource': true,
},Un-minified: https://github.com/rrrix/aws-iam-reference/blob/master/var/policies.js I found a little tool that extracts every IAM Action and puts it into a text file: https://github.com/rrrix/aws-iam-reference/blob/master/all-actions.txt Forked from https://github.com/rvedotrc/aws-iam-reference (which uses an outdated js file reference from the "old" policygen). |
|
I just ran into this using the service called "ResourceGroupsTaggingAPI" in Javascript, but the IAM action prefix is "tag". Is there a workaround for this that doesn't allow all actions? |
|
Escalating to |
|
Summary: We need API Prefix to tell CloudFormation what API to call for the custom resource, and need the IAM prefix to create permissions. These are often not the same value. Further, there is no authoritative mapping between Service Name, API Prefix, and IAM Action Prefix. (in fact there does not appear to even be a definitive way to list services, or a definitive name for what to call a service) Proposed Solution: Short of AWS providing a definitive list of their services, I think the only way to handle this will be to maintain a mapping in code somewhere. The Options:
Deceptively, the various service names/codes are often subtly different between sources - preventing a trivial mapping. However, there's enough overlap by coincidence that most of the services appear to work. Examples:
|
|
I'm not sure if this is an issue people are still struggling with, but here's a potential workaround I found (I did not write it) that works for me: Create a policy with the action as it should be in IAM (ex: 'ses:verifyEmailIdentity'), then attach that policy to your custom resource, instead of having it create the policy using fromSdkCalls. I'm sorry if this was already common knowledge/unhelpful, but I hope it's helpful to someone as a workaround in the meantime! |
|
In 2023, a new method to look up the IAM action was added: #27313 I believe this issue no longer applies, at least for the SES case. The specific line with SES IAM prefix mapping: I will wait for a week before closing in case anyone find this still an issue. |
|
Comments on closed issues and PRs are hard for our team to see. |
Hello,
When I try to create the
AwsCustomResourceshow as example in the documentation : https://docs.aws.amazon.com/cdk/api/latest/docs/custom-resources-readme.html, I got a cloudformation error.Reproduction Steps
Error Log
Environment
Other
When I look the cloudformation created by cdk, the
Actionproperty is wrong. I got ``"Action": "email:VerifyDomainIdentity"instead of"Action": "ses:VerifyDomainIdentity"`I think mapping define here is not correct for all services.
Workaround
This is 🐛 Bug Report
The text was updated successfully, but these errors were encountered: