(aws-cdk-lib.aws_s3_notifications): (Configuration is ambigously defined even through prefixes are not overlapping) #29951
Assignees
Labels
aws-cdk-lib
Related to the aws-cdk-lib package
bug
This issue is a bug.
closed-for-staleness
This issue was automatically closed because it hadn't received any attention in a while.
needs-reproduction
This issue needs reproduction.
response-requested
Waiting on additional info and feedback. Will move to "closing-soon" in 7 days.
Comments
thenamanpatwari
added
bug
ashishdhingra
added
needs-reproduction
|
@thenamanpatwari Good morning. Please share the following:
Using the below CDK TypeScript code, I was able to deploy CF stack (first deploying one event notification and then another one as part of same stack): import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as s3n from 'aws-cdk-lib/aws-s3-notifications'
export class TypescriptStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const lambdaFunction = new lambda.Function(this, 'TestLambdaFunction',{
runtime: lambda.Runtime.NODEJS_18_X,
code: lambda.Code.fromAsset('lambda'),
handler: 'hello.handler'
});
const s3Bucket = new s3.Bucket(this, 'TestBucket');
s3Bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(lambdaFunction), { prefix: "bounding_box_2d/" });
s3Bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(lambdaFunction), { prefix: "kv_rule_engine/input/" });
}
}
const app = new cdk.App();
new TypescriptStack(app, 'TypescriptStack');Thereafter, when I upload an object to S3, I see no error in the CloudWatch logs for BucketNotificationsHandler created by CDK stack (runtime Python): Thanks, |
ashishdhingra
added
the
response-requested
This was referenced Apr 26, 2024
|
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
github-actions
bot
added
the
closing-soon
|
Thank you for your response Ashish.
|
github-actions
bot
removed
closing-soon
|
I'm having the same issue, with the below changes cdk version 2.139.1 (build b88f959) stack is failing with the below: |
|
@airmonitor Thank you for the detailed description. Can confirm I am getting the same error. |
|
@ashishdhingra any updates? This is blocking some of our critical deployments. |
|
@thenamanpatwari As mentioned in the #29951 (comment), I'm unable to reproduce the issue. Please share the sample end-to-end code (including Python code for your lambda function) and any other configuration that needs to be done, to reproduce the issue. From the error in the issue description, it appears that you have overlapping prefixes for the same event type. You also mentioned that there are lot of prefixes already defined. Are these defined for same event type? Also the error in the issue description appears to be in Lambda event handler, looks like it is reported when the Lambda is invoked per event notification, not when CDK stack is deployed. Kindly correct if I'm wrong. Kindly note that once the CDK deployment is initiated, it is CloudFormation that takes over. Thanks, |
|
Hello @ashishdhingra I think you missed one variation in your tests. To use the same prefix but for different actions. prefix .jpg for s3:ObjectCreated:* this is failing. Are you able to replicate now the issue or this is still not clear? Regards |
@airmonitor Unable to reproduce using below code: import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as s3n from 'aws-cdk-lib/aws-s3-notifications';
export class TypescriptStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const lambdaFunction = new lambda.Function(this, 'TestLambdaFunction',{
runtime: lambda.Runtime.NODEJS_18_X,
code: lambda.Code.fromAsset('lambda'),
handler: 'hello.handler'
});
const s3Bucket = new s3.Bucket(this, 'TestBucket');
s3Bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(lambdaFunction), { prefix: ".jpg" });
s3Bucket.addEventNotification(s3.EventType.OBJECT_REMOVED, new s3n.LambdaDestination(lambdaFunction), { prefix: ".jpg" });
}
}Output of Output of It also works if I use the equivalent Python code: from aws_cdk import (
# Duration,
Stack,
aws_s3 as s3,
aws_lambda as lam,
aws_s3_notifications as s3n
)
from constructs import Construct
class PythonStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
lambdaFunction = lam.Function(
self, "TestLambdaFunction",
runtime=lam.Runtime.NODEJS_18_X,
code=lam.Code.from_asset("lambda"),
handler="hello.handler"
)
s3Bucket = s3.Bucket(
self, "TestBucket"
)
s3Bucket.add_event_notification(s3.EventType.OBJECT_CREATED, s3n.LambdaDestination(lambdaFunction), s3.NotificationKeyFilter(prefix=".jpg"))
s3Bucket.add_event_notification(s3.EventType.OBJECT_REMOVED, s3n.LambdaDestination(lambdaFunction), s3.NotificationKeyFilter(prefix=".jpg"))Output of Thanks, |
ashishdhingra
added
the
response-requested
|
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
github-actions
bot
added
the
closing-soon
|
Still having the same issue cdk --version 2.142.1 (build ed4e152) |
github-actions
bot
removed
closing-soon
@airmonitor Good afternoon. Could you please share self reproducible code sample to troubleshoot the issue? As per my previous comments, we are unable to reproduce the issue at our end, hence code sample from your side would be helpful. Thanks, |
ashishdhingra
added
the
response-requested
|
Hello I am wondering if you have covered a case when you are trying to add notification configuration to existing bucket which already has multiple events notifications configured. The example which you showed above with your typescript code covers a happy path when you are creating new bucket and adding notification configuration to it. So there is no existing configuration applied over there. I feel that the lambda function which is adding new notification configuration it is trying to apply existing notification configuration to it by using overwrite Approach. But in the reality that will not work because first you need to remove all of the notification configuration and apply the new one if old and new notification configurations have the same rules. What do you think? |
github-actions
bot
removed
the
response-requested
|
@airmonitor While running my example, I had first applied notification configuration for first prefix (commenting the code that adds bucket configuration for 2nd prefix). Thereafter, I uncommented the code that adds bucket configuration for 2nd prefix and deployed changes again. It would be great if you could elaborate your scenario with step-by-step example. Thanks, |
ashishdhingra
added
the
response-requested
|
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
github-actions
bot
added
closing-soon
|
Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Describe the bug
The S3 notification module is throwing an configuration is ambiguously defined error even though the prefixes do not overlap. This issue is resolved if I remove all currently defined notifications and re-add them along with the new one.
Example code
Expected Behavior
I expect the relevant event notifications to be created on the relevant S3 bucket
Current Behavior
CloudFromation is throwing an error
Reproduction Steps
Possible Solution
No response
Additional Information/Context
Recently upgraded to "aws-cdk-lib": "^2.138.0" which might be causing this issue
CDK CLI Version
2.138.0
Framework Version
No response
Node.js Version
10.8.1
OS
Linux
Language
TypeScript
Language Version
No response
Other information
No response
The text was updated successfully, but these errors were encountered: