bootstrap_extra_args are not passed to userdata when using eks_managed_node_group in root module

[youwalther65]

Description

When creating node groups in EKS module using MNG with bootstrap_extra_args = "--container-runtime containerd" these are not merged into bootstrap config.

Versions

• Terraform: v1.0.11

• Provider(s): registry.terraform.io/hashicorp/aws v3.72.0

• Module: eks 18.2.0

Code Snippet to Reproduce

eks_managed_node_groups = {
...
bootstrap_extra_args = "--container-runtime containerd"
...

Expected behavior

userdata should contain "--container-runtime containerd" as an argument passed to "/etc/eks/bootstrap.sh" script. Using this the EC2 instance will use containerd instead of docker ascontainer runtime

Actual behavior

argument is not passed into userdata. docker is running.

Additional considerations

The same inout works for self-MNG and is indeed mentioned as an input for MNG referenced here:
https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/latest/submodules/eks-managed-node-group?tab=inputs

bootstrap_extra_args string

Description: Additional arguments passed to the bootstrap script. When platform = bottlerocket; these are additional settings that are provided to the Bottlerocket user data

[youwalther65]

Just saw the PR 1789 which will hopefully address this as well:

[bryantbiggs]

bottlerocket does not use a bootstrap.sh script but instead is a TOML file so bootstrap_extra_args is not valid for bottlerocket based instances

for use with EKS managed node groups using the default AMI (details are very important), there is an issue with the way the EKS managed node group service handles bootstrap configurations - see issue here awslabs/amazon-eks-ami#844

[youwalther65]

I use AL2 and I have to admit that I don't see your point. bootstrap.sh has added an option "--container-runtime" which can be set to "--container-runtime containerd". And this can be used just with custom LT modifying the userdata but one can leave the standard AMI in place and can still benefits form MNG. So do you really think my approach above is not handled by EKS API?

[bryantbiggs]

As an exercise, I would recommend you launch a node with an EKS managed node group using the default AMI provided by the service and look at the user data generated. Then, try to envision how you are going to influence that user data

[youwalther65]

Bryant, thank you, you are right, I wasn't aware of the fact that modifying userdata really requires a custom AMI as referenced here:
https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html
But I can live with that. So I will try bootstrap_extra_args in conjunction with ami_id (just referencing to most_recent = true aws_ami resource) in "eks_managed_node_groups" . Appreciate your efforts in the module here! And I did a +1 for the EKS roadmap issue which covers this topic " [EKS] [request]: Simplify EKS managed worker bootstrap args #1178 " aws/containers-roadmap#1178

[bryantbiggs]

Thank you! There are lots of discussions going on around user data - Kubernetes is a complex beast, add on to that is the way EKS manages the cluster, then throw in all the different ways of managing nodes/node groups, etc. - it gets pretty hairy pretty quickly!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.