Make containerd config configurable

[ncsibra]

What would you like to be added:
Make containerd config configurable, similarly to --docker-config-json option.

Why is this needed:
By default max_container_log_line_size set to 16384 which means when a line is bigger than this size will be split into multiple log messages.
Our problem with this, that it's based on byte size and sometimes it splits a UTF-8 character containing multiple bytes, which makes it invalid and unable to parse by fluent-bit/fluentd.
E.g.:

...,"custom3":{"label":"🥱🤭😓🤑👥🖖🤲🤲🤠🤠🤠🤠🤠👊👨👨�
2021-12-10T09:59:12.166751315Z stderr F ���‍♂️🧝🧝🧞🧩🎗️🏐🎫🎰🎇🏡🏯1️⃣🆕🈹",

It's possible to modify the config in a custom userdata script, but would be nicer to do this through a parameter to the bootstrap script.

[jfrancisco0]

This would also be useful to configure a registry mirror, which seems to be possible with docker only, currently.

[bryantbiggs]

does #790 resolve this?

[ncsibra]

@bryantbiggs I think it's not, because max_container_log_line_size option should be under the [plugins."io.containerd.grpc.v1.cri"] table, which is already defined in the default configuration and Toml does not allow to redefine tables.

So the solution implemented in #790 does not support adding or overwriting any keys under tables that are already defined by the default configuration.
But I'm not very familiar with Toml, so maybe there's a solution.

A similar option like --docker-config-json would be better, which overwrites the whole configuration file.
Maybe harder to maintain in the long run, but wouldn't cause any Toml merge/validation issues and would be consistent in behavior with the already existing option to docker.

[rishabhToshniwal]

@ncsibra Sorry I am late here, but did you find the solution for it. I am facing a similar issue

[cartermckinnon]

@rishabhToshniwal the options for this are better on AL2023-based EKS AMIs, see: https://awslabs.github.io/amazon-eks-ami/nodeadm/doc/examples/#configuring-containerd

This doesn't rely on containerd's own merge/import process, which has limitations discussed above.