Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate the cron container from sidecar to K8s CronJob #703

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

MartinKirchner
Copy link

@MartinKirchner MartinKirchner commented Mar 3, 2025

Description of the change

With this change we no longer deploy a sidecar container to execute the cronjob, instead we deploy a Kubernetes CronJob object.

Benefits

  • For me this has the major advantage that the CronJob can run as user www-data (uid 33) and execute the php command directly whereas crond in the sidecar has to run as root since otherwise the impersonation of user www-data fails (crond: can't set groups: Operation not permitted).
  • History of failed and successful executions
  • Apart from that I think that is also more robust if the container fails for some reason.

Possible drawbacks

None obvious to me.

Applicable issues

Additional information

Checklist

As discussed in nextcloud#577

For me this has the major advantage that the CronJob can run as user
www-data (uid 33) and execute the php command directly whereas crond
in the sidecar has to run as root since otherwise the impersonation
of user www-data fails (crond: can't set groups: Operation not permitted).

Apart from that I think that is also more robust if the container fails
for some reason.

Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
@pschichtel
Copy link

One concern that I had while originally thinking about the implementation: This would allow the cronjob to run even while the rest of nextcloud is not running (e.g. during upgrades). I'm not entirely sure if this is actually a problem, but worth checking what the implications are. I'm also not entirely sure if the current setup properly handles this in all cases.

Martin Kirchner added 6 commits March 4, 2025 07:23
Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
Allows to extract more common definitions of cronJob and deployment.

Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
Such as nodeSelector, affinity, tolerations, volumnes, securityContext, ...

Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
This way it can be shared between deployment and cronJob.

Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
With that we can create a reusable template for the pod in the
deployment and cronJob more easily.

Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
@MartinKirchner
Copy link
Author

One concern that I had while originally thinking about the implementation: This would allow the cronjob to run even while the rest of nextcloud is not running (e.g. during upgrades). I'm not entirely sure if this is actually a problem, but worth checking what the implications are. I'm also not entirely sure if the current setup properly handles this in all cases.

Thanks for this thought. But I think using the CronJob is not worse (maybe better) than the current state. Yes, it might happen that a cronJob is executed while a new, updated pod is deployed and potentially migrate the data model. But I have no idea how to avoid this one way or the other.

@MartinKirchner MartinKirchner marked this pull request as ready for review March 4, 2025 14:40
@pschichtel
Copy link

I did come across a few apps that ask their users to setup additional cronjobs, I wonder if that could be supported.

@MartinKirchner
Copy link
Author

I did come across a few apps that ask their users to setup additional cronjobs, I wonder if that could be supported.

With the configurable command you can do basically whatever you want to, e.g. chain several commands, mount a script and execute this, ...

{{- end -}}


{{/*
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following is more common in helm-charts.
Where you have to add app.kubernetes.io/component: app manuelle.

{{/*
Common labels
*/}}
{{- define "nextcloud.labels" -}}
helm.sh/chart: {{ include "nextcloud.chart" . }}
{{ include "nextcloud.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "nextcloud.selectorLabels" -}}
app.kubernetes.io/name: {{ include "headscale.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

PS: Maybe you should move such changes into smaller PRs for easiert review (and merge)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wrenix
Thanks for your comment.

To make the review easier I used a commit for every step of the refactoring over smaller PRs as it makes only sense to me if get everything merged in. In commit a874bc4 you can see that I simply moved the labels as they were from the deployment.yamltohelpers.tpl`. So I guess someone had good reasoning to define the labels like this.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you must differentiate the labels used by the deployment to select its pod vs the labels on the cronjob pods, otherwise cronjob pods would show up as pods of a deployment and might get controlled by the replicaset controller, right? So app in app.kubernetes.io/component: app should probably be cronjob for cronjob pods or so.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pschichtel Good that you noticed this! Thank you.

Copy link
Collaborator

@wrenix wrenix Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MartinKirchner as a Co-Maintainer i could easier merge such single changes / small PRs.
For the change from sidecar to cronjob itself, i believe we need a longer discussion, see #683

PS: i like to have an _helper for the labels and that could be in this week if i got a easy to review.

Martin Kirchner added 3 commits March 5, 2025 19:36
So that we can keep as much as possible in sync between the deployment
and the cronJob.

Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
Signed-off-by: Martin Kirchner <martin.kirchner@cas.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: Use CronJob resources instead of a container with crond
3 participants