Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Chart] Secure chart for best practices #10000

Merged
merged 5 commits into from
Feb 23, 2022
Merged

[Chart] Secure chart for best practices #10000

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
09fa744
[Chart] Allow securityContexts, extraVolumes/extraVolumeMounts, extra…
atorrescogollo Feb 2, 2022
806274c
[Chart] Fixed broken metadata in values.yaml and generated README.md
atorrescogollo Feb 2, 2022
32ce381
[Chart] Fixed failing Probes
atorrescogollo Feb 2, 2022
7f7900c
Fixed some scheduler and webapp issues
atorrescogollo Feb 2, 2022
685f187
fix conflicts
alafanechere Feb 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
215 changes: 141 additions & 74 deletions charts/airbyte/README.md
View file
Open in desktop

Large diffs are not rendered by default.

35 changes: 35 additions & 0 deletions charts/airbyte/templates/pod-sweeper/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,20 +38,55 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
{{- if .Values.podSweeper.containerSecurityContext }}
securityContext: {{- toYaml .Values.podSweeper.containerSecurityContext | nindent 10 }}
{{- end }}
volumeMounts:
- mountPath: /script/sweep-pod.sh
subPath: sweep-pod.sh
name: sweep-pod-script
- mountPath: /.kube
name: kube-config
{{- if .Values.podSweeper.extraVolumeMounts }}
{{ toYaml .Values.podSweeper.extraVolumeMounts | nindent 8 }}
{{- end }}
command: ["/bin/bash", "-c", /script/sweep-pod.sh]
{{- if .Values.podSweeper.resources }}
resources: {{- toYaml .Values.podSweeper.resources | nindent 10 }}
{{- end }}
{{- if .Values.podSweeper.livenessProbe.enabled }}
livenessProbe:
exec:
command:
- /bin/sh
- -ec
- grep -aq sweep-pod.sh /proc/1/cmdline
initialDelaySeconds: {{ .Values.podSweeper.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.podSweeper.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.podSweeper.livenessProbe.timeoutSeconds }}
successThreshold: {{ .Values.podSweeper.livenessProbe.successThreshold }}
failureThreshold: {{ .Values.podSweeper.livenessProbe.failureThreshold }}
{{- end }}
{{- if .Values.podSweeper.readinessProbe.enabled }}
readinessProbe:
exec:
command:
- /bin/sh
- -ec
- grep -aq sweep-pod.sh /proc/1/cmdline
initialDelaySeconds: {{ .Values.podSweeper.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.podSweeper.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.podSweeper.readinessProbe.timeoutSeconds }}
successThreshold: {{ .Values.podSweeper.readinessProbe.successThreshold }}
failureThreshold: {{ .Values.podSweeper.readinessProbe.failureThreshold }}
{{- end }}
volumes:
- name: kube-config
emptyDir: {}
- name: sweep-pod-script
configMap:
name: {{ include "airbyte.fullname" . }}-sweep-pod-script
defaultMode: 0755
{{- if .Values.podSweeper.extraVolumes }}
{{ toYaml .Values.podSweeper.extraVolumes | nindent 6 }}
{{- end }}
35 changes: 35 additions & 0 deletions charts/airbyte/templates/scheduler/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,11 +212,46 @@ spec:
{{- if .Values.scheduler.resources }}
resources: {{- toYaml .Values.scheduler.resources | nindent 10 }}
{{- end }}
{{- if .Values.scheduler.containerSecurityContext }}
securityContext: {{- toYaml .Values.scheduler.containerSecurityContext | nindent 10 }}
{{- end }}
{{- if .Values.scheduler.livenessProbe.enabled }}
livenessProbe:
exec:
command:
- /bin/sh
- -ec
- grep -qa airbyte.scheduler.app.SchedulerApp /proc/1/cmdline
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please explain what the liveness probes commands are checking exactly? 🙏

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! Since the scheduler is not opening any port, the only thing we can do is to ensure that the entrypoint was the airbyte.scheduler.app.SchedulerApp java class and not another random command. In /proc/1/cmdline we can find the command executed for the entrypoint (pid=1) since there is no ps command inside the container.

This is basically needed because probes are a requisite for some admission controllers (e.g.: Kyverno require_probes policy).

initialDelaySeconds: {{ .Values.scheduler.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.scheduler.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.scheduler.livenessProbe.timeoutSeconds }}
successThreshold: {{ .Values.scheduler.livenessProbe.successThreshold }}
failureThreshold: {{ .Values.scheduler.livenessProbe.failureThreshold }}
{{- end }}
{{- if .Values.scheduler.readinessProbe.enabled }}
readinessProbe:
exec:
command:
- /bin/sh
- -ec
- grep -qa airbyte.scheduler.app.SchedulerApp /proc/1/cmdline
initialDelaySeconds: {{ .Values.scheduler.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.scheduler.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.scheduler.readinessProbe.timeoutSeconds }}
successThreshold: {{ .Values.scheduler.readinessProbe.successThreshold }}
failureThreshold: {{ .Values.scheduler.readinessProbe.failureThreshold }}
{{- end }}
volumeMounts:
- name: gcs-log-creds-volume
mountPath: /secrets/gcs-log-creds
readOnly: true
{{- if .Values.scheduler.extraVolumeMounts }}
{{ toYaml .Values.scheduler.extraVolumeMounts | nindent 8 }}
{{- end }}
volumes:
- name: gcs-log-creds-volume
secret:
secretName: gcs-log-creds
{{- if .Values.scheduler.extraVolumes }}
{{ toYaml .Values.scheduler.extraVolumes | nindent 6 }}
{{- end }}
9 changes: 9 additions & 0 deletions charts/airbyte/templates/server/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,9 @@ spec:
{{- if .Values.server.resources }}
resources: {{- toYaml .Values.server.resources | nindent 10 }}
{{- end }}
{{- if .Values.server.containerSecurityContext }}
securityContext: {{- toYaml .Values.server.containerSecurityContext | nindent 10 }}
{{- end }}
volumeMounts:
- name: airbyte-data
mountPath: /configs
Expand All @@ -224,10 +227,16 @@ spec:
- name: gcs-log-creds-volume
mountPath: /secrets/gcs-log-creds
readOnly: true
{{- if .Values.server.extraVolumeMounts }}
{{ toYaml .Values.server.extraVolumeMounts | nindent 8 }}
{{- end }}
volumes:
- name: airbyte-data
persistentVolumeClaim:
claimName: {{ include "common.names.fullname" . }}-data
- name: gcs-log-creds-volume
secret:
secretName: gcs-log-creds
{{- if .Values.server.extraVolumes }}
{{ toYaml .Values.server.extraVolumes | nindent 6 }}
{{- end }}
39 changes: 39 additions & 0 deletions charts/airbyte/templates/temporal/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ spec:
{{- if .Values.temporal.affinity }}
affinity: {{- include "common.tplvalues.render" (dict "value" .Values.temporal.affinity "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.temporal.extraInitContainers }}
initContainers:
{{- toYaml .Values.temporal.extraInitContainers | nindent 6 }}
{{- end }}
containers:
- name: airbyte-temporal
image: {{ include "airbyte.temporalImage" . }}
Expand Down Expand Up @@ -64,16 +68,51 @@ spec:
{{- end }}
ports:
- containerPort: 7233
{{- if .Values.temporal.containerSecurityContext }}
securityContext: {{- toYaml .Values.temporal.containerSecurityContext | nindent 10 }}
{{- end }}
volumeMounts:
- name: airbyte-temporal-dynamicconfig
mountPath: "/etc/temporal/config/dynamicconfig/"
{{- if .Values.temporal.resources }}
resources: {{- toYaml .Values.temporal.resources | nindent 10 }}
{{- end }}
{{- if .Values.temporal.extraVolumeMounts }}
{{ toYaml .Values.temporal.extraVolumeMounts | nindent 8 }}
{{- end }}
{{- if .Values.temporal.livenessProbe.enabled }}
livenessProbe:
exec:
command:
- /bin/sh
- -ec
- 'test $(ps -ef | grep -v grep | grep temporal-server | wc -l) -eq 1'
initialDelaySeconds: {{ .Values.temporal.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.temporal.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.temporal.livenessProbe.timeoutSeconds }}
successThreshold: {{ .Values.temporal.livenessProbe.successThreshold }}
failureThreshold: {{ .Values.temporal.livenessProbe.failureThreshold }}
{{- end }}
{{- if .Values.temporal.readinessProbe.enabled }}
readinessProbe:
exec:
command:
- /bin/sh
- -ec
- 'test $(ps -ef | grep -v grep | grep temporal-server | wc -l) -eq 1'
initialDelaySeconds: {{ .Values.temporal.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.temporal.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.temporal.readinessProbe.timeoutSeconds }}
successThreshold: {{ .Values.temporal.readinessProbe.successThreshold }}
failureThreshold: {{ .Values.temporal.readinessProbe.failureThreshold }}
{{- end }}
volumes:
- name: airbyte-temporal-dynamicconfig
configMap:
name: airbyte-temporal-dynamicconfig
items:
- key: development.yaml
path: development.yaml
{{- if .Values.temporal.extraVolumes }}
{{ toYaml .Values.temporal.extraVolumes | nindent 6 }}
{{- end }}
32 changes: 32 additions & 0 deletions charts/airbyte/templates/webapp/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,42 @@ spec:
{{- if .Values.webapp.extraEnv }}
{{ .Values.webapp.extraEnv | toYaml | nindent 8 }}
{{- end }}
{{- if .Values.webapp.livenessProbe.enabled }}
livenessProbe:
tcpSocket:
port: http
initialDelaySeconds: {{ .Values.webapp.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.webapp.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.webapp.livenessProbe.timeoutSeconds }}
successThreshold: {{ .Values.webapp.livenessProbe.successThreshold }}
failureThreshold: {{ .Values.webapp.livenessProbe.failureThreshold }}
{{- end }}
{{- if .Values.webapp.readinessProbe.enabled }}
readinessProbe:
httpGet:
path: /api/v1/health
port: http
initialDelaySeconds: {{ .Values.webapp.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.webapp.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.webapp.readinessProbe.timeoutSeconds }}
successThreshold: {{ .Values.webapp.readinessProbe.successThreshold }}
failureThreshold: {{ .Values.webapp.readinessProbe.failureThreshold }}
{{- end }}
ports:
- name: http
containerPort: 80
protocol: TCP
{{- if .Values.webapp.resources }}
resources: {{- toYaml .Values.webapp.resources | nindent 10 }}
{{- end }}
{{- if .Values.webapp.containerSecurityContext }}
securityContext: {{- toYaml .Values.webapp.containerSecurityContext | nindent 10 }}
{{- end }}
volumeMounts:
{{- if .Values.webapp.extraVolumeMounts }}
{{ toYaml .Values.webapp.extraVolumeMounts | nindent 8 }}
{{- end }}
volumes:
{{- if .Values.webapp.extraVolumes }}
{{ toYaml .Values.webapp.extraVolumes | nindent 6 }}
{{- end }}
9 changes: 9 additions & 0 deletions charts/airbyte/templates/worker/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,11 +287,20 @@ spec:
{{- if .Values.worker.resources }}
resources: {{- toYaml .Values.worker.resources | nindent 10 }}
{{- end }}
{{- if .Values.worker.containerSecurityContext }}
securityContext: {{- toYaml .Values.worker.containerSecurityContext | nindent 10 }}
{{- end }}
volumeMounts:
- name: gcs-log-creds-volume
mountPath: /secrets/gcs-log-creds
readOnly: true
{{- if .Values.worker.extraVolumeMounts }}
{{ toYaml .Values.worker.extraVolumeMounts | nindent 8 }}
{{- end }}
volumes:
- name: gcs-log-creds-volume
secret:
secretName: gcs-log-creds
{{- if .Values.worker.extraVolumes }}
{{ toYaml .Values.worker.extraVolumes | nindent 6 }}
{{- end }}
Loading