Document Kustomization.spec.kubeConfig for Cluster API

fluxcd · Oct 15, 2020 · e33e7a4 · e33e7a4
1 parent 75f1b05
commit e33e7a4
Show file tree

Hide file tree

Showing 3 changed files with 111 additions and 10 deletions.
diff --git a/api/v1beta1/kustomization_types.go b/api/v1beta1/kustomization_types.go
@@ -50,6 +50,7 @@ type KustomizationSpec struct {
 	Interval metav1.Duration `json:"interval"`
 
 	// The KubeConfig for reconciling the Kustomization on a remote cluster.
+	// Apply, Prune, HealthCheck, and Delete are all functional.
 	// +optional
 	KubeConfig *KubeConfig `json:"kubeConfig,omitempty"`
 
@@ -125,9 +126,19 @@ type Decryption struct {
 // KubeConfig references a Kubernetes secret generated by CAPI.
 // that contains a kubeconfig file.
 type KubeConfig struct {
-	// The secret name containing a 'value' key
-	// with the kubeconfig file as the value.
-	// Ref: https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24
+	// The secret name containing a 'value' key with the kubeconfig file as the value.
+	// This secret must be in the same Namespace as the Kustomization.
+	// KubeConfig secrets maintained by Cluster API bootstrap providers can be used here.
+	// (ex: If your CAPI Cluster's name is `stage`, set this to `stage-kubeconfig`.
+	// Ensure the Kustomization is in the same Namespace as the Cluster object.
+	// Ref: https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24)
+	// The reconciliation clients are regularly refreshed from the Secret, so
+	// rotating kubeconfigs for KaaS control-planes from cloud-providers are supported.
+	// These kubeconfigs follow the same design constraints as Cluster API.
+	// It is recommended that kubeconfigs be self-contained, and the Secret be
+	// regularly updated if credentials such as a cloud-access-token expire.
+	// Cloud-specific `cmd-path` auth helpers will not function without adding
+	// binaries and credentials to the kustomize-controller Pod.
 	// +required
 	SecretRef corev1.LocalObjectReference `json:"secretRef,omitempty"`
 }

diff --git a/docs/api/kustomize.md b/docs/api/kustomize.md
@@ -122,7 +122,8 @@ KubeConfig
 </td>
 <td>
 <em>(Optional)</em>
-<p>The KubeConfig for reconciling the Kustomization on a remote cluster.</p>
+<p>The KubeConfig for reconciling the Kustomization on a remote cluster.
+Apply, Prune, HealthCheck, and Delete are all functional.</p>
 </td>
 </tr>
 <tr>
@@ -471,9 +472,19 @@ Kubernetes core/v1.LocalObjectReference
 </em>
 </td>
 <td>
-<p>The secret name containing a &lsquo;value&rsquo; key
-with the kubeconfig file as the value.
-Ref: <a href="https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24">https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24</a></p>
+<p>The secret name containing a &lsquo;value&rsquo; key with the kubeconfig file as the value.
+This secret must be in the same Namespace as the Kustomization.
+KubeConfig secrets maintained by Cluster API bootstrap providers can be used here.
+(ex: If your CAPI Cluster&rsquo;s name is <code>stage</code>, set this to <code>stage-kubeconfig</code>.
+Ensure the Kustomization is in the same Namespace as the Cluster object.
+Ref: <a href="https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24)">https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24)</a>
+The reconciliation clients are regularly refreshed from the Secret, so
+rotating kubeconfigs for KaaS control-planes from cloud-providers are supported.
+These kubeconfigs follow the same design constraints as Cluster API.
+It is recommended that kubeconfigs be self-contained, and the Secret be
+regularly updated if credentials such as a cloud-access-token expire.
+Cloud-specific <code>cmd-path</code> auth helpers will not function without adding
+binaries and credentials to the kustomize-controller Pod.</p>
 </td>
 </tr>
 </tbody>
@@ -551,7 +562,8 @@ KubeConfig
 </td>
 <td>
 <em>(Optional)</em>
-<p>The KubeConfig for reconciling the Kustomization on a remote cluster.</p>
+<p>The KubeConfig for reconciling the Kustomization on a remote cluster.
+Apply, Prune, HealthCheck, and Delete are all functional.</p>
 </td>
 </tr>
 <tr>

diff --git a/docs/spec/v1beta1/kustomization.md b/docs/spec/v1beta1/kustomization.md
@@ -88,12 +88,24 @@ type Decryption struct {
 }
 ```
 
-KubeConfig references a Kubernetes secret generated by CAPI:
+KubeConfig references a Kubernetes Secret for applying to another cluster.
+This can be used with Cluster API:
 
 ```go
 type KubeConfig struct {
 	// The secret name containing a 'value' key with the kubeconfig file as the value.
-	// Ref: https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24
+	// This secret must be in the same Namespace as the Kustomization.
+	// KubeConfig secrets maintained by Cluster API bootstrap providers can be used here.
+	// (ex: If your CAPI Cluster's name is `stage`, set this to `stage-kubeconfig`.
+	// Ensure the Kustomization is in the same Namespace as the Cluster object.
+	// Ref: https://github.com/kubernetes-sigs/cluster-api/blob/release-0.3/util/secret/consts.go#L24)
+	// The reconciliation clients are regularly refreshed from the Secret, so
+	// rotating kubeconfigs for KaaS control-planes from cloud-providers are supported.
+	// These kubeconfigs follow the same design constraints as Cluster API.
+	// It is recommended that kubeconfigs be self-contained, and the Secret be
+	// regularly updated if credentials such as a cloud-access-token expire.
+	// Cloud-specific `cmd-path` auth helpers will not function without adding
+	// binaries and credentials to the kustomize-controller Pod.
 	// +required
 	SecretRef corev1.LocalObjectReference `json:"secretRef,omitempty"`
 }
@@ -486,6 +498,72 @@ account. If the kustomization contains cluster level objects like CRDs or object
 namespace, the reconciliation will fail since the account it runs under has no permissions to alter objects
 outside of the `webapp` namespace.
 
+## Cluster-API / Remote Clusters
+
+If the `kubeConfig` field is set, objects will be applied, health-checked, pruned, and deleted for the default
+cluster specified in that KubeConfig instead of using the in-cluster ServiceAccount.
+
+This is done by specifying a KubeConfig from a Secret in the same Namespace as the Kustomization.
+The KubeConfig bytes are loaded from the `value` key of the Secret's data.
+This Secret can be regularly updated if cluster-access-tokens have to rotate due to expiration.
+
+This composes well with Cluster API bootstrap providers such as CAPBK(kubeadm) as well as the CAPA(aws) EKS integration.
+
+To reconcile a kustomization to a CAPI controlled cluster, put the `Kustomization` in the same Namespace as your
+`Cluster` object, and set the `kubeConfig.secretName` to `<cluster-name>-kubeconfig`:
+
+```yaml
+apiVersion: cluster.x-k8s.io/v1alpha3
+kind: Cluster
+metadata:
+  name: stage  # the kubeconfig Secret will contain the Cluster name
+  namespace: capi-stage
+spec:
+  clusterNetwork:
+    pods:
+      cidrBlocks:
+      - 10.100.0.0/16
+    serviceDomain: stage-cluster.local
+    services:
+      cidrBlocks:
+      - 10.200.0.0/12
+  controlPlaneRef:
+    apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
+    kind: KubeadmControlPlane
+    name: stage-control-plane
+    namespace: capi-stage
+  infrastructureRef:
+    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
+    kind: DockerCluster
+    name: stage
+    namespace: capi-stage
+---
+# ... unrelated Cluster API objects omitted for brevity ...
+---
+apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
+kind: Kustomization
+metadata:
+  name: backend
+  namespace: capi-stage
+spec:
+  interval: 5m
+  path: "./webapp/backend/"
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: webapp
+  kubeConfig:
+    secretName: stage-kubeconfig  # Cluster API creates this for the matching Cluster
+```
+
+The Cluster and Kustomization can be created at the same time.
+The Kustomization will eventually reconcile once the cluster is available.
+
+> **Note** that the KubeConfig should be self-contained and not rely on binaries, environment, or credential files
+> from the kustomize-controller Pod.
+> This matches the constraints of KubeConfigs from current Cluster API providers.
+> KubeConfigs with `cmd-path` in them likely won't work without a custom, per-cluster installation of kustomize-controller.
+
 ## Secrets decryption
 
 In order to store secrets safely in a public or private Git repository,