diff --git a/.github/ISSUE_TEMPLATE/kubernetes_bump.md b/.github/ISSUE_TEMPLATE/kubernetes_bump.md index ac1733c14a2f..e9fcce9f5fd5 100644 --- a/.github/ISSUE_TEMPLATE/kubernetes_bump.md +++ b/.github/ISSUE_TEMPLATE/kubernetes_bump.md @@ -26,9 +26,11 @@ changes should be cherry-picked to all release series that will support the new * `test/*`: search for occurrences of the previous Kubernetes version * `Tiltfile` * Ensure the latest available kind version is used (including the latest images for this kind release) - * Add new images in the [kind mapper.go](https://github.com/kubernetes-sigs/cluster-api/blob/48ae58e51f9723ab7b9635d0e05ee54c4843707a/test/infrastructure/kind/mapper.go#L79). + * Add new images in the [kind mapper.go](https://github.com/kubernetes-sigs/cluster-api/blob/0f47a19e038ee6b0d3b1e7675a62cdaf84face8c/test/infrastructure/kind/mapper.go#L79). * See the [kind releases page](https://github.com/kubernetes-sigs/kind/releases) for the list of released images. - * Set new default image for the [test framework](https://github.com/kubernetes-sigs/cluster-api/blob/48ae58e51f9723ab7b9635d0e05ee54c4843707a/test/framework/bootstrap/kind_provider.go#L40) + * Set new default image for the [test framework](https://github.com/kubernetes-sigs/cluster-api/blob/0f47a19e038ee6b0d3b1e7675a62cdaf84face8c/test/framework/bootstrap/kind_provider.go#L40) + * If code changes are required for CAPD to incorporate the new Kind version, update [kind latestMode](https://github.com/kubernetes-sigs/cluster-api/blob/0f47a19e038ee6b0d3b1e7675a62cdaf84face8c/test/infrastructure/kind/mapper.go#L66) + * Prior art: #10094 * Verify the quickstart manually * Bump `InitWithKubernetesVersion` and `WorkloadKubernetesVersion` in `clusterctl_upgrade_test.go` * Note: Only bump for Cluster API versions that will support the new Kubernetes release. diff --git a/.github/workflows/pr-dependabot.yaml b/.github/workflows/pr-dependabot.yaml index 4099a664bd18..b8b3bbf792fc 100644 --- a/.github/workflows/pr-dependabot.yaml +++ b/.github/workflows/pr-dependabot.yaml @@ -27,7 +27,7 @@ jobs: uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # tag=v5.0.0 with: go-version: ${{ steps.vars.outputs.go_version }} - - uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # tag=v4.0.0 + - uses: actions/cache@ab5e6d0c87105b4c9c2047343972218f562e4319 # tag=v4.0.1 name: Restore go cache with: path: | diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index f76f0585341a..f825e4d1bd1e 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -22,7 +22,7 @@ jobs: fetch-depth: 0 - name: Get changed files id: changed-files - uses: tj-actions/changed-files@90a06d6ba9543371ab4df8eeca0be07ca6054959 # tag=v42.0.2 + uses: tj-actions/changed-files@aa08304bd477b800d468db44fe10f6c61f7f7b11 # tag=v42.1.0 - name: Get release version id: release-version run: | @@ -105,7 +105,7 @@ jobs: curl -L "https://raw.githubusercontent.com/${{ github.repository }}/main/CHANGELOG/${{ env.RELEASE_TAG }}.md" \ -o "${{ env.RELEASE_TAG }}.md" - name: Release - uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # tag=v1 + uses: softprops/action-gh-release@3198ee18f814cdf787321b4a32a26ddbf37acc52 # tag=v2.0.3 with: draft: true files: out/* diff --git a/CHANGELOG/v1.5.6.md b/CHANGELOG/v1.5.6.md new file mode 100644 index 000000000000..86ca91ad340f --- /dev/null +++ b/CHANGELOG/v1.5.6.md @@ -0,0 +1,24 @@ +## 👌 Kubernetes version support + +- Management Cluster: v1.24.x -> v1.28.x +- Workload Cluster: v1.22.x -> v1.28.x + +[More information about version support can be found here](https://cluster-api.sigs.k8s.io/reference/versions.html) + +## Changes since v1.5.5 +## :chart_with_upwards_trend: Overview +- 6 new commits merged +- 1 bug fixed 🐛 + +## :bug: Bug Fixes +- ClusterCacheTracker: Fix ClusterCacheTracker memory leak (#10065) + +## :seedling: Others +- clusterctl: Bump cert-manager to v1.14.2 (#10121) (#10128) +- Community meeting: Promote chrischdi to Cluster API maintainer (#10090) +- Dependency: Bump Go to 1.21.5 (#10153) + +:book: Additionally, there has been 1 contribution to our documentation and book. (#10117) + + +_Thanks to all our contributors!_ 😊 diff --git a/CHANGELOG/v1.6.2.md b/CHANGELOG/v1.6.2.md new file mode 100644 index 000000000000..5ab1ec8d97e7 --- /dev/null +++ b/CHANGELOG/v1.6.2.md @@ -0,0 +1,34 @@ +## 👌 Kubernetes version support + +- Management Cluster: v1.25.x -> v1.29.x +- Workload Cluster: v1.23.x -> v1.29.x + +[More information about version support can be found here](https://cluster-api.sigs.k8s.io/reference/versions.html) + +## Highlights +* :warning: Warning: This release fixes a bug (#10051) that was introduced in v1.6.0, which caused a regression in the conversion of v1alpha3/v1alpha4 objects. It is recommended to upgrade to v1.6.2 to avoid the issue. + +## Changes since v1.6.1 +## :chart_with_upwards_trend: Overview +- 16 new commits merged +- 3 bugs fixed 🐛 + +## :bug: Bug Fixes +- [API/e2e]: Restore v1alpha3/v1alpha4 conversion to fix SSA issue & add e2e test coverage (#10151) + - :warning: Warning: This change is a fix for the conversion bug that was introduced in v1.6.0. +- ClusterCacheTracker: Fix ClusterCacheTracker memory leak (#10064) +- Machine: Watch external objects for machine before deleting (#10177) + +## :seedling: Others +- clusterctl: Bump cert-manager to v1.14.2 (#10120) (#10127) +- clusterctl: Clarify rules for adding new clusterctl default providers (#10109) +- Community meeting: Promote chrischdi to Cluster API maintainer (#10089) +- Dependency: Bump controller runtime v0.16.5 (#10163) +- Dependency: Bump Go to 1.21.5 (#10152) +- e2e: Use manager in test extension (#10106) +- Testing: Print conformance image used in kubetest (#10081) + +:book: Additionally, there have been 4 contributions to our documentation and book. (#10024, #10047, #10105, #10116) + + +_Thanks to all our contributors!_ 😊 diff --git a/Makefile b/Makefile index dea7187b21b9..b3a63b577d98 100644 --- a/Makefile +++ b/Makefile @@ -23,7 +23,7 @@ SHELL:=/usr/bin/env bash # # Go. # -GO_VERSION ?= 1.21.5 +GO_VERSION ?= 1.21.8 GO_CONTAINER_IMAGE ?= docker.io/library/golang:$(GO_VERSION) # Use GOPROXY environment variable if set @@ -108,7 +108,7 @@ KUSTOMIZE_BIN := kustomize KUSTOMIZE := $(abspath $(TOOLS_BIN_DIR)/$(KUSTOMIZE_BIN)-$(KUSTOMIZE_VER)) KUSTOMIZE_PKG := sigs.k8s.io/kustomize/kustomize/v4 -SETUP_ENVTEST_VER := v0.0.0-20231012212722-e25aeebc7846 +SETUP_ENVTEST_VER := v0.0.0-20240215143116-d0396a3d6f9f SETUP_ENVTEST_BIN := setup-envtest SETUP_ENVTEST := $(abspath $(TOOLS_BIN_DIR)/$(SETUP_ENVTEST_BIN)-$(SETUP_ENVTEST_VER)) SETUP_ENVTEST_PKG := sigs.k8s.io/controller-runtime/tools/setup-envtest @@ -123,7 +123,7 @@ GOTESTSUM_BIN := gotestsum GOTESTSUM := $(abspath $(TOOLS_BIN_DIR)/$(GOTESTSUM_BIN)-$(GOTESTSUM_VER)) GOTESTSUM_PKG := gotest.tools/gotestsum -CONVERSION_GEN_VER := v0.29.0 +CONVERSION_GEN_VER := v0.29.2 CONVERSION_GEN_BIN := conversion-gen # We are intentionally using the binary without version suffix, to avoid the version # in generated files. @@ -145,7 +145,7 @@ HADOLINT_FAILURE_THRESHOLD = warning SHELLCHECK_VER := v0.9.0 -TRIVY_VER := 0.47.0 +TRIVY_VER := 0.49.1 KPROMO_VER := v4.0.5 KPROMO_BIN := kpromo @@ -158,7 +158,7 @@ YQ_BIN := yq YQ := $(abspath $(TOOLS_BIN_DIR)/$(YQ_BIN)-$(YQ_VER)) YQ_PKG := github.com/mikefarah/yq/v4 -PLANTUML_VER := 1.2023.10 +PLANTUML_VER := 1.2024.3 GINKGO_BIN := ginkgo GINKGO_VER := $(call get_go_version,github.com/onsi/ginkgo/v2) @@ -183,7 +183,7 @@ IMPORT_BOSS_PKG := k8s.io/code-generator/cmd/import-boss CONVERSION_VERIFIER_BIN := conversion-verifier CONVERSION_VERIFIER := $(abspath $(TOOLS_BIN_DIR)/$(CONVERSION_VERIFIER_BIN)) -OPENAPI_GEN_VER := 5e7f5fd +OPENAPI_GEN_VER := 70dd376 OPENAPI_GEN_BIN := openapi-gen # We are intentionally using the binary without version suffix, to avoid the version # in generated files. @@ -546,10 +546,11 @@ generate-go-openapi: $(OPENAPI_GEN) $(CONTROLLER_GEN) ## Generate openapi go cod (cd ../ && $(MAKE) clean-generated-openapi-definitions SRC_DIRS="./$${pkg}"); \ echo "** Generating openapi schema for types in ./$${pkg} **"; \ $(OPENAPI_GEN) \ - --input-dirs=sigs.k8s.io/cluster-api/$${pkg} \ - --output-file-base=zz_generated.openapi \ - --output-package=sigs.k8s.io/cluster-api/$${pkg} \ - --go-header-file=../hack/boilerplate/boilerplate.generatego.txt; \ + --output-dir=../$${pkg} \ + --output-file=zz_generated.openapi.go \ + --output-pkg=sigs.k8s.io/cluster-api/$${pkg} \ + --go-header-file=../hack/boilerplate/boilerplate.generatego.txt \ + sigs.k8s.io/cluster-api/$${pkg}; \ done; \ rm sigs.k8s.io/cluster-api @@ -1166,6 +1167,10 @@ release-notes: release-notes-tool test-release-notes-tool: go test -C hack/tools -v -tags tools,integration sigs.k8s.io/cluster-api/hack/tools/release/notes +.PHONY: release-provider-issues-tool +release-provider-issues-tool: # Creates GitHub issues in a pre-defined list of CAPI provider repositories + @go run ./hack/tools/release/internal/update_providers/provider_issues.go + .PHONY: release-weekly-update-tool release-weekly-update-tool: go build -C hack/tools -o $(ROOT_DIR)/bin/weekly -tags tools sigs.k8s.io/cluster-api/hack/tools/release/weekly diff --git a/Tiltfile b/Tiltfile index fa5b97684163..6dbc6565b120 100644 --- a/Tiltfile +++ b/Tiltfile @@ -3,7 +3,7 @@ envsubst_cmd = "./hack/tools/bin/envsubst" clusterctl_cmd = "./bin/clusterctl" kubectl_cmd = "kubectl" -kubernetes_version = "v1.29.0" +kubernetes_version = "v1.29.2" load("ext://uibutton", "cmd_button", "location", "text_input") @@ -184,7 +184,7 @@ def load_provider_tiltfiles(): tilt_helper_dockerfile_header = """ # Tilt image -FROM golang:1.21.5 as tilt-helper +FROM golang:1.21.8 as tilt-helper # Install delve. Note this should be kept in step with the Go release minor version. RUN go install github.com/go-delve/delve/cmd/dlv@v1.21 # Support live reloading with Tilt @@ -195,7 +195,7 @@ RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com """ tilt_dockerfile_header = """ -FROM golang:1.21.5 as tilt +FROM golang:1.21.8 as tilt WORKDIR / COPY --from=tilt-helper /process.txt . COPY --from=tilt-helper /start.sh . @@ -342,23 +342,24 @@ def enable_provider(name, debug): port_forwards, links = get_port_forwards(debug) - build_go_binary( - context = p.get("context"), - reload_deps = p.get("live_reload_deps"), - debug = debug, - go_main = p.get("go_main", "main.go"), - binary_name = "manager", - label = label, - ) + if p.get("image"): + build_go_binary( + context = p.get("context"), + reload_deps = p.get("live_reload_deps"), + debug = debug, + go_main = p.get("go_main", "main.go"), + binary_name = "manager", + label = label, + ) - build_docker_image( - image = p.get("image"), - context = p.get("context"), - binary_name = "manager", - additional_docker_helper_commands = p.get("additional_docker_helper_commands", ""), - additional_docker_build_commands = p.get("additional_docker_build_commands", ""), - port_forwards = port_forwards, - ) + build_docker_image( + image = p.get("image"), + context = p.get("context"), + binary_name = "manager", + additional_docker_helper_commands = p.get("additional_docker_helper_commands", ""), + additional_docker_build_commands = p.get("additional_docker_build_commands", ""), + port_forwards = port_forwards, + ) additional_objs = [] p_resources = p.get("additional_resources", []) diff --git a/api/v1beta1/common_types.go b/api/v1beta1/common_types.go index 017f7f675791..b25985f1eccb 100644 --- a/api/v1beta1/common_types.go +++ b/api/v1beta1/common_types.go @@ -68,6 +68,10 @@ const ( // update that disallows a pre-existing Cluster to be populated with Topology information and Class. ClusterTopologyUnsafeUpdateClassNameAnnotation = "unsafe.topology.cluster.x-k8s.io/disable-update-class-name-check" + // ClusterTopologyUnsafeUpdateVersionAnnotation can be used to disable the webhook checks on + // update that disallows updating the .topology.spec.version on certain conditions. + ClusterTopologyUnsafeUpdateVersionAnnotation = "unsafe.topology.cluster.x-k8s.io/disable-update-version-check" + // ProviderNameLabel is the label set on components in the provider manifest. // This label allows to easily identify all the components belonging to a provider; the clusterctl // tool uses this label for implementing provider's lifecycle operations. diff --git a/api/v1beta1/zz_generated.openapi.go b/api/v1beta1/zz_generated.openapi.go index 073de884ba90..a3644d79f97a 100644 --- a/api/v1beta1/zz_generated.openapi.go +++ b/api/v1beta1/zz_generated.openapi.go @@ -19,8 +19,6 @@ limitations under the License. // Code generated by openapi-gen. DO NOT EDIT. -// This file was autogenerated by openapi-gen. Do not edit it manually! - package v1beta1 import ( @@ -856,7 +854,6 @@ func schema_sigsk8sio_cluster_api_api_v1beta1_ClusterVariable(ref common.Referen "value": { SchemaProps: spec.SchemaProps{ Description: "Value of the variable. Note: the value will be validated against the schema of the corresponding ClusterClassVariable from the ClusterClass. Note: We have to use apiextensionsv1.JSON instead of a custom JSON type, because controller-tools has a hard-coded schema for apiextensionsv1.JSON which cannot be produced by another type via controller-tools, i.e. it is not possible to have no type field. Ref: https://github.com/kubernetes-sigs/controller-tools/blob/d0e03a142d0ecdd5491593e941ee1d6b5d91dba6/pkg/crd/known_types.go#L106-L111", - Default: map[string]interface{}{}, Ref: ref("k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1.JSON"), }, }, @@ -902,7 +899,6 @@ func schema_sigsk8sio_cluster_api_api_v1beta1_Condition(ref common.ReferenceCall "lastTransitionTime": { SchemaProps: spec.SchemaProps{ Description: "Last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.", - Default: map[string]interface{}{}, Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.Time"), }, }, @@ -1385,8 +1381,7 @@ func schema_sigsk8sio_cluster_api_api_v1beta1_JSONSchemaProps(ref common.Referen Items: &spec.SchemaOrArray{ Schema: &spec.Schema{ SchemaProps: spec.SchemaProps{ - Default: map[string]interface{}{}, - Ref: ref("k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1.JSON"), + Ref: ref("k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1.JSON"), }, }, }, @@ -3558,8 +3553,7 @@ func schema_sigsk8sio_cluster_api_api_v1beta1_UnhealthyCondition(ref common.Refe }, "timeout": { SchemaProps: spec.SchemaProps{ - Default: 0, - Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.Duration"), + Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.Duration"), }, }, }, diff --git a/bootstrap/kubeadm/config/manager/manager.yaml b/bootstrap/kubeadm/config/manager/manager.yaml index d5e12b4572e8..dae2a9b6c615 100644 --- a/bootstrap/kubeadm/config/manager/manager.yaml +++ b/bootstrap/kubeadm/config/manager/manager.yaml @@ -26,6 +26,19 @@ spec: - "--bootstrap-token-ttl=${KUBEADM_BOOTSTRAP_TOKEN_TTL:=15m}" image: controller:latest name: manager + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_UID + valueFrom: + fieldRef: + fieldPath: metadata.uid ports: - containerPort: 9440 name: healthz diff --git a/cmd/clusterctl/client/config/providers_client.go b/cmd/clusterctl/client/config/providers_client.go index 3adad5719b5c..476738e3d561 100644 --- a/cmd/clusterctl/client/config/providers_client.go +++ b/cmd/clusterctl/client/config/providers_client.go @@ -45,6 +45,7 @@ const ( DOProviderName = "digitalocean" GCPProviderName = "gcp" HetznerProviderName = "hetzner" + HivelocityProviderName = "hivelocity-hivelocity" OutscaleProviderName = "outscale" IBMCloudProviderName = "ibmcloud" InMemoryProviderName = "in-memory" @@ -91,6 +92,11 @@ const ( K0smotronControlPlaneProviderName = "k0sproject-k0smotron" ) +// IPAM providers. +const ( + InClusterIPAMProviderName = "in-cluster" +) + // Add-on providers. const ( HelmAddonProviderName = "helm" @@ -234,6 +240,11 @@ func (p *providersClient) defaults() []Provider { url: "https://github.com/syself/cluster-api-provider-hetzner/releases/latest/infrastructure-components.yaml", providerType: clusterctlv1.InfrastructureProviderType, }, + &provider{ + name: HivelocityProviderName, + url: "https://github.com/hivelocity/cluster-api-provider-hivelocity/releases/latest/infrastructure-components.yaml", + providerType: clusterctlv1.InfrastructureProviderType, + }, &provider{ name: OutscaleProviderName, url: "https://github.com/outscale/cluster-api-provider-outscale/releases/latest/infrastructure-components.yaml", @@ -369,6 +380,13 @@ func (p *providersClient) defaults() []Provider { providerType: clusterctlv1.ControlPlaneProviderType, }, + // IPAM providers + &provider{ + name: InClusterIPAMProviderName, + url: "https://github.com/kubernetes-sigs/cluster-api-ipam-provider-in-cluster/releases/latest/ipam-components.yaml", + providerType: clusterctlv1.IPAMProviderType, + }, + // Add-on providers &provider{ name: HelmAddonProviderName, diff --git a/cmd/clusterctl/client/config_test.go b/cmd/clusterctl/client/config_test.go index 864b894f9277..4ee7a56796a3 100644 --- a/cmd/clusterctl/client/config_test.go +++ b/cmd/clusterctl/client/config_test.go @@ -82,6 +82,7 @@ func Test_clusterctlClient_GetProvidersConfig(t *testing.T) { config.DockerProviderName, config.GCPProviderName, config.HetznerProviderName, + config.HivelocityProviderName, config.IBMCloudProviderName, config.InMemoryProviderName, config.K0smotronProviderName, @@ -101,6 +102,7 @@ func Test_clusterctlClient_GetProvidersConfig(t *testing.T) { config.VclusterProviderName, config.VirtinkProviderName, config.VSphereProviderName, + config.InClusterIPAMProviderName, config.HelmAddonProviderName, }, wantErr: false, @@ -139,6 +141,7 @@ func Test_clusterctlClient_GetProvidersConfig(t *testing.T) { config.DockerProviderName, config.GCPProviderName, config.HetznerProviderName, + config.HivelocityProviderName, config.IBMCloudProviderName, config.InMemoryProviderName, config.K0smotronProviderName, @@ -158,6 +161,7 @@ func Test_clusterctlClient_GetProvidersConfig(t *testing.T) { config.VclusterProviderName, config.VirtinkProviderName, config.VSphereProviderName, + config.InClusterIPAMProviderName, config.HelmAddonProviderName, }, wantErr: false, diff --git a/cmd/clusterctl/client/repository/overrides.go b/cmd/clusterctl/client/repository/overrides.go index 1a6c858347bf..bd14f035d61b 100644 --- a/cmd/clusterctl/client/repository/overrides.go +++ b/cmd/clusterctl/client/repository/overrides.go @@ -28,6 +28,7 @@ import ( "github.com/pkg/errors" "sigs.k8s.io/cluster-api/cmd/clusterctl/client/config" + logf "sigs.k8s.io/cluster-api/cmd/clusterctl/log" ) const ( @@ -107,7 +108,11 @@ func (o *overrides) Path() (string, error) { // getLocalOverride return local override file from the config folder, if it exists. // This is required for development purposes, but it can be used also in production as a workaround for problems on the official repositories. func getLocalOverride(info *newOverrideInput) ([]byte, error) { + log := logf.Log + overridePath, err := newOverride(info).Path() + log.V(5).Info("Potential override file", "SearchFile", overridePath, "Provider", info.provider.ManifestLabel(), "Version", info.version) + if err != nil { return nil, err } diff --git a/cmd/clusterctl/cmd/config_repositories_test.go b/cmd/clusterctl/cmd/config_repositories_test.go index 3f522a203a26..a5ef73c9b8f4 100644 --- a/cmd/clusterctl/cmd/config_repositories_test.go +++ b/cmd/clusterctl/cmd/config_repositories_test.go @@ -102,55 +102,57 @@ providers: type: "CoreProvider" ` -var expectedOutputText = `NAME TYPE URL FILE -cluster-api CoreProvider https://github.com/myorg/myforkofclusterapi/releases/latest/ core_components.yaml -another-provider BootstrapProvider ./ bootstrap-components.yaml -k0sproject-k0smotron BootstrapProvider https://github.com/k0sproject/k0smotron/releases/latest/ bootstrap-components.yaml -kubeadm BootstrapProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ bootstrap-components.yaml -kubekey-k3s BootstrapProvider https://github.com/kubesphere/kubekey/releases/latest/ bootstrap-components.yaml -microk8s BootstrapProvider https://github.com/canonical/cluster-api-bootstrap-provider-microk8s/releases/latest/ bootstrap-components.yaml -ocne BootstrapProvider https://github.com/verrazzano/cluster-api-provider-ocne/releases/latest/ bootstrap-components.yaml -rke2 BootstrapProvider https://github.com/rancher-sandbox/cluster-api-provider-rke2/releases/latest/ bootstrap-components.yaml -talos BootstrapProvider https://github.com/siderolabs/cluster-api-bootstrap-provider-talos/releases/latest/ bootstrap-components.yaml -k0sproject-k0smotron ControlPlaneProvider https://github.com/k0sproject/k0smotron/releases/latest/ control-plane-components.yaml -kamaji ControlPlaneProvider https://github.com/clastix/cluster-api-control-plane-provider-kamaji/releases/latest/ control-plane-components.yaml -kubeadm ControlPlaneProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ control-plane-components.yaml -kubekey-k3s ControlPlaneProvider https://github.com/kubesphere/kubekey/releases/latest/ control-plane-components.yaml -microk8s ControlPlaneProvider https://github.com/canonical/cluster-api-control-plane-provider-microk8s/releases/latest/ control-plane-components.yaml -nested ControlPlaneProvider https://github.com/kubernetes-sigs/cluster-api-provider-nested/releases/latest/ control-plane-components.yaml -ocne ControlPlaneProvider https://github.com/verrazzano/cluster-api-provider-ocne/releases/latest/ control-plane-components.yaml -rke2 ControlPlaneProvider https://github.com/rancher-sandbox/cluster-api-provider-rke2/releases/latest/ control-plane-components.yaml -talos ControlPlaneProvider https://github.com/siderolabs/cluster-api-control-plane-provider-talos/releases/latest/ control-plane-components.yaml -aws InfrastructureProvider my-aws-infrastructure-components.yaml -azure InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-azure/releases/latest/ infrastructure-components.yaml -byoh InfrastructureProvider https://github.com/vmware-tanzu/cluster-api-provider-bringyourownhost/releases/latest/ infrastructure-components.yaml -cloudstack InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack/releases/latest/ infrastructure-components.yaml -coxedge InfrastructureProvider https://github.com/coxedge/cluster-api-provider-coxedge/releases/latest/ infrastructure-components.yaml -digitalocean InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-digitalocean/releases/latest/ infrastructure-components.yaml -docker InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ infrastructure-components-development.yaml -gcp InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-gcp/releases/latest/ infrastructure-components.yaml -hetzner InfrastructureProvider https://github.com/syself/cluster-api-provider-hetzner/releases/latest/ infrastructure-components.yaml -ibmcloud InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud/releases/latest/ infrastructure-components.yaml -in-memory InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ infrastructure-components-in-memory-development.yaml -k0sproject-k0smotron InfrastructureProvider https://github.com/k0sproject/k0smotron/releases/latest/ infrastructure-components.yaml -kubekey InfrastructureProvider https://github.com/kubesphere/kubekey/releases/latest/ infrastructure-components.yaml -kubevirt InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-kubevirt/releases/latest/ infrastructure-components.yaml -maas InfrastructureProvider https://github.com/spectrocloud/cluster-api-provider-maas/releases/latest/ infrastructure-components.yaml -metal3 InfrastructureProvider https://github.com/metal3-io/cluster-api-provider-metal3/releases/latest/ infrastructure-components.yaml -my-infra-provider InfrastructureProvider /home/.config/cluster-api/overrides/infrastructure-docker/latest/ infrastructure-components.yaml -nested InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-nested/releases/latest/ infrastructure-components.yaml -nutanix InfrastructureProvider https://github.com/nutanix-cloud-native/cluster-api-provider-nutanix/releases/latest/ infrastructure-components.yaml -oci InfrastructureProvider https://github.com/oracle/cluster-api-provider-oci/releases/latest/ infrastructure-components.yaml -openstack InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-openstack/releases/latest/ infrastructure-components.yaml -outscale InfrastructureProvider https://github.com/outscale/cluster-api-provider-outscale/releases/latest/ infrastructure-components.yaml -packet InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-packet/releases/latest/ infrastructure-components.yaml -proxmox InfrastructureProvider https://github.com/ionos-cloud/cluster-api-provider-proxmox/releases/latest/ infrastructure-components.yaml -sidero InfrastructureProvider https://github.com/siderolabs/sidero/releases/latest/ infrastructure-components.yaml -vcd InfrastructureProvider https://github.com/vmware/cluster-api-provider-cloud-director/releases/latest/ infrastructure-components.yaml -vcluster InfrastructureProvider https://github.com/loft-sh/cluster-api-provider-vcluster/releases/latest/ infrastructure-components.yaml -virtink InfrastructureProvider https://github.com/smartxworks/cluster-api-provider-virtink/releases/latest/ infrastructure-components.yaml -vsphere InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/releases/latest/ infrastructure-components.yaml -helm AddonProvider https://github.com/kubernetes-sigs/cluster-api-addon-provider-helm/releases/latest/ addon-components.yaml +var expectedOutputText = `NAME TYPE URL FILE +cluster-api CoreProvider https://github.com/myorg/myforkofclusterapi/releases/latest/ core_components.yaml +another-provider BootstrapProvider ./ bootstrap-components.yaml +k0sproject-k0smotron BootstrapProvider https://github.com/k0sproject/k0smotron/releases/latest/ bootstrap-components.yaml +kubeadm BootstrapProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ bootstrap-components.yaml +kubekey-k3s BootstrapProvider https://github.com/kubesphere/kubekey/releases/latest/ bootstrap-components.yaml +microk8s BootstrapProvider https://github.com/canonical/cluster-api-bootstrap-provider-microk8s/releases/latest/ bootstrap-components.yaml +ocne BootstrapProvider https://github.com/verrazzano/cluster-api-provider-ocne/releases/latest/ bootstrap-components.yaml +rke2 BootstrapProvider https://github.com/rancher-sandbox/cluster-api-provider-rke2/releases/latest/ bootstrap-components.yaml +talos BootstrapProvider https://github.com/siderolabs/cluster-api-bootstrap-provider-talos/releases/latest/ bootstrap-components.yaml +k0sproject-k0smotron ControlPlaneProvider https://github.com/k0sproject/k0smotron/releases/latest/ control-plane-components.yaml +kamaji ControlPlaneProvider https://github.com/clastix/cluster-api-control-plane-provider-kamaji/releases/latest/ control-plane-components.yaml +kubeadm ControlPlaneProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ control-plane-components.yaml +kubekey-k3s ControlPlaneProvider https://github.com/kubesphere/kubekey/releases/latest/ control-plane-components.yaml +microk8s ControlPlaneProvider https://github.com/canonical/cluster-api-control-plane-provider-microk8s/releases/latest/ control-plane-components.yaml +nested ControlPlaneProvider https://github.com/kubernetes-sigs/cluster-api-provider-nested/releases/latest/ control-plane-components.yaml +ocne ControlPlaneProvider https://github.com/verrazzano/cluster-api-provider-ocne/releases/latest/ control-plane-components.yaml +rke2 ControlPlaneProvider https://github.com/rancher-sandbox/cluster-api-provider-rke2/releases/latest/ control-plane-components.yaml +talos ControlPlaneProvider https://github.com/siderolabs/cluster-api-control-plane-provider-talos/releases/latest/ control-plane-components.yaml +aws InfrastructureProvider my-aws-infrastructure-components.yaml +azure InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-azure/releases/latest/ infrastructure-components.yaml +byoh InfrastructureProvider https://github.com/vmware-tanzu/cluster-api-provider-bringyourownhost/releases/latest/ infrastructure-components.yaml +cloudstack InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack/releases/latest/ infrastructure-components.yaml +coxedge InfrastructureProvider https://github.com/coxedge/cluster-api-provider-coxedge/releases/latest/ infrastructure-components.yaml +digitalocean InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-digitalocean/releases/latest/ infrastructure-components.yaml +docker InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ infrastructure-components-development.yaml +gcp InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-gcp/releases/latest/ infrastructure-components.yaml +hetzner InfrastructureProvider https://github.com/syself/cluster-api-provider-hetzner/releases/latest/ infrastructure-components.yaml +hivelocity-hivelocity InfrastructureProvider https://github.com/hivelocity/cluster-api-provider-hivelocity/releases/latest/ infrastructure-components.yaml +ibmcloud InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud/releases/latest/ infrastructure-components.yaml +in-memory InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api/releases/latest/ infrastructure-components-in-memory-development.yaml +k0sproject-k0smotron InfrastructureProvider https://github.com/k0sproject/k0smotron/releases/latest/ infrastructure-components.yaml +kubekey InfrastructureProvider https://github.com/kubesphere/kubekey/releases/latest/ infrastructure-components.yaml +kubevirt InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-kubevirt/releases/latest/ infrastructure-components.yaml +maas InfrastructureProvider https://github.com/spectrocloud/cluster-api-provider-maas/releases/latest/ infrastructure-components.yaml +metal3 InfrastructureProvider https://github.com/metal3-io/cluster-api-provider-metal3/releases/latest/ infrastructure-components.yaml +my-infra-provider InfrastructureProvider /home/.config/cluster-api/overrides/infrastructure-docker/latest/ infrastructure-components.yaml +nested InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-nested/releases/latest/ infrastructure-components.yaml +nutanix InfrastructureProvider https://github.com/nutanix-cloud-native/cluster-api-provider-nutanix/releases/latest/ infrastructure-components.yaml +oci InfrastructureProvider https://github.com/oracle/cluster-api-provider-oci/releases/latest/ infrastructure-components.yaml +openstack InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-openstack/releases/latest/ infrastructure-components.yaml +outscale InfrastructureProvider https://github.com/outscale/cluster-api-provider-outscale/releases/latest/ infrastructure-components.yaml +packet InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-packet/releases/latest/ infrastructure-components.yaml +proxmox InfrastructureProvider https://github.com/ionos-cloud/cluster-api-provider-proxmox/releases/latest/ infrastructure-components.yaml +sidero InfrastructureProvider https://github.com/siderolabs/sidero/releases/latest/ infrastructure-components.yaml +vcd InfrastructureProvider https://github.com/vmware/cluster-api-provider-cloud-director/releases/latest/ infrastructure-components.yaml +vcluster InfrastructureProvider https://github.com/loft-sh/cluster-api-provider-vcluster/releases/latest/ infrastructure-components.yaml +virtink InfrastructureProvider https://github.com/smartxworks/cluster-api-provider-virtink/releases/latest/ infrastructure-components.yaml +vsphere InfrastructureProvider https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/releases/latest/ infrastructure-components.yaml +in-cluster IPAMProvider https://github.com/kubernetes-sigs/cluster-api-ipam-provider-in-cluster/releases/latest/ ipam-components.yaml +helm AddonProvider https://github.com/kubernetes-sigs/cluster-api-addon-provider-helm/releases/latest/ addon-components.yaml ` var expectedOutputYaml = `- File: core_components.yaml @@ -261,6 +263,10 @@ var expectedOutputYaml = `- File: core_components.yaml Name: hetzner ProviderType: InfrastructureProvider URL: https://github.com/syself/cluster-api-provider-hetzner/releases/latest/ +- File: infrastructure-components.yaml + Name: hivelocity-hivelocity + ProviderType: InfrastructureProvider + URL: https://github.com/hivelocity/cluster-api-provider-hivelocity/releases/latest/ - File: infrastructure-components.yaml Name: ibmcloud ProviderType: InfrastructureProvider @@ -341,6 +347,10 @@ var expectedOutputYaml = `- File: core_components.yaml Name: vsphere ProviderType: InfrastructureProvider URL: https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/releases/latest/ +- File: ipam-components.yaml + Name: in-cluster + ProviderType: IPAMProvider + URL: https://github.com/kubernetes-sigs/cluster-api-ipam-provider-in-cluster/releases/latest/ - File: addon-components.yaml Name: helm ProviderType: AddonProvider diff --git a/cmd/clusterctl/cmd/generate_cluster.go b/cmd/clusterctl/cmd/generate_cluster.go index 7562a90380b8..9b95911e5831 100644 --- a/cmd/clusterctl/cmd/generate_cluster.go +++ b/cmd/clusterctl/cmd/generate_cluster.go @@ -116,8 +116,9 @@ func init() { "The Kubernetes version to use for the workload cluster. If unspecified, the value from OS environment variables or the $XDG_CONFIG_HOME/cluster-api/clusterctl.yaml config file will be used.") generateClusterClusterCmd.Flags().Int64Var(&gc.controlPlaneMachineCount, "control-plane-machine-count", 1, "The number of control plane machines for the workload cluster.") + // Remove default from hard coded text if the default is ever changed from 0 since cobra would then add it generateClusterClusterCmd.Flags().Int64Var(&gc.workerMachineCount, "worker-machine-count", 0, - "The number of worker machines for the workload cluster.") + "The number of worker machines for the workload cluster. (default 0)") // flags for the repository source generateClusterClusterCmd.Flags().StringVarP(&gc.infrastructureProvider, "infrastructure", "i", "", diff --git a/cmd/clusterctl/cmd/init.go b/cmd/clusterctl/cmd/init.go index e19eee9796da..18dd3edab328 100644 --- a/cmd/clusterctl/cmd/init.go +++ b/cmd/clusterctl/cmd/init.go @@ -103,7 +103,7 @@ func init() { initCmd.PersistentFlags().StringSliceVarP(&initOpts.controlPlaneProviders, "control-plane", "c", nil, "Control plane providers and versions (e.g. kubeadm:v1.1.5) to add to the management cluster. If unspecified, the Kubeadm control plane provider's latest release is used.") initCmd.PersistentFlags().StringSliceVar(&initOpts.ipamProviders, "ipam", nil, - "IPAM providers and versions (e.g. infoblox:v0.0.1) to add to the management cluster.") + "IPAM providers and versions (e.g. in-cluster:v0.1.0) to add to the management cluster.") initCmd.PersistentFlags().StringSliceVar(&initOpts.runtimeExtensionProviders, "runtime-extension", nil, "Runtime extension providers and versions to add to the management cluster; please note that clusterctl doesn't include any default runtime extensions and thus it is required to use custom configuration files to register runtime extensions.") initCmd.PersistentFlags().StringSliceVar(&initOpts.addonProviders, "addon", nil, diff --git a/cmd/clusterctl/cmd/root.go b/cmd/clusterctl/cmd/root.go index 6e54068fd813..89366a78ddd4 100644 --- a/cmd/clusterctl/cmd/root.go +++ b/cmd/clusterctl/cmd/root.go @@ -106,6 +106,8 @@ var RootCmd = &cobra.Command{ // Execute executes the root command. func Execute() { + handlePlugins() + if err := RootCmd.Execute(); err != nil { if verbosity != nil && *verbosity >= 5 { if err, ok := err.(stackTracer); ok { @@ -146,8 +148,6 @@ func init() { RootCmd.SetCompletionCommandGroupID(groupOther) cobra.OnInitialize(initConfig, registerCompletionFuncForCommonFlags) - - handlePlugins() } func initConfig() { diff --git a/controllers/remote/cluster_cache_tracker.go b/controllers/remote/cluster_cache_tracker.go index 3c1c67248d60..5060729ce1a4 100644 --- a/controllers/remote/cluster_cache_tracker.go +++ b/controllers/remote/cluster_cache_tracker.go @@ -189,6 +189,11 @@ func (t *ClusterCacheTracker) GetClient(ctx context.Context, cluster client.Obje return accessor.client, nil } +// GetReader returns a cached read-only client for the given cluster. +func (t *ClusterCacheTracker) GetReader(ctx context.Context, cluster client.ObjectKey) (client.Reader, error) { + return t.GetClient(ctx, cluster) +} + // GetRESTConfig returns a cached REST config for the given cluster. func (t *ClusterCacheTracker) GetRESTConfig(ctc context.Context, cluster client.ObjectKey) (*rest.Config, error) { accessor, err := t.getClusterAccessor(ctc, cluster, t.indexes...) diff --git a/controlplane/kubeadm/internal/controllers/controller_test.go b/controlplane/kubeadm/internal/controllers/controller_test.go index aaf7c09a180b..6459174a918a 100644 --- a/controlplane/kubeadm/internal/controllers/controller_test.go +++ b/controlplane/kubeadm/internal/controllers/controller_test.go @@ -1279,7 +1279,8 @@ dns: type: CoreDNS imageRepository: registry.k8s.io kind: ClusterConfiguration -kubernetesVersion: metav1.16.1`, +kubernetesVersion: metav1.16.1 +`, }, } g.Expect(env.Create(ctx, kubeadmCM)).To(Succeed()) diff --git a/controlplane/kubeadm/internal/controllers/fakes_test.go b/controlplane/kubeadm/internal/controllers/fakes_test.go index 3c7348bc4831..cf9fcbafe66e 100644 --- a/controlplane/kubeadm/internal/controllers/fakes_test.go +++ b/controlplane/kubeadm/internal/controllers/fakes_test.go @@ -108,11 +108,11 @@ func (f fakeWorkloadCluster) ReconcileKubeletRBACBinding(_ context.Context, _ se return nil } -func (f fakeWorkloadCluster) UpdateKubernetesVersionInKubeadmConfigMap(_ context.Context, _ semver.Version) error { +func (f fakeWorkloadCluster) UpdateKubernetesVersionInKubeadmConfigMap(semver.Version) func(*bootstrapv1.ClusterConfiguration) { return nil } -func (f fakeWorkloadCluster) UpdateEtcdVersionInKubeadmConfigMap(_ context.Context, _, _ string, _ semver.Version) error { +func (f fakeWorkloadCluster) UpdateEtcdLocalInKubeadmConfigMap(*bootstrapv1.LocalEtcd) func(*bootstrapv1.ClusterConfiguration) { return nil } @@ -132,13 +132,17 @@ func (f fakeWorkloadCluster) EtcdMembers(_ context.Context) ([]string, error) { return f.EtcdMembersResult, nil } +func (f fakeWorkloadCluster) UpdateClusterConfiguration(context.Context, semver.Version, ...func(*bootstrapv1.ClusterConfiguration)) error { + return nil +} + type fakeMigrator struct { migrateCalled bool migrateErr error migratedCorefile string } -func (m *fakeMigrator) Migrate(_, _, _ string, _ bool) (string, error) { +func (m *fakeMigrator) Migrate(string, string, string, bool) (string, error) { m.migrateCalled = true if m.migrateErr != nil { return "", m.migrateErr diff --git a/controlplane/kubeadm/internal/controllers/upgrade.go b/controlplane/kubeadm/internal/controllers/upgrade.go index 6abf136947ab..651d0c2a798c 100644 --- a/controlplane/kubeadm/internal/controllers/upgrade.go +++ b/controlplane/kubeadm/internal/controllers/upgrade.go @@ -23,6 +23,7 @@ import ( "github.com/pkg/errors" ctrl "sigs.k8s.io/controller-runtime" + bootstrapv1 "sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1beta1" controlplanev1 "sigs.k8s.io/cluster-api/controlplane/kubeadm/api/v1beta1" "sigs.k8s.io/cluster-api/controlplane/kubeadm/internal" "sigs.k8s.io/cluster-api/util" @@ -73,9 +74,8 @@ func (r *KubeadmControlPlaneReconciler) upgradeControlPlane( return ctrl.Result{}, errors.Wrap(err, "failed to set cluster-admin ClusterRoleBinding for kubeadm") } - if err := workloadCluster.UpdateKubernetesVersionInKubeadmConfigMap(ctx, parsedVersion); err != nil { - return ctrl.Result{}, errors.Wrap(err, "failed to update the kubernetes version in the kubeadm config map") - } + kubeadmCMMutators := make([]func(*bootstrapv1.ClusterConfiguration), 0) + kubeadmCMMutators = append(kubeadmCMMutators, workloadCluster.UpdateKubernetesVersionInKubeadmConfigMap(parsedVersion)) if controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration != nil { // We intentionally only parse major/minor/patch so that the subsequent code @@ -84,38 +84,30 @@ func (r *KubeadmControlPlaneReconciler) upgradeControlPlane( if err != nil { return ctrl.Result{}, errors.Wrapf(err, "failed to parse kubernetes version %q", controlPlane.KCP.Spec.Version) } + // Get the imageRepository or the correct value if nothing is set and a migration is necessary. imageRepository := internal.ImageRepositoryFromClusterConfig(controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration, parsedVersionTolerant) - if err := workloadCluster.UpdateImageRepositoryInKubeadmConfigMap(ctx, imageRepository, parsedVersion); err != nil { - return ctrl.Result{}, errors.Wrap(err, "failed to update the image repository in the kubeadm config map") - } - } - - if controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration != nil && controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Etcd.Local != nil { - meta := controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Etcd.Local.ImageMeta - if err := workloadCluster.UpdateEtcdVersionInKubeadmConfigMap(ctx, meta.ImageRepository, meta.ImageTag, parsedVersion); err != nil { - return ctrl.Result{}, errors.Wrap(err, "failed to update the etcd version in the kubeadm config map") - } - - extraArgs := controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Etcd.Local.ExtraArgs - if err := workloadCluster.UpdateEtcdExtraArgsInKubeadmConfigMap(ctx, extraArgs, parsedVersion); err != nil { - return ctrl.Result{}, errors.Wrap(err, "failed to update the etcd extra args in the kubeadm config map") + kubeadmCMMutators = append(kubeadmCMMutators, + workloadCluster.UpdateImageRepositoryInKubeadmConfigMap(imageRepository), + workloadCluster.UpdateFeatureGatesInKubeadmConfigMap(controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.FeatureGates), + workloadCluster.UpdateAPIServerInKubeadmConfigMap(controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.APIServer), + workloadCluster.UpdateControllerManagerInKubeadmConfigMap(controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.ControllerManager), + workloadCluster.UpdateSchedulerInKubeadmConfigMap(controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Scheduler)) + + // Etcd local and external are mutually exclusive and they cannot be switched, once set. + if controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Etcd.Local != nil { + kubeadmCMMutators = append(kubeadmCMMutators, + workloadCluster.UpdateEtcdLocalInKubeadmConfigMap(controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Etcd.Local)) + } else { + kubeadmCMMutators = append(kubeadmCMMutators, + workloadCluster.UpdateEtcdExternalInKubeadmConfigMap(controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Etcd.External)) } } - if controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration != nil { - if err := workloadCluster.UpdateAPIServerInKubeadmConfigMap(ctx, controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.APIServer, parsedVersion); err != nil { - return ctrl.Result{}, errors.Wrap(err, "failed to update api server in the kubeadm config map") - } - - if err := workloadCluster.UpdateControllerManagerInKubeadmConfigMap(ctx, controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.ControllerManager, parsedVersion); err != nil { - return ctrl.Result{}, errors.Wrap(err, "failed to update controller manager in the kubeadm config map") - } - - if err := workloadCluster.UpdateSchedulerInKubeadmConfigMap(ctx, controlPlane.KCP.Spec.KubeadmConfigSpec.ClusterConfiguration.Scheduler, parsedVersion); err != nil { - return ctrl.Result{}, errors.Wrap(err, "failed to update scheduler in the kubeadm config map") - } + // collectively update Kubeadm config map + if err = workloadCluster.UpdateClusterConfiguration(ctx, parsedVersion, kubeadmCMMutators...); err != nil { + return ctrl.Result{}, err } if err := workloadCluster.UpdateKubeletConfigMap(ctx, parsedVersion); err != nil { diff --git a/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane.go b/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane.go index 4338b8b11b15..d848d4616bba 100644 --- a/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane.go +++ b/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane.go @@ -150,6 +150,7 @@ const ( ntp = "ntp" ignition = "ignition" diskSetup = "diskSetup" + featureGates = "featureGates" ) const minimumCertificatesExpiryDays = 7 @@ -176,6 +177,8 @@ func (webhook *KubeadmControlPlane) ValidateUpdate(_ context.Context, oldObj, ne {spec, kubeadmConfigSpec, clusterConfiguration, "dns", "imageRepository"}, {spec, kubeadmConfigSpec, clusterConfiguration, "dns", "imageTag"}, {spec, kubeadmConfigSpec, clusterConfiguration, "imageRepository"}, + {spec, kubeadmConfigSpec, clusterConfiguration, featureGates}, + {spec, kubeadmConfigSpec, clusterConfiguration, featureGates, "*"}, {spec, kubeadmConfigSpec, clusterConfiguration, apiServer}, {spec, kubeadmConfigSpec, clusterConfiguration, apiServer, "*"}, {spec, kubeadmConfigSpec, clusterConfiguration, controllerManager}, diff --git a/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane_test.go b/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane_test.go index 699d3776d0e1..9c62b22ad4e6 100644 --- a/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane_test.go +++ b/controlplane/kubeadm/internal/webhooks/kubeadm_control_plane_test.go @@ -886,8 +886,8 @@ func TestKubeadmControlPlaneValidateUpdate(t *testing.T) { kcp: imageRepository, }, { - name: "should fail when making a change to the cluster config's featureGates", - expectErr: true, + name: "should succeed when making a change to the cluster config's featureGates", + expectErr: false, before: before, kcp: featureGates, }, diff --git a/controlplane/kubeadm/internal/workload_cluster.go b/controlplane/kubeadm/internal/workload_cluster.go index d4c41eb89f4c..9034dd1e05e8 100644 --- a/controlplane/kubeadm/internal/workload_cluster.go +++ b/controlplane/kubeadm/internal/workload_cluster.go @@ -105,13 +105,14 @@ type WorkloadCluster interface { // Upgrade related tasks. ReconcileKubeletRBACBinding(ctx context.Context, version semver.Version) error ReconcileKubeletRBACRole(ctx context.Context, version semver.Version) error - UpdateKubernetesVersionInKubeadmConfigMap(ctx context.Context, version semver.Version) error - UpdateImageRepositoryInKubeadmConfigMap(ctx context.Context, imageRepository string, version semver.Version) error - UpdateEtcdVersionInKubeadmConfigMap(ctx context.Context, imageRepository, imageTag string, version semver.Version) error - UpdateEtcdExtraArgsInKubeadmConfigMap(ctx context.Context, extraArgs map[string]string, version semver.Version) error - UpdateAPIServerInKubeadmConfigMap(ctx context.Context, apiServer bootstrapv1.APIServer, version semver.Version) error - UpdateControllerManagerInKubeadmConfigMap(ctx context.Context, controllerManager bootstrapv1.ControlPlaneComponent, version semver.Version) error - UpdateSchedulerInKubeadmConfigMap(ctx context.Context, scheduler bootstrapv1.ControlPlaneComponent, version semver.Version) error + UpdateKubernetesVersionInKubeadmConfigMap(version semver.Version) func(*bootstrapv1.ClusterConfiguration) + UpdateImageRepositoryInKubeadmConfigMap(imageRepository string) func(*bootstrapv1.ClusterConfiguration) + UpdateFeatureGatesInKubeadmConfigMap(featureGates map[string]bool) func(*bootstrapv1.ClusterConfiguration) + UpdateEtcdLocalInKubeadmConfigMap(localEtcd *bootstrapv1.LocalEtcd) func(*bootstrapv1.ClusterConfiguration) + UpdateEtcdExternalInKubeadmConfigMap(externalEtcd *bootstrapv1.ExternalEtcd) func(*bootstrapv1.ClusterConfiguration) + UpdateAPIServerInKubeadmConfigMap(apiServer bootstrapv1.APIServer) func(*bootstrapv1.ClusterConfiguration) + UpdateControllerManagerInKubeadmConfigMap(controllerManager bootstrapv1.ControlPlaneComponent) func(*bootstrapv1.ClusterConfiguration) + UpdateSchedulerInKubeadmConfigMap(scheduler bootstrapv1.ControlPlaneComponent) func(*bootstrapv1.ClusterConfiguration) UpdateKubeletConfigMap(ctx context.Context, version semver.Version) error UpdateKubeProxyImageInfo(ctx context.Context, kcp *controlplanev1.KubeadmControlPlane, version semver.Version) error UpdateCoreDNS(ctx context.Context, kcp *controlplanev1.KubeadmControlPlane, version semver.Version) error @@ -121,6 +122,7 @@ type WorkloadCluster interface { ForwardEtcdLeadership(ctx context.Context, machine *clusterv1.Machine, leaderCandidate *clusterv1.Machine) error AllowBootstrapTokensToGetNodes(ctx context.Context) error AllowClusterAdminPermissions(ctx context.Context, version semver.Version) error + UpdateClusterConfiguration(ctx context.Context, version semver.Version, mutators ...func(*bootstrapv1.ClusterConfiguration)) error // State recovery tasks. ReconcileEtcdMembers(ctx context.Context, nodeNames []string, version semver.Version) ([]string, error) @@ -173,20 +175,30 @@ func (w *Workload) getConfigMap(ctx context.Context, configMap ctrlclient.Object } // UpdateImageRepositoryInKubeadmConfigMap updates the image repository in the kubeadm config map. -func (w *Workload) UpdateImageRepositoryInKubeadmConfigMap(ctx context.Context, imageRepository string, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { +func (w *Workload) UpdateImageRepositoryInKubeadmConfigMap(imageRepository string) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { if imageRepository == "" { return } + c.ImageRepository = imageRepository - }, version) + } +} + +// UpdateFeatureGatesInKubeadmConfigMap updates the feature gates in the kubeadm config map. +func (w *Workload) UpdateFeatureGatesInKubeadmConfigMap(featureGates map[string]bool) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { + // Even if featureGates is nil, reset it to ClusterConfiguration + // to override any previously set feature gates. + c.FeatureGates = featureGates + } } // UpdateKubernetesVersionInKubeadmConfigMap updates the kubernetes version in the kubeadm config map. -func (w *Workload) UpdateKubernetesVersionInKubeadmConfigMap(ctx context.Context, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { +func (w *Workload) UpdateKubernetesVersionInKubeadmConfigMap(version semver.Version) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { c.KubernetesVersion = fmt.Sprintf("v%s", version.String()) - }, version) + } } // UpdateKubeletConfigMap will create a new kubelet-config-1.x config map for a new version of the kubelet. @@ -270,24 +282,24 @@ func (w *Workload) UpdateKubeletConfigMap(ctx context.Context, version semver.Ve } // UpdateAPIServerInKubeadmConfigMap updates api server configuration in kubeadm config map. -func (w *Workload) UpdateAPIServerInKubeadmConfigMap(ctx context.Context, apiServer bootstrapv1.APIServer, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { +func (w *Workload) UpdateAPIServerInKubeadmConfigMap(apiServer bootstrapv1.APIServer) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { c.APIServer = apiServer - }, version) + } } // UpdateControllerManagerInKubeadmConfigMap updates controller manager configuration in kubeadm config map. -func (w *Workload) UpdateControllerManagerInKubeadmConfigMap(ctx context.Context, controllerManager bootstrapv1.ControlPlaneComponent, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { +func (w *Workload) UpdateControllerManagerInKubeadmConfigMap(controllerManager bootstrapv1.ControlPlaneComponent) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { c.ControllerManager = controllerManager - }, version) + } } // UpdateSchedulerInKubeadmConfigMap updates scheduler configuration in kubeadm config map. -func (w *Workload) UpdateSchedulerInKubeadmConfigMap(ctx context.Context, scheduler bootstrapv1.ControlPlaneComponent, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { +func (w *Workload) UpdateSchedulerInKubeadmConfigMap(scheduler bootstrapv1.ControlPlaneComponent) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { c.Scheduler = scheduler - }, version) + } } // RemoveMachineFromKubeadmConfigMap removes the entry for the machine from the kubeadm configmap. @@ -350,11 +362,11 @@ func (w *Workload) updateClusterStatus(ctx context.Context, mutator func(status }) } -// updateClusterConfiguration gets the ClusterConfiguration kubeadm-config ConfigMap, converts it to the +// UpdateClusterConfiguration gets the ClusterConfiguration kubeadm-config ConfigMap, converts it to the // Cluster API representation, and then applies a mutation func; if changes are detected, the // data are converted back into the Kubeadm API version in use for the target Kubernetes version and the // kubeadm-config ConfigMap updated. -func (w *Workload) updateClusterConfiguration(ctx context.Context, mutator func(*bootstrapv1.ClusterConfiguration), version semver.Version) error { +func (w *Workload) UpdateClusterConfiguration(ctx context.Context, version semver.Version, mutators ...func(*bootstrapv1.ClusterConfiguration)) error { return retry.RetryOnConflict(retry.DefaultBackoff, func() error { key := ctrlclient.ObjectKey{Name: kubeadmConfigKey, Namespace: metav1.NamespaceSystem} configMap, err := w.getConfigMap(ctx, key) @@ -373,7 +385,9 @@ func (w *Workload) updateClusterConfiguration(ctx context.Context, mutator func( } updatedObj := currentObj.DeepCopy() - mutator(updatedObj) + for i := range mutators { + mutators[i](updatedObj) + } if !reflect.DeepEqual(currentObj, updatedObj) { updatedData, err := kubeadmtypes.MarshalClusterConfigurationForVersion(updatedObj, version) @@ -382,7 +396,7 @@ func (w *Workload) updateClusterConfiguration(ctx context.Context, mutator func( } configMap.Data[clusterConfigurationKey] = updatedData if err := w.Client.Update(ctx, configMap); err != nil { - return errors.Wrap(err, "failed to upgrade the kubeadmConfigMap") + return errors.Wrap(err, "failed to upgrade cluster configuration in the kubeadmConfigMap") } } return nil diff --git a/controlplane/kubeadm/internal/workload_cluster_coredns.go b/controlplane/kubeadm/internal/workload_cluster_coredns.go index 5699c9c06656..deb5d712d708 100644 --- a/controlplane/kubeadm/internal/workload_cluster_coredns.go +++ b/controlplane/kubeadm/internal/workload_cluster_coredns.go @@ -145,7 +145,7 @@ func (w *Workload) UpdateCoreDNS(ctx context.Context, kcp *controlplanev1.Kubead } // Perform the upgrade. - if err := w.updateCoreDNSImageInfoInKubeadmConfigMap(ctx, &clusterConfig.DNS, version); err != nil { + if err := w.UpdateClusterConfiguration(ctx, version, w.updateCoreDNSImageInfoInKubeadmConfigMap(&clusterConfig.DNS)); err != nil { return err } if err := w.updateCoreDNSCorefile(ctx, info); err != nil { @@ -270,11 +270,11 @@ func (w *Workload) updateCoreDNSDeployment(ctx context.Context, info *coreDNSInf } // updateCoreDNSImageInfoInKubeadmConfigMap updates the kubernetes version in the kubeadm config map. -func (w *Workload) updateCoreDNSImageInfoInKubeadmConfigMap(ctx context.Context, dns *bootstrapv1.DNS, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { +func (w *Workload) updateCoreDNSImageInfoInKubeadmConfigMap(dns *bootstrapv1.DNS) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { c.DNS.ImageRepository = dns.ImageRepository c.DNS.ImageTag = dns.ImageTag - }, version) + } } // updateCoreDNSClusterRole updates the CoreDNS ClusterRole when necessary. diff --git a/controlplane/kubeadm/internal/workload_cluster_coredns_test.go b/controlplane/kubeadm/internal/workload_cluster_coredns_test.go index 12bf01c42863..fd68d0a1595e 100644 --- a/controlplane/kubeadm/internal/workload_cluster_coredns_test.go +++ b/controlplane/kubeadm/internal/workload_cluster_coredns_test.go @@ -32,7 +32,7 @@ import ( bootstrapv1 "sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1beta1" controlplanev1 "sigs.k8s.io/cluster-api/controlplane/kubeadm/api/v1beta1" - "sigs.k8s.io/cluster-api/util/yaml" + utilyaml "sigs.k8s.io/cluster-api/util/yaml" ) func TestUpdateCoreDNS(t *testing.T) { @@ -124,7 +124,7 @@ func TestUpdateCoreDNS(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - "ClusterConfiguration": yaml.Raw(` + "ClusterConfiguration": utilyaml.Raw(` apiServer: apiVersion: kubeadm.k8s.io/v1beta2 dns: @@ -140,7 +140,7 @@ func TestUpdateCoreDNS(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - "ClusterConfiguration": yaml.Raw(` + "ClusterConfiguration": utilyaml.Raw(` apiServer: apiVersion: kubeadm.k8s.io/v1beta2 dns: @@ -1410,7 +1410,7 @@ func TestUpdateCoreDNSImageInfoInKubeadmConfigMap(t *testing.T) { }{ { name: "it should set the DNS image config", - clusterConfigurationData: yaml.Raw(` + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration `), @@ -1420,7 +1420,7 @@ func TestUpdateCoreDNSImageInfoInKubeadmConfigMap(t *testing.T) { ImageTag: "v1.2.3", }, }, - wantClusterConfiguration: yaml.Raw(` + wantClusterConfiguration: utilyaml.Raw(` apiServer: {} apiVersion: kubeadm.k8s.io/v1beta2 controllerManager: {} @@ -1451,7 +1451,7 @@ func TestUpdateCoreDNSImageInfoInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.updateCoreDNSImageInfoInKubeadmConfigMap(ctx, &tt.newDNS, semver.MustParse("1.19.1")) + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.updateCoreDNSImageInfoInKubeadmConfigMap(&tt.newDNS)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap diff --git a/controlplane/kubeadm/internal/workload_cluster_etcd.go b/controlplane/kubeadm/internal/workload_cluster_etcd.go index bb4c4d417960..48c06bc3f567 100644 --- a/controlplane/kubeadm/internal/workload_cluster_etcd.go +++ b/controlplane/kubeadm/internal/workload_cluster_etcd.go @@ -92,23 +92,22 @@ loopmembers: return removedMembers, errs } -// UpdateEtcdVersionInKubeadmConfigMap sets the imageRepository or the imageTag or both in the kubeadm config map. -func (w *Workload) UpdateEtcdVersionInKubeadmConfigMap(ctx context.Context, imageRepository, imageTag string, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { +// UpdateEtcdLocalInKubeadmConfigMap sets etcd local configuration in the kubeadm config map. +func (w *Workload) UpdateEtcdLocalInKubeadmConfigMap(etcdLocal *bootstrapv1.LocalEtcd) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { if c.Etcd.Local != nil { - c.Etcd.Local.ImageRepository = imageRepository - c.Etcd.Local.ImageTag = imageTag + c.Etcd.Local = etcdLocal } - }, version) + } } -// UpdateEtcdExtraArgsInKubeadmConfigMap sets extraArgs in the kubeadm config map. -func (w *Workload) UpdateEtcdExtraArgsInKubeadmConfigMap(ctx context.Context, extraArgs map[string]string, version semver.Version) error { - return w.updateClusterConfiguration(ctx, func(c *bootstrapv1.ClusterConfiguration) { - if c.Etcd.Local != nil { - c.Etcd.Local.ExtraArgs = extraArgs +// UpdateEtcdExternalInKubeadmConfigMap sets etcd external configuration in the kubeadm config map. +func (w *Workload) UpdateEtcdExternalInKubeadmConfigMap(etcdExternal *bootstrapv1.ExternalEtcd) func(*bootstrapv1.ClusterConfiguration) { + return func(c *bootstrapv1.ClusterConfiguration) { + if c.Etcd.External != nil { + c.Etcd.External = etcdExternal } - }, version) + } } // RemoveEtcdMemberForMachine removes the etcd member from the target cluster's etcd cluster. diff --git a/controlplane/kubeadm/internal/workload_cluster_etcd_test.go b/controlplane/kubeadm/internal/workload_cluster_etcd_test.go index e80e869a51ba..3c8f8736ae0d 100644 --- a/controlplane/kubeadm/internal/workload_cluster_etcd_test.go +++ b/controlplane/kubeadm/internal/workload_cluster_etcd_test.go @@ -32,58 +32,69 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client/fake" clusterv1 "sigs.k8s.io/cluster-api/api/v1beta1" + bootstrapv1 "sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1beta1" "sigs.k8s.io/cluster-api/controlplane/kubeadm/internal/etcd" fake2 "sigs.k8s.io/cluster-api/controlplane/kubeadm/internal/etcd/fake" - "sigs.k8s.io/cluster-api/util/yaml" + utilyaml "sigs.k8s.io/cluster-api/util/yaml" ) -func TestUpdateEtcdVersionInKubeadmConfigMap(t *testing.T) { +func TestUpdateEtcdExternalInKubeadmConfigMap(t *testing.T) { tests := []struct { name string clusterConfigurationData string - newImageRepository string - newImageTag string + externalEtcd *bootstrapv1.ExternalEtcd wantClusterConfiguration string }{ { - name: "it should set etcd version when local etcd", - clusterConfigurationData: yaml.Raw(` + name: "it should set external etcd configuration with external etcd", + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration etcd: - local: {} + external: {} `), - newImageRepository: "example.com/k8s", - newImageTag: "v1.6.0", - wantClusterConfiguration: yaml.Raw(` + externalEtcd: &bootstrapv1.ExternalEtcd{ + Endpoints: []string{"1.2.3.4"}, + CAFile: "/tmp/ca_file.pem", + CertFile: "/tmp/cert_file.crt", + KeyFile: "/tmp/key_file.key", + }, + wantClusterConfiguration: utilyaml.Raw(` apiServer: {} apiVersion: kubeadm.k8s.io/v1beta2 controllerManager: {} dns: {} etcd: - local: - imageRepository: example.com/k8s - imageTag: v1.6.0 + external: + caFile: /tmp/ca_file.pem + certFile: /tmp/cert_file.crt + endpoints: + - 1.2.3.4 + keyFile: /tmp/key_file.key kind: ClusterConfiguration networking: {} scheduler: {} `), }, { - name: "no op when external etcd", - clusterConfigurationData: yaml.Raw(` + name: "no op when local etcd configuration already exists", + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration etcd: - external: {} + local: {} `), - newImageRepository: "example.com/k8s", - newImageTag: "v1.6.0", - wantClusterConfiguration: yaml.Raw(` + externalEtcd: &bootstrapv1.ExternalEtcd{ + Endpoints: []string{"1.2.3.4"}, + CAFile: "/tmp/ca_file.pem", + CertFile: "/tmp/cert_file.crt", + KeyFile: "/tmp/key_file.key", + }, + wantClusterConfiguration: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration etcd: - external: {} + local: {} `), }, } @@ -104,7 +115,7 @@ func TestUpdateEtcdVersionInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.UpdateEtcdVersionInKubeadmConfigMap(ctx, tt.newImageRepository, tt.newImageTag, semver.MustParse("1.19.1")) + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.UpdateEtcdExternalInKubeadmConfigMap(tt.externalEtcd)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap @@ -118,25 +129,31 @@ func TestUpdateEtcdVersionInKubeadmConfigMap(t *testing.T) { } } -func TestUpdateEtcdExtraArgsInKubeadmConfigMap(t *testing.T) { +func TestUpdateEtcdLocalInKubeadmConfigMap(t *testing.T) { tests := []struct { name string clusterConfigurationData string - newExtraArgs map[string]string + localEtcd *bootstrapv1.LocalEtcd wantClusterConfiguration string }{ { - name: "it should set etcd extraArgs when local etcd", - clusterConfigurationData: yaml.Raw(` + name: "it should set local etcd configuration with local etcd", + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration etcd: local: {} `), - newExtraArgs: map[string]string{ - "foo": "bar", + localEtcd: &bootstrapv1.LocalEtcd{ + ImageMeta: bootstrapv1.ImageMeta{ + ImageRepository: "example.com/k8s", + ImageTag: "v1.6.0", + }, + ExtraArgs: map[string]string{ + "foo": "bar", + }, }, - wantClusterConfiguration: yaml.Raw(` + wantClusterConfiguration: utilyaml.Raw(` apiServer: {} apiVersion: kubeadm.k8s.io/v1beta2 controllerManager: {} @@ -145,23 +162,31 @@ func TestUpdateEtcdExtraArgsInKubeadmConfigMap(t *testing.T) { local: extraArgs: foo: bar + imageRepository: example.com/k8s + imageTag: v1.6.0 kind: ClusterConfiguration networking: {} scheduler: {} `), }, { - name: "no op when external etcd", - clusterConfigurationData: yaml.Raw(` + name: "no op when external etcd configuration already exists", + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration etcd: external: {} `), - newExtraArgs: map[string]string{ - "foo": "bar", + localEtcd: &bootstrapv1.LocalEtcd{ + ImageMeta: bootstrapv1.ImageMeta{ + ImageRepository: "example.com/k8s", + ImageTag: "v1.6.0", + }, + ExtraArgs: map[string]string{ + "foo": "bar", + }, }, - wantClusterConfiguration: yaml.Raw(` + wantClusterConfiguration: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration etcd: @@ -186,7 +211,7 @@ func TestUpdateEtcdExtraArgsInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.UpdateEtcdExtraArgsInKubeadmConfigMap(ctx, tt.newExtraArgs, semver.MustParse("1.19.1")) + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.UpdateEtcdLocalInKubeadmConfigMap(tt.localEtcd)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap @@ -534,7 +559,7 @@ func TestReconcileEtcdMembers(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterStatusKey: yaml.Raw(` + clusterStatusKey: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -610,7 +635,7 @@ func TestReconcileEtcdMembers(t *testing.T) { client.ObjectKey{Name: kubeadmConfigKey, Namespace: metav1.NamespaceSystem}, &actualConfig, )).To(Succeed()) - expectedOutput := yaml.Raw(` + expectedOutput := utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -702,7 +727,7 @@ func TestRemoveNodeFromKubeadmConfigMap(t *testing.T) { { name: "removes the api endpoint", apiEndpoint: "ip-10-0-0-2.ec2.internal", - clusterStatusData: yaml.Raw(` + clusterStatusData: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -713,7 +738,7 @@ func TestRemoveNodeFromKubeadmConfigMap(t *testing.T) { apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterStatus `), - wantClusterStatus: yaml.Raw(` + wantClusterStatus: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -725,7 +750,7 @@ func TestRemoveNodeFromKubeadmConfigMap(t *testing.T) { { name: "no op if the api endpoint does not exists", apiEndpoint: "ip-10-0-0-2.ec2.internal", - clusterStatusData: yaml.Raw(` + clusterStatusData: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -733,7 +758,7 @@ func TestRemoveNodeFromKubeadmConfigMap(t *testing.T) { apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterStatus `), - wantClusterStatus: yaml.Raw(` + wantClusterStatus: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 diff --git a/controlplane/kubeadm/internal/workload_cluster_test.go b/controlplane/kubeadm/internal/workload_cluster_test.go index bb15b6d99da2..eb475a89ca21 100644 --- a/controlplane/kubeadm/internal/workload_cluster_test.go +++ b/controlplane/kubeadm/internal/workload_cluster_test.go @@ -30,12 +30,13 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/client/fake" + "sigs.k8s.io/yaml" clusterv1 "sigs.k8s.io/cluster-api/api/v1beta1" bootstrapv1 "sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1beta1" controlplanev1 "sigs.k8s.io/cluster-api/controlplane/kubeadm/api/v1beta1" "sigs.k8s.io/cluster-api/util/version" - "sigs.k8s.io/cluster-api/util/yaml" + utilyaml "sigs.k8s.io/cluster-api/util/yaml" ) func TestGetControlPlaneNodes(t *testing.T) { @@ -262,7 +263,7 @@ func TestRemoveMachineFromKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterStatusKey: yaml.Raw(` + clusterStatusKey: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -327,7 +328,7 @@ func TestRemoveMachineFromKubeadmConfigMap(t *testing.T) { machine: machine, objs: []client.Object{kubeadmConfig}, expectErr: false, - expectedEndpoints: yaml.Raw(` + expectedEndpoints: utilyaml.Raw(` apiEndpoints: ip-10-0-0-2.ec2.internal: advertiseAddress: 10.0.0.2 @@ -397,7 +398,7 @@ func TestUpdateKubeletConfigMap(t *testing.T) { ResourceVersion: "some-resource-version", }, Data: map[string]string{ - kubeletConfigKey: yaml.Raw(` + kubeletConfigKey: utilyaml.Raw(` apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration foo: bar @@ -416,7 +417,7 @@ func TestUpdateKubeletConfigMap(t *testing.T) { ResourceVersion: "some-resource-version", }, Data: map[string]string{ - kubeletConfigKey: yaml.Raw(` + kubeletConfigKey: utilyaml.Raw(` apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration foo: bar @@ -435,7 +436,7 @@ func TestUpdateKubeletConfigMap(t *testing.T) { ResourceVersion: "some-resource-version", }, Data: map[string]string{ - kubeletConfigKey: yaml.Raw(` + kubeletConfigKey: utilyaml.Raw(` apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration foo: bar @@ -453,7 +454,7 @@ func TestUpdateKubeletConfigMap(t *testing.T) { ResourceVersion: "some-resource-version", }, Data: map[string]string{ - kubeletConfigKey: yaml.Raw(` + kubeletConfigKey: utilyaml.Raw(` apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration foo: bar @@ -473,7 +474,7 @@ func TestUpdateKubeletConfigMap(t *testing.T) { ResourceVersion: "some-resource-version", }, Data: map[string]string{ - kubeletConfigKey: yaml.Raw(` + kubeletConfigKey: utilyaml.Raw(` apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: cgroupfs @@ -576,7 +577,7 @@ func TestUpdateUpdateClusterConfigurationInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterConfigurationKey: yaml.Raw(` + clusterConfigurationKey: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v1.16.1 @@ -590,7 +591,7 @@ func TestUpdateUpdateClusterConfigurationInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterConfigurationKey: yaml.Raw(` + clusterConfigurationKey: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v1.16.1 @@ -607,7 +608,7 @@ func TestUpdateUpdateClusterConfigurationInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterConfigurationKey: yaml.Raw(` + clusterConfigurationKey: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v1.16.1 @@ -623,7 +624,7 @@ func TestUpdateUpdateClusterConfigurationInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterConfigurationKey: yaml.Raw(` + clusterConfigurationKey: utilyaml.Raw(` apiServer: {} apiVersion: kubeadm.k8s.io/v1beta2 controllerManager: {} @@ -646,7 +647,7 @@ func TestUpdateUpdateClusterConfigurationInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterConfigurationKey: yaml.Raw(` + clusterConfigurationKey: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v1.16.1 @@ -662,7 +663,7 @@ func TestUpdateUpdateClusterConfigurationInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterConfigurationKey: yaml.Raw(` + clusterConfigurationKey: utilyaml.Raw(` apiServer: {} apiVersion: kubeadm.k8s.io/v1beta3 controllerManager: {} @@ -686,7 +687,7 @@ func TestUpdateUpdateClusterConfigurationInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.updateClusterConfiguration(ctx, tt.mutator, tt.version) + err := w.UpdateClusterConfiguration(ctx, tt.version, tt.mutator) if tt.wantErr { g.Expect(err).To(HaveOccurred()) return @@ -754,7 +755,7 @@ func TestUpdateUpdateClusterStatusInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterStatusKey: yaml.Raw(` + clusterStatusKey: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -771,7 +772,7 @@ func TestUpdateUpdateClusterStatusInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterStatusKey: yaml.Raw(` + clusterStatusKey: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -791,7 +792,7 @@ func TestUpdateUpdateClusterStatusInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterStatusKey: yaml.Raw(` + clusterStatusKey: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -810,7 +811,7 @@ func TestUpdateUpdateClusterStatusInKubeadmConfigMap(t *testing.T) { Namespace: metav1.NamespaceSystem, }, Data: map[string]string{ - clusterStatusKey: yaml.Raw(` + clusterStatusKey: utilyaml.Raw(` apiEndpoints: ip-10-0-0-1.ec2.internal: advertiseAddress: 10.0.0.1 @@ -859,7 +860,7 @@ func TestUpdateKubernetesVersionInKubeadmConfigMap(t *testing.T) { { name: "updates the config map and changes the kubeadm API version", version: semver.MustParse("1.17.2"), - clusterConfigurationData: yaml.Raw(` + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v1.16.1`), @@ -882,7 +883,8 @@ func TestUpdateKubernetesVersionInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.UpdateKubernetesVersionInKubeadmConfigMap(ctx, tt.version) + + err := w.UpdateClusterConfiguration(ctx, tt.version, w.UpdateKubernetesVersionInKubeadmConfigMap(tt.version)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap @@ -905,7 +907,7 @@ func TestUpdateImageRepositoryInKubeadmConfigMap(t *testing.T) { }{ { name: "it should set the image repository", - clusterConfigurationData: yaml.Raw(` + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration`), newImageRepository: "example.com/k8s", @@ -913,7 +915,7 @@ func TestUpdateImageRepositoryInKubeadmConfigMap(t *testing.T) { }, { name: "it should preserve the existing image repository if then new value is empty", - clusterConfigurationData: yaml.Raw(` + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration imageRepository: foo.bar/baz.io`), @@ -938,7 +940,7 @@ func TestUpdateImageRepositoryInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.UpdateImageRepositoryInKubeadmConfigMap(ctx, tt.newImageRepository, semver.MustParse("1.19.1")) + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.UpdateImageRepositoryInKubeadmConfigMap(tt.newImageRepository)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap @@ -961,7 +963,7 @@ func TestUpdateApiServerInKubeadmConfigMap(t *testing.T) { }{ { name: "it should set the api server config", - clusterConfigurationData: yaml.Raw(` + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration `), @@ -980,7 +982,7 @@ func TestUpdateApiServerInKubeadmConfigMap(t *testing.T) { }, }, }, - wantClusterConfiguration: yaml.Raw(` + wantClusterConfiguration: utilyaml.Raw(` apiServer: extraArgs: bar: baz @@ -1016,7 +1018,7 @@ func TestUpdateApiServerInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.UpdateAPIServerInKubeadmConfigMap(ctx, tt.newAPIServer, semver.MustParse("1.19.1")) + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.UpdateAPIServerInKubeadmConfigMap(tt.newAPIServer)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap @@ -1039,7 +1041,7 @@ func TestUpdateControllerManagerInKubeadmConfigMap(t *testing.T) { }{ { name: "it should set the controller manager config", - clusterConfigurationData: yaml.Raw(` + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration `), @@ -1056,7 +1058,7 @@ func TestUpdateControllerManagerInKubeadmConfigMap(t *testing.T) { }, }, }, - wantClusterConfiguration: yaml.Raw(` + wantClusterConfiguration: utilyaml.Raw(` apiServer: {} apiVersion: kubeadm.k8s.io/v1beta2 controllerManager: @@ -1092,7 +1094,7 @@ func TestUpdateControllerManagerInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.UpdateControllerManagerInKubeadmConfigMap(ctx, tt.newControllerManager, semver.MustParse("1.19.1")) + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.UpdateControllerManagerInKubeadmConfigMap(tt.newControllerManager)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap @@ -1115,7 +1117,7 @@ func TestUpdateSchedulerInKubeadmConfigMap(t *testing.T) { }{ { name: "it should set the scheduler config", - clusterConfigurationData: yaml.Raw(` + clusterConfigurationData: utilyaml.Raw(` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration `), @@ -1132,7 +1134,7 @@ func TestUpdateSchedulerInKubeadmConfigMap(t *testing.T) { }, }, }, - wantClusterConfiguration: yaml.Raw(` + wantClusterConfiguration: utilyaml.Raw(` apiServer: {} apiVersion: kubeadm.k8s.io/v1beta2 controllerManager: {} @@ -1167,7 +1169,7 @@ func TestUpdateSchedulerInKubeadmConfigMap(t *testing.T) { w := &Workload{ Client: fakeClient, } - err := w.UpdateSchedulerInKubeadmConfigMap(ctx, tt.newScheduler, semver.MustParse("1.19.1")) + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.UpdateSchedulerInKubeadmConfigMap(tt.newScheduler)) g.Expect(err).ToNot(HaveOccurred()) var actualConfig corev1.ConfigMap @@ -1260,6 +1262,70 @@ func TestClusterStatus(t *testing.T) { } } +func TestUpdateFeatureGatesInKubeadmConfigMap(t *testing.T) { + tests := []struct { + name string + clusterConfigurationData string + newFeatureGates map[string]bool + wantFeatureGates map[string]bool + }{ + { + name: "it updates feature gates", + clusterConfigurationData: utilyaml.Raw(` + apiVersion: kubeadm.k8s.io/v1beta2 + kind: ClusterConfiguration`), + newFeatureGates: map[string]bool{"EtcdLearnerMode": true}, + wantFeatureGates: map[string]bool{"EtcdLearnerMode": true}, + }, + { + name: "it should override feature gates even if new value is nil", + clusterConfigurationData: utilyaml.Raw(` + apiVersion: kubeadm.k8s.io/v1beta2 + kind: ClusterConfiguration + featureGates: + EtcdLearnerMode: true + `), + newFeatureGates: nil, + wantFeatureGates: nil, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + g := NewWithT(t) + fakeClient := fake.NewClientBuilder().WithObjects(&corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Name: kubeadmConfigKey, + Namespace: metav1.NamespaceSystem, + }, + Data: map[string]string{ + clusterConfigurationKey: tt.clusterConfigurationData, + }, + }).Build() + + w := &Workload{ + Client: fakeClient, + } + err := w.UpdateClusterConfiguration(ctx, semver.MustParse("1.19.1"), w.UpdateFeatureGatesInKubeadmConfigMap(tt.newFeatureGates)) + g.Expect(err).ToNot(HaveOccurred()) + + var actualConfig corev1.ConfigMap + g.Expect(w.Client.Get( + ctx, + client.ObjectKey{Name: kubeadmConfigKey, Namespace: metav1.NamespaceSystem}, + &actualConfig, + )).To(Succeed()) + + actualConfiguration := bootstrapv1.ClusterConfiguration{} + err = yaml.Unmarshal([]byte(actualConfig.Data[clusterConfigurationKey]), &actualConfiguration) + if err != nil { + return + } + g.Expect(actualConfiguration.FeatureGates).Should(Equal(tt.wantFeatureGates)) + }) + } +} + func getProxyImageInfo(ctx context.Context, c client.Client) (string, error) { ds := &appsv1.DaemonSet{} diff --git a/docs/book/book.toml b/docs/book/book.toml index ea3f1591c7f4..0f07025c6a7a 100644 --- a/docs/book/book.toml +++ b/docs/book/book.toml @@ -13,11 +13,11 @@ additional-css = ["theme/css/custom.css"] [output.html.redirect] "/tasks/upgrade.html" = "/tasks/upgrading-cluster-api-versions.html" "/agenda.html" = "/agenda/2024.html" -"/agenda/2024.html" = "https://docs.google.com/document/d/1X5_qNUvoY0Tk3BwXODWHAl72L-APM_GiA-IE4WoitYY" +"/agenda/2024.html" = "https://docs.google.com/document/d/1GgFbaYs-H6J5HSQ6a7n4aKpk0nDLE2hgG2NSOM9YIRw" "/agenda/2023.html" = "https://docs.google.com/document/d/1Ov_rbxOV_O4HeAuwTev1lSwkiLJoBc_HDS1NyI93AcY" -"/agenda/2022.html" = "" +"/agenda/2022.html" = "https://docs.google.com/document/d/1Bk_1581mriNUlwOFpt7SAC3WnaM_gBqLy8aWoiWG-9A" "/agenda/2021.html" = "" -"/agenda/2020.html" = "" +"/agenda/2020.html" = "https://docs.google.com/document/d/1r8ydd2g2NFE4giD9MqLspv0CJcjzC17e4UUQyyNLx2M" "/agenda/2019.html" = "https://docs.google.com/document/d/1Ys-DOR5UsgbMEeciuG0HOgDQc8kZsaWIWJeKJ1-UfbY" [preprocessor.tabulate] diff --git a/docs/book/src/clusterctl/provider-contract.md b/docs/book/src/clusterctl/provider-contract.md index da52a09dfc46..b1ad3e261685 100644 --- a/docs/book/src/clusterctl/provider-contract.md +++ b/docs/book/src/clusterctl/provider-contract.md @@ -321,6 +321,7 @@ providers. | CAPDO | cluster.x-k8s.io/provider=infrastructure-digitalocean | | CAPG | cluster.x-k8s.io/provider=infrastructure-gcp | | CAPH | cluster.x-k8s.io/provider=infrastructure-hetzner | +| CAPHV | cluster.x-k8s.io/provider=infrastructure-hivelocity | | CAPIBM | cluster.x-k8s.io/provider=infrastructure-ibmcloud | | CAPKK | cluster.x-k8s.io/provider=infrastructure-kubekey | | CAPK | cluster.x-k8s.io/provider=infrastructure-kubevirt | @@ -336,6 +337,7 @@ providers. | CAPZ | cluster.x-k8s.io/provider=infrastructure-azure | | CAPOSC | cluster.x-k8s.io/provider=infrastructure-outscale | | CAPK0S | cluster.x-k8s.io/provider=infrastructure-k0smotron | +| CAIPAMIC | cluster.x-k8s.io/provider=ipam-in-cluster | ### Workload cluster templates diff --git a/docs/book/src/developer/providers/implementers-guide/controllers_and_reconciliation.md b/docs/book/src/developer/providers/implementers-guide/controllers_and_reconciliation.md index a5029886c79a..836532204490 100644 --- a/docs/book/src/developer/providers/implementers-guide/controllers_and_reconciliation.md +++ b/docs/book/src/developer/providers/implementers-guide/controllers_and_reconciliation.md @@ -175,7 +175,7 @@ if err != nil { But wait, this isn't quite right. `Reconcile()` gets called periodically for updates, and any time any updates are made. That would mean we're potentially sending an email every few minutes! -This is an important thing about controllers: they need to be [*idempotent*][idempotent]. +This is an important thing about controllers: they need to be idempotent. This means a controller must be able to repeat actions on the same inputs without changing the effect of those actions. So in our case, we'll store the result of sending a message, and then check to see if we've sent one before. @@ -209,7 +209,6 @@ return ctrl.Result{}, nil [cluster]: https://godoc.org/sigs.k8s.io/cluster-api/api/v1beta1#Cluster [getowner]: https://godoc.org/sigs.k8s.io/cluster-api/util#GetOwnerMachine -[idempotent]: https://stackoverflow.com/questions/1077412/what-is-an-idempotent-operation #### A note about the status diff --git a/docs/book/src/developer/providers/migrations/v1.4-to-v1.5.md b/docs/book/src/developer/providers/migrations/v1.4-to-v1.5.md index ba37d8d8884d..44b42de695e5 100644 --- a/docs/book/src/developer/providers/migrations/v1.4-to-v1.5.md +++ b/docs/book/src/developer/providers/migrations/v1.4-to-v1.5.md @@ -25,7 +25,7 @@ maintainers of providers and consumers of our Go API. ### Removals - API version `v1alpha3` is not served in v1.5 (users can enable it manually in case they are lagging behind with deprecation cycles). Important: `v1alpha3` will be completely removed in 1.6. -- The lazy restmapper feature gate was removed in controller-runtime and lazy restmapper is now the default restmapper. Accordingly the `EXP_LAZY_RESTMAPPER` feature gate was removed in Cluster API. +- The lazy restmapper feature gate was removed in controller-runtime and lazy restmapper is now the default restmapper. Accordingly the `EXP_LAZY_RESTMAPPER` feature gate was removed in Cluster API. ### API Changes @@ -34,32 +34,34 @@ maintainers of providers and consumers of our Go API. ### Other - clusterctl move is adding the new annotation `clusterctl.cluster.x-k8s.io/delete-for-move` before object deletion. -- Providers running CAPI release-0.3 clusterctl upgrade tests should set `WorkloadKubernetesVersion` field to the maximum workload cluster kubernetes version supported by the old providers in `ClusterctlUpgradeSpecInput`. For more information, please see: https://github.com/kubernetes-sigs/cluster-api/pull/8518#issuecomment-1508064859 +- Providers running CAPI release-0.3 clusterctl upgrade tests should set `WorkloadKubernetesVersion` field to the maximum workload cluster kubernetes version supported by the old providers in `ClusterctlUpgradeSpecInput`. For more information, please see: https://github.com/kubernetes-sigs/cluster-api/pull/8518#issuecomment-1508064859 - Introduced function `CollectInfrastructureLogs` at the `ClusterLogCollector` interface in `test/framework/cluster_proxy.go` to allow collecting infrastructure related logs during tests. - A `GetTypedConfigOwner` function has been added to the `sigs.k8s.io./cluster-api/bootstrap/util` package. It is equivalent to `GetConfigOwner` except that it uses the cached typed client instead of the uncached unstructured client, so `GetTypedConfigOwner` is expected to be more performant. - `ClusterToObjectsMapper` in `sigs.k8s.io./cluster-api/util` has been deprecated, please use `ClusterToTypedObjectsMapper` instead. - The generated `kubeconfig` by the Control Plane providers must be labelled with the key-value pair `cluster.x-k8s.io/cluster-name=${CLUSTER_NAME}`. - This is required for the CAPI managers caches to store and retrieve them for the required operations. + This is required for the CAPI managers caches to store and retrieve them for the required operations. +- When using custom certificates, the certificates must be labeled with the key-value pair `cluster.x-k8s.io/cluster-name=${CLUSTER_NAME}`. + This is required for the CAPI managers caches to store and retrieve them for the required operations. ### Suggested changes for providers -- +- ## Notes about the controller-runtime bump This section shares our learnings of bumping controller-runtime to v0.15 in core Cluster API. It highlights the most relevant changes and pitfalls for Cluster API providers. For the full list of changes please see the [controller-runtime release notes](https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.15.0). -* Webhooks can now also return warnings, this requires adding an additional `admission.Warnings` return parameter to all webhooks. +* Webhooks can now also return warnings, this requires adding an additional `admission.Warnings` return parameter to all webhooks. * Manager options have been refactored and old fields have been deprecated. * Manager now has a builtin profiler server which can be enabled via `Options.PprofBindAddress`, this allows us to remove our profiler server. * Controller builder has been refactored, this requires small changes to our controller setup code. * The EventHandler interface has been modified to also take a context, which affects our mapping functions (e.g. `ClusterToInfrastructureMapFunc`). * Controller-runtime now uses a lazy restmapper per default, i.e. API groups and resources are only fetched when they are actually used. This should drastically reduce the amount of API calls in clusters with a lot of CRDs. -* Some wait utils in `k8s.io/apimachinery/pkg/util/wait` have been deprecated. The migration is relatively straightforward except that passing in `0` +* Some wait utils in `k8s.io/apimachinery/pkg/util/wait` have been deprecated. The migration is relatively straightforward except that passing in `0` as a timeout in `wait.PollUntilContextTimeout` is treated as a timeout with 0 seconds, in `wait.PollImmediateWithContext` it is interpreted as infinity. * The fake client has been improved to handle status properly. In tests that write the CRD status, the CRDs should be added to the fake client via `WithStatusSubresource`. -* Ensure that the e2e test suite is setting a logger (e.g. via `ctrl.SetLogger(klog.Background())` in `TestE2E`. Otherwise logs are not visible and controller-runtime will print a warning. +* Ensure that the e2e test suite is setting a logger (e.g. via `ctrl.SetLogger(klog.Background())` in `TestE2E`. Otherwise logs are not visible and controller-runtime will print a warning. For reference, please see the [Bump to CR v0.15 PR](https://github.com/kubernetes-sigs/cluster-api/pull/8007) in core Cluster API. diff --git a/docs/book/src/developer/providers/migrations/v1.6-to-v1.7.md b/docs/book/src/developer/providers/migrations/v1.6-to-v1.7.md index e7999a0869a0..d361011c8bfd 100644 --- a/docs/book/src/developer/providers/migrations/v1.6-to-v1.7.md +++ b/docs/book/src/developer/providers/migrations/v1.6-to-v1.7.md @@ -10,6 +10,7 @@ maintainers of providers and consumers of our Go API. ## Dependencies **Note**: Only the most relevant dependencies are listed, `k8s.io/` and `ginkgo`/`gomega` dependencies in Cluster API are kept in sync with the versions used by `sigs.k8s.io/controller-runtime`. +- sigs.k8s.io/kind: v0.20.x => v0.22.x ## Changes by Kind diff --git a/docs/book/src/developer/tilt.md b/docs/book/src/developer/tilt.md index 037a518721ac..3dec91b63380 100644 --- a/docs/book/src/developer/tilt.md +++ b/docs/book/src/developer/tilt.md @@ -8,7 +8,7 @@ workflow that offers easy deployments and rapid iterative builds. ## Prerequisites 1. [Docker](https://docs.docker.com/install/): v19.03 or newer -2. [kind](https://kind.sigs.k8s.io): v0.20.0 or newer +2. [kind](https://kind.sigs.k8s.io): v0.22.0 or newer 3. [Tilt](https://docs.tilt.dev/install.html): v0.30.8 or newer 4. [kustomize](https://github.com/kubernetes-sigs/kustomize): provided via `make kustomize` 5. [envsubst](https://github.com/drone/envsubst): provided via `make envsubst` @@ -337,7 +337,7 @@ Custom values for variable substitutions can be set using `kustomize_substitutio ```yaml kustomize_substitutions: NAMESPACE: "default" - KUBERNETES_VERSION: "v1.29.0" + KUBERNETES_VERSION: "v1.29.2" CONTROL_PLANE_MACHINE_COUNT: "1" WORKER_MACHINE_COUNT: "3" # Note: kustomize substitutions expects the values to be strings. This can be achieved by wrapping the values in quotation marks. diff --git a/docs/book/src/images/bootstrap-controller.png b/docs/book/src/images/bootstrap-controller.png index 21db2d26587c..242385e69001 100644 Binary files a/docs/book/src/images/bootstrap-controller.png and b/docs/book/src/images/bootstrap-controller.png differ diff --git a/docs/book/src/images/bootstrap-provider.png b/docs/book/src/images/bootstrap-provider.png index 1a58524bfced..086e1f076dd6 100644 Binary files a/docs/book/src/images/bootstrap-provider.png and b/docs/book/src/images/bootstrap-provider.png differ diff --git a/docs/book/src/images/cluster-admission-cluster-controller.png b/docs/book/src/images/cluster-admission-cluster-controller.png index 8a08df94be4e..31d75eacae5e 100644 Binary files a/docs/book/src/images/cluster-admission-cluster-controller.png and b/docs/book/src/images/cluster-admission-cluster-controller.png differ diff --git a/docs/book/src/images/cluster-admission-machine-controller.png b/docs/book/src/images/cluster-admission-machine-controller.png index 5f9c9b520792..67c21f1d6328 100644 Binary files a/docs/book/src/images/cluster-admission-machine-controller.png and b/docs/book/src/images/cluster-admission-machine-controller.png differ diff --git a/docs/book/src/images/cluster-admission-machinedeployment-controller.png b/docs/book/src/images/cluster-admission-machinedeployment-controller.png index 5e2a9fe01f57..dcc81260a1d6 100644 Binary files a/docs/book/src/images/cluster-admission-machinedeployment-controller.png and b/docs/book/src/images/cluster-admission-machinedeployment-controller.png differ diff --git a/docs/book/src/images/cluster-admission-machinepool-controller.png b/docs/book/src/images/cluster-admission-machinepool-controller.png index 8c7b582dc6e1..892393c7ea6f 100644 Binary files a/docs/book/src/images/cluster-admission-machinepool-controller.png and b/docs/book/src/images/cluster-admission-machinepool-controller.png differ diff --git a/docs/book/src/images/cluster-admission-machineset-controller.png b/docs/book/src/images/cluster-admission-machineset-controller.png index e9ea358c154e..4f19976661b0 100644 Binary files a/docs/book/src/images/cluster-admission-machineset-controller.png and b/docs/book/src/images/cluster-admission-machineset-controller.png differ diff --git a/docs/book/src/images/cluster-infra-provider.png b/docs/book/src/images/cluster-infra-provider.png index fc58b3dff732..567f10a01693 100644 Binary files a/docs/book/src/images/cluster-infra-provider.png and b/docs/book/src/images/cluster-infra-provider.png differ diff --git a/docs/book/src/images/cluster-resource-set-controller.png b/docs/book/src/images/cluster-resource-set-controller.png index 57716645eb59..9a85c12734c0 100644 Binary files a/docs/book/src/images/cluster-resource-set-controller.png and b/docs/book/src/images/cluster-resource-set-controller.png differ diff --git a/docs/book/src/images/cluster-topology-controller.png b/docs/book/src/images/cluster-topology-controller.png index e8015a62c2ca..8a3358f7129b 100644 Binary files a/docs/book/src/images/cluster-topology-controller.png and b/docs/book/src/images/cluster-topology-controller.png differ diff --git a/docs/book/src/images/cluster-topology-reconciller.png b/docs/book/src/images/cluster-topology-reconciller.png index aeb20addb722..d875df566b0e 100644 Binary files a/docs/book/src/images/cluster-topology-reconciller.png and b/docs/book/src/images/cluster-topology-reconciller.png differ diff --git a/docs/book/src/images/control-plane-controller.png b/docs/book/src/images/control-plane-controller.png index 7ea511cd917e..2693c13bcf57 100644 Binary files a/docs/book/src/images/control-plane-controller.png and b/docs/book/src/images/control-plane-controller.png differ diff --git a/docs/book/src/images/kubeadm-control-plane-machines-resources.png b/docs/book/src/images/kubeadm-control-plane-machines-resources.png index 2e2c7a1b495c..160dbfc22fd9 100644 Binary files a/docs/book/src/images/kubeadm-control-plane-machines-resources.png and b/docs/book/src/images/kubeadm-control-plane-machines-resources.png differ diff --git a/docs/book/src/images/machine-infra-provider.png b/docs/book/src/images/machine-infra-provider.png index 2b0d8eec9463..792279645b9a 100644 Binary files a/docs/book/src/images/machine-infra-provider.png and b/docs/book/src/images/machine-infra-provider.png differ diff --git a/docs/book/src/images/machinehealthcheck-controller.png b/docs/book/src/images/machinehealthcheck-controller.png index 2a0affef6702..78f235b3b537 100644 Binary files a/docs/book/src/images/machinehealthcheck-controller.png and b/docs/book/src/images/machinehealthcheck-controller.png differ diff --git a/docs/book/src/images/management-workload-same-cluster.png b/docs/book/src/images/management-workload-same-cluster.png index 8d631fd39452..ac7bfe4a235d 100644 Binary files a/docs/book/src/images/management-workload-same-cluster.png and b/docs/book/src/images/management-workload-same-cluster.png differ diff --git a/docs/book/src/images/management-workload-separate-clusters.png b/docs/book/src/images/management-workload-separate-clusters.png index a3ec82376931..b10623077f56 100644 Binary files a/docs/book/src/images/management-workload-separate-clusters.png and b/docs/book/src/images/management-workload-separate-clusters.png differ diff --git a/docs/book/src/images/runtime-sdk-topology-mutation.png b/docs/book/src/images/runtime-sdk-topology-mutation.png index 99f3fb976a9e..92da0abe6f53 100644 Binary files a/docs/book/src/images/runtime-sdk-topology-mutation.png and b/docs/book/src/images/runtime-sdk-topology-mutation.png differ diff --git a/docs/book/src/images/worker-machines-resources.png b/docs/book/src/images/worker-machines-resources.png index 74d24f9d8416..0e2d05971aa8 100644 Binary files a/docs/book/src/images/worker-machines-resources.png and b/docs/book/src/images/worker-machines-resources.png differ diff --git a/docs/book/src/reference/glossary.md b/docs/book/src/reference/glossary.md index dd22362c30af..2b6d7103049c 100644 --- a/docs/book/src/reference/glossary.md +++ b/docs/book/src/reference/glossary.md @@ -66,6 +66,9 @@ Cluster API Google Cloud Provider ### CAPH Cluster API Provider Hetzner +### CAPHV +Cluster API Provider Hivelocity + ### CAPIBM Cluster API Provider IBM Cloud @@ -108,6 +111,9 @@ Cluster API Provider VMware Cloud Director ### CAPZ Cluster API Provider Azure +### CAIPAMIC +Cluster API IPAM Provider In Cluster + ### Cloud provider Or __Cloud service provider__ diff --git a/docs/book/src/reference/labels_and_annotations.md b/docs/book/src/reference/labels_and_annotations.md index e1eb0de5900a..3d20686f853c 100644 --- a/docs/book/src/reference/labels_and_annotations.md +++ b/docs/book/src/reference/labels_and_annotations.md @@ -2,7 +2,7 @@ | Label | Note | -| :---------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +|:------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | cluster.x-k8s.io/cluster-name | It is set on machines linked to a cluster and external objects(bootstrap and infrastructure providers). | | topology.cluster.x-k8s.io/owned | It is set on all the object which are managed as part of a ClusterTopology. | | topology.cluster.x-k8s.io/deployment-name | It is set on the generated MachineDeployment objects to track the name of the MachineDeployment topology it represents. | @@ -21,11 +21,12 @@ **Supported Annotations:** | Annotation | Note | -| :--------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +|:-----------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | clusterctl.cluster.x-k8s.io/skip-crd-name-preflight-check | Can be placed on provider CRDs, so that clusterctl doesn't emit an error if the CRD doesn't comply with Cluster APIs naming scheme. Only CRDs that are referenced by core Cluster API CRDs have to comply with the naming scheme. | | clusterctl.cluster.x-k8s.io/delete-for-move | DeleteForMoveAnnotation will be set to objects that are going to be deleted from the source cluster after being moved to the target cluster during the clusterctl move operation. It will help any validation webhook to take decision based on it. | | clusterctl.cluster.x-k8s.io/block-move | BlockMoveAnnotation prevents the cluster move operation from starting if it is defined on at least one of the objects in scope. Provider controllers are expected to set the annotation on resources that cannot be instantaneously paused and remove the annotation when the resource has been actually paused. | | unsafe.topology.cluster.x-k8s.io/disable-update-class-name-check | It can be used to disable the webhook check on update that disallows a pre-existing Cluster to be populated with Topology information and Class. | +| unsafe.topology.cluster.x-k8s.io/disable-update-version-check | It can be used to disable the webhook checks on update that disallows updating the `.topology.spec.version` on certain conditions. | | cluster.x-k8s.io/cluster-name | It is set on nodes identifying the name of the cluster the node belongs to. | | cluster.x-k8s.io/cluster-namespace | It is set on nodes identifying the namespace of the cluster the node belongs to. | | cluster.x-k8s.io/machine | It is set on nodes identifying the machine the node belongs to. | diff --git a/docs/book/src/reference/providers.md b/docs/book/src/reference/providers.md index 3589f794739a..7f368ae83ead 100644 --- a/docs/book/src/reference/providers.md +++ b/docs/book/src/reference/providers.md @@ -33,8 +33,9 @@ updated info about which API version they are supporting. - [CoxEdge](https://github.com/coxedge/cluster-api-provider-coxedge) - [DigitalOcean](https://github.com/kubernetes-sigs/cluster-api-provider-digitalocean) - [Equinix Metal (formerly Packet)](https://github.com/kubernetes-sigs/cluster-api-provider-packet) -- [Google Cloud Platform (GCP)](https://github.com/kubernetes-sigs/cluster-api-provider-gcp) +- [Google Cloud Platform (GCP)](https://cluster-api-gcp.sigs.k8s.io/) - [Hetzner](https://github.com/syself/cluster-api-provider-hetzner) +- [Hivelocity](https://github.com/hivelocity/cluster-api-provider-hivelocity) - [IBM Cloud](https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud) - [KubeKey](https://github.com/kubesphere/kubekey) - [KubeVirt](https://github.com/kubernetes-sigs/cluster-api-provider-kubevirt) diff --git a/docs/book/src/tasks/certs/using-custom-certificates.md b/docs/book/src/tasks/certs/using-custom-certificates.md index b89373cd162f..267ac83abdf9 100644 --- a/docs/book/src/tasks/certs/using-custom-certificates.md +++ b/docs/book/src/tasks/certs/using-custom-certificates.md @@ -11,6 +11,7 @@ Each certificate must be stored in a single secret named one of: | *[cluster name]***-proxy** | CA | openssl req -x509 -subj "/CN=Front-End Proxy" -new -newkey rsa:2048 -nodes -keyout tls.key -sha256 -days 3650 -out tls.crt | | *[cluster name]***-sa** | Key Pair | openssl genrsa -out tls.key 2048 && openssl rsa -in tls.key -pubout -out tls.crt | +The certificates *must* also be labeled with the key-value pair `cluster.x-k8s.io/cluster-name=[cluster name]` (where `[cluster name]` is the name of the cluster it should be used with).