Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamp bootstrap documentation #1226

Merged
merged 6 commits into from
Apr 8, 2021
Merged

Revamp bootstrap documentation #1226

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d8d0809
Move Azure DevOps bootstrap to Azure docs
stefanprodan Apr 7, 2021
f4adfc3
Add bootstrap git to install docs
stefanprodan Apr 7, 2021
719ef3c
Add flux CLI container image to docs
stefanprodan Apr 7, 2021
1ea5d4d
Remove sourceignore from SOPS guide
stefanprodan Apr 8, 2021
65d5cad
Update the alert providers list in notifications guide
stefanprodan Apr 8, 2021
a58c40f
Add note about providing a SSH key to bootstrap
stefanprodan Apr 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ Arch Linux (AUR) packages:
Binaries for macOS, Windows and Linux AMD64/ARM are available to download on the
[release page](https://github.com/fluxcd/flux2/releases).

A container image with `kubectl` and `flux` is available on Docker Hub and GitHub:

* `docker.io/fluxcd/flux-cli:<version>`
* `ghcr.io/fluxcd/flux-cli:<version>`

Verify that your cluster satisfies the prerequisites with:

```sh
Expand Down
240 changes: 60 additions & 180 deletions docs/guides/installation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@ are also supported with their own sub-commands.
Binaries for macOS, Windows and Linux AMD64/ARM are available for download on the
[release page](https://github.com/fluxcd/flux2/releases).

A container image with `kubectl` and `flux` is available on DockerHub and GitHub:

* `docker.io/fluxcd/flux-cli:<version>`
* `ghcr.io/fluxcd/flux-cli:<version>`

Verify that your cluster satisfies the prerequisites with:

```sh
Expand All @@ -42,36 +47,61 @@ flux check --pre
Using the `flux bootstrap` command you can install Flux on a
Kubernetes cluster and configure it to manage itself from a Git
repository.

The bootstrap creates a Git repository if one doesn't exist and
commits the Flux components manifests to the main branch. Then it
configures the target cluster to synchronize with that repository by
setting up SSH deploy keys.

If the Flux components are present on the cluster, the bootstrap
command will perform an upgrade if needed. The bootstrap is
idempotent, it's safe to run the command as many times as you want.

You can choose what components to install and for which cluster with:
The Flux component images are published to DockerHub and GitHub Container Registry
as [multi-arch container images](https://docs.docker.com/docker-for-mac/multi-arch/)
with support for Linux `amd64`, `arm64` and `armv7` (e.g. 32bit Raspberry Pi)
architectures.

If your Git provider is **GitHub**, **GitLab** or **Azure DevOps** please follow the specific bootstrap procedure:

* [github.com and GitHub Enterprise](#github-and-github-enterprise)
* [GitLab.com and GitLab Enterprise](#gitlab-and-gitlab-enterprise)
* [Azure DevOps](../use-cases/azure.md#flux-installation-for-azure-devops)

### Generic Git Server

The `bootstrap git` command takes an existing Git repository, clones it and
commits the Flux components manifests to the specified branch. Then it
configures the target cluster to synchronize with that repository.

Run bootstrap for a Git repository and authenticate with your SSH agent:

```sh
flux bootstrap <GIT-PROVIDER> \
--components=source-controller,kustomize-controller,helm-controller,notification-controller \
--components-extra=image-reflector-controller,image-automation-controller \
flux bootstrap git \
--url=ssh://git@<host>/<org>/<repository> \
--branch=<my-branch> \
--path=clusters/my-cluster
```

!!! hint "Multi-arch images"
The component images are published as [multi-arch container images](https://docs.docker.com/docker-for-mac/multi-arch/)
with support for Linux `amd64`, `arm64` and `armv7` (e.g. 32bit Raspberry Pi)
architectures.
The above command will generate a SSH key (defaults to RSA 2048 but can be changed with `--ssh-key-algorithm`),
and it will prompt you to add the SSH public key as a deploy key to your repository.

If you want to use your own SSH key, you can provide a **passwordless** private key using
`--private-key-file=<path/to/private.key>`.
This option can also be used if no SSH agent is available on your machine.

!!! hint "Bootstrap options"
There are many options available when bootstrapping Flux, such as installing a subset of Flux components,
setting the Kubernetes context, changing the Git author name and email, enabling Git submodules, and more.
To list all the available options run `flux bootstrap git --help`.

If your Git server doesn't support SSH, you can run bootstrap for Git over HTTPS:

If you wish to install a specific version, use the Flux
[release tag](https://github.com/fluxcd/flux2/releases) e.g. `--version=v0.9.0`.
```sh
flux bootstrap git \
--url=https://<host>/<org>/<repository> \
--username=<my-username> \
--password=<my-password> \
--token-auth=true \
--path=clusters/my-cluster
```

If you wish to deploy the Flux components onto
[tainted Kubernetes nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/),
you can specify the toleration keys with `--toleration-keys=node.kubernetes.io/dedicated-to-flux`.
If your Git server uses a self-signed TLS certificate, you can specify the CA file with
`--ca-file=<path/to/ca.crt>`.

With `--path` you can configure the directory which will be used to reconcile the target cluster.
To control multiple clusters from the same Git repository, you have to set a unique path per
Expand All @@ -84,24 +114,25 @@ cluster e.g. `clusters/staging` and `clusters/production`:
│   ├── gotk-components.yaml
│   ├── gotk-sync.yaml
│   └── kustomization.yaml
└── production-cluster # <- path=clusters/production
└── production # <- path=clusters/production
└── flux-system
```

After running bootstrap you can place Kubernetes YAMLs inside a dir under path
e.g. `clusters/staging/my-app`, and Flux will reconcile them on your cluster.

!!! hint "Change the default branch"
If you wish to change the branch to something else than main, create the repository manually,
push a branch to origin and then use `flux bootstrap <GIT-PROVIDER> --branch=your-branch`.

For examples on how you can structure your Git repository see:

* [flux2-kustomize-helm-example](https://github.com/fluxcd/flux2-kustomize-helm-example)
* [flux2-multi-tenancy](https://github.com/fluxcd/flux2-multi-tenancy)

### GitHub and GitHub Enterprise

The `bootstrap github` command creates a GitHub repository if one doesn't exist and
commits the Flux components manifests to specified branch. Then it
configures the target cluster to synchronize with that repository by
setting up a SSH deploy key or by using token-based authentication.

Generate a [personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line)
that can create repositories by checking all permissions under `repo`.

Expand Down Expand Up @@ -166,6 +197,11 @@ flux bootstrap github \

### GitLab and GitLab Enterprise

The `bootstrap gitlab` command creates a GitLab repository if one doesn't exist and
commits the Flux components manifests to specified branch. Then it
configures the target cluster to synchronize with that repository by
setting up a SSH deploy key or by using token-based authentication.

Generate a [personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html)
that grants complete read/write access to the GitLab API.

Expand Down Expand Up @@ -274,162 +310,6 @@ the CLI will use the manifests embedded in its binary instead of downloading
them from GitHub. You can determine which version you'll be installing,
with `flux --version`.

### Generic Git Server

For other Git providers such as Bitbucket, Gogs, Gitea, Azure DevOps, etc
you can manually setup the repository and deploy key.

Create a Git repository and clone it locally:

```sh
git clone ssh://<host>/<org>/my-repository
cd my-repository
```

Create a directory inside the repository:

```sh
mkdir -p ./clusters/my-cluster/flux-system
```

Generate the Flux manifests with:

```sh
flux install \
--export > ./clusters/my-cluster/flux-system/gotk-components.yaml
```

Commit and push the manifest to the master branch:

```sh
git add -A && git commit -m "add components" && git push
```

Apply the manifests on your cluster:

```sh
kubectl apply -f ./clusters/my-cluster/flux-system/gotk-components.yaml
```

Verify that the controllers have started:

```sh
flux check
```

Create a `GitRepository` object on your cluster by specifying the SSH address of your repo:

```sh
flux create source git flux-system \
--url=ssh://git@<host>/<org>/<repository> \
--ssh-key-algorithm=ecdsa \
--ssh-ecdsa-curve=p521 \
--branch=master \
--interval=1m
```

You will be prompted to add a deploy key to your repository.
If you don't specify the SSH algorithm, then `flux` will generate an RSA 2048 bits key.

!!! hint "Azure DevOps"
Azure DevOps requires a non-default Git implementation (`libgit2`) to be enabled, so that the Git v2 protocol is supported.
Note that this implementation does not support shallow cloning, and it is therefore advised to only resort to this option if a
connection fails with the default configuration.

Azure DevOps [only supports RSA SSH keys](https://developercommunity.visualstudio.com/t/support-non-rsa-keys-for-ssh-authentication/365980),
you cannot use elliptic curve SSH keys like ecdsa or ed25519.

Here is how to specify the `libgit2` implementation and generate a proper RSA key:

```sh
flux create source git flux-system \
--git-implementation=libgit2 \
--ssh-key-algorithm=rsa \
--ssh-rsa-bits=4096 \
--url=ssh://git@ssh.dev.azure.com/v3/<org>/<project>/<repository> \
--branch=main \
--interval=1m
```

This config uses the `main` branch, but your repo may be older and need to specify `master` instead.

Note that unlike `git`, Flux does not support the
["shorter" scp-like syntax for the SSH protocol](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#_the_ssh_protocol)
(e.g. `ssh.dev.azure.com:v3`).
Use the [RFC 3986 compatible syntax](https://tools.ietf.org/html/rfc3986#section-3) instead: `ssh.dev.azure.com/v3`.

The `flux create source git` command will prompt you to add a deploy key to your repository, but Azure DevOps
[does not support repository or org-specific deploy keys](https://developercommunity.visualstudio.com/t/allow-the-creation-of-ssh-deploy-keys-for-vsts-hos/365747).
You may add the deploy key to a user's personal SSH keys being mindful that removing them from the repo may revoke Flux's access.
As an alternative, create a machine-user whose sole purpose is to store credentials for automation.
Using a machine-user also has the benefit of being able to be read-only or restricted to specific repositories if that is needed.

If you wish to use Git over HTTPS, then generate a personal access token and supply it as the password:

```sh
flux create source git flux-system \
--git-implementation=libgit2 \
--url=https://dev.azure.com/<org>/<project>/_git/<repository> \
--branch=master \
--username=git \
--password=${AZ_PAT_TOKEN} \
--interval=1m
```

Please consult the [Azure DevOps documentation](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page)
on how to generate personal access tokens for Git repositories.
Azure DevOps PAT's always have an expiration date, so be sure to have some process for renewing or updating these tokens.
Similar to the lack of repo-specific deploy keys, a user needs to generate a user-specific PAT.
If you are using a machine-user, you can generate a PAT or simply use the machine-user's password which does not expire.

If your Git server supports basic auth, you can set the URL to HTTPS and specify the credentials with:

```sh
flux create source git flux-system \
--url=https://<host>/<org>/my-repository \
--username=my-username \
--password=my-password \
--branch=master \
--interval=1m
```

Create a `Kustomization` object on your cluster:

```sh
flux create kustomization flux-system \
--source=flux-system \
--path="./clusters/my-cluster" \
--prune=true \
--interval=10m
```

Export both objects, generate a `kustomization.yaml`, commit and push the manifests to Git:

```sh
flux export source git flux-system \
> ./clusters/my-cluster/flux-system/gotk-sync.yaml

flux export kustomization flux-system \
>> ./clusters/my-cluster/flux-system/gotk-sync.yaml

cd ./clusters/my-cluster/flux-system && kustomize create --autodetect

git add -A && git commit -m "add sync manifests" && git push
```

To upgrade the Flux components to a newer version, download the latest `flux` binary,
run the install command and commit the changes:

```sh
flux install \
--export > ./clusters/my-cluster/flux-system/gotk-components.yaml

git add -A && git commit -m "update flux" && git push
```

The source-controller will pull the changes on the cluster, then the kustomize-controller
will perform a rolling update of all Flux components including itself.

## Bootstrap with Terraform

The bootstrap procedure can be implemented with Terraform using the Flux provider published on
Expand Down
12 changes: 1 addition & 11 deletions docs/guides/mozilla-sops.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,23 +143,13 @@ Multiple directories can use separate SOPS configs.
Contributors using the `sops` CLI to create and encrypt files
won't have to worry about specifying the proper key for the target cluster or namespace.

`encrypted_regex` helps encrypt the the proper `data` and `stringData` fields for Secrets.
`encrypted_regex` helps encrypt the `data` and `stringData` fields for Secrets.
You may wish to add other fields if you are encrypting other types of Objects.

!!! hint
Note that you should encrypt only the `data` or `stringData` section. Encrypting the Kubernetes
secret metadata, kind or apiVersion is not supported by kustomize-controller.

Ignore all `.sops.yaml` files in a [`.sourceignore`](../components/source/gitrepositories#excluding-files)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is ignoring .sops.yaml files not a recommended/required step anymore?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest source-controller ignores .sops.yaml by default. See fluxcd/source-controller#329

file at the root of your repo.

```sh
touch .sourceignore
echo '**/.sops.yaml' >> .sourceignore
```

You can now commit your SOPS config.

## Encrypt secrets

Generate a Kubernetes secret manifest with kubectl:
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/notifications.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ spec:
name: slack-url
```

The provider type can be `slack`, `msteams`, `discord`, `rocket`, `github`, `gitlab` or `generic`.
The provider type can be `slack`, `msteams`, `discord`, `rocket`, `googlechat`, `webex`, `sentry` or `generic`.

When type `generic` is specified, the notification controller will post the incoming
[event](../components/notification/event.md) in JSON format to the webhook address.
Expand Down
Loading