Store workspace outputs as secrets

[koikonom]

Currently the operator stores outputs as configmaps. This can be
undesirable for some environments since output values can be considered
sensitive. This change creates k8s secrets instead of ConfigMaps to
satisfy this use case.

[koikonom]

How to test:

Create a secret called workspacesecrets that holds your AWS credentials:

❯ kubectl create -n awstest secret generic workspacesecrets \
 --from-literal=AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
 --from-literal=AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
 --from-literal=AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}

Apply the following (or a similar) workspace

---
apiVersion: app.terraform.io/v1alpha1
kind: Workspace
metadata:
  name: test123
spec:
  secretsMountPath: "/tmp/secrets"
  module:
    source: "git::https://github.com/koikonom/tfc-operator-test.git"
  outputs:
    - key: aws_ebs_volume_id
      moduleOutputName: aws_ebs_volume_id
  variables:
    - key: region
      sensitive: false
      environmentVariable: false
      valueFrom:
        configMapKeyRef:
          name: aws-configuration
          key: region
    - key: zone
      value: us-west-2a
      sensitive: false
      environmentVariable: false
    - key: size
      value: "1"
      sensitive: false
      environmentVariable: false
    - key: AWS_DEFAULT_REGION
      valueFrom:
        configMapKeyRef:
          name: aws-configuration
          key: region
      sensitive: false
      environmentVariable: true
    - key: AWS_ACCESS_KEY_ID
      sensitive: true
      environmentVariable: true
    - key: AWS_SECRET_ACCESS_KEY
      sensitive: true
      environmentVariable: true
    - key: AWS_SESSION_TOKEN
      sensitive: true
      environmentVariable: true
    - key: CONFIRM_DESTROY
      value: "1"
      sensitive: false
      environmentVariable: true

Wait for a few minutes while the operator works its magic, and run

❯ kubectl -n awstest describe workspace/test123
[.... lots of output ...]
Events:
  Type    Reason          Age   From       Message
  ----    ------          ----  ----       -------
  Normal  WorkspaceEvent  12m   workspace  Started new Terraform job with id run-NbDXabkBo8tt6csJ
  Normal  WorkspaceEvent  11m   workspace  Updated outputs for run run-NbDXabkBo8tt6csJ

Once you see the Updated outputs message in the events you can then verify the secrets are there:

❯ kubectl -n awstest get secrets/test123-outputs -o json  | jq -r .data.aws_ebs_volume_id | base64 -D
vol-027e309f2a67be949

[koikonom]

Example output when upgrading:

Up until the previous version, once the workspace was done applying we could get the outputs from a config map:

❯ kubectl -n awstest describe configmaps/demo-outputs
Name:         demo-outputs
Namespace:    awstest
[.....]
Data
====
pet:
----
readily-actually-electric-wahoo
[....]

Once we install the latest version:

❯ helm install --devel --namespace awstest --generate-name --set global.imageK8S=terraform-k8s-dev:latest .
[...]

❯ kubectl -n awstest get secret/demo-outputs
NAME           TYPE     DATA   AGE
demo-outputs   Opaque   2      69s

[dak1n1]

I started testing this today. It looks like there is a regression in the CRD:

$ helm install -n test operator .
Error: failed to install CRD crds/app.terraform.io_workspaces_crd.yaml: CustomResourceDefinition.apiextensions.k8s.io "workspaces.app.terraform.io" is invalid: [spec.versions[0].schema.openAPIV3Schema.properties[spec].properties[terraformVersion].default: Forbidden: must not be set (cannot set default values in apiextensions.k8s.io/v1beta1 CRDs, must use apiextensions.k8s.io/v1

I'm working around it by setting the API version to v1 instead of v1beta1, but the CRD should probably be updated so the installation process isn't broken.

[dak1n1]

Nice, it's working! I had to grab the latest commit in your test repo (which btw, thanks for adding that!). Now my test app is up-to-date enough for the secrets to work:

$ kube get secrets -n test test-outputs -o json |jq .data.aws_ebs_volume_id
"dm9sLTBhZTc0OTJhYzI0YzkzM2Nj"

$ kube get secrets -n test test-outputs -o json |jq -r .data.aws_ebs_volume_id |base64 -d
vol-0ae7492ac24c933cc

Next I'll just give the code a quick read-through.

[dak1n1]

Thanks for adding this feature! This looks great. The only issue I ran into was with the CRD, and maybe running go mod tidy to clean up go.mod, but other than that, everything is working well.

[dak1n1]

Nice upgrade notes! I like that users can take this at their own pace, rather than having the ConfigMap automatically deleted.

[dak1n1]

It looks like we need to run go mod tidy since the modules were in a bit of an inconsistent state when I looked at them locally. When I ran it, it cleaned up some things.

[koikonom]

Rebased against master.

[koikonom]

@dak1n1 see commit b77cb65 for the solution I chose for the CRD schema version issue you pointed out. (Thanks for reporting this btw :) )

[dak1n1]

Nice fix! It's all working as expected 😄

[aareet]

@koikonom does this resolve #39 ?

[koikonom]

Yes I believe it does.