how do I set oidc-ca-file?

[mlbiam]

I'm trying to get a CA cert available for supporting my IdP. I can't seem to get the cert available. I setup the following machine patch:

cluster:
  apiServer:
    extraArgs:
      "oidc-issuer-url": "https://ou.192-168-2-204.nip.io/auth/idp/k8sIdp"
      "oidc-client-id": "kubernetes"
      "oidc-username-claim": "sub"
      "oidc-groups-claim": "groups"
      "oidc-ca-file": "/etc/oidc/oidc-ca.crt"
    extraVolumes:
    - mountPath: /etc/oidc
      hostPath: /var/oidc
machine:
  files:
    - path: /var/oidc/oidc-ca.crt
      op: create
      content: |
        -----BEGIN CERTIFICATE-----
        MIIDETCCAfmgAwIBAgIUbkbKfQ7oeurnTzrygH/GCKI36E0wDQYJKoZIhvcNAQEL
        BQAwGDEWMBQGA1UEAwwNZW50ZXJwcmlzZS1jYTAeFw0yMjExMDcxNDQ1MjJaFw0z
        MjExMDQxNDQ1MjJaMBgxFjAUBgNVBAMMDWVudGVycHJpc2UtY2EwggEiMA0GCSqG
        SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCnqVwyQo22rG6nUZcSe/GmVfr90Kzgux09
        468pSSQdpDq9RTQUOvfATPBW81wBRfP1/ryEhshruAKa9/5h+5B/x8ncxTXpm8B7 
        vD7etv8WuryKPsILuidOD0GQSETo77AXM7FfiROr01j7w6QPwuPvBJSp3ikiC/Dc
        vE66lvIEXN7dSgDddvuvGQNDWOYlGZhf5FHW/5drPHuO9zuyUG+MMi1iP+RBMPRg
        Ie6v8BpOgrsgdtmXLa4VMsPM+0XfD0H8cSf/2H6WS4/7D8AulnPIoKcY+FLJPAm3
        ITR7/l6Q0IQuMSw6BLKafBFnBVcTQSH7yJdAJ5gH4VYDr2jkUZL3AgMBAAGjUzBR
        MB0GA1UdDgQWBBSf9D5FKwHIF7xWqF/48n+r/RTQ3jAfBgNVHSMEGDAWgBSf9D5F
        KwHIF7xWqF/48n+r/RTQ3jAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA
        A4IBAQB7Pl29+rRvxp+TxKOtBddKxHaE2UELnbidhU/16QmnuViBAXbuERHAvcJa
        naoZecBUARtiLXOjh91A6Ao5ZDOdDNYNRCgLb6s7CUXR+3KzvVFcITTRtkSNLJ15 
        g4hjYrAKDY1H3OswQ/SrhLoFBwgxbICQyxSKit49Dk+exszn1BE716iiIVgYOGZO
        yIayzBYumFss40jkmhlnk5enab8IL4TqpCe/qbvm5wNjKZUZ3jbl3d1UemqYNuYV
        cEcZ4QymABYKy4VE3TRYRbIdet4V6uYHF5YPyEEiY0TUe+XURZVAmiOcrkjnUHOx
        1bjzRqJZL5TwoFCg5eeDzuY4ZTcc
        -----END CERTIFICATE-----

but the API server won't start and talosctl logs -k kube-system/kube-apiserver-x produces no logs.

talosctl  --nodes 192.168.2.204 --talosconfig ./talosconfig  -e 192.168.2.204 containers -k                                    
NODE            NAMESPACE   ID                                                                                          IMAGE                                             PID    STATUS
192.168.2.204   k8s.io      kube-system/kube-apiserver-x                                                    registry.k8s.io/pause:3.10                        2511   SANDBOX_READY
192.168.2.204   k8s.io      └─ kube-system/kube-apiserver-x:kube-apiserver:892d8f9eb7cf                     registry.k8s.io/kube-apiserver:v1.32.0            0      CONTAINER_EXITED
192.168.2.204   k8s.io      kube-system/kube-controller-manager-x                                           registry.k8s.io/pause:3.10                        2512   SANDBOX_READY
192.168.2.204   k8s.io      └─ kube-system/kube-controller-manager-x-kube-controller-manager:dd0e18d084a5   registry.k8s.io/kube-controller-manager:v1.32.0   0      CONTAINER_EXITED
192.168.2.204   k8s.io      kube-system/kube-scheduler-x                                                    registry.k8s.io/pause:3.10                        2513   SANDBOX_READY
192.168.2.204   k8s.io      └─ kube-system/kube-scheduler-x:kube-scheduler:3be6ffbd672f                     registry.k8s.io/kube-scheduler:v1.32.0            0      CONTAINER_EXITED

I previously tried adding

apiVersion: v1alpha1
kind: TrustedRootsConfig
name: openunison-ca
certificates: |-
    -----BEGIN CERTIFICATE-----
    MIIDETCCAfmgAwIBAgIUbkbKfQ7oeurnTzrygH/GCKI36E0wDQYJKoZIhvcNAQEL
    BQAwGDEWMBQGA1UEAwwNZW50ZXJwcmlzZS1jYTAeFw0yMjExMDcxNDQ1MjJaFw0z
    MjExMDQxNDQ1MjJaMBgxFjAUBgNVBAMMDWVudGVycHJpc2UtY2EwggEiMA0GCSqG
    SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCnqVwyQo22rG6nUZcSe/GmVfr90Kzgux09
    468pSSQdpDq9RTQUOvfATPBW81wBRfP1/ryEhshruAKa9/5h+5B/x8ncxTXpm8B7
    vD7etv8WuryKPsILuidOD0GQSETo77AXM7FfiROr01j7w6QPwuPvBJSp3ikiC/Dc
    vE66lvIEXN7dSgDddvuvGQNDWOYlGZhf5FHW/5drPHuO9zuyUG+MMi1iP+RBMPRg
    Ie6v8BpOgrsgdtmXLa4VMsPM+0XfD0H8cSf/2H6WS4/7D8AulnPIoKcY+FLJPAm3
    ITR7/l6Q0IQuMSw6BLKafBFnBVcTQSH7yJdAJ5gH4VYDr2jkUZL3AgMBAAGjUzBR
    MB0GA1UdDgQWBBSf9D5FKwHIF7xWqF/48n+r/RTQ3jAfBgNVHSMEGDAWgBSf9D5F
    KwHIF7xWqF/48n+r/RTQ3jAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA
    A4IBAQB7Pl29+rRvxp+TxKOtBddKxHaE2UELnbidhU/16QmnuViBAXbuERHAvcJa
    naoZecBUARtiLXOjh91A6Ao5ZDOdDNYNRCgLb6s7CUXR+3KzvVFcITTRtkSNLJ15
    g4hjYrAKDY1H3OswQ/SrhLoFBwgxbICQyxSKit49Dk+exszn1BE716iiIVgYOGZO
    yIayzBYumFss40jkmhlnk5enab8IL4TqpCe/qbvm5wNjKZUZ3jbl3d1UemqYNuYV
    cEcZ4QymABYKy4VE3TRYRbIdet4V6uYHF5YPyEEiY0TUe+XURZVAmiOcrkjnUHOx
    1bjzRqJZL5TwoFCg5eeDzuY4ZTcc
    -----END CERTIFICATE-----

but that still got me TLS errors from oidc.

what am I missing?

[smira]

If the kube-apiserver doesn't run, it usually makes sense to look more into the Troubleshooting.

Also you can grab talosctl support bundle and attach it here for us to look into it.

[trunet]

I'm having the same issue trying to make talos work with keycloak. I think this is a valid use case for people relying on SSO and centralized user management.

Custom Certificate Authorities documentation lacks information about this use-case. When we apply TrustedRootsConfig it ends up in the OS, it should work for containers, pulling images (containerd, CRI works), but not in kubernetes control planes components.

As the kubernetes configurations are also handled by talos, would be nice to also include in that page, how we can have Trusted Roots Certificate in the apiserver to be able to use OIDC.

[trunet]

@mlbiam I manage to make it work. I don't know why your apiserver pods aren't running, but it's working for me.

@smira feel free to add it to the Custom Certificate Authorities documentation.

machine:
  files:
    - path: /var/local/oidc/oidc-ca.crt
      op: create
      permissions: 0o644
      content: |
        [REDACTED]
        
cluster:
  apiServer:
       extraArgs:
         oidc-ca-file: "/var/local/oidc/oidc-ca.crt"
         oidc-issuer-url: "https://[REDACTED]/realms/clients"
         oidc-client-id: kubernetes
         oidc-groups-claim: groups
         oidc-groups-prefix: "oidc:"
         oidc-username-claim: username
         oidc-username-prefix: "oidc:"
       extraVolumes:
         - hostPath: /var/local/oidc
           mountPath: /var/local/oidc
           readonly: true

Describing kube-apiserver pods, I can see:

Containers:
  kube-apiserver:
    Command:
...
      --oidc-ca-file=/var/local/oidc/oidc-ca.crt
      --oidc-client-id=kubernetes
      --oidc-groups-claim=groups
      --oidc-groups-prefix=oidc:
      --oidc-issuer-url=https://[REDACTED]/realms/clients
      --oidc-username-claim=username
      --oidc-username-prefix=oidc:
...
Volumes:
  var-local-oidc:
    Type:          HostPath (bare host directory volume)
    Path:          /var/local/oidc
    HostPathType:
...

[mlbiam]

Thanks @trunet . I updated my config and it seems to be working:

cluster:
  apiServer:
    extraArgs:
      "oidc-issuer-url": "https://ou.192-168-2-204.nip.io/auth/idp/k8sIdp"
      "oidc-client-id": "kubernetes"
      "oidc-username-claim": "sub"
      "oidc-groups-claim": "groups"
      "oidc-ca-file": "/var/local/oidc/oidc-ca.crt"
    extraVolumes:
    - mountPath: /var/local/oidc
      hostPath: /var/local/oidc
machine:
  files:
    - path: /var/local/oidc/oidc-ca.crt
      op: create
      permissions: 0o644
      content: |
        -----BEGIN CERTIFICATE-----
        MIIDETCCAfmgAwIBAgIUbkbKfQ7oeurnTzrygH/GCKI36E0wDQYJKoZIhvcNAQEL
        BQAwGDEWMBQGA1UEAwwNZW50ZXJwcmlzZS1jYTAeFw0yMjExMDcxNDQ1MjJaFw0z
        MjExMDQxNDQ1MjJaMBgxFjAUBgNVBAMMDWVudGVycHJpc2UtY2EwggEiMA0GCSqG
        SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCnqVwyQo22rG6nUZcSe/GmVfr90Kzgux09
        468pSSQdpDq9RTQUOvfATPBW81wBRfP1/ryEhshruAKa9/5h+5B/x8ncxTXpm8B7 
        vD7etv8WuryKPsILuidOD0GQSETo77AXM7FfiROr01j7w6QPwuPvBJSp3ikiC/Dc
        vE66lvIEXN7dSgDddvuvGQNDWOYlGZhf5FHW/5drPHuO9zuyUG+MMi1iP+RBMPRg
        Ie6v8BpOgrsgdtmXLa4VMsPM+0XfD0H8cSf/2H6WS4/7D8AulnPIoKcY+FLJPAm3
        ITR7/l6Q0IQuMSw6BLKafBFnBVcTQSH7yJdAJ5gH4VYDr2jkUZL3AgMBAAGjUzBR
        MB0GA1UdDgQWBBSf9D5FKwHIF7xWqF/48n+r/RTQ3jAfBgNVHSMEGDAWgBSf9D5F
        KwHIF7xWqF/48n+r/RTQ3jAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA
        A4IBAQB7Pl29+rRvxp+TxKOtBddKxHaE2UELnbidhU/16QmnuViBAXbuERHAvcJa
        naoZecBUARtiLXOjh91A6Ao5ZDOdDNYNRCgLb6s7CUXR+3KzvVFcITTRtkSNLJ15 
        g4hjYrAKDY1H3OswQ/SrhLoFBwgxbICQyxSKit49Dk+exszn1BE716iiIVgYOGZO
        yIayzBYumFss40jkmhlnk5enab8IL4TqpCe/qbvm5wNjKZUZ3jbl3d1UemqYNuYV
        cEcZ4QymABYKy4VE3TRYRbIdet4V6uYHF5YPyEEiY0TUe+XURZVAmiOcrkjnUHOx
        1bjzRqJZL5TwoFCg5eeDzuY4ZTcc
        -----END CERTIFICATE-----

The only changes were where I copied the cert to and where it's mounted to. Is there something in the talos docs that says "use /var/local for added files"?

[zombiemaker]

@mlbiam: Thanks for asking this question! This help me solve this problem.
@trunet: Thanks for posting your solution!