Support http01 challenge for Auto TLS feature

[ZhiminXiang]

In what area(s)?

/area networking
/kind proposal

Describe the feature

Currently Auto TLS only supports DNS01 challenge when integrating with cert-manager.
It would be useful for Auto TLS to also support HTTP01 challenge because:

1. not all users have permission to get the credential of their DNS servers in order to fulfill the DNS01 challenge.
2. not all DNS server providers expose API to manipulate DNS records.

How http01 challenge work

Per ACME, with HTTP validation, the client in an ACME transaction proves its control over a domain name by proving that it can provision HTTP resources on a server accessible under that domain name.

Specifically in the k8s world, the requests sent from Certificate Issuer to verify the ownership of the domain need to be routed a specific http01 k8s service in the cluster for fulfilling the http01 challenge. The http01 k8s service needs to return a token obtained from the Certificate Issuer. Per ACME, those requests have specific path /.well-known/acme-challenge/token.

Proposal

I propose below auto TLS workflow to support http01 certificate. This workflow is based on current auto TLS framework.

1. When a Route is created, Route controller creates a Knative Certificate to request a TLS certificates. This is the step already existed in the auto TLS flow.

2. When the Certificate is created, the Certificate controller will be responsible for getting the http01 challenge service. (new step)

3. After the Certificate controller gets the service, it will add the service name and the corresponding path (/.well-known/acme-challenge/token) in the Knative Certificate status. (new step).

4. After the Certificate status is updated, Route controller will add a new Path in the ClusterIngress which routes the requests matching the path in the Certificate status to the service in the Certificate status. (need to modify existing auto TLS step)

5. After the http01 challenge is fulfilled, the challenge information needs to be deleted in the Certificate status. As a result, Route controller will remove the Path for http01 challenge from the ClusterIngress.

Note that

1. this proposal is not specific for any Certificate CRD implementation. It is based on ACME contract.

2. In order to be able to fulfill http01 challenge, the ClusterIngress CRD implementation needs to support routing requests based on URI path. I assume this would not be a strong requirement as it is a basic feature of k8s Ingress.

/cc @mattmoor @tcnghia @mysterytony @greghaynes @ilackarms

[mpetason]

I'm interested in taking this.

[vagababov]

/assign @ZhiminXiang

[ZhiminXiang]

/assign @mpetason

[knative-prow-robot]

@ZhiminXiang: GitHub didn't allow me to assign the following users: mpetason.

Note that only knative members and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @mpetason

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tcnghia]

@mpetason we are planning this for 0.8.0 which is August 06. Would you be able to target that timeline? Thanks

[mpetason]

@tcnghia I am working on it but I don't think I will have it done in time for 0.8.0. I'm getting help from Ryan Moe.

[cpu]

Per ACME

One quick note: I think RFC 8555 §8.3 is a better citation to reference for this work instead of the obsolete draft-ietf-acme-acme-12.

As one concrete example draft-ietf-acme-acme-12 predates the "POST-as-GET" construction described in RFC 8555 §6.3 and would be affected by an upcoming Let's Encrypt API change.

[ZhiminXiang]

Thanks @cpu for sharing the information! I updated the description of this issue accordingly.

We are using cert-manager to interact with LetsEncrypt to get TLS certs. Looks like cert-manager already uses the "POST-as-GET" way to communicate with LetsEncrypt (https://github.com/jetstack/cert-manager/blob/a1fa913d4835497434e5905d20df1a15d7d4d9a6/third_party/crypto/acme/acme.go#L178).

@cpu @munnerz can you help to confirm this?

[cpu]

Looks like cert-manager already uses the "POST-as-GET" way to communicate with LetsEncrypt (https://github.com/jetstack/cert-manager/blob/a1fa913d4835497434e5905d20df1a15d7d4d9a6/third_party/crypto/acme/acme.go#L178)

@ZhiminXiang The specific line of code you linked to here isn't a POST-as-GET request (it's a more standard POST to create an order) but it looks like there are POST-as-GET requests being made to poll the order. The nil body parameter gives it away: https://github.com/jetstack/cert-manager/blob/a1fa913d4835497434e5905d20df1a15d7d4d9a6/third_party/crypto/acme/acme.go#L262

I don't believe cert-manager will have trouble with the unauthenticated GET deprecation.