'new-account' vs 'new-reg' in discovery info (aka supporting multiple versions of spec)

[munnerz]

Hi there

I'm working on switching to using pebble for our e2e tests in cert-manager (previously we've used letsencrypt/boulder, which is quite expensive to run!)

However my e2e tests fail against Pebble due to differences between the two. I understand that Pebble is meant to closely follow the ACME specification, whereas boulder makes some departures.

When querying https://acme-v01.api.letsencrypt.org/directory, the following discovery info is returned:

{
  "hsX66jWA6lI": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change",
  "meta": {
    "terms-of-service": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf"
  },
  "new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
  "new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",
  "new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",
  "revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"
}

However when doing the same against my pebble instance, I get:

{
   "meta": {
      "terms-of-service": "data:text/plain,Do%20what%20thou%20wilt"
   },
   "new-account": "http://test-pebble/sign-me-up",
   "new-nonce": "http://test-pebble/nonce-plz",
   "new-order": "http://test-pebble/order-plz"
}

From what I can tell, Pebble implements acme-08 (the latest draft of the spec), however boulder seems to be using some older implementation that has different field names. Is there some way I can discovery the version in use? As it stands I'm not sure how I can support older ACME servers at the same time as being up to date 😢

I'm using golang.org/x/crypto/acme as my ACME library, which appears to implement a pre-08 version of the spec (as it does not recognise new-account, new-nonce or new-order)

/cc @cpu (as I've seen you working on this repo and you are very helpful!! 😄 )

[munnerz]

Ah - I see from discussion here and looking through this projects history that this is meant to support the ACME v2 protocol only, so looks like I can't use pebble just yet in my tests.

I assume LE will start serving acme-v2.api... once it's ready to roll out, and then support both versions of the protocol for a period of time? If so, cool 👍 I'll just need to punt this task to a later date when everything supports v2!

[cpu]

From what I can tell, Pebble implements acme-08 (the latest draft of the spec), however boulder seems to be using some older implementation that has different field names. Is there some way I can discovery the version in use?

You got it. Pebble is meant to be as close to the current draft as possible. Unfortunately since Boulder evolved alongside ACME there isn't a specific draft version that it matches. The best way to understand Boulder's implementation of ACME is to read draft-07 next to the list of divergences. It's admittedly awkward
:-(

I'm using golang.org/x/crypto/acme as my ACME library, which appears to implement a pre-08 version of the spec (as it does not recognise new-account, new-nonce or new-order)

Yup, almost every existing ACME client implements "V1" (the Boulder ACME that isn't an exact draft version). I think that's a side-effect of our popularity and the draft status of the RFC 😊

Beyond the directory endpoint URL differences Pebble/V2 change the certificate issuance flow pretty substantial and also rework the JWS validation for most endpoints

I assume LE will start serving acme-v2.api... once it's ready to roll out, and then support both versions of the protocol for a period of time? If so, cool 👍 I'll just need to punt this task to a later date when everything supports v2!

We're targeting having a staging environment for V2 available in December. Both it & the eventual production endpoint will be available via a separate API URL. We have no intention of deprecating/removing the existing V1 API anytime soon (there's way too many clients using it!).

You should be very safe punting on V2 unless you or your users are especially interested in having Wildcard support ASAP. That will only be available via the V2 API in January.

Hope that helps clarify! I'm going to close this issue since we don't intend to add V1 support to Pebble at the current time.

[munnerz]

Thanks again!

I'm going to keep an eye on progress, and try and get some testing against v2 API as soon as the golang library supports it. Exactly how I model this in cert-manager (to allow users to decide between different versions) I'm not sure, but I've had quite a lot of requests for wildcard support in the past so I'd like to support it soon after January!

Looks like I'm back to running a full boulder for e2e tests! Thanks 😄

[cpu]

Looks like I'm back to running a full boulder for e2e tests! Thanks 😄

Happy to help! If you find something particularly painful about using Boulder for integration tests please let us know. Within reason we'd be happy to try and make it easier.

[munnerz]

It's been working well for me so far - I've used a simple git clone and docker-compose up up until now.

We're moving all of our e2e testing into Kubernetes however, so will be converting this compose definition into a Kubernetes deployment and running boulder within k8s itself.

The one thing I've not found, is any official images pushed to any public registries. Am I right in thinking that you just don't publish images and instead expect users to build their own? 👍 if so!

[cpu]

so will be converting this compose definition into a Kubernetes deployment and running boulder within k8s itself.

Neat!

The one thing I've not found, is any official images pushed to any public registries. Am I right in thinking that you just don't publish images and instead expect users to build their own? 👍 if so!

Yup, that's the case today. We have an open issue to publish images (letsencrypt/boulder#2865) but it isn't on anyone's plate. I don't think there's any philosophical opposition, just a matter of time and TODO wrangling.

[jsha]

The one thing I've not found, is any official images pushed to any public registries. Am I right in thinking that you just don't publish images and instead expect users to build their own?

That's mostly correct. We do build a boulder-tools image that has all the precursor stuff for boulder, then that gets pulled down in local testing and on Travis, and the actual Boulder image gets built on top of that.

Note that the resulting built Boulder image is way big, because it includes all the tools. We've been meaning to publish an image that's just "Boulder, as built." But it's a little complicated; for instance, right now our Boulder image entrypoint configures the HSM (in another container) and runs the database migrations if needed. Both of those require files that are in the source distro. Not an overwhelming obstacle, just a little annoyance. Long story short: If you'd like to contribute the code to auto-publish Boulder images, we'd be open to that!