Add infrastructure information to trusted root

[loosebazooka]

For a given trusted root (especially with sigstore public good), the fulio/ca instance should optionally be allowed to specify a(or many?) sigstore specific trusted oidc provider(s).

Why?
This allows clients to fully initialize their state from a single trusted_root.json, currently the only information missing is what would be required to initialize the oidc client for dex.

For example:

  "certificateAuthorities": [
    {
      "subject": {
        "organization": "sigstore.dev",
        "commonName": "sigstore"
      },
      "uri": "https://fulcio.sigstore.dev",
      //... new oidc providers associated with an instance of fulcio
      oidcProviders: [
        {
           "uri": "oauth2.sigstore.dev/auth"
           "clientId": "sigstore" // should default to this
           "wellKnownConfiguration": "/.well-known/openid-configuration"
        }
      ]
    },

1. This should be optional?
   • Yes? some instances may not want a web oidc option. Also allows us to clear the config is archived CAs.
2. should it be a list?
   • Yes? Allows for extensibility or single config for internal/external usecases
   • No? Client logic now has to check for existence/availability of endpoints as it decides which oidc endpoint to contact
3. should it be per CA?
   • Yes? It's not particularly harder for clients to parse this information.
   • Yes? As the trusted_root is updated, old CAs that are no longer active, will not need to define this. So repetition will be minimal.
4. is URI enough or should we have a bunch of other fields like client_id, well_known_configuration
   • Yes? Extensibility, custom configs

[loosebazooka]

@woodruffw @jku @kommendorkapten @haydentherapper @codysoyland @steiza I'm been wanting to have this feature for a while because it really makes sigstore-java's initialization code fully tied to the trusted_root.json. Additionally, for private instances, this means we don't have to expose a bunch of API bits in tooling (maven/gradle).

[woodruffw]

Big +1 from me! I think the desired functionality here is almost identical to what @jku has highlighted in #183; we want the OIDC information in the trust root for similar reasons on sigstore-python 🙂

[jku]

Yes, 👍 , I obviously also like this.

This should be optional?

Feels like that: I think it is probably a suggestion in any case?

should it be a list?

Current use cases involve taking the first item on the list and then throwing the list away (essentially replacing the hard coded dex URL that everyone now uses)... but I guess it can be a list.

is URI enough or should we have a bunch of other fields like client_id, well_known_configuration

can you expand on the potential use cases, I don't know what these are good for.

[woodruffw]

Current use cases involve taking the first item on the list and then throwing the list away (essentially replacing the hard coded dex URL that everyone now uses)... but I guess it can be a list.

Yeah, my soft preference would be for this being a single member, unless we know of a specific case where a Sigstore configuration/deployment is going to need/use multiple OIDC endpoints. I can see multiple endpoints getting pretty confusing 🙂

can you expand on the potential use cases, I don't know what these are good for.

client_id might be useful for some Sigstore deployments, e.g. if the staging instance of Sigstore suddenly switched to requiring sigstage instead of sigstore. But it currently doesn't, so this is an unknown I think.

IMO well_known_configuration might be an anti-pattern in the trust root -- OIDC says that the well-known URI is always at {TLD}/.well-known/openid-configuration, so including it as a potential configuration knob might encourage people to do non-standard OIDC things 🙂

Edit: Correcting myself: OIDC discovery says {TLD}/{ANY_PATH_COMPONENTS}/.well-known/openid-configuration, which is still fully discoverable given the auth endpoint itself.

[jku]

Yeah, my soft preference would be for this being a single member

This does mesh well with the meaning "this certificate issuer suggests using this OIDC endpoint".

[loosebazooka]

oh I forgot about #183 😞

Anyway, so... to summarize current sentiment:

1. Lets keep it a single optional oidcProvider (not a list)
2. It will be part of the CA configuration (only needs to exist on active logs)
3. Required fields uri, and optional fields of client_id and well_known_configuration

[jku]

optional fields of client_id and well_known_configuration

I think william made a good case against well_known_configuration

[loosebazooka]

ah I misread that. kk, lemme update the summary.

[haydentherapper]

should it be per CA?

I think it should not be per-CA and should be its own configuration. A CA can accept multiple OIDC providers, Fulcio does this. In Sigstore's trust root, we would not be distributing this list (or at least, I don't think we should because that list updates frequently and having to re-sign TUF targets frequently with our current setup is not ideal. Maybe a question for later). Sigstore's trust root would only contain the Dex instance, which can update independent of the CA.

is URI enough or should we have a bunch of other fields like client_id, well_known_configuration

client_id - Yea, for private deployments. I don't think well_known_configuration is needed. {TLD}/{path components} should be what uri is, correct? You could have uri = {TLD} and well_known_configuration = {path} + .well-known/..., but I don't see why, since just the TLD isn't usable by an OIDC client, right?

[loosebazooka]

I think it should not be per-CA and should be its own configuration. A CA can accept multiple OIDC providers, Fulcio does this. In Sigstore's trust root, we would not be distributing this list (or at least, I don't think we should because that list updates frequently and having to re-sign TUF targets frequently with our current setup is not ideal. Maybe a question for later). Sigstore's trust root would only contain the Dex instance, which can update independent of the CA.

Fuclio can accept multiple oidc providers, but fulcio is explicitly configured to accept a dex instance. What I want to provide a reference for is the dex instance -- so maybe odicProvider is a bad name?

[woodruffw]

What I want to provide a reference for is the dex instance -- so maybe odicProvider is a bad name?

Yeah, I could see this being confusing. Then again the provider isn't necessarily a Dex instance either 🤔 -- maybe we could call it hostedOidcProvider or similar? But I don't feel great about that name either.

[kommendorkapten]

I' thinking if this should be a separate config file, e.g trusted_op.json?
Do we even need to care from the client perspective that Fulcio trusts Dex, i,e have a link between them in the config (I think not, they are listed as trusted and Fulcio trusts dex, and we trust Fulcio).
If not it would be greatly simplified if we can just manage two files for this.

[woodruffw]

I' thinking if this should be a separate config file, e.g trusted_op.json?

At least for sigstore-python, the motivation for this change is to reduce the number of independent inputs needed to establish the "total" trusted state for both signing and verifying, so that we can get rid of all of the flags and configuration knobs that we currently have (e.g. individual knobs for Rekor URL, Rekor pubkey, Fulcio cert chain, OIDC issuer URL, etc.). Putting this into a separate config file/message wouldn't be the worst thing, but it would still mean two configuration states rather than one 🙂

Do we even need to care from the client perspective that Fulcio trusts Dex, i,e have a link between them in the config (I think not, they are listed as trusted and Fulcio trusts dex, and we trust Fulcio).

I don't think we need to know that Fulcio "trusts" Dex, per se, although maybe that's implied. What signing clients need is a way to go from an arbitrary trusted_root.json (or similar) to a state that's capable of making OIDC requests, which is currently not possible in the general case (we currently hardcode the URLs for Sigstore staging and PGI).

[loosebazooka]

Yeah I strongly prefer a single file. The dex instance is something that is very much coupled with fulcio. Its not config that has to be used by clients either. It just simply exists to inform people that public-good uses https://oauth2.sigstore.dev/auth and staging uses https://oauth2.sigstage.dev/auth which would otherwise have to be baked into the clients (or passed in to them). If a client doesn't support web based auth, then the parameter is ignored. To me, its optional but very useful information.

[steiza]

There's a lot here, but I think I caught up - the main problem we're trying to solve is how can signing libraries get information about which OIDC provider(s) to offer to a user, without making the user supply that information as an additional flag?

Today that information lives inside of Fulcio's configuration. We distribute some information to the client already via the trusted root, but which OIDC provider(s) to use is more of a hint to client libraries and users, and I think we really want to keep the concerns of the trust root as narrowly defined as possible.

Maybe it would make sense for a Fulcio instance to advertise its trusted OIDC provider(s) publicly, via some well-known URL? Then client libraries would only need to know which Fulcio instance to use to get this information, which they need to know anyways.

On the verification side, there's really two parts to verification: does the signature verify with my trust root and then does the signature / metadata comply with my policies. I think what OIDC provider(s) you trust when verifying is really a policy question, and again mixing this into the trust root feels like a mixing of concerns.

[haydentherapper]

We actually do advertise which OIDC providers we trust! curl https://fulcio.sigstore.dev/api/v2/configuration will return this list. However, this doesn't tell you which OIDC provider is the one that is also managed by Sigstore.

For the discussion of whether this should be in its own file or not, I think what's different here is this is only going to be used for signing, never verification. The token is never verified during verification because it's no longer present. I think it'd be fine to still bundle this in the existing trusted root spec.

Fuclio can accept multiple oidc providers, but fulcio is explicitly configured to accept a dex instance. What I want to provide a reference for is the dex instance -- so maybe odicProvider is a bad name?

For history, the primary reason Sigstore is operating its own oidc provider is so that we can control the lifetime of the issued tokens. As a client, I would recommend users use the Sigstore-managed Dex instance for this reason, but we know some won't use it, either because they're doing workload signing or if their federated identity provider is not supported by our Dex instance.

[loosebazooka]

So to avoid overloading the trusted_root, should we just have an infrastructure.json as a target that has all the infra urls?

This would make initializing clients very straight forward.

[kommendorkapten]

So to avoid overloading the trusted_root, should we just have an infrastructure.json as a target that has all the infra urls?

Yes, this is my preference. As it's only the signers that need this, I think we should avoid mixing concerns in trusted_root.json.

What else configuration would we want to have in this file? I'm thinking about naming.
We have updated the list of supported signing algorithms, but my gut feeling is against advertising this via the TUF repo as this may be information that changes more frequently. What we advertise via the TUF repo should be the bare minimum information, that is more or less static for a deployment.

[jku]

So to avoid overloading the trusted_root, should we just have an infrastructure.json as a target that has all the infra urls?

I'm not a fan

• It prevents the whole idea that a single file defines the infrastructure for the client: this is useful for non-TUF cases where you can still bootstrap a client with a single file
• It seems to not prevent overloading at all since the same people (root-signing signers) will still be vouching for the content -- the URL will now in practice be part of the trusted root but is confusingly not contained in the same file

The idea that fulcio servers can have an optional wellknown file that may contain a "suggested interactive OIDC URL" seems much better than just another file in the TUF repository (assuming that the extra field in trusted_root.json does not sound appealing).

[kommendorkapten]

It seems to not prevent overloading at all since the same people (root-signing signers) will still be vouching for the content -- the URL will now in practice be part of the trusted root but is confusingly not contained in the same file

The overloading IMHO comes from the content in the file, not how it's managed. The trusted_root.json was created to provide the public trusted root to use during verification.
Now adding adding configuration into the trust root, describing how some signers (when using the interactive email flow) should discover the OPs is where I say we don't agree.
From my view, it would be easier to reason about the system with two files, one for the public trust root, and one for any config used during signing.

I don't think it would be that much harder for a client to load two files instead of one when a signing operation should happen? They can still be merged into a single struct in-memory if desired.

[loosebazooka]

This would make initializing clients very straight forward.

I meant to add to this, but I guess I forgot. The current trusted_root.json is actually structured a little strange for initialization. We can have multiple "current" targets for CT, Rekor and Fulcio, because of they key rotation hints -- we don't have different URLs but we technically could. In addition to some private usecase where the trusted_root.json consists of multiple different current verification targets from different sigstore deployments. I think the client logic gets really weird.

[kommendorkapten]

The idea that fulcio servers can have an optional wellknown file that may contain a "suggested interactive OIDC URL" seems much better than just another file in the TUF repository

Agreed! Storing data in a TUF repository has a bit of slowness built into it, so if we can be more data-driven by the services this is preferable. There is one problem though, there can be many OPs that have the same challenge claim (e.g. email) for interactive signing.

An example entry looks like this:

    {
      "issuerUrl": "https://oauth2.sigstore.dev/auth",
      "audience": "sigstore",
      "challengeClaim": "email",
      "spiffeTrustDomain": ""
    },

Could we extend this data with more parameters that could guide the client for using the public good instance, like a default: true parameter.

[jku]

I don't think it would be that much harder for a client to load two files instead of one when a signing operation should happen?

It's not about programming difficulty, but user experience and CLI design.

imagine a setup where TUF is not used: the user is trying to bootstrap their client to use some specific infrastructure (that is not the public good one):

• currently people who do this provide half a dozen arguments to their client. This leads to likely wrong arguments being used accidentally and almost certainly client bugs (because no-one tests all of the combinations of the possible arguments)
• providing just two arguments (trusted_root and another config file) is certainly better but still not good
• providing a single trusted root file is much easier to manage

But again, trusted root + a wellknown file in fulcio domain does not have these problems so I'm fine with that.

[woodruffw]

As it's only the signers that need this, I think we should avoid mixing concerns in trusted_root.json.

I agree with this in the abstract, but I think in practice these concerns are already mixed 😅 -- signing is defined as also including a verification step over each signed artifact (although most clients don't currently do this), and signers also independently depend on the trust root for key material during SCT and CSR response verification.

One random idea that I just had that could be a suitable compromise: maybe the we could define a new message that encompasses the trusted root, but that also includes a single reference OIDC endpoint for signing purposes? Sigstore could then continue to serve the "normal" trust root via TUF, and clients could accept the "super" one via the CLI. But that might cause even more confusion 😕

[kommendorkapten]

One random idea that I just had that could be a suitable compromise: maybe the we could define a new message that encompasses the trusted root, but that also includes a single reference OIDC endpoint for signing purposes? Sigstore could then continue to serve the "normal" trust root via TUF, and clients could accept the "super" one via the CLI. But that might cause even more confusion

I have been thinking in these terms too, like this:

message TrustedRoot {
    ...
}

message OIDCProviders {
    ...
}

message ClientConfig {
    tr TrustedRoot
    op OIDCProviders
}

This would give us pretty good flexibility, we can ship two files via TUF, signing clients can assemble them in to a ClientConfig in memory. When providing a config via a command line parameter, users can chose to manage a ClientConfig file and provide that.

Maybe it is a bit hacky if we ship to files and clients assembles them from the TUF root if not provided as a command line parameter? I'm not sure tbh 🤔 I think this may work and be "good enough" 😬

[woodruffw]

Maybe it is a bit hacky if we ship to files and clients assembles them from the TUF root if not provided as a command line parameter? I'm not sure tbh 🤔 I think this may work and be "good enough" 😬

Yeah, this is the case I'm worried about 😅 -- the goal here is to reduce user confusion around "what" instance of Sigstore they're communicating with/what materials they're trusting, and two inputs instead of one means 3 error states, one valid state in the worst case.

(I agree with @jku that a well-known file also solves these issues adequately, so I'm good with that!)

[kommendorkapten]

I can prepare a PR that updates the comment section for the trust root, that Fulcio CAs exposes the OIDC config at /api/v2/configuration.

But there is still one problem I believe as I mentioned in the above comment, currently there are three OPs that use the email challenge claim. Google, Eclipse and DEX. We may want to provide some guidance here?

[steiza]

Using a well-known Fulcio endpoint for this is my preferred solution, but like @kommendorkapten is saying, I'm not sure that the existing /api/v2/configuration is sufficient.

One option is to have the Fulcio configuration optionally specify something like preferredUserOIDCProvider that in practice points at the DEX instance. Another option would be to add another Fulcio public endpoint, but I'm not sure about the overhead of another endpoint for one piece of information.

[woodruffw]

(Accidental close, sorry!)

What's missing from the client spec for signing is a standardized way to specify the URLs when not hardcoded with a default. Do we recommend all clients support an API or CLI flags to specify each dependency? Or something like an infrastructure file that takes in {rekor_url = x, fulcio_url = y, oidc_provider_url = z, tuf_repo_url = r, timestamp_url = t}?

Agreed with this -- my 0.02c preference would be for these being an "atomic" file/message, to avoid the proliferation of CLI states like we've seen with clients so far (most of which end up being invalid/non-functional). A user could of course still populate that "infrastructure" input with incompatible fields, but at least the API/CLI facing component is simplified.

In practice, the current TrustedRoot message has 95% of what we need here, so long as we can all agree on a way to interpret it (and add the last missing bits) 🙂 -- in particular:

1. Each TransparencyLogInstance contains a base_url; when loaded via --trusted-root, clients could mandate its presence.
2. Each CertificateAuthority (both for timestamping and issuance purposes) contains a uri; clients could similarly mandate their presence. For CSR purposes, the selected uri will be the first one with a live TimeRange.
3. The main missing piece for the --trusted-root use case is an oidc_provider_url or similar, which could become a new field (or a field of a sub-message named Infrastructure or similar). This field would be optional like the others, with the expectation that clients loading via --trusted-root would require it.

Does this sound generally workable to everyone? I think this fulfills the original use case with only a very small addition to the TrustedRoot itself.

[loosebazooka]

Sgtm; I'm not super fond of the "first" active tlog/ca is the signing item, since conventions are less explicit. Is it too weird for that to be an index in the infra/signer submessage referencing an item in the ca/log/ts fields?

[haydentherapper]

It might be better to require the client specify which CA or tlog will be used in signing when there are multiple (or via an index or key hint or something).

[woodruffw]

Sgtm; I'm not super fond of the "first" active tlog/ca is the signing item, since conventions are less explicit. Is it too weird for that to be an index in the infra/signer submessage referencing an item in the ca/log/ts fields?

Agreed that the implicitness isn't ideal.

Bouncing off @haydentherapper's idea: maybe we could add a bool for_signing to each, with the semantics then being "first active && for_signing? That will make things more explicit while also keeping the client interface simple (IMO it might be pretty confusing to signal an index into these via the CLI, especially for multiple 🙂)

[kommendorkapten]

I'm not sure the bool for_signing will make a single instance unique. Rotations will happens, and during an interval, more than one instance will be active.

Some filtering will need to happen by the client, and some operations may fail, as during the overlapping time, both instances may not actually be concurrently active.

Or is the idea to use bool for_signing to find the instance to use if multiple differently instances are found, like this example:

"certificateAuthorities": [
    {
      "for_signing": true,
      "subject": {
        "organization": "sigstore.dev",
        "commonName": "sigstore"
      },
      "uri": "https://fulcio.sigstage.dev",
      "certChain": {
        "certificates": [...]
      }
    },
    {
      "for_signing": false,
      "subject": {
        "organization": "foo.bar",
        "commonName": "baz"
      },
      "uri": "https://foo.bar.test",
      "certChain": {
        "certificates": [...]
      }
    },
]

In this case the client can verify signatures from the CA foo.bar but only use sigstore.dev to sign.

[woodruffw]

Or is the idea to use bool for_signing to find the instance to use if multiple differently instances are found, like this example:

Yep, exactly!

[loosebazooka]

So, there's another potential issue with defining signing cues in trusted_root. If there are two trusted_roots that need to be merged -- say an org wants to trust the public instance and their own private infra. Merging is pretty seamless if the trust root follows the current design -- in that merge conflicts are basically non existent. Clients may even be able to just handle this in the future --trusted-root is repeatable or something.

With the bool injected into the trusted root, now someone has to handle the conflict of two potential signing infras -- maybe processing this each time. I think I still lean to the simplicity of signing.json just being a single separate file with like 4 urls in it (or a few url groups -- to handle log lists or whatever)

[kommendorkapten]

I think I still lean to the simplicity of signing.json just being a single separate file with like 4 urls in it (or a few url groups -- to handle log lists or whatever)

Yes, this is what I would prefer as well (as I have mentioned before). Separation of concerns, I think the design is easier to reason about in that way and we can specialize each file for the use-case.

[woodruffw]

Merging is pretty seamless if the trust root follows the current design -- in that merge conflicts are basically non existent. Clients may even be able to just handle this in the future --trusted-root is repeatable or something.

I understand this, but I think it might be the wrong thing to optimize for: IMO merging cryptographic materials from composite should always be manually cross-checked, so optimizing for ease of merge isn't quite as valuable as reducing the number of error states that a user can surface.

That being said, I agree that the current TrustedRoot message isn't well-equipped for this, and I can see the argument that we shouldn't extend it for the signing use case at the confusing its use during verification. At the same time, I don't think having a separate file here would be ideal: the ideal end UX here is sigstore --trust-root foo.json ..., where foo.json has sufficient state for both verification and signing (which itself includes verification, per the client spec).

Given that, maybe @kommendorkapten's proposal in #259 (comment) is the most workable? With that, a user could pass in --trust-config foo.json, where foo.json contains the JSON-encoded ClientConfig.

[loosebazooka]

IMO merging cryptographic materials from composite should always be manually cross-checked

yeah good point :)

In the case of #259 (comment) I think I would still like to see SigningTargets (or something named like that) instead of just OIDCProviders that has fully populated signing information.

where it might look more like:

message TrustedRoot {
    ...
}

message SigningProviders {
    ... // complex structure here?
}

message ClientConfig {
    tr TrustedRoot
    sp SigningProviders
}

[woodruffw]

Yeah, I'm okay with that -- I can start on the PR here and we can iterate on naming/ultimate structure 🙂

[woodruffw]

Opened #277 to iterate on this!

[loosebazooka]

I don't want to muddy this conversation even more... but how do we define which log to contact when a rekor entry is not provided during verification? Or is this just a valid usecase anymore. The java client's initial design was to go looking for the entry in rekor.sigstore.dev. But which that being configurable, it makes less sense.

[haydentherapper]

Should we say this is solved by v2+ bundles because inclusion proof is required, so offline verification should only be done? If a v1 bundle is presented without a proof, then default to the public instance or a provided URL? And if you want to force online verification, then we should require a URL presented?

[loosebazooka]

Sure, but are we asserting that all verification must include a rekor entry? Is (sig, cert, artifact) an incomplete tuple?

I guess now it's (sig, cert, artifact, log(default = rekor.sigstore.dev))

[haydentherapper]

No, you're correct, a rekor entry is not required - code.

This depends on policy, particularly whether we want the client to assert a default policy (what you're suggesting here would be at least one log contains the entry with a default log) vs the user presenting a policy (at which point they can specify which logs to query).

[loosebazooka]

or should it just query ALL logs it knows about (in the trusted_root) until it finds it? Which I think probably makes sense... so maybe this wasn't confusing at all and java just had a weird design.

[haydentherapper]

All as a default, with the option to specify a specific log if you know where it is, seems reasonable.

[kommendorkapten]

I think this may need further clarifications in the client spec, but I would treat it like this.

Signing

When a client is signing an artifact it should publish the signature on every transparency log in the SigningConfig.tlog_urls, unless explicitly instructed via signing options config to only publish to a specific set of uris. This approach will work until we start to see a "massive number" of transparency logs, which I'm not certain will happen in the near time. And also, each trust root config always controls the content of the SigningConfig, so even if there are 100+ tlogs out there, the operators of the trust root can always decide which X logs they would like to utilize. Here I would think X is a small number, maybe no larger than 5.

Verification

Here we don't need any update IMO. We already have support for M tlogs in the bundle, and N trusted logs in the trusted_root. The verification allows for a threshold value x to set. This way we can verify M entries against the N trusted logs and count the valid ones and compare against x. This logic is already present in a few client implementations.

[woodruffw]

I don't want to muddy this conversation even more... but how do we define which log to contact when a rekor entry is not provided during verification?

For sigstore-python, we're moving towards requiring an embedded inclusion proof, full stop. We accept v1 bundles only if they contain an inclusion proof, and reject them otherwise. Given that this is formally required in v2+ bundles anyways, IMO it's the simplest/most workable forwards path 🙂

That being said, I think the changes here don't affect the verification path anyways: the TrustedRoot already lists one or more tlogs for verifying, so a verifying client should not consult SigningConfig.tlog_urls anyways.

[loosebazooka]

Should clients be removing the option of verification separates? Only bundles now?

[woodruffw]

Should clients be removing the option of verification separates? Only bundles now?

That's the direction we're slowly moving to with sigstore-python, since "detached" verification is not really well specified. However, both the conformance suite and the spec itself currently allow for detached materials, so that probably requires an extensive separate discussion 🙂

[haydentherapper]

Here we don't need any update IMO. We already have support for M tlogs in the bundle, and N trusted logs in the trusted_root. The verification allows for a threshold value x to set. This way we can verify M entries against the N trusted logs and count the valid ones and compare against x. This logic is already present in a few client implementations.

As an aside, one thing we don't have currently that ct log lists do is the notion of operators, so that you can group a set of logs together under the same operator. Policy is defined by the number of operators, not unique logs, so you can't submit to two sharded logs by the same operator and pass a policy. We don't need this right now since we freeze old log shards and we don't have multiple logs operated by one shard though, and it should be straightforward to add later on.