Use TUF metadata formats for signing index commits

[tarcieri]

Rendered

This PR adapts rust-lang#2474 to use file schemas (serialized as TOML) and role names from The Update Framework (TUF):

• keys TOML => root.toml
• can-rotate privilege => TUF root role
• can-commit privilege => TUF timestamp role

It should be a fairly close to a 1:1 mapping of TUF concepts to the original approach, and otherwise continues to use OpenPGP public keys, fingerprints, and signatures on git commits. The main differences are:

• keys TOML (i.e. root.toml) has a different schema
• root.toml is checked into the repository as opposed to being stored in an annotated tag (tagged commits can be signed ala git tag -s)
• when @bors commits to the index, it would also modify a timestamp.json file, bumping a version number and expires date
• key rotation tags are prefixed with root-rotation- and the counter starts from 1 instead of 0 per the TUF spec

Other than that, it should pretty much be the same idea.

Omitted for now are TUF's targets role and snapshot roles. The targets role could provide end-to-end crate integrity, but is a much more substantial undertaking in hopes of keeping an initial index signing simple. The value of the snapshot role in a git-based system is debatable. It might be worth including for compatibility with existing TUF implementations (e.g. rust-tuf), but this PR omits it for now (if you feel strongly about including it, please leave a note and I can update the PR).

[trishankatdatadog]

Thanks very much, Tony! 👍

I would strongly recommend leaving the snapshot, and targets roles (with empty targets) just so you would not have to add special cases to the TUF client update workflow.

In any case, this is a great start :)

[withoutboats]

Other than that, it should pretty much be the same idea.

Then how is it better? Given that almost all of these changes are superficial and inconsequential for security, this is why I am honestly very suspicious of all the TUF advocacy I have experienced: I am inclined to believe that this is about a win for TUF rather than improving the security of our infrastructure.

[tarcieri]

Then how is it better?

The full benefits would come after the targets role is implemented and can be used for end-to-end-crate integrity (via delegated targets).

Beyond that though, TUF is well-specified and has detailed descriptions of e.g. its verification workflows, for example:

https://github.com/theupdateframework/specification/blob/master/tuf-spec.md#5-detailed-workflows

[arielb1]

If the timestamp key is used to ensure "transparency" (i.e., to prevent the creation of 2 separate commit chains), then we have to require an "old timestamp" signature on all commits.

[tarcieri]

In general preventing duplicity in systems like this is difficult (see similar concerns regarding CT logs, auditors, gossip protocols, etc).

That said, I don't follow what you're saying. A signature over a git commit covers the current tree in addition to the parent OID, which is the SHA-1 digest of the previous commit's tree, parent, and the full OpenPGP signature over that commit (provided it's there), i.e. each commit signature is over the entire repository history including the signature on the previous commit.

[arielb1]

The idea is to have the timestamp key is stored in a manner that makes it hard for an attacker to create unlogged signatures (I think Amazon offers an HSM module that does that). If we can find such a service, it would be worth to put the timestamp key in it.

In that case, we don't want a compromised root key to bypass the "anti-duplicity" feature of the timestamp key.

[tarcieri]

Using cloud HSM or KMS services on any of the major cloud vendors (e.g. AWS, GCP, Azure) would likely require using a different signature algorithm than Ed25519, e.g. ECDSA over P-256. ECDSA has wide support among cloud KMS services, HSMs, and PIV devices such as YubiKey 4s.

That said, there are several hardware options for Ed25519 key storage. YubiHSM2s support Ed25519, as do a number of cryptocurrency hardware wallets (e.g. Ledger), although whether that's a practical deployment option for something like @bors seems questionable.

[arielb1]

In that case, why not allow for verification with that signature algorithm? It feels like something we can set up basically today.

It seems likely that a threshold of the root keys will not have a strong audit mechanism, and setting up separate "root", "root+audit", and "timestamp+audit" keys feels like a PITA.

[tarcieri]

I think ECDSA support is worth considering. FWIW I support it in Signatory.

[arielb1]

I suppose the verification rules here are as following:

1. Define the "principal chain" of "principal commits" to be the list of commits starting from HEAD and extending by following the first ancestor of each commit, ordered such that the first commit is the "verification initial commit", whose hash is pre-distributed, and the final/last commit is HEAD.
2. The hash of one of the principal commits, the "verification initial commit", is pre-distributed (that does not have to be the same as the Git initial commit, i.e. it can have a parent). It forms the root of trust of the system.
3. Every principal commit must have a git commit signature. The key that signed the commit must be a timestamp key in that commit's root.toml, and in all principal commits other than the verification initial commit, in the previous commit's root.toml.
4. If root.toml in a commit differs from root.toml in that commit's parent, that commit must be signed by a tag signature by a root keys in the parent's root.toml.
5. timestamp.toml & root.toml must be valid TOML files of the right format in every principal commit.
6. The version field in timestamp.toml must increase by 1 between succeeding commits. The version field in root.toml must increase (not necessarily by 1) every time root.toml changes.
7. Non-principal commits (i.e., commits that are only the "second" ancestors of a principal commit) don't have to be signed, but are verified indirectly because they are a part of the Merkle tree.

I could write them in pseudo- code.

[Eh2406]

Thank you very much for this RFC. It was submitted four years before I was in a place to understand it. But reading it over today, with my unusual collection of knowledge and knowledge gaps, it was the most effective explanation of TUF I have ever found. Thank you.

There is a long road to get to adopting a solution like this. Nonetheless, I am completely convinced that a solution like this is the direction we should be heading. (Modulo a large number of details that need to be negotiated.) I am generally in favor of incremental solutions.

Sparse indexes took us three years and counting, and was a step back on the ability to sign snapshots. There is an issue to re-add snapshot like abilities to sparse indexes rust-lang/cargo#10928 Do you have resources I could read about how TUF snapshots work, or would be willing to write up an explanation? Whatever solution we come up with should be informed by the careful thought put into TUFs design.