Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Snark keys management #6341

Merged
merged 6 commits into from
Oct 27, 2020
Merged

[RFC] Snark keys management #6341

merged 6 commits into from
Oct 27, 2020

Conversation

mrmr1993
Copy link
Member

@mrmr1993 mrmr1993 commented Oct 9, 2020

Checklist:

  • Document code purpose, how to use it
    • Mention expected invariants, implicit constraints
  • Tests were added for the new behavior
    • Document test purpose, significance of failures
    • Test names should reflect their purpose
  • All tests pass (CI will check this if you didn't)
  • Serialized types are in stable-versioned modules
  • Does this close issues? List them:


### Add a tool for generating keys outside of the build process

This tool could be an `internal`/`advanced` subcommand of the main executable
Copy link
Contributor

@lk86 lk86 Oct 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of value in having this as a distinct executable, especially in making sure that most tests/jobs are not dependent on building coda.exe. Is the time/effort cost of splitting it out much higher? Alternatively, do we save effort by adding this command / these features to src/lib/snark_keys/gen_keys/gen_keys.exe instead as it is already focused on snark key generation/management?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can build a separate executable that sits at the same layer of the dependency graph as snark_keys does (and library-ise the command so that it can still be exposed from the CLI too), if that would be helpful. I will update the text to add this, and it should be no extra work to do it this way.

I'll note though that the snark keys depend on code quite far up the coda.exe dependency chain, so we'll have to recompile it for each commit, and we likely won't end up saving much time vs using compiling and using coda.exe directly.

Copy link
Member

@bkase bkase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good idea!

zexe repos used when generating the keys
* the length of the binary data following the header, so that we can validate
that the file was successfully downloaded in full, where applicable
* the date when the file's contents were created
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't include generation date, this (in theory) should be reproducible and I think that's worth it. We're going to want easy reproducibility for the more security-conscious folks and it makes it easy for us to go back to old commits to recover data if needed.

Since we have the commit sha, this can act as a timestamp

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the medium term, it would be good to have a ls-style tool that lets us list all of the cached keys in different ways, and especially to have list-keys --sort=date so that we could delete out-of-date keys more easily and reliably.

I've updated the rfc to use the date from the current commit, so we have reproducibility and some notion of time, and mentioned these reasons as sub-bullets.

, "supercharged_coinbase_factor": 2
, "account_creation_fee": "0.001" }
, "commits":
{ "coda": "COMMIT_SHA_HERE"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*mina

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the marlin/zexe commits should be redundant since they're specified by the submodule checked out at that commit, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the marlin/zexe commits should be redundant

I've removed the zexe commit. Keeping the marlin one because we're using it as an input to the 'identifying hash', and it would be nice if we could compute/check that from the header without needing access to the full git history. I've added a sub-bullet with this justification.

@mrmr1993 mrmr1993 marked this pull request as ready for review October 12, 2020 18:17
@mrmr1993 mrmr1993 added ci-build-me Add this label to trigger a circle+buildkite build for this branch ready-to-merge-into-develop labels Oct 27, 2020
@netlify
Copy link

netlify bot commented Oct 27, 2020

Deploy preview for mina-staging ready!

Built with commit a120d7f

https://deploy-preview-6341--mina-staging.netlify.app

@mergify mergify bot merged commit 307863d into develop Oct 27, 2020
@mergify mergify bot deleted the rfc/snark-keys-management branch October 27, 2020 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-build-me Add this label to trigger a circle+buildkite build for this branch ready-to-merge-into-develop
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants