Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make deployment atomic #505

Merged
merged 1 commit into from
Mar 7, 2016
Merged

Make deployment atomic #505

merged 1 commit into from
Mar 7, 2016

Conversation

tobias
Copy link
Member

@tobias tobias commented Feb 22, 2016

Deployments aren't currently atomic, so if a deploy is interrupted, it
can leave the repo in an inconsistent state.

This adds the basics of atomic deploys, with the following additional
changes:

  • an update to pomegranate (0.0.13 -> 0.3.0) - I'm using aether to
    deploy from the tmp upload dir to the repo, and need the :artifact-map
    functionality that doesn't exist in 0.0.13
  • replacing the auth helper macros with functions - this is to make it
    easier to debug, and easier to determine where values come
    from (instead of doing magic binding, like we were doing with
    'account)

The atomic deploy functionality uses sessions (since aether honors
session cookies) to store uploads in a tmp dir that is scoped to the
deploy. Once we see a non-snapshot maven-metadata.xml, we finalize the
deployment by verifying the contents and then deploying them to the
actual repo. This process ignores checksum files, since the redeploy
recreates them.

The finalize process also validates the deploy, which includes the
existing validations (that were once per-artifact), plus adds some new
ones that operate over the full set of artifacts. The validations
are (existing validations marked with ^):

  • verify a pom was uploaded
  • verify the pom parses
  • verify the group, name, and version of a valid format ^
  • verify the gav values in the pom match the gav in the url
  • verify this isn't a redeploy of a non-SNAPSHOT version ^
  • verify that a jar was uploaded if the pom packaging is jar
  • verify that the provided checksums match the artifacts
  • verify that if any signature is uploaded, then every artifact has a
    signature

Now that we are validating after all the artifacts are pushed, we no
longer fail-fast on redeploys or invalid gav's, but I don't think that
should cause issues.

The change to require valid poms is debatable, given that it will
prevent deploys of projects that are affected by #233.

Deployments aren't currently atomic, so if a deploy is interrupted, it
can leave the repo in an inconsistent state.

This adds the basics of atomic deploys, with the following additional
changes:

* an update to pomegranate (0.0.13 -> 0.3.0) - I'm using aether to
  deploy from the tmp upload dir to the repo, and need the :artifact-map
  functionality that doesn't exist in 0.0.13
* replacing the auth helper macros with functions - this is to make it
  easier to debug, and easier to determine where values come
  from (instead of doing magic binding, like we were doing with
  'account)

The atomic deploy functionality uses sessions (since aether honors
session cookies) to store uploads in a tmp dir that is scoped to the
deploy. Once we see a non-snapshot maven-metadata.xml, we finalize the
deployment by verifying the contents and then deploying them to the
actual repo. This process ignores checksum files, since the redeploy
recreates them.

The finalize process also validates the deploy, which includes the
existing validations (that were once per-artifact), plus adds some new
ones that operate over the full set of artifacts. The validations
are (existing validations marked with ^):

* verify a pom was uploaded
* verify the pom parses
* verify the group, name, and version of a valid format ^
* verify the gav values in the pom match the gav in the url
* verify this isn't a redeploy of a non-SNAPSHOT version ^
* verify that a jar was uploaded if the pom packaging is jar
* verify that the provided checksums match the artifacts
* verify that if any signature is uploaded, then every artifact has a
  signature

Now that we are validating after all the artifacts are pushed, we no
longer fail-fast on redeploys or invalid gav's, but I don't think that
should cause issues.

The change to require valid poms is debatable, given that it will
prevent deploys of projects that are affected by #233.
tobias added a commit that referenced this pull request Mar 7, 2016
@tobias tobias merged commit 66a0097 into master Mar 7, 2016
@danielcompton
Copy link
Member

👏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants