Separate local release actions from publishing

[laughedelic]

First a concrete use case: I want to run Auto with the git-tag plugin, but I need to do things after the tag is created, but before it is pushed. Similar to the setup with exec plugin I mentioned here, but this time my actions don't fit in a single script that I can call from exec.

I see that git-tag plugin bumps the version and creates a tag in the version hook and pushes it in the publish hook. But how can I trigger these hooks from the CLI? I want to do this:

• run what's in git-tag's auto.hooks.version
• ...do my stuff...
• run what's in git-tag's auto.hooks.publish

Unintuitively, auto version is not related to the auto.hooks.version in the code, so calling auto version tells me, e.g. patch, but doesn't create the tag locally. Even if I get the version by running auto latest --dry-run --quiet and create the tag manually (as a workaround), how do I call the auto.hooks.publish from the CLI?

---

More generally: is it possible to do a release preparation (all local actions) separately from the publishing/pushing those changes? Something like

auto shipit --no-publish
# ... do things
auto shipit --carry-on 

And if it's not possible at the moment, can this be implemented with the current Auto architecture? I would be glad to contribute it, if you can give me some pointers 🙂

---

P.S. motivation for this:

For the majority of "package manager" plugins you should only use one at a time. Using multiple will lead to undesired results.

(quoting docs) and my comment here: #917 (comment).

I need to publish two different kinds of artifacts from the same project (not npm, not a monorepo), versioning is simple and is based on git tags and git describe (for canaries), so I don't really need publishing plugins and would be happy with the git-tag plugin, if the functionality described above was supported: bump version, create tag, do my custom publishing, push the tag, publish GitHub release, etc.

Having this would actually eliminate the need to write plugins for those use-cases where there is no manifest file with the version value: sbt, Docker, GitHub Actions, custom artifact upload, etc.

[hipstersmoothie]

I think the flags you describe are cool but hard with the current code.

I feel like you em use case could mostly be covered with the afterVersion hook in the exec plugin.

Have your thoughts changed at all since writing a plugin?

[laughedelic]

I feel like you em use case could mostly be covered with the afterVersion hook in the exec plugin.

I mentioned above:

setup with exec plugin I mentioned here, but this time my actions don't fit in a single script that I can call from exec.

Using exec plugin is very restrictive: everything needs to be done in a single script. If I'm running release process on GitHub Actions, I want to be able to use actions as part of the release process, something along the lines of

jobs:
  release:
    steps:
      # ... setup ...

      - run: auto shipit --no-publish

      # my custom publishing actions with the newly created tag
      - run: sbt publish
      - uses: docker/build-push-action@v2
        with:
          ...
      - uses: sonatype-nexus-community/[email protected]
        with:
          ...
      # etc. other steps/actions

      - run: auto shipit --publish

Sure, everything can be done in a bash script, but that kind of destroys the point of using GitHub Actions.

[bnigh]

I'm not sure where would be best to put my thoughts, so please feel free to move to another thread if more appropriate. But for now will put here.

I've actually had a similar pain point in a use case I've come across where my build & release process / pipeline (using Jenkins 2) would look something along the lines of:

stage('build') {...}
stage('validate') {...}
stage('release') {...}

In this use case, versioning is solely based upon git tags (so git-tags plugin is used). According to docs, the recommended approach would be to invoke auto shipit from the release stage. However, the build and validate stages need to be aware of the release version to operate correctly.

Similar to the example note here: #1965 (reply in thread), the build and validate stages could be implemented / invoked as part of a hook in the release stage. But this loses many benefits from ci tooling (ie: reporting of builds, ability to restart intermediate stages, ability to invoke stages in parallel, ...).

The workaround I have been using has been to:

• calculate next version by:
  • use custom logic to get previous release version from git tags in ci
  • use auto version to get semver bump
  • use custom logic to calculate next version
• invoke build & validate stages
• invoke release stage
  • use custom logic to publish artifacts
  • use auto release to publish release notes

While this approach allows me to maintain the benefits of the ci tooling, there is a lot of custom logic that duplicates existing code from auto (ie: get previous release version from git tags in ci & use custom logic to calculate next version - including logic for canaries, prerelease branches, and version branches).

From the thread here (#1867), it looks like I may be able to replace the calculate next version logic with auto shipit -dq. I'll have to give this a try, as it should simplify the pipeline I'm using now.

I'm not sure if my existing use case could be simplified further, but if I ever wanted to add additional logic that is already implemented in auto (ie: publish artifact to brew), then with current auto architecture I would need to duplicate that logic outside of auto myself.

Looking at this comment about cli command (#1867 (reply in thread)), it seems in regards to "atomic" commands that Package manager plugin (via shipit) or user scripts facilitate a release (ie: hooks) are the primary logic components that are not currently exposed via cli.

Perhaps there is room to refactor some of the hooks and expose them via cli (although this would likely require a number of breaking changes, perhaps it would simplify plugins and encourage better dryness)?

[laughedelic]

(separate thread, since it's a more general topic)

Have your thoughts changed at all since writing a plugin?

To be honest, I spent much more time than I expected for writing the plugin. There is a lot of code duplication, all git tag/push logic is replicated in each publishing plugin. See the last part of my comment here on this topic.

Having a plugin system is great and was the main attraction point for me. But I really don't see the advertised "each command only does one thing really well". In reality, it's more of a monolith release process with the ability to hook into it and do some extra things on the way. But then again, several publishing plugins can all hook in together and will create a mess because they don't interplay well. Maybe if those hooks were exposed as the release steps that can be run from the CLI, it would be more flexible? 🤔 (just thinking aloud)

If the usage described above was possible, I wouldn't need any new plugins at all for any of my use cases: git-tag with the separate "prepare release"/"publish release" would allow me to squeeze any custom publishing logic between them.

[jBouyoud]

I'm really interested in this and I'll be glad to create a PR to handle this.

One thing seems really strange in core auto package : core is responsible to create a github release.
But a github release doesn't work if the tag is not created before.
That means, Auto doesn't work without a plugin that create this tag.

When you run auto with the following configuration (https://intuit.github.io/auto/docs/configuration/plugins#no-plugins) :

{
  "plugins": []
}

You will have the error :

RequestError [HttpError]: Validation Failed: {"resource":"Release","code":"custom","field":"tag_name","message":"tag_name is not a valid tag"}, {"resource":"Release","code":"custom","message":"Published releases must have a valid tag"}, {"resource":"Release","code":"invalid","field":"target_commitish"}
    at /home/runner/work/test-release-config/test-release-config/node_modules/@octokit/request/dist-node/index.js:86:21
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Job.doExecute (/home/runner/work/test-release-config/test-release-config/node_modules/bottleneck/light.js:405:18) {
  status: 422,

That's why you need to handle this on each plugins (and re-implement the same following features) :

• looking for a previous version from git tags
• compute the version from semver bump, and tag it
• push the tag

That raise an issue with the "each command does only one thing really well" principle.

• Why docker, npm, sbt, chrome plugins need to look for, create and push a git tag ?

I believe one solution could be :

• re-integrate the git-tag plugin in core package as the default behavior
• remove the "look for, create and push a git tag" features from docker, npm, sbt, chrome plugins