Publish pre-release extensions in the store

[JVApen]

Recently the creation of pre-release versions of the extension was added. This publishes to GitHub.

Visual Studio Code has the concept of pre-release built-in, so it would be useful to make use of that. As such, we would get a larger reach and reduce the burden for people to test this.

Documentation about this can be found here

[HighCommander4]

I'm trying to think through the logistics of how this should work.

Currently, we have two CI workflows for creating releases:

• The one for publishing a stable release. This runs automatically on every commit, checks whether the commit updated the version field in package.json, and if so publishes a new stable release to Github, the VSCode Marketplace, and OpenVSX.
• The one for publishing an unstable release, newly added in Add a Github workflow for creating an unstable release #748. This is manually triggered, and publishes a new pre-release to Github only. Unlike the stable releases, the release name (and package name) contain a commit SHA rather than a version number.

If we're going to publish unstable releases to the VSCode Marketplace, I guess we'll want to identify them with a version number rather than a SHA? And maybe it doesn't need to be a dedicated version number, maybe we can "promote" an unstable release package itself to stable?

One potential way it could work is:

• Change the automatic workflow to create an unstable release (pre-release) when the version number is bumped. Now, every new version starts as unstable.
• Have a manually triggered workflow that promotes the latest unstable release to stable?

I'm open to suggestions / alternatives.

[JVApen]

I see 2 flows here:

• either release based on an existing pre-release when a newer pre-release was created
• only allow releasing the last pre-release

I'd be inclined to use the second flow as it reduces complexity while releasing an older pre-release seems rather exceptional. (A revert can be used in that case)

As such, I think that:

• the release notes should contain a new header for the pre-release
• each PR should extend the release notes (if relevant)
• each PR results in an automatic pre-release build (bumps the patch version automatically?)
• releasing requires a special commit (like today) that renames the pre-release header in the release notes and bumps the major version number
• releasing results in a new PR that adds the pre-release header to the release notes and bumps the version number once again (no pre-release build has to be created here)

With this, the release flow can be automated and create 2 sequential PRs. (Or 1, if the releasing occurs with data different from checkout)

As suggested by the documentation, we should use even numbers for releases and odd numbers for pre-release. Alternatively, we can use the patch number 0 as a release.

An example, each line is a PR:

• 0.50.0 release
• 0.51.0 no build
• 0.51.1 pre-release
• 0.51.2 pre-release
• 0.51.3 pre-release
• 0.52.0 release
• 0.53.0 no build
  or
• 0.50.0 release
• no build
• 0.50.1 pre-release
• 0.50.2 pre-release
• 0.50.3 pre-release
• 0.51.0 release
• no build

I feel more for the first flow of version numbering.

[HighCommander4]

• each PR results in an automatic pre-release build (bumps the patch version automatically?)

I'm a bit concerned about it working this way, as it means every PR merge automatically pushes out a new version to the VSCode Marketplace; even if it's just a pre-release version, users who have opted into the pre-release channel can be upgraded to that automatically.

I think it would be safer if pushing a new version to the VSCode Marketplace required an explicit action opting into that, i.e. either a manual Github Action invocation, or a special commit that touches a version number.

[JVApen]

I'm a bit afraid that with manual work, the pre-release will only happen when we actually want a release. Though at the same time, I think it would be a small change if we change our minds. So let's start with the manual triggers and we can always see how that goes.

[HighCommander4]

In either case, implementing these workflows will take some hacking on the Github Actions scripts. I'm not sure when I'll get around to that.

@JVApen, if you have the time and interest in trying to make these Github Actions changes, that would of course be appreciated.

However, if you're keen for the scriptAsExecutable patch to appear in a stable release without necessarily blocking on this work, we could take the approach suggested earlier of asking the users who reported regressions from earlier attempts to try the Github pre-release that's currently published. I'd be fine with making that a stable release if we have confirmation from those users that the regressions are resolved.

[JVApen]

I don't have any experience with GitHub Actions, though with the manual trigger it feels like a small adjustment of the existing script.
Give me some time to look into it.