ci: add prerelease #2553

btaillon-coveo · 2022-11-16T21:38:58Z

I'm merging this in master since in theory it shouldn't affect it. Changes from #2548 were copied here.

New prerelease process

To start a major prerelease (e.g., `1.2.3` → `2.0.0-pre.0`):

Make sure that HEAD refers to a version bump on master.
Create a new branch prefixed with prerelease/ (e.g., prerelease/headless_atomic_v2)
Run npm run bump:version:major-prerelease -- @coveo/package1 @coveo/package2 @coveo/package3.
Push the new commit and its tags to the branch.

Updating a prerelease branch:

Wait for master to reference a version bump.
Pull changes from master into the prerelease branch.

Adding new changes to the prerelease branch:

Open pull requests towards the release branch
Squash as usual.

Officially releasing (e.g., `2.0.0-pre.15` → `2.0.0`):

Create a pull request from the prerelease branch to “master”.
- Associate with a JIRA which QA will use to validate.
- Update the changelogs manually
- Update all dependents to use the prerelease version (include the -pre. suffix)
Squash after a version bump on master.

Changes in this PR

Added a bump:version:major-prerelease script to the root package.json
- This script will create a major prerelease version bump on the current branch, and will give it all the appropriate tags, but won't push it.
- The commit will contain the same message that lerna gives version bumps, including [skip-ci].
- Changelogs aren't affected.
Non-bump commits on a prerelease/** branch now trigger the same CI as master, except:
- Only packages matching /pre\.[0-9]+$/ are upgraded.
  - If a dependent is out of scope (doesn't match the above version pattern), its dependencies won't be upgraded.
- Changelogs aren't updated.
Bump commits on a prerelease/** branch now trigger the same CI as master, except:
- Only packages matching /pre\.[0-9]+$/ are published to NPM.
- The alpha tag on NPM isn't updated.
- The package rollout stops at the dev environment.
Some scripts were converted from CommonJS (.js) to ESM (.cjs)
Various calls to execSync were replaced with a utility execute function
- execute, in addition to returning its output, copies stdout & stderr to the console to make debugging easier. I caught some issues early thanks to this.

Other details

Packages are going to be published to the CDN based on their major/minor/patch version, the prerelease version will be ignored. Therefor, 2.0.0 will already be available on the dev CDN, and will be overwritten every time we update the prerelease. Out of scope packages will also update their CDN, but in theory the files should be the same anyway.

https://coveord.atlassian.net/browse/KIT-2114

btaillon-coveo · 2022-11-16T21:42:20Z

scripts/deploy/manual-major-prerelease.mjs

+/**
+ * @typedef {import('../packages.mjs').PackageDefinition} PackageDefinition
+ */


This is the JSDoc equivalent to import type { PackageDefinition } from '../packages';.

btaillon-coveo · 2022-11-16T21:43:25Z

scripts/git.mjs

+    )
+    .join('\n');
+  const message = `${lernaMessageSection}\n\n${updatedPackagesMessageSection}`;
+  await execute('git', ['commit', '--no-verify', '-m', message]);


I tested this, and multi-line commits work.

github-actions · 2022-11-16T21:53:40Z

Pull Request Report

PR Title

✅ Title follows the conventional commit spec.

Bundle Size

File	Old (kb)	New (kb)
case-assist	216	216
search	312.1	312.1
insight	258.1	258.1
product-listing	229.5	229.5
product-recommendation	213.7	213.7
recommendation	213.8	213.8

louis-bompart

Nothing seems particularly wrong, tho there's too many things for me to judge/give an approve given my limited knowledge of your CI/CD

louis-bompart · 2022-11-16T22:40:15Z

scripts/packages.mjs

+export function updatePackageVersion(
+  packageName,
+  newVersion,
+  depdendenciesPackageDirs
+) {
+  depdendenciesPackageDirs.forEach((packageDir) => {
+    const fullPath = resolve('.', 'packages', packageDir, 'package.json');
+    const originalContentAsText = readFileSync(fullPath).toString();
+    let newContent = originalContentAsText.replace(
+      new RegExp(`("${packageName.replace('/', '\\/')}"\\s*:\\s*")([^"]*)`),
+      '$1' + newVersion
+    );
+    if (packageName === JSON.parse(originalContentAsText).name) {
+      newContent = newContent.replace(
+        /("version"\s*:\s*\")([^"]*)/,
+        '$1' + newVersion
+      );
+    }
+    writeFileSync(fullPath, newContent);
+  });
+}


Why not parse, edit then, write? It'll be more maintainable than a RegExp nah?

Ah I should've clarified this part. Unfortunately, parsing, editing then reformatting gives a pretty ugly JSON, and I wanted to maintain the current structure of the JSON.

https://github.com/sindresorhus/detect-indent

ThibodeauJF · 2022-11-17T15:15:18Z

scripts/exec.mjs

+ * @param {readonly string[]} [args]
+ * @returns {Promise<string>}
+ */
+export function execute(command, args = []) {


Could we add a package to do this type of thing? I feel like it's a bit out of our scope

I wanted to see the command & output of every command in the console while also being able to use the output, which by default isn't the case with spawn and exec (setting stdio: 'inherit' makes them return nothing). I could've tried a bunch of third-party exec libraries to do it, but it doesn't seem to be worth the time exploring them, and this is a fairly simple utility function that mostly just wraps the native spawn function.

https://coveord.atlassian.net/browse/KIT-2114

ThibodeauJF · 2022-11-17T15:24:58Z

Nice work!

It would be nice to update the readme for upcoming pre-releases.

Some nitpick: doing the refactor to modules in the same PR as adding the pre-release specific code & the migration from Lerna... it will make it hard to discern changes when looking back. These kind of things were a bit annoying when fixing something in the dep. process in the past.

I'd wait for Olivier's feedback too before merging

btaillon-coveo · 2022-11-17T15:34:23Z

Nice work!

It would be nice to update the readme for upcoming pre-releases.

Some nitpick: doing the refactor to modules in the same PR as adding the pre-release specific code & the migration from Lerna... it will make it hard to discern changes when looking back. These kind of things were a bit annoying when fixing something in the dep. process in the past.

I'd wait for Olivier's feedback too before merging

Agreed, I ideally would've liked to do it separately, but unfortunately I wasn't able to use the semantic monorepo tools with commonJS.

ThibodeauJF · 2022-11-17T15:58:55Z

What if it was

PR1: switch to monorepo tools for our current release process + mjs conversion
Test that the main release process still works properly & iterate
PR2: add prerelease
Test, iterate...

btaillon-coveo · 2022-11-17T16:03:20Z

What if it was

PR1: switch to monorepo tools for our current release process + mjs conversion

Test that the main release process still works properly & iterate

PR2: add prerelease

Test, iterate...

We can do that, but in this case PR1 may be as big as this PR (possibly bigger since we'd need to add changelogs and conventional graduating). It would be more consistent across both releases though. Is this what you prefer?

louis-bompart · 2022-11-17T17:05:30Z

PR1: cjs -> mjs
PR2: monorepo tools
PR3: prerelease
Etc?

ci: add prerelease

9e9d40c

https://coveord.atlassian.net/browse/KIT-2114

btaillon-coveo commented Nov 16, 2022

View reviewed changes

btaillon-coveo requested review from wmannard, lvu285 and louis-bompart November 16, 2022 21:45

btaillon-coveo marked this pull request as ready for review November 16, 2022 21:45

btaillon-coveo requested review from a team as code owners November 16, 2022 21:45

btaillon-coveo requested review from olamothe and ThibodeauJF November 16, 2022 21:45

louis-bompart reviewed Nov 16, 2022

View reviewed changes

ThibodeauJF reviewed Nov 17, 2022

View reviewed changes

fixed invalid imports

fcf54cc

https://coveord.atlassian.net/browse/KIT-2114

btaillon-coveo closed this Nov 17, 2022

This was referenced Nov 17, 2022

chore(auth,bueno,headless,atomic): converted all CJS scripts to ESM #2556

Merged

ci: add prerelease process #2561

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: add prerelease #2553

ci: add prerelease #2553

btaillon-coveo commented Nov 16, 2022 •

edited

Loading

btaillon-coveo Nov 16, 2022 •

edited

Loading

ThibodeauJF Nov 17, 2022

btaillon-coveo Nov 16, 2022

github-actions bot commented Nov 16, 2022

louis-bompart left a comment

louis-bompart Nov 16, 2022

btaillon-coveo Nov 17, 2022 •

edited

Loading

louis-bompart Nov 17, 2022

ThibodeauJF Nov 17, 2022

btaillon-coveo Nov 17, 2022

ThibodeauJF commented Nov 17, 2022 •

edited

Loading

btaillon-coveo commented Nov 17, 2022

ThibodeauJF commented Nov 17, 2022

btaillon-coveo commented Nov 17, 2022

louis-bompart commented Nov 17, 2022 •

edited

Loading

ci: add prerelease #2553

ci: add prerelease #2553

Conversation

btaillon-coveo commented Nov 16, 2022 • edited Loading

New prerelease process

To start a major prerelease (e.g., 1.2.3 → 2.0.0-pre.0):

Updating a prerelease branch:

Adding new changes to the prerelease branch:

Officially releasing (e.g., 2.0.0-pre.15 → 2.0.0):

Changes in this PR

Other details

btaillon-coveo Nov 16, 2022 • edited Loading

Choose a reason for hiding this comment

ThibodeauJF Nov 17, 2022

Choose a reason for hiding this comment

btaillon-coveo Nov 16, 2022

Choose a reason for hiding this comment

github-actions bot commented Nov 16, 2022

louis-bompart left a comment

Choose a reason for hiding this comment

louis-bompart Nov 16, 2022

Choose a reason for hiding this comment

btaillon-coveo Nov 17, 2022 • edited Loading

Choose a reason for hiding this comment

louis-bompart Nov 17, 2022

Choose a reason for hiding this comment

ThibodeauJF Nov 17, 2022

Choose a reason for hiding this comment

btaillon-coveo Nov 17, 2022

Choose a reason for hiding this comment

ThibodeauJF commented Nov 17, 2022 • edited Loading

btaillon-coveo commented Nov 17, 2022

ThibodeauJF commented Nov 17, 2022

btaillon-coveo commented Nov 17, 2022

louis-bompart commented Nov 17, 2022 • edited Loading

btaillon-coveo commented Nov 16, 2022 •

edited

Loading

To start a major prerelease (e.g., `1.2.3` → `2.0.0-pre.0`):

Officially releasing (e.g., `2.0.0-pre.15` → `2.0.0`):

btaillon-coveo Nov 16, 2022 •

edited

Loading

btaillon-coveo Nov 17, 2022 •

edited

Loading

ThibodeauJF commented Nov 17, 2022 •

edited

Loading

louis-bompart commented Nov 17, 2022 •

edited

Loading