SGD version 5.4.0 failure

[scolladon]

Hi there,

We would like to open up the discussion here about the version 5.4.0 failure
We would like to be transparent with the community about what happen.
We would like to be transparent with the community about what we did to fix and what we did to ensure it will not happen anymore.
We would like to take information for you about what version behaviour you would like to use.

First of all, we are very sorry and disappointed for what happen in production for many of you, in your CI.
It was our biggest breakage so far and we apologize for that.
Breakage happen, they are most of the time the combination of multiple located events.
We want to make sure you still feel confident using the plugin and we would like to reassure everyone they can still trust our work.

The release 5.4.0 has been shipped in Friday 26 of August 2022.
First thing in next monday (29 of August) we were fixing this issue.
Multiple issues where created #340 #334 and #335.
A discussion has been opened
And a linkedin post has been published
Lot of traction from the community!

TL;DR

postinstall script with devDependencies broke the plugin installation.
We removed it as it is already done in prepare script.
We improved our CI to better check for installation.
We have other release checks to implement.
We are considering different release channel to make sure critical environment have issues more rarely.

What happen with 5.4.0 failure

We shipped a new minor version containing multiple feature including one resolving a functional deployment issue about flow and flow translation (huge improvement).
We did as always:

• Upgrade every dependencies to latest (for general health)
• Make sure new unit test are added to cover the spec of the addition
• Make sure test coverage is ok
• Integration test in js.
• Functional test locally with our big not yet anonymised repository
• Installation test in the CI
• Local plugin linking to make sure installation is ok

During the upgrade of husky, the package.json has been modified and a postinstall script has been added to install husky. We already have a prepare script doing the husky install and we did not notice.

In our CI, the installation of the plugin is done by doing a full install of the repo (dependencies and devDependencies), so the installation test we had went fine.

What we discover is the way sfdx plugin install a plugin is by not installing the devDependencies (makes sense).
So when doing a sfdx plugins:install it call a yarn install --prod and does not install the devDependencies.
yarn install also execute every script in the lifecycle, including postinstall.
As husky is part of our devDependencies it was not install and so the postinstall script failed the plugin installation.

And that passes every protection we had...

How we fixed that

After spending some time locally to reproduce and isolate the issue, we quickly defined it was related to the postinstall script for husky.
The fix was just to removed the postinstall script.
We do not need that, it is is already done in the prepare script, that was working fine previously like that.

We also automated further our installation test in the pull request in order to make sure it better simulate the installation behavior.
We talked to sfdx-cli to ensure the steps were accurate and they told us it is ok.
Issue with plugins:link and plugins:install is they are greatly dependent to npm/yarn version, os version, node version, current sfdx-cli version. Lot of dimension to take in account.

What to put in place to improve release quality

We want to improve the release flow to do an installation check post release.
This way we can make sure the new version is installable.
We need to have a script to rollback a version in case it is not installable post release.

We are also considering having different release channel.
And we need your help here.

Shipping a stable and a latest channel version

The goal here is to continue shipping fast and often and keep a simple to have a stable version for sure.
Those who are not in critical environment can install the plugin using sfdx plugins:install sfdx-git-delta or sfdx plugins:install sfdx-git-delta@latest
Those who are in critical environment can install the plugin using ``sfdx plugins:install sfdx-git-delta@stable`

No changes for most of the people as latest is the same as the default installation.
Only critical installation need to change their script to use the stable channel

Shipping a latest and a beta channel version

The goal here is to make sure the latest is the most stable and provide a way for the community to test the latest feature before everyone.
Those who are not in critical environment can install the plugin using sfdx plugins:install sfdx-git-delta@beta
Those who are in critical environment can install the plugin using sfdx plugins:install sfdx-git-delta or sfdx plugins:install sfdx-git-delta@latest

Having new feature first requires switching to beta channel installation

So what is your opinion here ? stable channel or beta channel or your own proposition below ?

That's it folks !
Let's talk the future way to release this plugin !

[scolladon]

What do you think guys in term of release channel ?
What are your preferences ?
Do you have any suggestions ?
Do you have other ideas to improve the release quality ?

I mention you guys because you were the most reactive ones @mehdisfdc @aveshsingh172 @mostafascoober @LeoCordiuS @reszkaj @jonny-harte @gaelCangy @Dinesh-Talreja @vaibhavdangar-woolies @shabu74 @arafesthain @jdkgabri

[shabu74]

Since we will have stable version (latest) plus new features (beta), I stand for beta channel.

[mehdicherf]

I vote for a beta channel. This way the "default" sfdx plugins:install sfdx-git-delta will install the most stable (non beta) version. I assume this what most users want.

[gaelCangy]

I vote for the beta channel too so by default we get the most stable version and if we want we have the choice to install the beta version for testing purposes.

[vaibhavdangar-woolies]

my vote: 'Shipping a latest and a beta channel version' however, could we not call it as stable (default) and beta (explicit specify) instead of 'latest' and 'beta', since beta would usually be latest !

[jdkgabri]

I believe we could use a similar convention than the one that Salesforce uses for its docker images: latest and latest-rc (relese candidate). This last one instead of beta.

[scolladon]

Thanks, that's a good idea.
It folllows a standard and use a less scary word than beta

[scolladon]

So, we have two version pointers, one in the front of the versions and one somewhere in the back.
The discussion here is: Do the latest default version pointer is the one in the front (rc) or the one in the back (stable)

So far the choice would be to go for latest as the last stable known version, and for beta or latest-rc as the newest version.
One drawback I can see here is that the latest-rc needs to be explicitly asked to be installed because not specifying a tag version means latest for npm.
This means it could lead to a latest-rc version not enough installed to be known as stable and could lead to either: move the stable pointer not so often, and also not detecting issue before getting stable...

I like the stable and latest channel solution because latest will still be widely used (it does not require to tell a version to sfdx in order to install it, it is the default installation) and so we will be able to detect issue very soon and improve the quality. My hypothesis here is that we will have more usage / data of the latest version (which is currently the case).
Those who are ok with the current state of the plugin can use the stable channel and trust us to move it when it is stable enough. As latest will be default we will have enough data.

Next: we need to determine when the last version is stable enough to move to the past ?

• After a certain number of downloads without issues related to the version ? Number is a constant ? Or a percentage of the weekly download at that time ?
• After a certain amount of time without issues related to the version ? 1 week ? 2 weeks ? a month ?

[scolladon]

Here are the latest news !

We decided to move to a bit of both suggestion.

• latest-rc channel. New release start by this tag
  • Easy to detect packaging issue or regression
  • Do not impact latest user (or those that do not specify version)
  • Allows enduser to test latest feature inside CI/CD (for issue logger user per example)
• latest channel. Release stable enough to include latest development.
  • New feat/fix are included after an incubation period to prevent regression or outage
• stable channel. Release stable for a certain amount of time goes there
  • Ensure the plugin version as reached a broader audience and is stable
  • A bit like fixing a version with regular enhancement

We will put it in place soon.

[aveshsingh172]

Hi Scolladon/Sfdx-Git-Delta, I have seen an issue. We are not able to delete List Views, Queues and flows. Just wanted to check if these are supported Appreciate your help on this matter. Thanks. Kind Regards, Avesh Singh

…

On Thu, Sep 29, 2022, 3:16 PM Sebastien ***@***.***> wrote: Here are the latest news ! We decided to move to a bit of both suggestion. - latest-rc channel. New release start by this tag - Easy to detect packaging issue or regression - Do not impact latest user (or those that do not specify version) - Allows enduser to test latest feature inside CI/CD (for issue logger user per example) - latest channel. Release stable enough to include latest development. - New feat/fix are included after an incubation period to prevent regression or outage - stable channel. Release stable for a certain amount of time goes there - Ensure the plugin version as reached a broader audience and is stable - A bit like fixing a version with regular enhancement We will put it in place soon. — Reply to this email directly, view it on GitHub <#346 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJV4FMUO2LVRR6LDOORVL6DWAWQFTANCNFSM6AAAAAAQCOWGAE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[jdkgabri]

For List Views an Queues I can not say much, but for flows:

The problem is that you do not delete flows, instead, you are only able to delete flow versions. A flow version can be specified in the destructiveChanges.xml by appending "-VersionNumber" to the flow name. For example, if you want to delete version 1 and 2 of a flow, you would have the following in you destructiveChanges.xml:

<types>
        <members>MyFlowName-1</members> 
        <members>MyFlowName-2</members> 
        <name>Flow</name>
</types>

sfdx-git-delta can not know which versions you want to delete just by deleting the MyFlowName.flow-meta.xml file from git.

However, I have created a lightweight sfdx cli plugin with a package:destructive:versionobsoleteflows command to be used in this very same sceneario. It will replace any reference of MyFlowName in your destructiveChanges.xml by entries of the versions that have been deactivated in the target org of your deployment.

Apologies for the advertising :)

[scolladon]

@aveshsingh172 Could you log an issue please ?
I'm not sure this belongs in this discussion