String replacement mechanism not working correctly under Windows

[micha79x]

Summary

String replacement mechanism as documented here is not working as expected in Windows. Depending on the Shell sfdx is run in, the replacement is behaving quite differently.

In Git Bash, when specifying a single file based on the filename property, this file is never recognized. Changing to glob, the replacement is happening.

In PowerShell or cmd.exe, replacement never takes place.

In Unix, the replacement is working as expected.

Steps To Reproduce:

The following steps are done in Git Bash in Windows (10):

The steps are following the example of replacing the API Version in Apex Classes Metadata files from the official documentation

Repository to reproduce: sfdx-replacement-issue

1. Checkout the repo in branch master
2. Set the environment variable for testing the replacement without the need for a real deployment according to the documentation of Test String Replacements
   Set the environment variable SF_APPLY_REPLACEMENTS_ON_CONVERT to true
3. Run sfdx force:source:convert --outputdir mdapiOut --sourcepath force-app

Expected result

If the replacement mechanism would actually kick in, the CLI would complain about the missing file latest-api-version.txt which doesn't exist for demonstrating the error. The file is referenced in the sfdx-project.json:
"replaceWithFile": "replacementFiles/latest-api-version.txt"

The expected behaviour can be triggered by switching to the branch with-glob-property

1. git checkout with-glob-property
2. Again, run sfdx force:source:convert --outputdir mdapiOut --sourcepath force-app
   It now errors out with ERROR running force:source:convert: Component conversion failed: The file "replacementFiles/latest-api-version.txt" specified in the "replacements" property of sfdx-project.json could not be read.

Actual result

Nothing is happening, the replacement doesn't take place when running on master branch. It looks like, the CLI just doesn't recognize single files when specified by filename property.

System Information

• This is happening on Windows in Git Bash

{
"cliVersion": "sfdx-cli/7.182.1",
"architecture": "win32-x64",
"nodeVersion": "node-v16.17.1",
"pluginVersions": [
"@oclif/plugin-autocomplete 1.3.6 (core)",
"@oclif/plugin-commands 2.2.1 (core)",
"@oclif/plugin-help 5.1.19 (core)",
"@oclif/plugin-not-found 2.3.9 (core)",
"@oclif/plugin-plugins 2.1.8 (core)",
"@oclif/plugin-search 0.0.4 (core)",
"@oclif/plugin-update 3.0.7 (core)",
"@oclif/plugin-version 1.1.3 (core)",
"@oclif/plugin-warn-if-update-available 2.0.15 (core)",
"@oclif/plugin-which 2.2.2 (core)",
"alias 2.1.12 (core)",
"apex 1.3.0 (core)",
"auth 2.3.3 (core)",
"community 2.0.18 (core)",
"config 1.4.22 (core)",
"custom-metadata 2.0.5 (core)",
"data 2.1.17 (core)",
"generator 2.0.13 (core)",
"info 2.2.13 (core)",
"limits 2.1.3 (core)",
"org 2.2.19 (core)",
"packaging 1.10.10 (core)",
"schema 2.1.19 (core)",
"signups 1.2.12 (core)",
"source 2.3.8 (core)",
"telemetry 2.0.5 (core)",
"templates 55.1.0 (core)",
"trust 2.2.2 (core)",
"user 2.1.22 (core)",
"@salesforce/sfdx-plugin-lwc-test 1.0.1 (core)",
"salesforce-alm 54.8.4 (core)",
"sfpowerkit 6.0.1 (user)"
],
"osVersion": "Windows_NT 10.0.19044",
"shell": "C:\###\Git\usr\bin\bash.exe",
"rootPath": "C:\Users\###\AppData\Roaming\npm\node_modules\sfdx-cli"
}

Additional information

• In PowerShell and cmd.exe, the replacement is not happening at all, in none of both branches. Even though the official docu states: "Always use forward slashes for directories (/), even on Windows.", the command will never recognize the specified file as file which needs the replacement.
• In Linux (CentOS 7.7) the replacement is triggered on both the master and with-glob-property branches, just as expected.

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[WillieRuemmele]

Hi @micha79x - thanks for reporting this, so it's working fine when you're specifying glob but fails with filepath, so the workaround for now is to use a glob

[micha79x]

Hi @WillieRuemmele, yes, this workaround at least is working in a Git Bash. The project members using any native Windows Shell do not have a workaround at the moment.

[git2gus]

This issue has been linked to a new work item: W-12398244

[micha79x]

Update with another workaround: I searched for the actual implementation of the logic the plugin uses, and I think the line in doubt is this one: https://github.com/forcedotcom/source-deploy-retrieve/blob/af6c40444e5cfbbc66699de57b472c540ef9d4d4/src/convert/replacements.ts#L184

However, I recognized that for the filename parameter the plugin is doing a .endsWith(...) comparison. So in my config I tried to remove the path and it actually works as expected. See https://github.com/micha79x/sfdx-replacement-issue/compare/workaround for a branch demonstrating this.

This also works on Windows Shells.

As soon as path separators are introduced, the whole thing stops working consistently.

[RCMoonpie]

@micha79x : Thank you for filing this and providing the removal of the file path as a workaround.

I can verify another workaround on Windows 11 using Powershell and sfdx-cli version 7.184.2.

The workaround is to ignore the documentation where it states:

Always use forward slashes for directories (/), even on Windows.

---

In my own deployment (not the demo), when I replaced this...

"filename": "force-app/main/default/namedCredentials/MyNamedCredential.namedCredential-meta.xml"

...with this:

"filename": "force-app\\main\\default\\namedCredentials\\MyNamedCredential.namedCredential-meta.xml"

...the replacement worked, and the deployment succeeded.

[mshanemc]

@micha79x does using the \\ on windows for filename work for git-bash?

Reminder: git-bash isn't officially supported (we run our windows tests using pwsh) and it confuses things a lot.

[micha79x]

Hi @mshanemc, yes, that would actually work. I added another branch with the changed path separator here:

https://github.com/micha79x/sfdx-replacement-issue/blob/with-double-backslashes/sfdx-project.json

Based on the changes, the described expected result is met.

However, our projects usually are platform agnostic and are built on Windows, Mac and Linux. So the sfdx-project.json should not have something that only works on Windows machines. ;)

[mshanemc]

ok, I understand. So you'd want everything to use posix-style separators.