Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

force:source:deploy -x manifest/package.xml failing #1112

Closed
tetter36 opened this issue Aug 9, 2021 · 41 comments
Closed

force:source:deploy -x manifest/package.xml failing #1112

tetter36 opened this issue Aug 9, 2021 · 41 comments

Comments

@tetter36
Copy link

tetter36 commented Aug 9, 2021

Summary

I'm getting deploy errors after updating to v. 7.112.*

Everything worked on v 7.111.*

I'm getting this error when attempting to run sfdx force:source:deploy -x manifest/package.xml:

*** Deploying with SOAP API ***
ERROR running force:source:deploy: Component conversion failed: Cannot read property 'id' of undefined

I've tried running with verbose flag, run in VS Code but I'm not getting any more info than "Cannot read property 'id' of undefined."

Steps To Reproduce:

Navigate to a repo with a manifest. run sfdx:force:source:deploy -x path/to/package.xml

Expected result

Should run the deployment without errors.

Actual result

Getting error:
*** Deploying with SOAP API ***
ERROR running force:source:deploy: Component conversion failed: Cannot read property 'id' of undefined

System Information

{
"cliVersion": "sfdx-cli/7.112.1",
"architecture": "darwin-x64",
"nodeVersion": "node-v14.17.4",
"pluginVersions": [
"@oclif/plugin-autocomplete 0.3.0 (core)",
"@oclif/plugin-commands 1.3.0 (core)",
"@oclif/plugin-help 3.2.2 (core)",
"@oclif/plugin-not-found 1.2.4 (core)",
"@oclif/plugin-plugins 1.10.1 (core)",
"@oclif/plugin-update 1.4.0-3 (core)",
"@oclif/plugin-warn-if-update-available 1.7.0 (core)",
"@oclif/plugin-which 1.0.3 (core)",
"@salesforce/sfdx-plugin-lwc-test 0.1.7 (core)",
"@salesforce/sfdx-trust 3.6.0 (core)",
"alias 1.1.10 (core)",
"apex 0.2.3 (core)",
"auth 1.7.1 (core)",
"config 1.2.14 (core)",
"custom-metadata 1.0.12 (core)",
"data 0.6.0 (core)",
"generator 1.1.7 (core)",
"limits 1.2.1 (core)",
"org 1.6.7 (core)",
"salesforce-alm 52.2.3 (core)",
"schema 1.0.8 (core)",
"sfdx-cli 7.112.1 (core)",
"source 1.0.6 (core)",
"telemetry 1.2.3 (core)",
"templates 52.1.0 (core)",
"texei-sfdx-plugin 1.10.1",
"user 1.4.0 (core)"
],
"osVersion": "Darwin 20.5.0"
}

@tetter36 tetter36 added the investigating We're actively investigating this issue label Aug 9, 2021
@github-actions
Copy link

github-actions bot commented Aug 9, 2021

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

@WillieRuemmele WillieRuemmele added plugin-source and removed investigating We're actively investigating this issue labels Aug 9, 2021
@WillieRuemmele
Copy link
Member

Hi @tetter36 the new source plugin is more strict about the location and organization of metadata in your project... checkout the threads on #1106 to see if any of the workarounds there might help you

@AndrewRayCode
Copy link

I'm getting a similar issue after fully reinstalling sfdx from trying to fix the snafu of the "no such file or directory" bug - and now I'm getting:

sfdx force:source:deploy -u MySandbox -x manifest/package.xml -w 60
*** Deploying with SOAP API ***
ERROR running force:source:deploy: Cannot set property 'content' of undefined

These deploys have worked fine in the past. Since this is a fresh install how do I work around it?

@tetter36
Copy link
Author

tetter36 commented Aug 9, 2021

@WillieRuemmele thanks for the quick reply! I ran through and checked the objects folder. I was hoping I'd be able to attempt to deploy the objects in chunks but no such luck. I'll post any updates here as I go through and troubleshoot.

@AndrewRayCode
Copy link

It looks like sfdx update had an update, but the issue persists:

sfdx update
sfdx-cli: Updating CLI from 7.112.1 to 7.112.1-b99fa53... done
sfdx-cli: Updating CLI... done

$ sfdx force:source:deploy -u Sandbox -x manifest/package.xml -w 60
*** Deploying with SOAP API ***
ERROR running force:source:deploy:  Cannot set property 'content' of undefined

@mshanemc
Copy link
Contributor

mshanemc commented Aug 9, 2021

@AndrewRayCode you can definitely install an older version if you need to get back to work right away (and turn off auto-update). URLs here: https://github.com/forcedotcom/cli/blob/main/releasenotes/README.md#51140-may-27-2021---cli-71030

If you're able to share your repo where the error is happening, or isolate it to a particular component, that helps us fix the bug in the current version.

@tetter36
Copy link
Author

tetter36 commented Aug 9, 2021

It seems like there's something else that's wrong with deploying with the -x flag. I can deploy all the other source (except the Objects) in my project without errors using VS Code (right click + Deploy Source to Org) or with the sfdx force:source:deploy -p path/to/folder.

When I deploy with sfdx force:source:deploy -x path/to/package.xml I get same error as before after removing the CustomObject from the package.xml file.

*** Deploying with SOAP API ***
ERROR running force:source:deploy: Cannot set property 'content' of undefined

Here's the --dev-debug error:

*** Internal Diagnostic ***

TypeError: Cannot read property 'id' of undefined
    at ComponentSet.simpleKey (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:303:116)
    at ComponentSet.has (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:237:58)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:81:49)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPath (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:43:25)
    at Function.fromSource (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:67:46)
    at Function.<anonymous> (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:95:49)
Outer stack:
    at Function.wrap (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/plugin-source/node_modules/@salesforce/command/node_modules/@salesforce/core/lib/sfdxError.js:171:27)
    at Deploy.catch (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/plugin-source/node_modules/@salesforce/command/lib/sfdxCommand.js:248:67)
    at async Deploy._run (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/plugin-source/node_modules/@salesforce/command/lib/sfdxCommand.js:85:13)
    at async Config.runCommand (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/config/lib/config.js:173:24)
    at async SfdxMain.run (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/command/lib/main.js:27:9)
    at async SfdxMain._run (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/command/lib/command.js:43:20)
    at async Object.run (/Users/bpete/.local/share/sfdx/client/7.112.1-b99fa53/dist/cli.js:162:47)
******

@tetter36
Copy link
Author

tetter36 commented Aug 9, 2021

@AndrewRayCode I tried going to the json file in the link you added but I'm getting an Access Denied error on the page for sfdx-linux-arm-tar.xz.json.
Screen Shot 2021-08-09 at 5 16 53 PM

@mshanemc
Copy link
Contributor

@tetter36 that's a typo--it should be -xz not .xz to match all the other files. Apologies.

@jshackell-sfdc FYI for RN and docs.

@mshanemc
Copy link
Contributor

Any chance y'all have the standard Task object in your repo? We've seen elsewhere an error with that.

You can try just deploying that object (source:deploy -m) to see if it causes a problem.

@mshanemc mshanemc added the more information required Issue requires more information or a response from the customer label Aug 10, 2021
@tetter36
Copy link
Author

tetter36 commented Aug 10, 2021

I'm still getting the same issue even when removing all objects from the package.xml file to deploy. It deploys correctly when I run sfdx force:source:deploy -p force-app when I have the forceignores for **/objects/*

Here's my package.xml file that no longer works even with the CustomObject removed:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
	<types>
		<members>*</members>
		<name>ApexClass</name>
	</types>
	<types>
		<members>*</members>
		<name>ApexComponent</name>
	</types>
	<types>
		<members>*</members>
		<name>ApexPage</name>
	</types>
	<types>
		<members>*</members>
		<name>ApexTrigger</name>
	</types>
	<types>
		<members>*</members>
		<name>AuraDefinitionBundle</name>
	</types>
	<types>
		<members>*</members>
		<name>CorsWhitelistOrigin</name>
	</types>
	<types>
		<members>*</members>
		<name>CspTrustedSite</name>
	</types>
	<types>
		<members>*</members>
		<name>CustomLabels</name>
	</types>
	<types>
		<members>*</members>
		<name>CustomMetadata</name>
	</types>
	<types>
		<members>Case</members>
		<name>EscalationRules</name>
	</types>
	<types>
		<members>*</members>
		<name>Flow</name>
	</types>
	<types>
		<members>Display_Theme</members>
		<members>Music_category</members>
		<members>Song</members>
		<members>State</members>
		<members>Terms</members>
		<name>GlobalValueSet</name>
	</types>
	<types>
		<members>*</members>
		<name>LightningComponentBundle</name>
	</types>
	<types>
		<members>*</members>
		<name>Workflow</name>
	</types>
	<version>50.0</version>
</Package>

I also tested the VS Code deploy with each of the listed folders and they all deployed without errors.

@no-response no-response bot removed the more information required Issue requires more information or a response from the customer label Aug 10, 2021
@zankoav
Copy link

zankoav commented Aug 10, 2021

Hello, I have same problem.
With -p - work fine, but with -m or -x -failing.
sfdx force:source:deploy -p ./force-app/main/default/classes/Test.cls - worke fine
Our project has a lot of meta files more then 5k, that is why (sfdx force:source:deploy) doesn't work with deployment by -m or -x.
In small projects - all works fine, but not in large.

My logs:
sfdx force:source:deploy -m ApexClass:ETEAssets --loglevel=debug --json { "status": 1, "name": "TypeError", "message": "Cannot read property 'id' of undefined", "exitCode": 1, "commandName": "Deploy", "stack": "TypeError: Cannot read property 'id' of undefined\n at ComponentSet.simpleKey (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:303:116)\n at ComponentSet.has (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:237:58)\n at MetadataResolver.getComponentsFromPathRecursive (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:81:49)\n at MetadataResolver.getComponentsFromPathRecursive (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)\n at MetadataResolver.getComponentsFromPathRecursive (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)\n at MetadataResolver.getComponentsFromPathRecursive (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)\n at MetadataResolver.getComponentsFromPathRecursive (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)\n at MetadataResolver.getComponentsFromPath (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:43:25)\n at Function.fromSource (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:67:46)\n at Function.build (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/plugin-source/lib/componentSetBuilder.js:74:74)\nOuter stack:\n at Function.wrap (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/plugin-source/node_modules/@salesforce/command/node_modules/@salesforce/core/lib/sfdxError.js:171:27)\n at Deploy.catch (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/plugin-source/node_modules/@salesforce/command/lib/sfdxCommand.js:248:67)\n at async Deploy._run (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/plugin-source/node_modules/@salesforce/command/lib/sfdxCommand.js:85:13)\n at async Config.runCommand (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/config/lib/config.js:173:24)\n at async SfdxMain.run (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/command/lib/main.js:27:9)\n at async SfdxMain._run (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/command/lib/command.js:43:20)\n at async Object.run (/Users/aleksandrzanko/.local/share/sfdx/client/7.112.1-b99fa53/dist/cli.js:162:47)", "warnings": [] }

@EgorBosolyho
Copy link

EgorBosolyho commented Aug 10, 2021

I'm getting the similar issue when I'm trying to use:
sfdx force:source:deploy -m ApexClass:MyApexClass OR sfdx force:source:deploy -x path/to/package.xml
After some research, I found that it works if I delete most of the objects in the Objects folder. So I think this may be a problem related to large projects. This also works well if I use a deployment with a direct file path
sfdx force:source:deploy -p path/to/file

TypeError: Cannot read property 'id' of undefined
    at ComponentSet.simpleKey (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:303:116)
    at ComponentSet.has (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:237:58)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:81:49)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPathRecursive (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:96:37)
    at MetadataResolver.getComponentsFromPath (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/metadataResolver.js:43:25)
    at Function.fromSource (/Users/user1/.local/share/sfdx/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSet.js:67:46)
    at Function.build (/Users/user1/.local/share/sfdx/node_modules/@salesforce/plugin-source/lib/componentSetBuilder.js:74:74)\nOuter stack:
    at Function.wrap (/Users/user1/.local/share/sfdx/node_modules/@salesforce/core/lib/sfdxError.js:171:27)
    at Deploy.catch (/Users/user1/.local/share/sfdx/node_modules/@salesforce/command/lib/sfdxCommand.js:248:67)
    at async Deploy._run (/Users/user1/.local/share/sfdx/node_modules/@salesforce/command/lib/sfdxCommand.js:85:13)
    at async Config.runCommand (/Users/user1/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/config/lib/config.js:173:24)
    at async SfdxMain.run (/Users/user1/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/command/lib/main.js:27:9)
    at async SfdxMain._run (/Users/user1/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@oclif/command/lib/command.js:43:20)
    at async Object.run (/Users/user1/.local/share/sfdx/client/7.112.1-b99fa53/dist/cli.js:162:47)

@mshanemc
Copy link
Contributor

@EgorBosolyho to make sure I understand, sfdx force:source:deploy -p works with the entire repo, but -m will fail even when it's only pointed at apexClasses. And -x will fail unless you remove a lot of objects from the objects folder.

@AndrewRayCode
Copy link

AndrewRayCode commented Aug 10, 2021

I found my first issue, which was that there are *.cls files inside an object folder. After moving those, I'm now getting

Expected source files for type 'EmailTemplate'

Not sure why yet.

@mshanemc you released breaking API changes to sfdx (like requiring certain files not to be in certain folders) with no warning, and rolling back doesn't fix it. Would you consider reverting the recent feature work and cut a new sfdx version until you're able to triage the root cause of these issues? This incident has been going on for several days now, still no formal messaging from the team

@EgorBosolyho
Copy link

EgorBosolyho commented Aug 10, 2021

@EgorBosolyho to make sure I understand, sfdx force:source:deploy -p works with the entire repo, but -m will fail even when it's only pointed at apexClasses. And -x will fail unless you remove a lot of objects from the objects folder.

Yes, it is.
-m and -x will fail unless I remove a lot of objects from the objects folder or whole folder.
And it doesn't depend on which objects I left, because I thought I had "bad" objects, and tried different cases

-p works well

@AndrewRayCode
Copy link

AndrewRayCode commented Aug 10, 2021

I found two issues (so far):

  • We had *.cls files in our classes/objects/task/fields/ folder. This is a breaking change in sfdx with no warning/messaging, and I was only able to find it by poking around in ~/.local/share/sfdx/client/7.112.1-b99fa53/node_modules/@salesforce/source-deploy-retrieve/lib/src/resolve/adapters/decomposedSourceAdapter.js
  • There was no associated .email file with a email-meta.xml file, because we deleted one file but not the other. This one is a little less worrying because sfdx should have been doing that all along, and there was an actual error message for it, even though it was a cryptic error.

It doesn't look like these changes are mentioned in the release notes https://github.com/forcedotcom/cli/blob/main/releasenotes/README.md, except this which you have to admit is ironic considering the current issues:

we've greatly improved the performance and reliability of these commands:
force:source:deploy

I'm now getting additional deploy -x failures, it's unclear if these are from sfdx or from misconfiguration with salesforce, we weren't getting them before:

=== Component Failures [3]
Type   Name                                        Problem
─────  ──────────────────────────────────────────  ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Error  Some_File           Error parsing file: Element {http://soap.sforce.com/2006/04/metadata}someThing invalid at this location in type Group (3:29)
Error  Some_File           Error parsing file: Element {http://soap.sforce.com/2006/04/metadata}someFile invalid at this location in type Group (3:29)
Error  Somefolder/Somereport  Cannot find folder:Somefolder

Please consider rolling back the latest release and triaging these issues!

@AndrewRayCode
Copy link

At least one more breaking change with no messaging / warning, I found that we have line in our package.xml like:

<members>Reportfolder/My_Report_xyz</members>

and on disk, the report is in the folder

force-app/main/default/reports/SomeParentFolder/Reportfolder/My_Report_xyz.report-meta.xml

It appears the source of this breaking change is now requiring the full relative path from reports/ to be present in package.xml. For some reason this error doesn't show up until the end of a deploy.

@mshanemc
Copy link
Contributor

@mshanemc you released breaking API changes to sfdx (like requiring certain files not to be in certain folders) with no warning, and rolling back doesn't fix it. Would you consider reverting the recent feature work and cut a new sfdx version until you're able to triage the root cause of these issues? This incident has been going on for several days now, still no formal messaging from the team

@AndrewRayCode apologies for the breaking changes. We've heard from a few people that had non very non-standard folder structures (like your example of classes inside object folders). We didn't have any sample projects with structures like that to test with, and wouldn't have imagined it would have worked on the old version, either. We didn't/don't intend to support that structure.

We did provide notice of the beta via a pinned issue #1057 for ~6 weeks to let users experiment with and report bugs in the new version of the source commands and announced the change in the release notes. https://github.com/forcedotcom/cli/blob/main/releasenotes/README.md#71120-august-4-2021

We've been monitoring the exception telemetry around this release and haven't seen any increase to exceptions for the source:deploy commands that would warrant rolling it back for everyone. It's acutely painful for a subset of users, but solves a lot of bugs and performance issues for others. It's also (incidentally) the same library that VSCode started using earlier this year, and keeping the deploy/retrieve consistent with that tool is important to us.

Can you say more about what makes using 7.110 unworkable for you until you're able to reorganize some metadata?

@mshanemc
Copy link
Contributor

@EgorBosolyho @zankoav I know you probably can't provide your objects folder so we can experiment with -x/-m but can you provide a ballpark of how many objects were talking about? We're going to have to build a reproducer, find someone who is able to share, OR have one of you do some local code modifications so we can find the problem.

@AndrewRayCode
Copy link

AndrewRayCode commented Aug 10, 2021

Another breaking change found blocking deploys, We had a MyGroup.group-meta.xml file in queues/ instead of groups/ which explains the cryptic error:

Error  Some_File           Error parsing file: Element {http://soap.sforce.com/2006/04/metadata}someThing invalid at this location in type Group (3:29)

I'm not sure why all of these breaking change undocumented parse-time errors happen at the end of a deploy and not at the start, it's a long cycle to wait. This appears to be an uncaught runtime parse error which could be much more intuitive especially since you've consciously chosen to enforce new project structures.

apologies for the breaking changes. We've heard from a few people that had non very non-standard folder structures (like your example of classes inside object folders). We didn't have any sample projects with structures like that to test with, and wouldn't have imagined it would have worked on the old version, either. We didn't/don't intend to support that structure.

@mshanemc thanks, would love to see the postmortem on this one and how these breaking changes were released (including the one where sfdx self-deletes itself and requires a full reinstall) with no messaging and people having to hunt around github tickets to understand what's going on. Given these tickets have been open for almost a week now with no official messaging nor resolutions I think it would be a good gesture to the dx community to look into the rollout strategy for things like these.

Can you say more about what makes using 7.110

I tried manually installing 7.110 and below and still got the same blocking issues on deploys.

$ sfdx plugins:install [email protected]
Installing plugin sfdx-cli... installed v7.98.0

$ sfdx force:source:deploy -u Sandbox -x manifest/package.xml -w 60
*** Deploying with SOAP API ***
ERROR running force:source:deploy:  Cannot set property 'content' of undefined

sfdx doesn't seem to do clean deterministic installs so I'm not sure why downgrading didn't work. Releasing undocumented breaking changes with no downgrade strategy is really painful dx.

@AndrewRayCode
Copy link

I'm really looking forward to debugging my next blocking deploy error:

SOURCE PROGRESS | ████████████████████████████████████████ | 2571/2584 Components

=== Component Failures [0]
Type  Name  Problem
────  ────  ───────

ERROR running force:source:deploy:  Deploy failed.

@AndrewRayCode
Copy link

Two additional breaking changes from above:

  • Fields that were getting deployed before are not getting deployed if the API name doesn't match what's in the package. I'm not sure why sfdx doesn't report any issues about this, nor report the issues from Salesforce about it
  • API names in package.xml now have to match the case exactly of the file/custom field/etc, another breaking change. sfdx doesn't report these either nor does it report the error from salesforce

@AndrewRayCode
Copy link

These errors persist after sfdx plugins:install [email protected]

@shetzel Can you please advise how to downgrade sfdx, as there haven't been any downgrade instructions during this incident yet?

@AndrewRayCode
Copy link

Another breaking change: We had <members>Custom_Object__c.Name</members> in our package.xml and no corresponding field definition. I'm assuming we can't do this as it's a standard field. I'm removing the .Name fields from package.xml

@tetter36
Copy link
Author

@AndrewRayCode the sfdx plugins:install [email protected] doesn't work. I tried it before submitting this bug initially. You'll have to remove the sfdx version you have and then install it from one of the links that @mshanemc provided above.

I was able to work around the issue for now with my GitLab CI/CD pipeline by not upgrading sfdx. This allowed me to successfully deploy my code as is with no errors.

@EgorBosolyho
Copy link

@EgorBosolyho @zankoav I know you probably can't provide your objects folder so we can experiment with -x/-m but can you provide a ballpark of how many objects were talking about? We're going to have to build a reproducer, find someone who is able to share, OR have one of you do some local code modifications so we can find the problem.

@mshanemc , unfortunately, I can't provide you with our metadata, but I can do any local testing you need.
We have 442 entries in our objects folder. Here we store standard and custom objects.

objects folder info - 11 035 091 bytes (36,7 MB on disk) for 8 358 items

@AndrewRayCode
Copy link

AndrewRayCode commented Aug 11, 2021

More breaking changes:

  • a WorkflowAlert that references an alert in a workflow not in package.xml fails to deploy with An object 'WorkflowName.AlerkName' of type WorkflowAlert was named in package.xml, but was not found in zipped directory
  • You seem to have broken the ability to deploy reports in nested folders (or some other breaking change). Specifying the relative/path/Report_xyz name in package.xml, which was previously working fine, now fails the deploy with An object 'relative/path/Report_xyz' of type Report was named in package.xml, but was not found in zipped directory
  • It looks like fields from installed packages that are present in package.xml fail to deploy
  • Breaking change bug: Empty folders in the lwc/ directory are considered to be LWC bundles, even if they're not in the package.xml, and fail the deploy with: An object 'mylwcfoldername' of type LightningComponentBundle was named in package.xml, but was not found in zipped directory

Normally for libraries with such traction, backwards incompatible breaking changes are released with deprecation warnings for developers. You shouldn't expect developers to check the sfdx github every day looking for beta releases. Could you please add official messaging around the issues with this release? You introduced so many breaking changes at once.

@AndrewRayCode
Copy link

The broken report issue may be that the relative folder path in the source isn't listed as its own standalone folder and reportFolder. sfdx opaquely ignores the report that it used to upload, so there's no real way for end users to debug this.

@mshanemc
Copy link
Contributor

@AndrewRayCode thanks for all the findings...the nested folders change for reports/dashboards/emailTemplates is a nice catch. I think that's all the same bug.

You brought up a good question--what's your preferred method for hearing about upcoming CLI changes? We've discussed putting occasional notices in the stdout for non-CI use cases...would that be more helpful than just using release notes and github?

We did get a report of the empty-folder or invalid LWC bug (it'll bite empty/invalid Aura, too): #1099

@mshanemc
Copy link
Contributor

@shetzel @WillieRuemmele we've got a volunteer for some local test/debug. Thanks, @EgorBosolyho

@AndrewRayCode
Copy link

AndrewRayCode commented Aug 11, 2021

You brought up a good question--what's your preferred method for hearing about upcoming CLI changes?

@mshanemc from looking at the versions on npm, sfdx has been out for 4 years, and for 4 years it's supported non-standard project structure. In a single release you had dozens of breaking changes, including API changes. We're in day 7 of this incident/outage, it doesn't seem like it's being taken seriously. Users are left to fend for themselves with information scattered between Github, Salesforce Groups, and Stackexchange, and there's been no official support from the platform team. Even the rollback process is broken and the "right" way to rollback is hidden in Github comments (also how do you rollback vscode version vs terminal version?). I think it's up to your team to run a full post mortem on this and figure out how to improve developer messaging, and have the difficult conversation about sfdx auto-updating by default. Given there's no proper channels for it now, maybe communicate on every channel you have, to make sure folks see it? Including another sfdx update that uses a post-install step, or a one-time warning that users have to confirm, that there are massive changes in this version? And updating the release notes to document the dozens of breaking changes? Just some thoughts - I think this is really something to triage as an all-hands-on-deck incident.

These are just my thoughts, I don't know how your team wants to handle it, I just hope you take it seriously and do lots of formal messaging around it, given that this is the core tool of salesforce dx.

@WillieRuemmele
Copy link
Member

Hi @tetter36 @EgorBosolyho @zankoav @AndrewRayCode thanks for the continued help,

This thread is quickly growing to include multiple different bugs. In order to clarify things for our team, and other users, I'm going to break this thread up into the different bugs that I see by reading through it. I'll be copy/pasting things from here into their own threads

if you feel that I've missed any bugs feel free to create a new issue

@SCWells72
Copy link

I've seen this issue as well, and I've isolated it to an index on a BigObject, e.g., objects/MyBigObject/indexes/MyFieldIndex.indexe-meta.xml. If I remove that from my package directory, the deployment goes fine; if I add it back, it fails with TypeError - Cannot read property 'id' of undefined. Not sure if it's the same root cause for others here, but that's the specific metadata object that causes it for me. Hope that helps!

@mshanemc
Copy link
Contributor

I created #1141 for that bigObjectIndex problem. @tetter36 any chance you're using that metadata type in your project?

@EgorBosolyho
Copy link

Hi @mshanemc , I have tested this in my project, so if I delete all *.indexe-meta.xml files, the deployment will work well.
I guess this may be the root cause of this error.

@maggiben
Copy link
Contributor

maggiben commented Aug 27, 2021

this should be fixed in the latest-rc build of the cli v7.116.0, you can get it by either sfdx update stable-rc or npm install sfdx-cli@latest-rc --global depending on your preferred install method.

Please let us know if you're still seeing this in 7.116.0 of the CLI

@steals
Copy link

steals commented Aug 30, 2021

We have a few more errors related to non-basic folder structure and latest cli `7.116.0

  1. We have aura components used as quick actions placed into the quickActions subfolder under the aura these aura components are not deployed and error says An object 'xxx' of type AuraDefinitionBundle was named in package.xml, but was not found in zipped directory
  2. We have a bunch of reports in subfolders in the repo and all of them failing with An object 'folder_name/report_name' of type Report was named in package.xml, but was not found in zipped directory error
sfdx plugins --core
@oclif/plugin-autocomplete 0.3.0
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.2 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.10.1 (core)
@oclif/plugin-update 1.4.0-3 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-diff 0.0.6
@salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.10 (core)
apex 0.2.8 (core)
auth 1.7.1 (core)
config 1.2.24 (core)
custom-metadata 1.0.12 (core)
data 0.6.1 (core)
etcopydata 0.5.7
generator 1.1.7 (core)
isvte-sfdx-plugin 1.1.12
limits 1.2.1 (core)
org 1.7.0 (core)
salesforce-alm 52.3.1 (core)
schema 1.0.8 (core)
sfdmu 4.4.5
sfdx-cli 7.116.2 (core)
sfdx-falcon 0.0.93
sfdx-git-delta 4.8.1
sfdx-git-packager 0.0.0 (link) /Users/steals/Work/sfdx-git-packager
sfdx-migration-automatic 4.2.1
sfpowerkit 3.2.2
shane-sfdx-plugins 4.43.0
├─ @mshanemc/sfdx-sosl 1.1.0
└─ @mshanemc/plugin-streaming 1.1.7
source 1.0.12 (core)
telemetry 1.2.3 (core)
templates 52.1.0 (core)
user 1.4.0 (core)

@tetter36
Copy link
Author

tetter36 commented Sep 1, 2021

I created #1141 for that bigObjectIndex problem. @tetter36 any chance you're using that metadata type in your project?

@mshanemc my id issue was due to the bigObjectIndex as well. After I removed the index metadata file, everything worked correctly.

@SCWells72 great find!

@nvuillam
Copy link

nvuillam commented Sep 13, 2021

Tips that saved a sfdx project of mine (and probably all the others)

  • Upgrade to latest sfdx-cli
  • Change api version to 53.0 in sfdx-project.xml
  • Change api version to 53.0 in package.xml

If you have broken flexipages, it's because of broken ascending compatiblity -> https://salesforce.stackexchange.com/questions/357206/problem-when-deploying-flexipages-on-winter-22-org-the-xxxx-component-in

To fix your flexipages, you can either update all your XMLs one by one, or use the following sfdx-hardis command

sfdx hardis:project:fix:v53flexipages

Edit:

and this is not over... lots of items in folders are not found whereas they ARE in sfdx folders >_<

We have a bunch of reports in subfolders in the repo and all of them failing with An object 'folder_name/report_name' of type Report was named in package.xml, but was not found in zipped directory error

Same here than for @steals

@cristiand391
Copy link
Member

This was fixed in latest v7.120.0, release notes:
https://github.com/forcedotcom/cli/tree/main/releasenotes#71200-sept-30-2021

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests