[New Source Plugin] Empty folders added to package manifest, "of type AuraDefinitionBundle was named in package.xml, but was not found in zipped directory"

[aheber]

Summary

I'm getting several "An object '[Component name]' of type AuraDefinitionBundle was named in package.xml, but was not found in zipped directory" errors in my deployment while using the new Source Plugin.

I'm not sure why this one is happening but when I try and deploy my Aura and LWC components only, -m LightningComponentBundle,AuraDefinitionBundle, some of the Aura components seem to not be included in the deployed ZIP, the components that don't make it seem to change a bit between deployments but a large selection of them seem to be alphabetical, but not 100% in any failed deployment. Maybe file access limitations? I have used the SFDX_MDAPI_TEMP_DIR env var to confirm that the component files are extracted into the staging directory but I haven't been able to inspect the actual submitted ZIP file. I thought I could pull these out of the temp directory but I'm not seeing them.

Steps To Reproduce:

I'm not sure how to replicate this outside of my code base. Maybe I could try and create lots and lots of components and push them all together? (I have 607 Aura and 430 LWC components as of this testing)

Expected result

Components should have deployed successfully.

Actual result

Deployment fails with random failures.

System Information

{
"cliVersion": "sfdx-cli/7.110.0",
"architecture": "win32-x64",
"nodeVersion": "node-v14.17.3",
"pluginVersions": [
"@oclif/plugin-autocomplete 0.3.0 (core)",
"@oclif/plugin-commands 1.3.0 (core)",
"@oclif/plugin-help 3.2.2 (core)",
"@oclif/plugin-not-found 1.2.4 (core)",
"@oclif/plugin-plugins 1.10.0 (core)",
"@oclif/plugin-update 1.4.0-3 (core)",
"@oclif/plugin-warn-if-update-available 1.7.0 (core)",
"@oclif/plugin-which 1.0.3 (core)",
"@salesforce/sfdx-plugin-lwc-test 0.1.7 (core)",
"@salesforce/sfdx-trust 3.6.0 (core)",
"alias 1.1.10 (core)",
"apex 0.2.2 (core)",
"auth 1.7.1 (core)",
"config 1.2.13 (core)",
"custom-metadata 1.0.12 (core)",
"data 0.5.2 (core)",
"generator 1.1.7 (core)",
"limits 1.2.1 (core)",
"org 1.6.7 (core)",
"salesforce-alm 52.1.0 (core)",
"schema 1.0.8 (core)",
"sfdx-cli 7.110.0 (core)",
"sfdx-devops 0.4.3",
"sfdx-heber 0.0.2",
"sfdx-typegen 0.6.2",
"sfpowerkit 3.1.6",
"shane-sfdx-plugins 4.43.0",
"?????? @mshanemc/plugin-streaming 1.1.7",
"?????? @mshanemc/sfdx-sosl 1.1.0",
"source 1.0.3",
"telemetry 1.2.2 (core)",
"templates 52.1.0 (core)",
"user 1.3.1 (core)"
],
"osVersion": "Windows_NT 10.0.19042"
}

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[shetzel]

Hi @aheber - I tried to reproduce this using the ebikes_lwc repo but it worked as expected. I'm using v1.0.6 of the source plugin which is the latest release published several hours ago. I'm on a mac but will try on windows.

The plan is to bundle this version of the source plugin with the CLI release candidate today. Your issue has me concerned though. There isn't much debug output from the SDR library but you can try enabling it with the deploy command. SDR does not write the zip to the file system so it would just be the list of files in the temp dir.

[aheber]

@shetzel I just test with the v1.0.6 version, same problem. I just tested against an existing org instead of creating a new org that didn't hold the components yet. This let me separate the Aura and LWC components into separate deployments. Just deploying the Aura Components caused the same error.

I'll do some digging and see if I can get a reproduction available outside of my source repo. It could certainly be a Windows thing, I can test on my mac too.

If there are other specific tests worth running for details I'm happy to give them a try. Let me know.

[aheber]

OK, first my apologies. This looked like something else and I crossed names of things when evaluating the results so it was a bit of a goose chase. I have an old component with a name incredibly similar to a new component and I evaluated that for being present incorrectly. The changes in components present and missing can be attributes to changing branches between tests as I was letting things run while working on other tasks.

The problem is empty folders. Where we've deleted a component (at least Aura or LWC in my testing) GIT will remove the files but leave the folders around. SDR is seeing the folder and adding it to the manifest but it isn't coming through to the ZIP file so the deployment is getting confused.

Repro here: https://github.com/aheber/sourcePluginBreak-1099

The repro should work and should have two components of each type, one populated and one "empty" (.forceignore to make GIT persist the folder). Should cause the failure.

[aheber]

@shetzel (forgot to mention you)

[shetzel]

Thanks for the repro @aheber. We'll work on a fix unless you beat us to it. :)

[WillieRuemmele]

Hi @aheber just to confirm, you see this issue when you have an empty directory under force-app/main/default/lwc in your posted repo I should delete withOutContent/.forceignore and then attempt the deploy.

how come you have to add the .forceignore so that "GIT persist the folder" but "GIT will remove the files but leave the folders around" why is GIT leaving an empty folder in your demo, but not in your real project?

[aheber]

@WillieRuemmele, Git source tracking doesn't care about folders, it only manages them enough to create them to place files there.

On my local machine, I have legacy folders from deleted components. Either created and deleted "locally" and never committed or deleted via source tracking changes from another user, and Git never removed the associated folders on my machine. Because I've had this source folder active on my machine for several years I have a variety of of these folders that I haven't yet manually cleaned up and that source tracking doesn't care about. They have never been a problem until now.

I had to add the .forceignore files to my example repo because Git wouldn't have committed the folders because it only tracks files. I had to create a file at that directory so that Git would force the folder to be created because Git won't track or create "empty" folders.

Hope that helps explain why these folders exist.

[aheber]

@shetzel thanks! I don't think I'll have time this week so I don't know if I'll beat you do it. I haven't dug through the source on that repo yet to get my bearings. If it hangs around longer I'll likely try my hand at it.

[aheber]

I should also add to the Git convo, maybe there is a totally reasonable way to manage these empty folders with a Git config or something that I don't have turned on but I haven't ever come across it. Education welcome :)

[uip-robot-zz]

This issue has been linked to a new work item: W-9692466

[mshanemc]

I think the PRs associated with this issue resolved everything we saw. If not, please response with whatever is still wrong.

[aheber]

Yes, I believe it is all resolved now.