Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add retrievetargetdir to source:retrieve #624

Merged
merged 11 commits into from
Nov 1, 2022

Conversation

peternhale
Copy link
Contributor

What does this PR do?

add retrievetargetdir to source:retrieve

What issues does this PR fix or reference?

@W-10735446@

@peternhale peternhale requested a review from mshanemc October 25, 2022 17:59
@peternhale peternhale requested a review from shetzel October 26, 2022 19:21
Copy link
Contributor

@mshanemc mshanemc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only firm request is readdir withFileTypes

everything else is comments, questions, and style preferences.

src/commands/force/source/retrieve.ts Outdated Show resolved Hide resolved
let files: string[] = [];
const srcStat = await fs.promises.stat(src);
if (srcStat.isDirectory()) {
const contents = await fs.promises.readdir(src);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can save a lot of fs ops by using the withFileTypes option on readdir
https://nodejs.org/api/fs.html#fspromisesreaddirpath-options

which gives you the full dirEnt info https://nodejs.org/api/fs.html#class-fsdirent

src/commands/force/source/retrieve.ts Outdated Show resolved Hide resolved
Comment on lines 27 to 36
const nextResults = (await Promise.all(next.map(producer)))
.reduce<T[]>((acc, val) => {
if (Array.isArray(val)) {
acc.push(...val);
} else {
acc.push(val);
}
return acc;
}, [])
.filter((val) => val !== undefined);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const nextResults = (await Promise.all(ensureArray(next.map(producer)))).flatMap((val) => val);

sourceQueue: T[],
producer: (T) => Promise<T | T[]>,
concurrency: number,
queueResults = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd omit this since it isn't used. It's not clear what it might do and at-a-glance invites an infinite loop

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mshanemc it is used in mv function. Yes it can and yes I have :). Not being able to queue results defeats the ability to accomplish the dir traversal without recursion.

@@ -0,0 +1,43 @@
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall comment...there's some tools that do this kind of thing if we find a lot more use cases for it.

https://github.com/supercharge/promise-pool is fairly small, zero-dep. The error handling and abort looks nice.

return false;
});
if (overlapsProject) {
return;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this feels like a good place for a warning. Another option would be to make this "overlaps" validation a standalone function that could be used for your new flag's parse so that it doesn't go through all that retrieve work to fail this late in the game.

As is, it looks like the user would specify a targetDir and then if it overlapped, the retrieve would go to their default dir anyway without a warning?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mshanemc I think I am going to change direction on overlapping targets, fail the command if the target overlaps a package dir. There are other commands and variants of source:retrieve to get source into a package dir.

return;
}
let overlapsProject = true;
const resolvedTargetDir = resolve(this.flags.retrievetargetdir as string);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and what if I want to "peek" at the results, outside my normal project. So I say, --retrievetargetdir peek/main/default

what's going to happen? .replace(join('main', 'default') would eliminate the redundancy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll give that a try, but my suspect that it will only remove one main/default

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As predicted, the results land in peek/main/default

@peternhale peternhale requested a review from mshanemc October 27, 2022 21:22
@mshanemc
Copy link
Contributor

mshanemc commented Oct 31, 2022

QA notes:

✅ retrieve classes to new dir out
✅ retrieve classes to a folder inside existing dir out/main/default
✅ retrieve classes to new folder with several levels inside existing dir out2/main/default
✅ retrieve over an existing file --retrievetargetdir .gitignore
ERROR running force:source:retrieve: Metadata API request failed: Component conversion failed: ENOTDIR: not a directory, mkdir '.gitignore/main'
✅ retrieve non-existent type to new dir [doesn't leave empty dir]
✅ retrieve to existing non-project dir
✅ retrieve to overwrite pkgDir --retrievetargetdir force-app
ERROR running force:source:retrieve: The retrieve target directory [force-app] overlaps one of your package directories. Specify a different retrieve target directory and try again. (fails before mdapi call)

❌ retrieve to new child folder of pkgDir --retrievetargetdir force-app/main/default/test
I wish we blocked this, too. Makes a mess of tracking (still seems to deploy, though) Also, it put the files in force-app/test (not where I said to put them) and created an empty folder where I expected the files to appear force-app/main/default/test

✅ retrieve to same dir twice
../../plugin-source/bin/dev force:source:retrieve -m ApexClass --retrievetargetdir force-ap (partial match)
ERROR running force:source:retrieve: The retrieve target directory [force-ap] overlaps one of your package directories. Specify a different retrieve target directory and try again.

@mshanemc
Copy link
Contributor

mshanemc commented Nov 1, 2022

✅ force-app hits error
✅ force-ap works fine
✅ force-app/main hits error
✅ force-app/main/default/foo hits error
✅ force works fine

couldn't find any additional problems. 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants