Sm/zip reads readables earlier

[mshanemc]

What does this PR do?

zip operation is probably the most common thing SDR does (every push/deploy)
ZipWriter passed all the WriteInfos to Archiver as Readable
Before this PR, all those Readable remained open until the zip finalizes.
So that would hit windows file limits but only on the zip operations (source->metadata and metada->source formats worked fine).

This PR moves the reading of those files left in the pipeline, so that they're read into buffers as they come into ZipWriter (using existing utility code) and then passed to Archiver. Once a file is read into buffer, the stream closes and the file doesn't have to stay open.

Besides solving the bug for windows users, this also has some perf benefits for all users (compare the passing scale nut from the 2 circle jobs shown below, or the checked in perf stats for linux/mac) on the larger projects' sourceToZip

fix #578 .

What issues does this PR fix or reference?

@W-11492994@
fix forcedotcom/cli#1555

Functionality Before

https://app.circleci.com/pipelines/github/forcedotcom/source-deploy-retrieve/4345/workflows/e4e5f01c-5eaa-4b9c-becb-1d23dcf358a0/jobs/19752)

Functionality After

https://app.circleci.com/pipelines/github/forcedotcom/source-deploy-retrieve/4351/workflows/f664546c-54fd-430e-a8ab-4a16dfec1d87/jobs/19817

[salesforce-cla]

Thanks for the contribution! Unfortunately we can't verify the commit author(s): Ubuntu <c***@i***.e***.internal>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Salesforce.com Contributor License Agreement and this Pull Request will be revalidated.

[WillieRuemmele]

couldn't get those tests working?

any idea why the sourceToMdapi time increased across the board? While the other three decreased?

[mshanemc]

couldn't get those tests working?

got them back now. It's the typical UT scenario--the tests are deeply tied to the implementation. I just wanted the NUTs to run to tell me I wasn't crazy (and they caught a real bug in folder types) before investing the time rewriting the UT

[mshanemc]

any idea why the sourceToMdapi time increased across the board? While the other three decreased?

whatever it is, it's not affected by this change to the zip writer. anything that's < 1 second of difference on these larger projects I'm gonna consider noise (after observing these for a while).

It's probably that the initial run had some good luck to be on the lower end and everything looks (slightly) worse by comparison.

We could probably run the tests a few times to increase precision. I'm looking more for "wow, that's 2x or 50% less" vs. "didn't change much" on these.

[RodEsp]

I was successfully able to run the perf NUTs with the new code and would intermittently get the "too many files open" error with the old code. Something to note though is that in both scenarios they took 30+ minutes to run 😕

I just have two very small comments but after those are addressed feel free to merge this at your convenience @mshanemc.