CI - create archive of dependency sources

[kumavis]

status: experiment

the intention of this is to create an uncorruptible record of the dependencies used to create the build

[metamaskbot]

Builds ready [59874e6]

• builds: chrome, firefox, opera, edge
• bundles: background, ui, inpage, contentscript, libs, phishing-detect

[kumavis]

I've created a monster

[metamaskbot]

Builds ready [37471d4]

• builds: chrome, firefox, opera, edge
• bundles: background, ui, inpage, contentscript, libs, phishing-detect

[Gudahtt]

I see what you mean about the CircleCI conditional steps being not fun. It looks like they can't take environment variables at all, so it might not be possible to use them for this kinda thing.

Maybe we could do something like this instead: Gudahtt@269b6c0
It's not pretty, but it works (I think).
I was tempted to add a prep-deps job that included both to avoid having to list both as requirements, but adding another stage (even a "dummy" stage with just an 'echo' like all-tests-pass) has a substantial performance impact.

[Gudahtt]

We should probably also consider storing these somewhere aside from CircleCI, as they don't make guarantees about how long artifacts remain available.

[kumavis]

wow, uploading artifacts as tar.gz vs as files is a crazy difference
tar.gz took 2s, individual files has been running for 1+hr 😲

[kumavis]

@Gudahtt i think its probly best to do the conditions in shell scripts and not put too much into the circle config. one thing to note tho is that the store_artifacts must be in the config

[Gudahtt]

Yeah, that would probably be best. We can just store an empty deps-archive in the default case.

[kumavis]

need to verify if artifacts can be tampered with, not sure how they would be tho if it needs to be in the circleci config

[kumavis]

fyi archive size: yarn-cache.tar.gz (288 MB) link here

[kumavis]

oh! we could likely upload the archive to github releases (this may affect our manipulation risk)

[kumavis]

you seem to be able to slap a whole 300mb into the releases page

[Gudahtt]

The only limit I could find was 2 GB per file, which we're well within.

[kumavis]

lets get #7208 merged first, then I'll update this

[whymarrh]

@kumavis did you still want to merge this?

[whymarrh]

status: experiment

I'm closing this for now, we can re-open it when we update it