"Unzipping Cypress" takes four minutes

[kevinburkenotion]

I have a t2.xlarge instance in AWS. I am attempting to run npm install inside of a Docker container to install Cypress 3.1.5.

I noticed that 4 minutes elapse between the start of "Unzipping Cypress" and its completion.

[00:19:28]  Unzipping Cypress       [started]
[00:23:13]  Unzipping Cypress       [completed]

This seems like an awfully long time to spend unzipping a binary... any idea what is going on?

[bahmutov]

Yeah on Linux the unzipping is done in software which combined with probably slow file system takes minutes. Can you cache the unzipped binary to avoid doing it more than once?

…

Sent from my iPhone

On Mar 21, 2019, at 20:26, kevinburkenotion ***@***.***> wrote: I have a t2.xlarge instance in AWS. I am attempting to run npm install inside of a Docker container to install Cypress 3.1.5. I noticed that 4 minutes elapse between the start of "Unzipping Cypress" and its completion. [00:19:28] Unzipping Cypress [started] [00:23:13] Unzipping Cypress [completed] This seems like an awfully long time to spend unzipping a binary... any idea what is going on? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[kevinburkenotion]

It's inside of a npm install step in a Docker container, so we either get the cached image or nothing, I think, unless you have better ideas.

[kevinburkenotion]

Any reason why you are not using gunzip or tar or similar to unpack the binary?

[bahmutov]

Because those tools might not be installed

…

Sent from my iPhone

On Mar 21, 2019, at 20:30, kevinburkenotion ***@***.***> wrote: Any reason why you are not using gunzip or tar or similar to unpack the binary? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[kevinburkenotion]

Understood, but 4 minute long unpacking on an AWS VM that's as powerful as a high end Macbook is not really great either.

[bahmutov]

Agree, just like node modules though - it needs to be cached even if it took 30 seconds

…

Sent from my iPhone

On Mar 21, 2019, at 20:30, kevinburkenotion ***@***.***> wrote: Any reason why you are not using gunzip or tar or similar to unpack the binary? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[cvuorinen]

Even though it does not take 4 min on my machine, after adding cypress to a project it makes switching branches just painful to always wait few minutes for downloading & unzipping cypress. Can't you cache it somewhere like other npm packages? I mean when switching branches where just one dep has changed, run npm ci and before cypress it was just few seconds since (almost) everything comes from cache, since adding cypress it's a waiting game that I try to avoid. I mean I just downloaded & unzipped the thing few minutes ago, such a waste (of bandwidth, cpu and most importantly my time).

[flotwig]

@cvuorinen If you're downloading the same version of Cypress, it is actually cached so it doesn't need to be downloaded multiple times. Check cypress cache for more info.

[cvuorinen]

@flotwig thanks, seems things have improved. That project is not on v3.x yet. I'll look into upgrading it.

[goodliff]

@cvuorinen did you manage to get this down at all?

We are waiting 3+ minutes for the extraction and as we are blowing the instance away after the build and tests we need to be able to install/ extract again once the next instance starts up. I've already been asked to remove cypress as it's the only thing taking this long in the build process. (trying to avoid that)

[flotwig]

@goodliff configure your CI to cache the ~/.cache/Cypress directory, you won't need to unzip it every time. More information: https://docs.cypress.io/guides/guides/continuous-integration.html#Advanced-setup

---

The code for unzipping is here:

cypress/cli/lib/tasks/unzip.js

Line 16 in 26c8e7f

const unzip = ({ zipFilePath, installDir, progress }) => {

The unzip speed could be greatly improved if code that uses the native unzip binary was added, in addition to the macOS-specific ditto code.

[flotwig]

Opened a PR to use the native unzip utility on Linux, which should be faster: #5851

[cypress-bot]

The code for this is done in cypress-io/cypress#5851, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[goodliff]

Nice, looking forward to seeing whether this changes anything for us. Still looking at the CI caching options, but this will help some.

[flotwig]

I've created a separate issue for improving the unzip time on Windows: #5861

[cypress-bot]

Released in 3.8.0.

[mightyiam]

Thank you!

[goodliff]

Um.... "[14:09:00] Unzipping Cypress 200% -56s [title changed]"

Not entirely sure what happened here but I'm being told that cypress unzipped to 200% when building my docker image. This is with 3.8.0 which incorporated the zipping change I believe.

[mightyiam]

@goodliff perhaps better to open a new issue.

[trainoasis]

@goodliff configure your CI to cache the ~/.cache/Cypress directory, you won't need to unzip it every time. More information: https://docs.cypress.io/guides/guides/continuous-integration.html#Advanced-setup

The code for unzipping is here:

cypress/cli/lib/tasks/unzip.js

Line 16 in 26c8e7f

const unzip = ({ zipFilePath, installDir, progress }) => {

The unzip speed could be greatly improved if code that uses the native unzip binary was added, in addition to the macOS-specific ditto code.

AFAIK you cant cache anything outside of the repo that's being used. But, you can set the Cypress cache folder in "variables" section like this:

CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"

and cache this. This works, but saving and extracting this cache also takes ages in our case so It does not speed things up at all.

[jennifer-shehane]

This issue will be closed to further comment as the exact issue here was resolved and tested.

If you're experiencing a bug similar to this in Cypress, please open a new issue with a fully reproducible example that we can run. There may be a specific edge case with the issue that we need more detail to fix.