Tests slow down since upgrading cypress from 4.4.1 #7930
Comments
My Mac doesn't show this performance issue. If it were something in 4.5, it's maybe related to the Electron upgrade there? Nothing else seems like it would cause this. 4.4.1Total runtime: 1:08
4.5.0Total runtime: 1:14
|
jennifer-shehane
added
type: performance 🏃♀️
|
Yes, everyone else is using windows as well as the build server |
|
hi JSumner1992, if you care for performance, try using version 3.7.0. We are still stuck with this version as since 3.8.0 the performance hit is enorm. Also look at this issue #5987 |
|
We also have this issue, cypress now fails everytime, for random test, and became unusable in CI |
Thank you JSumner for the tip! I reverted to 3.7.0 and the performance increase is very noticeable when running locally on my Windows machine. |
jennifer-shehane
added
OS: windows
type: regression
|
The regression occurs once upgrading to electron 8.2.1 from 8.2.0 |
|
Looking at the commits I spotted this: which re-enables offscreen support. Now.. cypress sets offscreen to true: but I can't find any code using the 'paint' event - given it was disabled in 8.2.0 I can't think why cypress would need it on. Since offscreen support was enabled and then disabled and then enabled, it would fit with the pattern of performance in the cypress releases I describe in my original issue. 🤞 |
Offscreen rendering: https://www.electronjs.org/docs/tutorial/offscreen-rendering passes every frame to a 'paint' event, which cypress does not listen too. In electron 8.2.1 it was fixed and switched back on, so when cypress upgraded from 8.2.0 to 8.2.3, it caused a performance regression. The performance regression on slow computers is about 8x slower. On fast computers you don't see a difference - I expect the multi-process nature of electron and cypress means that more cores mitigate the extra cpu needed. The option was introduced here, 3 years ago: cypress-io@5bde862 My guess is it was a mistake. @brian-mann do you remember?
|
Confirmed the problem with a custom cypress build and then on all tests with v5 of cypress using a hack: Replace the paths with where you keep your cypress cache on your build server. Pr raised - #8351 |
cypress-bot
bot
added
stage: needs review
|
The code for this is done in cypress-io/cypress#8351, but has yet to be released. |
|
Released in This comment thread has been locked. If you are still experiencing this issue after upgrading to |
Current behaviour:
Since upgrading from 4.4.1 to 4.8.0 the tests I run have gone from taking ~10 minutes to ~27 minutes, with some random failures getting introduced due to timeouts. I think this is machine specific as other people on the team have not had similar problems and neither has the build server. I've also reset my laptop to default settings but the issue was still present.
I've included test runs on the cypress kitchen sink repo below using the cy:run command using different versions of cypress. Since 4.4.1 there seems to have been a noticeable slowdown going from 1:40 to roughly 2:30.
4.10.0
4.9.0
4.8.0
4.7.0
4.6.0
4.5.0
4.4.1
Desired behaviour:
Figure out what happened between 4.4.1 and 4.5.0 to cause such an increase in test time.
Test code to reproduce
https://github.com/cypress-io/cypress-example-kitchensink
Versions
Any version after 4.4.1
Electron headless
Lenovo P51s
Windows 10 Enterprise
The text was updated successfully, but these errors were encountered: