'Aw, Snap' Error in Test Runner consistently occurs every minute when a test is running during cypress open

[salty-blue-mango]

Current behavior:

Cypress Test Runner displays the chrome 'Aw, Snap' error on any commands run after 60 seconds

Desired behavior:

To hopefully never see the 'Aw, Snap' error ever again.

Steps to reproduce:

Am able to reproduce in my own environment when simply running any tests that run commands after a test has been running for at least 60 seconds.

These are the custom Cypress commands that I am running in multiple tests. If I break up the single test file into multiple test files that run less than 60 seconds all of the tests pass. Longer than that and 'Aw, Snap' error is returned consistently every time.

Cypress.Commands.add('navigateCreateJob', () => {
    cy.clearSessionStorage();
    cy.wait(2000);
    cy.visit(`${activeEndpoint}/#/jobs/create`);
    cy.url().should('contain', '/#/jobs');
});
Cypress.Commands.add('createJob', (jobName, jobFunction) => {
    cy.get(setupTestsMap.jobName).click().clear().type(jobName);
    cy.get(setupTestsMap.functionName).click().clear().type(jobFunction);
    cy.get(setupTestsMap.stateApplyDropDown).click();
    if(jobFunction === 'state.apply'){
        cy.get(setupTestsMap.selectFile).click().clear().type('CypressSleepJob40');
        cy.get(setupTestsMap.cypressSleepJob40).click();
    }
    cy.get(setupTestsMap.save).click();
});
Cypress.Commands.add('navigateCreateSchedule', () => {
    cy.clearSessionStorage();
    cy.wait(2000);
    cy.visit(`${activeEndpoint}/#/schedules/new/detail`);
    cy.url().should('contain', '/#/');
});
Cypress.Commands.add('createSchedule', (scheduleName, jobCommand, target, frequency, time) => {
    cy.get(setupTestsMap.scheduleName).click().clear().type(scheduleName);
    cy.get(setupTestsMap.jobCommand).click().clear().type(jobCommand);
    cy.get(setupTestsMap.clickDropDown).click();
    cy.get(setupTestsMap.target).click().clear().type(target);
    cy.get(setupTestsMap.clickDropDown).click();
    cy.get(setupTestsMap.frequency).click().clear().type(frequency);
    cy.get(setupTestsMap.timeDrop).select(time);
    cy.get(setupTestsMap.runNow).click();
    cy.get(setupTestsMap.saveSchedule).click();
});

describe('Scheduler List Page -- Sort', () => {
    it('Setup Testing Jobs', () => {
        cy.login();
        cy.wait(6000);
        cy.createSleepFile();
        cy.navigateCreateJob();
        cy.createJob('aa_CypressSleepJob', 'state.apply');
        cy.navigateCreateJob();
        cy.createJob('zz_CypressSleepJob', 'state.apply');
    });
    it('Setup Testing Jobs #2', () => {
        cy.login();
        cy.wait(6000);
        cy.navigateCreateJob();
        cy.createJob('Yao Ping #11', 'test.ping');
        cy.navigateCreateJob();
        cy.createJob('Disable Me', 'test.ping');
    })
});

Versions

Cypress 3.0.1
MacOs Sierra
Chrome 67

[kuceb]

@Nate-Watts this does seem strange that the tests consistently fail after 60 seconds. Could you not reproduce the failure with just cy.wait(60000)?
If not, it would be helpful for you to iterate over the failing test removing certain test code, to hopefully narrow this bug down, because it's too complex to reproduce on our end.

[salty-blue-mango]

@bkucera
Ok, so I went through the code and pinpointed where exactly it is failing at after rearranging and replacing some code. It is failing at the beforeEach hook where my custom login function is called.

It has been crashing and giving me an 'Aw, Snap' error, but two times I have gotten a failed test that says 'Cannot read property 'access' of undefined' which occurred in the beforeEach hook. It is only occurring on the third 'it' block.

Here is my custom login command and below is the clear session storage command that is being used.

What is happening in the login command is that we visit our login endpoint and a cookie is stored before giving it any credentials. The way we authenticate is by checking for this initial cookie as well as a token that is created once the 'POST' command is made with the username and password. This is why I have to request our login endpoint first, and then request it again with a post command.

I am not sure whether any of this information helps, but another thing I would like to mention is that for the past week I have been receiving this socket time out error. Whenever this was received I would just run the test again and it would work as expected. I haven't been getting this error anymore, but it seems that maybe chrome is just crashing now in its place? Not sure how, just a thought.

To add on to that, I just ran a test a noticed all these XHR requests are being aborted on each login command, but the first two 'it' blocks are not failing, only the last.

[faiz121]

I am having the exact same issue, says Aw, Snap after 60 seconds. I am on 3.0.1, which I recently upgraded. I am testing it on my local. Cannot reproduce if I were to just cy.wait(60000).
I have multiple tests and as it goes through them and after 60s it says Aw, snap. No matter which test, if it hits 60s mark it fails

[mechanical-turk]

• I've been getting this error too.
• On longer test suites, I almost always get it.
• However, sometimes I get it on short ones too. (~45 secs)
• I tried different versions of cypress: v.2.0.0 and v.3.0.1. No luck there either.

These errors started happening shortly after we migrated to node v.10. We also very recently switched to yarn v.1.7 from npm. Hope you'll find these data points useful. Best of luck Cypress Team <3

[brian-mann]

Can any of you explain if you are getting this via cypress open or cypress run or only one or on both?

That would be extremely helpful to know.

As it stands there is not enough information here for us to be able to reproduce.

An animated gif or video of the problem would be helpful to see.

If this is JUST happening on cypress open its likely because of the debugging features we enable that enable time traveling and other features. These debugging features effectively act as a controlled memory leak which can cause the browser to run out of memory.

Do any of you all use chromeWebSecurity: false by chance? Have you tried running it in Electron to see if it doesn't crash?

Has this always been happening? Did it recently start happening?

[faiz121]

@brian-mann

Can any of you explain if you are getting this via cypress open or cypress run or only one or on both?

Only with cypress open

An animated gif or video of the problem would be helpful to see.

I will see if I can post a gif

Do any of you all use chromeWebSecurity: false by chance?

Yes I do have chromeWebSecurity: false in cypress.json

Have you tried running it in Electron to see if it doesn't crash?

I have run cypress run (headless) couple of times, recently on my local, and I never saw that issue

Has this always been happening? Did it recently start happening?

Other developers in my team have occasionally ran in to this issue before (with cypress open) when we were on cypress v2 but since we upgraded to v3.0.1 this issue is widespread and can be easily replicated, with cypress open. Surprisingly we never saw this issue when running on CI (probably because it runs headless in Electron) both with cypress v2 and v3

Thanks for looking into this issue

[brian-mann]

Then I'm very confident its due to the debugging features we enable with cypress open.

During a cypress run we turn everything off since that mode is optimized for running to completion - whereas cypress open is about iterating and writing tests.

I would first say that you should really only be using cypress open to run a single test - doing anything more than that is potentially volatile. We've written about it in some issues and have plans to improve that experience, but that is honestly the truth of it today.

• Change the way "Run All Specs" are run in Interactive Mode (from the GUI) #1586

There are some older ones too like these...

• Tests slows down to a crawl and eventually times out/crashes #990
• Cypress slows down drastically over long runs - restart renderer between spec files? #431

You can turn off the debugging features of Cypress by writing { numTestsKeptInMemory: 0 }. That may immediately solve the issue.

As for the { chromeWebSecurity: false } bits - the reason I ask is that in Chrome 67, Google started running experiments enable site level isolation. The problem is that its a known issue they don't take into account turning off web security. When Cypress does its thing, I can see how its possible this can lead to a crash.

Read my comment here for a workaround you can pop in today: #1951 (comment)

When 3.0.3 comes out it will have this flag in there by default.

Can you also try running Electron 59 locally with cypress open ? That would help isolate whether this is a browser version specifically problem (with Chrome) or this is an application / Cypress specific issue. If it doesn't happen in Electron 59 (which is Chromium 59) then I'm confident its related to the Chrome 67 upgrades.

In the past you were likely seeing these messages for a completely different reason - possibly because you may also have an application that puts significant pressure on the browser's memory.

[faiz121]

@brian-mann

So I tried to run cypress open with Electron 59. After 60- 70 secs I see blank screen in Electron, also memory usage for cypress app around 2.7GB, and I see following error dump in cypress helper (in terminal), not sure if it will be helpful

[7833:0x7fbf1a01ae00]   198090 ms: Mark-sweep 2046.8 (2169.4) -> 2046.8 (2169.4) MB, 719.9 / 0.4 ms  allocation failure GC in old space requested
[7833:0x7fbf1a01ae00]   198963 ms: Mark-sweep 2046.8 (2169.4) -> 2046.6 (2131.9) MB, 872.8 / 68.0 ms  last resort
[7833:0x7fbf1a01ae00]   199761 ms: Mark-sweep 2046.6 (2131.9) -> 2046.6 (2127.9) MB, 797.9 / 54.3 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2307972cb5d1 <Window map = 0x3c90e360cd89>
    2: replace(this=0xb0e0b8b8141 <Very long string[2248993]>,0x2ed072dcf3d9 <JSRegExp <String[23]: url\((['"])([^'"]*)\1\)>>,0xb0e0b8b81c9 <JSFunction (sfi = 0x7e5f4e439e9)>)
    4: makePathsAbsoluteToStylesheet [http://local.entelo.com:3000/__cypress/runner/cypress_runner.js:61418] [bytecode=0x7e5f4e42dd9 offset=84](this=0x21b5711b28f1 <JSGlobal Object>,styles=0xb0e0b8b8141```

[brian-mann]

Yup this seems like a memory issue caused by how we hold onto references for debuggability with cypress open.

Likely will be fixed by what I said in my previous comment.

[MaxnzWZ]

{ numTestsKeptInMemory: 0 } works for me. I have suffered from this issue for a long time.

[Prashant-Kan]

@MaznzWz: Mine too problem was solved using numTestsKeptInMemory: 0. But, it comes to be temporarily solution. The problem comes again even if numTestsKeptInMemory is set to 0.

Facing the same Aw Snap!!.

Anyone has some different solution to this?

[Prashant-Kan]

@brian-mann: I am facing this issue. I have used. "numTestskeptInMemory: 0" in my cypress.json file. Also, Its coming cypress open command. I face this issue earlier too, and it gets solved using the mentioned property of memory. But recently, it comes back again and now its reproducible everytime. by steps of one testcase goes beyong 1000 steps.

we are running only ONE testcase and still browser crashes.

Can you please share the workaround for this ?

[jennifer-shehane]

@Prashant-Kan Please see this issue: #4164

[jennifer-shehane]

Since this issue hasn't had activity in a while, we'll close the issue until we can confirm this is still happening. Please comment if there is new information to provide concerning the original issue and we'd be happy to reopen.

[criess]

I can confirm it still exists for cypress 9.5.0 and versions:

• chromium v96 (Fedora 35 latest updates)
• chromium v101 (custom chromium from https://download-chromium.appspot.com/)
• electron v94 (default from cypress)

We have some complex app (large SPA based on react), that constantly fails within 60s after testing started on cypress open. numTestsKeptInMemory non-zero seems to wreak some havoc still (even as low as values 1-3). The whole snapshot/command history feature seems to hit the envs per tab js memory limit with chrome/chromium/electron pretty quick (what I can read from the cypress:server:util:process_profiler) and thus crashing it. cypress run works fine …

[criess]

coredump_chromium_101_cypress.dmp.txt
renderer process failing but sadly no real dump/trace attached …

[Vamsi-K-Reddy]

Issue still happening for us. Cypress crashes during execution with cypress open, with 'Aw,Snap' error when the lines of code are more( in our case 400-500) even with the below setting
"numTestsKeptInMemory": 0
"video": false
Cypress version -9.5.2 (Initially was happening with 5.6.0. Tried upgrading, but issue didn't get resolved)
Chrome version -98
Mac OS

S

[Prashant-Kan]

Yes, We also facing the same. when test runs for some time around 15 minutes.
The scenario is in such a way that require this long test. We cannot divide into different testcase because its a one scenario tests.

In the end we have to use the solution as "numTestsKeptInMemory" to 0 in cypress.json and then run that testcase.
it would be great if we get a solution for this.

various bugs we have come across after the cypress version 8.7. Due to which we are not able to update the version and use.

Cypress has become unstable after that (at least for our application )

[satyajeetpaygude]

This issue is still existing even today with cypress 12.3 version .. test was running for 15 mins and then suddenly it broke into aw snap