Reduce Memory Usage with window.gc() #8525

CoryDanielson · 2020-09-08T20:19:02Z

I recently discovered that a method can be exposed in Chrome and Electron to trigger garbage collection, and it seems to improve memory consumption significantly during test runs.

I'm wondering if adding this functionality to cypress by default (rather than a plugin) would be worthwhile, to hopefully reduce the number of crashes and improve performance.

Browser	Flag/Switch
Chrome	--js-flags=--expose-gc
Electron	--js-flags=--expose_gc

Once enabled, you can simply call window.gc() within javascript code.

I ran the same Cypress test with and without calling window.gc() in a global afterEach and saw a 30-50% reduction in peak memory usage from the same tests.

I suspect that this will also reduce the number of crashes due to hitting the max memory. For a large and complex app with lots of requests, I think the CPU is too busy and never has the downtime required for javascripts garbage collector to clean up stale references. GC does occur occasionally, but not often enough, and that eventually becomes a fatal mistake as the cypress process gets closer to the max memory threshold and GC is not run. In my case, starting a new test could increase memory usage by 100-300mb, and put the app over the 2gb limit.

To enable win.gc() for election, run cypress with the following with this environment variable set:

ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc cypress run

cypress/plugins/index.js

plugins/index.js will not work for Cypress v10+, see https://docs.cypress.io/guides/references/migration-guide#Plugins-File-Removed

// enables win.gc() for chrome

module.exports = (on, config) => {
	on('before:browser:launch', (browser, launchOptions) => {
		if (browser.name === 'chrome') {
			// exposes window.gc() function that will manually force garbage collection
			launchOptions.args.push('--js-flags=--expose-gc');
		}

		return launchOptions;
	});
};

cypress/support/index.js

afterEach(() => {
	cy.window().then(win => {
		// window.gc is enabled with --js-flags=--expose-gc chrome flag
		if (typeof win.gc === 'function') {
			// run gc multiple times in an attempt to force a major GC between tests
			win.gc();
			win.gc();
			win.gc();
			win.gc();
			win.gc();
		}
	});
});

The text was updated successfully, but these errors were encountered:

CoryDanielson · 2020-10-16T20:37:04Z

I have not been able to find a way to expose the garbage collector in Edge or FireFox. In the past there were a lot of different APIs for causing GC - the documentation around all of them is pretty sparse, or no longer exists.

It's possible to "force" garbage collection by manually increasing memory pressure (creating tons of objects/keys). It's probably going to be slower though. I'm not sure if this is worth implementing as a fallback, but it may provide some benefit for browsers where the a GC method is not available.

Example of manually forced garbage collection fallback

CoryDanielson · 2020-10-19T17:31:02Z

There's a chance that tests for low-memory apps would run slower with this, because calling gc is expensive. (I added a similar change with jest tests, and they were running noticeably slower. (Unit tests run a lot faster than e2e tests, and there were a lot more unit tests.)

I'm not sure if high memory apps also run slower too, in my case the tests would crash vs not crash so the performance hit was not noticeable and worth taking on.

flotwig · 2020-10-23T13:50:02Z

There's a chance that tests for low-memory apps would run slower with this, because calling gc is expensive.

@CoryDanielson that's the reason why forced GC has not been implemented in Cypress as a whole - it is expensive, and causes user tests to pause/stutter during garbage collection.

When Cypress initially introduced Firefox support, code was introduced that used the Firefox driver protocol to force garbage collection after tests (Cypress.on('test:before:run:async', ...)). This was necessary because Firefox <= 79 had a memory leak that was not getting GC'd properly unless we manually forced GC: https://docs.cypress.io/guides/references/configuration.html#firefoxGcInterval

You can manually trigger the Firefox GC routine (even in FF >= 80) by using Cypress.backend('firefox:force:gc').

The reason why forced GC was removed as soon as Firefox 80 became available was because it was causing user tests to run much more slowly than if we didn't force GC.

With this in mind, I still think that the functionality you're proposing is a good idea, because users do run into memory issues that can be helped with the use of forced GC. However, it is probably better-suited to being a user plugin than to being in Cypress core.

CoryDanielson · 2020-10-23T15:06:50Z

Thanks for the info! Does it make sense to expose chrome/electron gc in the same way as FF, or are you planning on removing the firefox:force:gc feature?

flotwig · 2020-10-28T16:01:14Z

firefox:force:gc will be around for the foreseeable future, since people will still be using versions of Firefox older than 80 for a while.

If you want to, you can open up a PR to add the --expose-gc flag by default, it doesn't seem to have any side-effects. Or users can pass ELECTRON_EXTRA_LAUNCH_ARGS=--expose-gc if they want to use window.gc in Electron.

Closing since this feature request will not be implemented in Cypress core.

bahunov · 2020-12-07T07:35:32Z

I recently discovered that a method can be exposed in Chrome and Electron to trigger garbage collection, and it seems to improve memory consumption significantly during test runs.

I'm wondering if adding this functionality to cypress by default (rather than a plugin) would be worthwhile, to hopefully reduce the number of crashes and improve performance.

Browser Flag/Switch
Chrome --js-flags=--expose-gc
Electron --js-flags=--expose_gc
Once enabled, you can simply call window.gc() within javascript code.

I ran the same Cypress test with and without calling window.gc() in a global afterEach and saw a 30-50% reduction in peak memory usage from the same tests.

I suspect that this will also reduce the number of crashes due to hitting the max memory. For a large and complex app with lots of requests, I think the CPU is too busy and never has the downtime required for javascripts garbage collector to clean up stale references. GC does occur occasionally, but not often enough, and that eventually becomes a fatal mistake as the cypress process gets closer to the max memory threshold and GC is not run. In my case, starting a new test could increase memory usage by 100-300mb, and put the app over the 2gb limit.

To enable win.gc() for election, run cypress with the following with this environment variable set:
ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc cypress run
cypress/plugins/index.js

enable win.gc() for chrome
// enables win.gc() for chrome

module.exports = (on, config) => {
	on('before:browser:launch', (browser, launchOptions) => {
		if (browser.name === 'chrome') {
			// exposes window.gc() function that will manually force garbage collection
			launchOptions.args.push('--js-flags=--expose-gc');
		}

		return launchOptions;
	});
};
cypress/support/index.js
afterEach(() => {
	cy.window().then(win => {
		// window.gc is enabled with --js-flags=--expose-gc chrome flag
		if (typeof win.gc === 'function') {
			// run gc multiple times in an attempt to force a major GC between tests
			win.gc();
			win.gc();
			win.gc();
			win.gc();
			win.gc();
		}
	});
});

Thanks alot @CoryDanielson !! I tried this fix in our jenkins-docker pipeline.. it seem to have fixed the stuck runs with chrome.

kamiloski · 2020-12-16T12:00:01Z

@CoryDanielson how can I check that flag is set in browse?

CoryDanielson · 2021-02-05T22:30:22Z

@kamiloski how can I check that flag is set in browse?

If it's working properly you can open the browse and type "window.gc" into the console. It should display a function.

If you want to test in Electron, simply do

if (win.gc) throw new Error("win.gc exists!")

MarcusElevait · 2022-08-30T15:49:03Z

I recently discovered that a method can be exposed in Chrome and Electron to trigger garbage collection, and it seems to improve memory consumption significantly during test runs.

I'm wondering if adding this functionality to cypress by default (rather than a plugin) would be worthwhile, to hopefully reduce the number of crashes and improve performance.

Browser Flag/Switch
Chrome --js-flags=--expose-gc
Electron --js-flags=--expose_gc
Once enabled, you can simply call window.gc() within javascript code.

I ran the same Cypress test with and without calling window.gc() in a global afterEach and saw a 30-50% reduction in peak memory usage from the same tests.

I suspect that this will also reduce the number of crashes due to hitting the max memory. For a large and complex app with lots of requests, I think the CPU is too busy and never has the downtime required for javascripts garbage collector to clean up stale references. GC does occur occasionally, but not often enough, and that eventually becomes a fatal mistake as the cypress process gets closer to the max memory threshold and GC is not run. In my case, starting a new test could increase memory usage by 100-300mb, and put the app over the 2gb limit.

To enable win.gc() for election, run cypress with the following with this environment variable set:
ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc cypress run
cypress/plugins/index.js

enable win.gc() for chrome
// enables win.gc() for chrome

module.exports = (on, config) => {
	on('before:browser:launch', (browser, launchOptions) => {
		if (browser.name === 'chrome') {
			// exposes window.gc() function that will manually force garbage collection
			launchOptions.args.push('--js-flags=--expose-gc');
		}

		return launchOptions;
	});
};
cypress/support/index.js
afterEach(() => {
	cy.window().then(win => {
		// window.gc is enabled with --js-flags=--expose-gc chrome flag
		if (typeof win.gc === 'function') {
			// run gc multiple times in an attempt to force a major GC between tests
			win.gc();
			win.gc();
			win.gc();
			win.gc();
			win.gc();
		}
	});
});

I exactly tried this and i also tried to set the ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc with export on my linux system, but neither can i use window.gc in the developer tools console nor does the gc work on the tests.
I'm using chrome version: 104.0.5112.101 (Official Build) (64-bit)
Does anyone of you know if this is not supported by chrome anymore?

bahunov · 2022-08-30T16:02:00Z

Yes, win.gc() definitely helped me too.

…

On Tue, 30 Aug 2022 at 19:49, Marcus Trepte ***@***.***> wrote: I recently discovered that a method can be exposed in Chrome and Electron to trigger garbage collection, and it seems to improve memory consumption significantly during test runs. *I'm wondering if adding this functionality to cypress by default (rather than a plugin) would be worthwhile, to hopefully reduce the number of crashes and improve performance.* Browser Flag/Switch Chrome --js-flags=--expose-gc Electron --js-flags=--expose_gc Once enabled, you can simply call window.gc() within javascript code. I ran the same Cypress test with and without calling window.gc() in a global afterEach and saw a *30-50%* reduction in peak memory usage <#350 (comment)> from the same tests. I suspect that this will also reduce the number of crashes due to hitting the max memory. For a large and complex app with lots of requests, I think the CPU is too busy and never has the downtime required for javascripts garbage collector to clean up stale references. GC does occur occasionally, but not often enough, and that eventually becomes a fatal mistake as the cypress process gets closer to the max memory threshold and GC is not run. In my case, starting a new test could increase memory usage by 100-300mb, and put the app over the 2gb limit. To enable win.gc() for election, run cypress with the following with this environment variable set: ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc cypress run *cypress/plugins/index.js* - enable win.gc() for chrome // enables win.gc() for chrome module.exports = (on, config) => { on('before:browser:launch', (browser, launchOptions) => { if (browser.name === 'chrome') { // exposes window.gc() function that will manually force garbage collection launchOptions.args.push('--js-flags=--expose-gc'); } return launchOptions; });}; *cypress/support/index.js* afterEach(() => { cy.window().then(win => { // window.gc is enabled with --js-flags=--expose-gc chrome flag if (typeof win.gc === 'function') { // run gc multiple times in an attempt to force a major GC between tests win.gc(); win.gc(); win.gc(); win.gc(); win.gc(); } });}); I exactly tried this and i also tried to set the ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc with export on my linux system, but neither can i use window.gc in the developer tools console nor does the gc work on the tests. I'm using chrome version: 104.0.5112.101 (Official Build) (64-bit) Does anyone of you know if this is not supported by chrome anymore? — Reply to this email directly, view it on GitHub <#8525 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFKDNFEXSA2RN64OOLVAMVLV3YUPVANCNFSM4RAORJVQ> . You are receiving this because you commented.Message ID: ***@***.***>

CoryDanielson · 2022-08-30T16:50:04Z

actly tried this and i also tried to set the ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc with export on my linux system, but neither can i use window.gc in the developer tools console nor does the gc work on the tests.
I'm using chrome version: 104.0.5112.101 (Official Build) (64-bit)
Does anyone of you know if this is not supported by chrome anym

@MarcusElevait window.gc still works fine for me in Chrome 104

CoryDanielson · 2022-08-30T16:52:41Z

@MarcusElevait in the latest version of Cypress, cypress/plugins/index.js does not work the same as it used to. If you copy/pasted the code from this ticket into plugins/index.js that may be the problem

See https://docs.cypress.io/guides/references/migration-guide#Plugins-File-Removed

JohnSmithyy · 2022-08-31T02:51:06Z

What is the command to collect garbage in chrome 105? Window/win.gc() no longer work for me :?

CoryDanielson · 2022-08-31T03:03:51Z

@JohnSmithyy window.gc() works fine for me in Chrome 105. (see screenshot below)

Have you followed the steps from the original post in this thread to enable window.gc()?

JohnSmithyy · 2022-08-31T03:10:34Z

@JohnSmithyy window.gc() works fine for me in Chrome 105. (see screenshot below)

Have you followed the steps from the original post in this thread to enable window.gc()?

I have and i get undefined errors but my cypress still crashes too :/

CoryDanielson · 2022-08-31T03:29:49Z

You'll have to debug and see if that plugin change to enable GC is being run, as you expect.

I'm on Cypress 9.3 still, so I can't give you exact code for how to load it in Cypress 10

MarcusElevait · 2022-08-31T11:16:19Z

So i found out what was doing wrong. I had another browser window open. So to try it out locally, you first have to close all chrome instances.

So i managed to do it while starting chrome from command line. But still struggling to use it with cypress. I can see, that the launchoptions.args now have the --js-flags=expose-gc option set and i also set the ELECTRON_EXTRA_LAUNCH_ARGS, but still the window.gc function is not present in the browser.

CoryDanielson · 2022-08-31T11:28:29Z

You may want to file a bug for window.gc not working. If you're using the code I provided in this GitHub thread it should work in Cypress 9 and below, but I don't know the exact code for Cypress 10

MarcusElevait · 2022-08-31T11:43:00Z

Okay now i managed to make it work. I guess it just works with cypress run, not with cypress open, right?
So we're using nx to run our cypress tests and i always tried it with the --watch option. But without it, it works fine.

Thanks for the help here.
Maybe some smaller related question. If i want every spec file to run the gc command after each test, is there a way to configure this? I don't want to add the afterEach in every spec file.

CoryDanielson · 2022-08-31T11:48:05Z

It works with Cypress open too. Maybe try -b chrome?

MarcusElevait · 2022-08-31T11:59:26Z

This was the command i've tried ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=expose_gc nx e2e explorait-e2e --browser=chrome --watch and it didn't work.

With this it worked ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=expose_gc nx e2e explorait-e2e --browser=chrome

But it's fine for me, i don't need it in the watch mode.

MarcusElevait · 2022-08-31T12:12:18Z

Sorry i was wrong, it doesn't work for both cases.
But i mean for you guys it is working, maybe i'm doing something wrong.

CoryDanielson · 2022-08-31T14:10:08Z

@MarcusElevait ELECTRON_EXTRA_LAUNCH_ARGS are for electron, not Chrome.

To set these flags for Chrome, you'll need to do it in your cypress.config.js (for Cypress 10).

Based on this section of the docs, it should look something like this for Cypress 10

// cypress.config.js
const { defineConfig } = require('cypress')

module.exports = defineConfig({
    // setupNodeEvents can be defined in either
    // the e2e or component configuration
    e2e: {
        setupNodeEvents(on, config) {
            on('before:browser:launch', (browser = {}, launchOptions) => {
                // Chrome is used by default for `cypress open`
                // Electron is used for `cypress run` but the command line flags are modified by ELECTRON_EXTRA_LAUNCH_ARGS environment variable
                if (browser.name === 'chrome') {
                    // exposes window.gc() function that will manually force garbage collection
                    launchOptions.args.push('--js-flags=--expose-gc');
                }

                return launchOptions;
            })
        }
    }
})

Once that is set up, you'll need to add a global afterEach hook to your Support File that calls window.gc().

// cypress/support/e2e.js
afterEach(() => {
    cy.window().then(win => {
        // window.gc is enabled with --js-flags=--expose-gc chrome flag
        // window.gc is enabled for electron with ELECTRON_EXTRA_LAUNCH_ARGS=--js-flags=--expose_gc
        if (typeof win.gc === 'function') {
            // run gc multiple times in an attempt to force a major GC between tests
            win.gc();
            win.gc();
            win.gc();
            win.gc();
            win.gc();
        }
    });
});

MarcusElevait · 2022-09-01T06:57:51Z

@CoryDanielson so much thanks for your help. I finally managed to make it work for our project and it really helps us a lot with damn bitbucket pipelines ;-)

praveenpandey02 · 2022-09-26T14:29:51Z

This was my plan B. Nice thread!

CoryDanielson · 2022-09-26T14:54:01Z

My browser still crashed even with window.gc being called. I ran all my tests with cypress open and it crashed after a few minutes. We don't have any issues with our tests running in Jenkins/CI where they are parallelized though.

Really just need to avoid this issue with the specific Chrome versions, until it's fixed by Chrome.

praveenpandey02 · 2022-09-26T14:57:14Z

I believe it is not a good idea to run all your tests in cypress open. It is actually not meant for it. I tried with a very simple solution first which was: break my spec file into smaller ones will less it blocks in them. Memory is flushed out when spec files are loaded but not when individual it blocks are loaded.

CoryDanielson changed the title ~~Add --js-flags=--expose-gc Chrome flag~~ Add --js-flags=--expose-gc Chrome flag and window.gc() calls Sep 8, 2020

CoryDanielson changed the title ~~Add --js-flags=--expose-gc Chrome flag and window.gc() calls~~ Enable and call window.gc by default via --js-flags=--expose-gc Sep 9, 2020

CoryDanielson changed the title ~~Enable and call window.gc by default via --js-flags=--expose-gc~~ Enable and call window.gc() by default via --js-flags=--expose-gc Sep 9, 2020

CoryDanielson changed the title ~~Enable and call window.gc() by default via --js-flags=--expose-gc~~ Reduce Memory Usage with window.gc() Sep 16, 2020

CoryDanielson mentioned this issue Oct 14, 2020

The Chromium Renderer process just crashed - 'sad face' Error #7204

Closed

jennifer-shehane added the type: performance 🏃‍♀️ Performance related label Oct 15, 2020

cypress-bot bot added the stage: proposal 💡 No work has been done of this issue label Oct 20, 2020

jennifer-shehane added the internal-priority label Oct 20, 2020

sync-by-unito bot assigned flotwig Oct 22, 2020

flotwig closed this as completed Oct 28, 2020

jennifer-shehane added stage: wontfix Cypress does not regard this as an issue or will not implement this feature and removed stage: proposal 💡 No work has been done of this issue internal-priority labels Dec 7, 2020

retypepassword mentioned this issue Jan 9, 2021

Cypress sometimes stalls/hangs with no output when running in Jenkins with Docker #8206

Closed

KeKs0r mentioned this issue Apr 7, 2021

Possible infinite loop while running in Jenkins with retries #14972

Closed

tt5015 mentioned this issue May 14, 2022

Cypress component tests go into a hanging state in CI environment #21505

Closed

CoryDanielson mentioned this issue Aug 31, 2022

'Aw, Snap' error screen shown when test runs over 250 seconds (approx) intermittant #1906

Closed

CoryDanielson mentioned this issue Oct 17, 2022

Out of memory issue after update a browsers to version 104+ (Chrome&Edge) #23391

Closed

mschile mentioned this issue Nov 22, 2022

After upgrading Docker image with chrome 107 tests started crashing #24552

Closed

abrugaro mentioned this issue Jan 31, 2023

[RFR] Fix tests for upstream konveyor/tackle-ui-tests#342

Merged

This was referenced Sep 11, 2024

Cypress 14: Breaking changes #29925

Open

breaking: remove log:memory:pressure and firefox:force:gc from the socket API #30222

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce Memory Usage with window.gc() #8525

Reduce Memory Usage with window.gc() #8525

CoryDanielson commented Sep 8, 2020 •

edited

Loading

CoryDanielson commented Oct 16, 2020 •

edited

Loading

CoryDanielson commented Oct 19, 2020 •

edited

Loading

flotwig commented Oct 23, 2020

CoryDanielson commented Oct 23, 2020

flotwig commented Oct 28, 2020

bahunov commented Dec 7, 2020 •

edited

Loading

kamiloski commented Dec 16, 2020

CoryDanielson commented Feb 5, 2021 •

edited

Loading

MarcusElevait commented Aug 30, 2022

bahunov commented Aug 30, 2022 via email

CoryDanielson commented Aug 30, 2022 •

edited

Loading

CoryDanielson commented Aug 30, 2022

JohnSmithyy commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022 •

edited

Loading

JohnSmithyy commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022

MarcusElevait commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022 •

edited

Loading

MarcusElevait commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022

MarcusElevait commented Aug 31, 2022

MarcusElevait commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022 •

edited

Loading

MarcusElevait commented Sep 1, 2022

praveenpandey02 commented Sep 26, 2022

CoryDanielson commented Sep 26, 2022 •

edited

Loading

praveenpandey02 commented Sep 26, 2022

Reduce Memory Usage with window.gc() #8525

Reduce Memory Usage with window.gc() #8525

Comments

CoryDanielson commented Sep 8, 2020 • edited Loading

CoryDanielson commented Oct 16, 2020 • edited Loading

CoryDanielson commented Oct 19, 2020 • edited Loading

flotwig commented Oct 23, 2020

CoryDanielson commented Oct 23, 2020

flotwig commented Oct 28, 2020

bahunov commented Dec 7, 2020 • edited Loading

kamiloski commented Dec 16, 2020

CoryDanielson commented Feb 5, 2021 • edited Loading

MarcusElevait commented Aug 30, 2022

bahunov commented Aug 30, 2022 via email

CoryDanielson commented Aug 30, 2022 • edited Loading

CoryDanielson commented Aug 30, 2022

JohnSmithyy commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022 • edited Loading

JohnSmithyy commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022

MarcusElevait commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022 • edited Loading

MarcusElevait commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022

MarcusElevait commented Aug 31, 2022

MarcusElevait commented Aug 31, 2022

CoryDanielson commented Aug 31, 2022 • edited Loading

MarcusElevait commented Sep 1, 2022

praveenpandey02 commented Sep 26, 2022

CoryDanielson commented Sep 26, 2022 • edited Loading

praveenpandey02 commented Sep 26, 2022

CoryDanielson commented Sep 8, 2020 •

edited

Loading

CoryDanielson commented Oct 16, 2020 •

edited

Loading

CoryDanielson commented Oct 19, 2020 •

edited

Loading

bahunov commented Dec 7, 2020 •

edited

Loading

CoryDanielson commented Feb 5, 2021 •

edited

Loading

CoryDanielson commented Aug 30, 2022 •

edited

Loading

CoryDanielson commented Aug 31, 2022 •

edited

Loading

CoryDanielson commented Aug 31, 2022 •

edited

Loading

CoryDanielson commented Aug 31, 2022 •

edited

Loading

CoryDanielson commented Sep 26, 2022 •

edited

Loading