Running large test sets with chrome headless

[cullsin]

Hi,

We are executing almost 2000 karma test cases. I am getting the following message. Can you please guide me.

12 06 2017 10:37:34.562:WARN [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Disconnected (1 times)
HeadlessChrome 0.0.0 (Linux 0.0.0) ERROR
Disconnectedundefined
HeadlessChrome 0.0.0 (Linux 0.0.0) ERROR
Disconnectedundefined
HeadlessChrome 0.0.0 (Linux 0.0.0): Executed 1900 of 2600 (skipped 68) DISCONNECTED (3 mins 30.945 secs / 50.046 secs)

Thanks,
Raja K

[rafinskipg]

same issue (running on , latest chrome and also with version 58)

[weikinhuang]

Happening for us as well, we have ~2000 tests. It passes completely fine when run in OSX, however when we run it in CI with chrome and karma in a docker container, it will fail about 1500 tests in.

[weikinhuang]

Additional notes: our Dockerfile is based off node:6.11-slim and google-chrome-stable installed via https://dl.google.com/linux/chrome/deb/

[javigd]

Hi @weikinhuang we experienced the exact same problem as you and managed to fix it by chance by defining a custom launcher for it:

                browsers: ["Chrome-headless"],
                customLaunchers: {
                    "Chrome-headless": {
                        base: 'Chrome',
                        flags: ['--headless', '--remote-debugging-port=9222', '--no-sandbox']
                    }
                }

still a mystery why this solves the problem. Hope this helps 👍

[dafuster]

I was able to overcome the Chrome disconnects by adding this additional flag in my custom launcher:
--js-flags="--max_old_space_size=4096"

[rogeriopvl]

@javigd the only difference between your custom launcher and the default one is that it has the --no-sandbox flag and doesn't have the --disable-gpu flag. So I would say that one of these may be causing the issue.

I'm going to need help investigating this, since I don't have large enough battery of tests to replicate
the issue 🤔

[martynchamberlin]

Just wanted to weigh in and say that we were having this issue with 445 tests. Passing --no-sandbox also seems to have fixed the problem.

[martynchamberlin]

Update: scratch that. Only about half the time does running our test suite cause this error, so it's hard to know when it's actually been fixed. Still getting this error.

[x3igh7]

@dafuster what exactly does your customer launcher look like with --js-flags="--max_old_space_size=4096" ? doesn't really seem to do anything for me

[macjohnny]

we are also experiencing the issue described.
Using karma-spec-reporter we get an information about which test causes an issue.

[mashhoodr]

Also experiencing this problem, we have around 5700 tests but it breaks on 5600ish. Started happening a 2-3 weeks ago. Running fine on OSX, causing problem inside headless chrome docker container.

[jressey]

This is probably not a satisfying response but I ran in to this today and solved by restarting my Mac.

[StoneMorKnight]

Also experiencing this issue with only 73 tests. Any movement on this? Restarting my Mac fixed it once but our environment is too heavy to rebuild constantly to get this working properly. Any other workarounds or solutions?

[cyrus-mc]

Experiencing this issue as well;

The oddity is it only fails during an actual docker build. If I exec into the intermediate container just prior to the RUN tests command and run the tests, it succeeds.

[ajorquera]

we had this issue the last couple of days. We solve it by using puppeteer. Hope it helps

// Karma configuration file, see link for more information
// https://karma-runner.github.io/1.0/config/configuration-file.html

process.env.CHROME_BIN = require('puppeteer').executablePath()

module.exports = function (config) {
  config.set({
    basePath: '',
    frameworks: ['jasmine', '@angular-devkit/build-angular'],
    plugins: [
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-jasmine-html-reporter'),
      require('karma-coverage-istanbul-reporter'),
      require('@angular-devkit/build-angular/plugins/karma'),
      require('karma-scss-preprocessor')
    ],
    client: {
      clearContext: false // leave Jasmine Spec Runner output visible in browser
    },
    files: [{
        pattern: './test.ts',
        watched: false
      },
      {
        pattern: './styles/**/*.*',
        watched: true,
        included: true,
        served: true
      },
      {
        pattern: '../node_modules/cookieconsent/build/cookieconsent.min.js',
        watched: true,
        included: true,
        served: true
      },
      {
        pattern: '../node_modules/cookieconsent/build/cookieconsent.min.css',
        watched: true,
        included: true,
        served: true
      }
    ],
    preprocessors: {
      './styles/**/*.*': ['scss']
    },
    coverageIstanbulReporter: {
      dir: require('path').join(__dirname, '../coverage'),
      reports: ['html', 'lcovonly'],
      fixWebpackSourcePaths: true
    },
    reporters: ['progress', 'kjhtml'],
    port: 9876,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: true,
    singleRun: false,
    browsers: ['ChromeHeadlessNoSandbox'],
    customLaunchers: {
      ChromeHeadlessNoSandbox: {
        base: 'ChromeHeadless',
        flags: ['--no-sandbox']
      }
    },
  });
};

[erwinheitzman]

We experienced the same and for us the cause was a memory leak in our tests which was causing these seemingly random errors while in reality we where sometimes hitting the memory limit. We fixed this by using the npm package karma-parallel after which we had the time to fix the actual memory leak. Hope this helps anyone struggling with this.

[ns89-web]

We experienced the same and for us the cause was a memory leak in our tests which was causing these seemingly random errors while in reality we where sometimes hitting the memory limit. We fixed this by using the npm package karma-parallel after which we had the time to fix the actual memory leak. Hope this helps anyone struggling with this.

Could you please explain how you tried to identify the memory leaks in your tests?

[hhyyg]

I faced the same problem. Removing the extra component and module imports from the test fixed it. I also referred to this article. https://www.forbes.com/sites/forbesdigitalgroup/2019/05/14/improve-your-angular-jasmine-unit-test-speeds-by-500/

[cullsin]

Thank you so much. Raja K

…

On Sun, 6 Dec 2020 at 07:49, Hiroka Yago ***@***.***> wrote: I faced the same problem. Removing the extra component and module imports from the test fixed it. I also referred to this article. https://www.forbes.com/sites/forbesdigitalgroup/2019/05/14/improve-your-angular-jasmine-unit-test-speeds-by-500/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#137 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHGJPUAOGEYGMTP5FT7PDTSTLS2PANCNFSM4DO5YLAQ> .