Puppeteer spawns a runaway amount of Chromium instances #1047

tnolet · 2017-10-16T16:35:25Z

Over the last week I've encountered this problem twice. Once on an Ubuntu server and once on my local Macbook Pro. Running a dead simple Puppeteer scripts locks up my machine or a server. After a quick top it shows a huge amount of Chromium instances being started (see screenshot...).

The only way to stop this is a killall Chromium

Just for completeness, I've pasted the script that triggered it below, but it runs fine 9 out of 10 times and it probably not the cause of this.

// Get the title of a page and print it to the console.
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()
  await page.goto('https://www.google.com/')
  const title = await page.title()
  console.log(title)
  await browser.close()
})()

Steps to reproduce

Tell us about your environment:

Puppeteer version: 0.12.0 and 0.11.0.
Platform / OS version:

MacOS Sierra 10.12.6
Ubuntu 17.04 (GNU/Linux 4.10.0-35-generic x86_64)

What steps will reproduce the problem?

Regretfully, I can't reproduce it consistently

What is the expected result?

Run the Puppeteer script in a single Chromium instance. Then close the Chromium process when browser.close() is called.

What happens instead?

More and more Chromium instances are started, locking up the machine and forcing a killall chromium command.

The text was updated successfully, but these errors were encountered:

tnolet · 2017-10-16T16:49:11Z

EDIT: This is not the reason for the runaway instances. Will keep this comment for history, but please ignore.

Aha, this is related to #405 adding a semicolon to the require statement fixes this, i.e. require('puppeteer');. However, if you don't do this, you end up with a machine consumed by Chromium instances. We could close this but leave it for other people that run into the same problem.

JoelEinbinder · 2017-10-16T23:36:12Z

Are you using a transpiler? Your code should just fail with Class constructor Puppeteer cannot be invoked without 'new' because you are trying to run puppeteer as a function.

sukrosono · 2017-10-18T00:09:59Z

IMO the behavior of duck typing in javascript should ignore the absence of ;
I am also facing this issue, with

I can't reproduce it consistently
require directly to lib\Puppeteer on v0.12.0 and v0.13.0 alpha

The script to reproduce the issue is not good choice, google will never that page throw an error, 😄
I will inform here if i can reproduce it consistently. Suggesting label need investigation

tnolet · 2017-10-18T07:57:21Z

@brutalcrozt @JoelEinbinder my initial assumption was mistaken. I was wrapping the script in some other code (which moved the require statement) which threw me off. This behaviour of runaway instances still occurs, seemingly at random. Pretty nasty bug as I found my laptop with fans blazing at max. Had to do a hard restart to even get it to work again.

sukrosono · 2017-10-19T13:53:53Z

@tnolet did you also terminate puppeteer via CLI? I just try to close puppeteer in cli via ctrl C, chromium instance are still running:

entrptaher · 2017-10-20T17:24:49Z

Experienced similar with windows 7 and puppeteer v 0.12.
Thought I was the only one.
Ended up using spawn and the workaround,

var kill  = require('tree-kill');
process.on('exit', (code) => {
  kill(child.pid, 'SIGTERM', function(err){
    kill(child.pid, 'SIGKILL', function(err){
      process.kill(-child.pid);
    });
  });
});

You can feel my frustration in the code :D

pemrouz · 2017-10-22T15:33:07Z

I'm also getting a similar result with trying to run lots of puppeteer tests in parallel. I believe you can reliably reproduce this by creating lots of copies of the same basic test that passes (and exits cleanly) on it's own, and then run tap ./*.test.js --jobs 999 to run them all in parallel. I get a proportional amount of "There is no running instance of the task" and stray Chromium processes left over.

tnolet · 2017-10-23T10:12:25Z

@brutalcrozt No, I did not terminate with the CLI. My runs are started and stopped programmatically, basically in a cron job. I'm actually a bit scared running Puppeteer in this way now. I blew up access to an EC2 instance that was not even responding to SSH anymore...

@pemrouz your use case is similar to mine. There is a chance in my scheduling that multiple of the same tests are running at the same time.

aslushnikov · 2017-10-24T22:47:37Z

@pemrouz @tnolet @entrptaher do you guys use custom user data dir to launch puppeteer? I have a fix for this case.

This patch starts force-killing chromium regardless of custom userdata directory when the node process exits. References puppeteer#1047

) This patch starts force-killing chromium regardless of custom userdata directory when the node process exits. References #1047

entrptaher · 2017-10-24T23:58:04Z

I don't use user data dir.

And, I think taskkill failed to close child processes every time only on a windows 7. If the process creates it's own child processes, then somehow windows needs more instruction on killing the child.

I see your fix, will try it and let know.

sukrosono · 2017-10-25T00:26:50Z

I also don't use user data dir. I will inform here if still get this after applying that patch.

aslushnikov · 2017-10-25T08:49:01Z

@brutalcrozt @entrptaher If you guys live inside docker, it might be related to #615. Check out it's proposed workaround to use dumb-init.

tnolet · 2017-10-25T11:34:46Z

@aslushnikov No Docker, no Windows, no User Data Dir. Just plain install MacOS and Ubuntu.

sukrosono · 2017-10-25T13:50:15Z

my case , no docker, on Windows, no user data dir.

pemrouz · 2017-10-25T20:16:22Z

@aslushnikov just tried out your latest, same issue. Windows, no docker. Running 10 tests (the same basic test) in parallel and I get a bunch of the following error messages at the start:

ERROR: The process with PID 3208 (child process of PID 7704) could not be terminated.
Reason: There is no running instance of the task.
ERROR: The process with PID 29340 (child process of PID 24324) could not be terminated.
Reason: The operation attempted is not supported.
ERROR: The process with PID 27488 (child process of PID 22536) could not be terminated.
Reason: There is no running instance of the task.
ERROR: The process with PID 10664 (child process of PID 12712) could not be terminated.
Reason: There is no running instance of the task.
ERROR: The process with PID 19276 (child process of PID 30112) could not be terminated.
Reason: There is no running instance of the task.
ERROR: The process with PID 26540 (child process of PID 23380) could not be terminated.
Reason: There is no running instance of the task.
ERROR: The process with PID 18768 (child process of PID 15748) could not be terminated.
Reason: There is no running instance of the task.

At the end of each test where I do: page.close().then(d => browser.close()) it rejects with the following:

Command failed: taskkill /pid 27560 /T /F ERROR: The process with PID 16016 (child process of PID 27560) could not be terminated.  Reason: The operation attempted is not supported.

Does puppeteer attempt any reuse? i.e. try to connect to an existing Chromium instance if one exists? Or perhaps the Chromium instances are attempting to reuse behind the scenes? If so, is there a way to force complete isolation across runs, similar to going incognito?

aslushnikov · 2017-10-25T20:35:54Z

@pemrouz can you please share your tests so that I can take a look (and maybe run them)?

entrptaher · 2017-10-25T21:02:18Z

Can we use some code from fkill or tree-kill package etc?

aslushnikov · 2017-10-25T21:08:29Z

@entrptaher both fkill and tree-kill are doing the same thing as we do on windows: calling taskkill /pid /T /F.

entrptaher · 2017-10-25T21:29:03Z

I am talking about the child processes. You know that Chrome is actually creates a lot of child processes. If we kill the child processes, it might help killing the parent process too. Even though the general logic is that killing parent process will kill child processes.

We can see sometimes on windows/mac, on rare case, it just leaves child processes. Can we kill child processes instead and then kill parent?

…ppeteer#1155) This patch starts force-killing chromium regardless of custom userdata directory when the node process exits. References puppeteer#1047

pemrouz · 2017-11-12T21:30:52Z

I can't share the application tests unfortunately, but I've just checked in an example on a branch on the underlying framework here (rijs/fullstack#puppeteer). I've duplicated the basic test 10 times. Running the following command highlights the issue every time:

tap ./tests/basic.test* --jobs 999

entrptaher · 2017-11-13T03:08:38Z

I found something more about the problem. Let me share one case. Just one of the many that could be more helpful to debug.

I have an electron app (or a nodeJS app),
where I am spawning a child script,
from that child script, I'm calling puppeteer.

Now, when I am loading a website on puppeteer, I suddenly wanted to close the electron app or the child script. The main program exits, the child program exits, but the chrome remains,

If the website loaded properly, or there is no evaluation/command going on, then chrome also exits.
But if there is some command going on, then no matter what state nodeJS has (crash/close), the chrome does not exit.

It could be due to async/await, but there should be a check for this kind of thing.

Hope the case helps understanding the problem. I'll try to see if I can provide a sample code for this.

Edit: I've created a seperate issue for this.

This patch starts listening to SIGTERM event to gracefully shutdown chromium instance. References puppeteer#1047.

aslushnikov · 2017-11-16T19:45:48Z

@pemrouz your issue should be fixed with #1402, given that you run under Mac or Linux. I'm not sure if this would help with windows though.

SIGTERM signal is widely used to notify application that it will be shut down. This patch starts listening to SIGTERM event to gracefully retire chromium instance. References #1047.

pemrouz · 2017-12-10T16:13:47Z

@pemrouz your issue should be fixed with #1402, given that you run under Mac or Linux. I'm not sure if this would help with windows though.

Thanks @aslushnikov. This should be useful for our CI at least. Do you know if it's possible this can be fixed on Windows too however? (that's where most dev happens)

aslushnikov · 2018-01-12T11:06:04Z

@pemrouz we've fixed windows since then; please let me know how puppeteer v1.0.0 works for you.

I believe this is fixed now. Please don't hesitate to write back or refile if the behavior still reproduces for you.

simoncpu · 2018-01-28T05:16:55Z

I'm using v1.0.0 and my script doesn't even need to crash... if the user presses Ctrl+C and bypasses browser.close(), ps will show that Chrome is still running.

EDIT: turns out that I set handleSIGINT to false. Please ignore my comment.

gorshkov-leonid · 2018-08-10T22:44:24Z

I saw error again.

{
  "name": "puppeteer",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "jest --detectOpenHandles"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "jest": "^23.4.2",
    "jest-puppeteer": "^3.2.1",
    "puppeteer": "^1.6.2"
  }
}

There are not zombie processes, but I don't know how to finish tests correctly. Error messages annoy me

ernestochaves · 2018-12-06T22:29:51Z

I found something more about the problem. Let me share one case. Just one of the many that could be more helpful to debug.

I have an electron app (or a nodeJS app),
where I am spawning a child script,
from that child script, I'm calling puppeteer.

Now, when I am loading a website on puppeteer, I suddenly wanted to close the electron app or the child script. The main program exits, the child program exits, but the chrome remains,

If the website loaded properly, or there is no evaluation/command going on, then chrome also exits.
But if there is some command going on, then no matter what state nodeJS has (crash/close), the chrome does not exit.

It could be due to async/await, but there should be a check for this kind of thing.

Hope the case helps understanding the problem. I'll try to see if I can provide a sample code for this.

Edit: I've created a seperate issue for this.

Hi.

Was this resolved for you ever on windows? I still get the errors running windows 10.

gorshkov-leonid · 2018-12-07T04:49:00Z

I don't remember how I fixed it, but I fixed it Problem was not in puppeteer.

Choongkyu · 2019-08-03T20:25:23Z

why is this issue closed? I'm still seeing a runaway amount of Chromium helper instances when running puppeteer rendering this completely useless.

JoelEinbinder · 2019-08-03T20:43:13Z

@zebralight can you please file a separate issue. While you might have the same symptom in this issue, it’s highly unlikely that you have the same root cause.

entrptaher mentioned this issue Oct 20, 2017

Puppeteer failing at killChrome #1045

Closed

aslushnikov mentioned this issue Oct 24, 2017

fix(Launcher): force-kill chrome on process exit and interruption #1155

Merged

aslushnikov added a commit that referenced this issue Oct 24, 2017

fix(Launcher): force-kill chrome on process exit and interruption (#1155

ab9b34c

) This patch starts force-killing chromium regardless of custom userdata directory when the node process exits. References #1047

This was referenced Nov 1, 2017

Expose underlying chrome process #1238

Closed

browser.close() didn't close browser #1245

Closed

entrptaher mentioned this issue Nov 13, 2017

Chromium process does not close if main nodejs process crashes #1367

Closed

aslushnikov added a commit to aslushnikov/puppeteer that referenced this issue Nov 16, 2017

fix(Launcher): handle SIGTERM by default

647d5f9

This patch starts listening to SIGTERM event to gracefully shutdown chromium instance. References puppeteer#1047.

aslushnikov mentioned this issue Nov 16, 2017

fix(Launcher): handle SIGTERM by default #1402

Merged

aslushnikov closed this as completed Jan 12, 2018

MartijnR mentioned this issue Jun 22, 2018

OC Only: Headless mode with autoqueries kobotoolbox/enketo-express#533

Closed

liza-mae mentioned this issue Aug 16, 2018

Issues exiting chromedriver on windows elastic/kibana#22094

Closed

enketo-issue-mover mentioned this issue Sep 10, 2018

OC Only: Headless mode with autoqueries OpenClinica/enketo-express-oc#47

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Puppeteer spawns a runaway amount of Chromium instances #1047

Puppeteer spawns a runaway amount of Chromium instances #1047

tnolet commented Oct 16, 2017 •

edited

Loading

tnolet commented Oct 16, 2017 •

edited

Loading

JoelEinbinder commented Oct 16, 2017

sukrosono commented Oct 18, 2017

tnolet commented Oct 18, 2017

sukrosono commented Oct 19, 2017

entrptaher commented Oct 20, 2017 •

edited

Loading

pemrouz commented Oct 22, 2017

tnolet commented Oct 23, 2017 •

edited

Loading

aslushnikov commented Oct 24, 2017

entrptaher commented Oct 24, 2017 •

edited

Loading

sukrosono commented Oct 25, 2017

aslushnikov commented Oct 25, 2017

tnolet commented Oct 25, 2017

sukrosono commented Oct 25, 2017

pemrouz commented Oct 25, 2017

aslushnikov commented Oct 25, 2017

entrptaher commented Oct 25, 2017

aslushnikov commented Oct 25, 2017

entrptaher commented Oct 25, 2017

pemrouz commented Nov 12, 2017

entrptaher commented Nov 13, 2017 •

edited

Loading

aslushnikov commented Nov 16, 2017

pemrouz commented Dec 10, 2017

aslushnikov commented Jan 12, 2018

simoncpu commented Jan 28, 2018 •

edited

Loading

gorshkov-leonid commented Aug 10, 2018 •

edited

Loading

ernestochaves commented Dec 6, 2018

gorshkov-leonid commented Dec 7, 2018

Choongkyu commented Aug 3, 2019

JoelEinbinder commented Aug 3, 2019

Puppeteer spawns a runaway amount of Chromium instances #1047

Puppeteer spawns a runaway amount of Chromium instances #1047

Comments

tnolet commented Oct 16, 2017 • edited Loading

Steps to reproduce

tnolet commented Oct 16, 2017 • edited Loading

JoelEinbinder commented Oct 16, 2017

sukrosono commented Oct 18, 2017

tnolet commented Oct 18, 2017

sukrosono commented Oct 19, 2017

entrptaher commented Oct 20, 2017 • edited Loading

pemrouz commented Oct 22, 2017

tnolet commented Oct 23, 2017 • edited Loading

aslushnikov commented Oct 24, 2017

entrptaher commented Oct 24, 2017 • edited Loading

sukrosono commented Oct 25, 2017

aslushnikov commented Oct 25, 2017

tnolet commented Oct 25, 2017

sukrosono commented Oct 25, 2017

pemrouz commented Oct 25, 2017

aslushnikov commented Oct 25, 2017

entrptaher commented Oct 25, 2017

aslushnikov commented Oct 25, 2017

entrptaher commented Oct 25, 2017

pemrouz commented Nov 12, 2017

entrptaher commented Nov 13, 2017 • edited Loading

aslushnikov commented Nov 16, 2017

pemrouz commented Dec 10, 2017

aslushnikov commented Jan 12, 2018

simoncpu commented Jan 28, 2018 • edited Loading

gorshkov-leonid commented Aug 10, 2018 • edited Loading

ernestochaves commented Dec 6, 2018

gorshkov-leonid commented Dec 7, 2018

Choongkyu commented Aug 3, 2019

JoelEinbinder commented Aug 3, 2019

tnolet commented Oct 16, 2017 •

edited

Loading

tnolet commented Oct 16, 2017 •

edited

Loading

entrptaher commented Oct 20, 2017 •

edited

Loading

tnolet commented Oct 23, 2017 •

edited

Loading

entrptaher commented Oct 24, 2017 •

edited

Loading

entrptaher commented Nov 13, 2017 •

edited

Loading

simoncpu commented Jan 28, 2018 •

edited

Loading

gorshkov-leonid commented Aug 10, 2018 •

edited

Loading