Latest testcafe docker image fails to instantiate chromium intermittently

[monicainde]

What is your Test Scenario?

Executing tests in testcafe docker image on chromium

What is the Current behavior?

We execute testcafe tests frequently with our CI tool and since the latest release of the image on docker hub, we see that many of our executions intermittently fail with the below error:

testcafe chromium:headless src --reporter xunit:reports/jReport.xml,html:reports/report.html --video reports/videos --video-options failedOnly=true -q
  ERROR Cannot establish one or more browser connections.
  1 of 1 browser connections have not been established:
  - chromium:headless
  
  Hints:
  - Increase the value of the "browserInitTimeout" option if it is too low (currently: 2 minutes for local browsers and 6 minutes for remote browsers). This option determines how long TestCafe waits for browsers to be ready.
  - The error can also be caused by network issues or remote device failure. Make sure that your network connection is stable and you can reach the remote device.
  
  Type "testcafe -h" for help.
  npm ERR! code ELIFECYCLE
  npm ERR! errno 1
  npm ERR! [email protected] e2e:ci-staging: `testcafe chromium:headless src --reporter xunit:reports/jReport.xml,html:reports/report.html --video reports/videos --video-options failedOnly=true -q`
  npm ERR! Exit status 1
  npm ERR! 
  npm ERR! Failed at the [email protected] e2e:ci-staging script.
  npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
  
  npm ERR! A complete log of this run can be found in:
  npm ERR! 

What is the Expected behavior?

The browser instance should be created without flakiness.

Steps to Reproduce:

1. Set up a test execution job which executes frequently e.g. every half an hour on any CI tool
2. CI job steps looks like below:

  checkout test code
  npm install
  docker run --volume /bamboo-agent-home/JOB1:/data --workdir /data --rm --entrypoint npm testcafe/testcafe run e2e:ci-prelive

4. Observe that some executions fail with above error in description.

Note: to validate that the issue was with the latest testcafe image, we changed docker image of the ci jobs to testcafe/testcafe:1.15.0 and this issue disappeared. As suggested in the error discription, we also tried increasing the browserInitTimeout but it made no difference.

Your Environment details:

• testcafe version: 1.15.1
• node.js version: node 12
• command-line arguments: --reporter xunit:reports/jReport.xml,html:reports/report.html --video reports/videos --video-options failedOnly=true -q
• browser name and version: chromium:headless
• platform and version: Linux

[avi0711]

Facing this issue too, using the latest image no browser connectivity and all testcases failing...

[avi0711]

this is also happening with firefox too...

[AlexKamaev]

I confirm that the Cannot establish one or more browser connections error appears in the latest TestCafe image. The error does not occur on every test run, but it occurs quite often. I was not able to reproduce it with the 1.15.0 image. I was not able to reproduce it with Firefox either.

[HHHartmann]

We are facing the same problem.
What response is needed to get this bug on? The issue is confirmed.

[cvietor]

+1

[AlexKamaev]

@HHHartmann The "Need Response" label is our internal label, which helps us track user activity in a thread.

I will increase the priority of this issue.

[Jeredian]

We have the same issue with the docker image of testcafe/testcafe:latest and for testcafe/testcafe:1.15.0 all works fine.
I compared the dockerfile for version 1.15.0 and for newer, there is a difference between port exposing
EXPOSE map[1337/tcp:{} 1338/tcp:{}] vs EXPOSE 1337 1338

Can it be the reason why it goes down?

[aleks-pro]

Hello @Jeredian ,

Thank you for the hint. We will check this.

[di5ko]

+1, also experiencing this in our Gitlab CI/CD pipeline, intermittently on random places. Not on each pipeline run, but randomly.

Environment details:

testcafe version: 1.15.0
node.js version: node 14
command-line arguments: testcafe "chromium --headless --no-sandbox --disable-dev-shm-usage --window-size='1920,1080'" /srv/tests/test-subdirectory/
browser name and version: chromium:headless
platform and version: Alpine Linux (node:14-alpine)

@aleks-pro it's been almost 2 months since the last update, is there any progress on this?

[github-actions]

No updates yet. Once we get any results, we will post them in this thread.

[mhanci92]

We are experiencing this issue with Docker & TestCafe Version 1.18.1. We run our end-2-end tests within an CI/CD environment (Jenkins) as well.

Environment details:

testcafe docker image version: 1.18.1
browser name and version: chromium
platform and version: Linux

Failing with the same error:

ERROR Cannot establish one or more browser connections.
1 of 1 browser connections have not been established:
- chromium

Hints:
- Increase the value of the "browserInitTimeout" option if it is too low (currently: 2 minutes for local browsers and 6 minutes for remote browsers). This option determines how long TestCafe waits for browsers to be ready.
- The error can also be caused by network issues or remote device failure. Make sure that your network connection is stable and you can reach the remote device.

Type "testcafe -h" for help.

[Aleksey28]

Hi @mhanci92,
Please create a new issue with a full description of the problem.