Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Only first test runs with Firefox 124.0 #29172

Closed
MikeMcC399 opened this issue Mar 20, 2024 · 8 comments · Fixed by #29179
Closed

Only first test runs with Firefox 124.0 #29172

MikeMcC399 opened this issue Mar 20, 2024 · 8 comments · Fixed by #29179
Assignees
@AtofStryker
Labels
browser: firefox type: bug

Comments

@MikeMcC399
Copy link
Contributor

MikeMcC399 commented Mar 20, 2024 •
edited
Loading

Current behavior

Cypress fails running tests with Firefox 124.0.

It is reproducible using the standard set of examples on https://github.com/cypress-io/cypress-example-kitchensink.

On Windows 11 after passing the first test, it fails running the next test with:

Error: connect ECONNREFUSED 127.0.0.1:62907
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1495:16)

On Ubuntu 22.04 it hung first before failing.

Desired behavior

Cypress should successfully run the set of example tests from https://github.com/cypress-io/cypress-example-kitchensink using the latest version of Firefox (currently 124.0).

Test code to reproduce

Execute

git clone https://github.com/cypress-io/cypress-example-kitchensink
cd cypress-example-kitchensink
npm ci
npm start

In separate terminal window

npx cypress run --browser firefox

Cypress Version

13.7.0

Browser / version

Firefox 124.0

Node version

v20.11.1 LTS

Operating System

Ubuntu 22.04.4 LTS and Windows 11

Debug Logs

$ npx cypress run --browser firefox
It looks like this is your first time using Cypress: 13.7.0

✔  Verified Cypress! C:\Users\mikem\AppData\Local\Cypress\Cache\13.7.0\Cypress

Opening Cypress...


DevTools listening on ws://127.0.0.1:62899/devtools/browser/b0d36021-cd42-42d3-9cce-8945a276ae10

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        13.7.0                                                                         │
  │ Browser:        Firefox 124 (headless)                                                         │
  │ Node Version:   v20.11.1 (C:\Program Files\nodejs\node.exe)                                    │
  │ Specs:          20 found (1-getting-started/todo.cy.js, 2-advanced-examples/actions.cy.js, 2-a │
  │                 dvanced-examples/aliasing.cy.js, 2-advanced-examples/assertions.cy.js, 2-advan │
  │                 ced-examples/connectors.cy.js, 2-advanced-examples/cookies.cy.js, 2-advanced-e │
  │                 xamples/cypress_api.cy....)                                                    │
  │ Searched:       cypress/e2e/**/*.cy.{js,jsx,ts,tsx}                                            │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  1-getting-started/todo.cy.js                                                   (1 of 20)


  example to-do app
    √ displays two todo items by default (278ms)
    √ can add new todo items (482ms)
    √ can check off an item as completed (270ms)
    with a checked task
      √ can filter for uncompleted tasks (361ms)
      √ can filter for completed tasks (364ms)
      √ can delete all completed tasks (341ms)


  6 passing (4s)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        6                                                                                │
  │ Passing:      6                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     3 seconds                                                                        │
  │ Spec Ran:     1-getting-started/todo.cy.js                                                     │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  2-advanced-examples/actions.cy.js                                              (2 of 20)
There was an error reconnecting to the Chrome DevTools protocol. Please restart the browser.

Error: connect ECONNREFUSED 127.0.0.1:62907
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1495:16)

Other

The issue does not occur with Electron v118 or Chrome 123.0.6312.59.

Related issues

Workaround

For environments able to run Cypress Docker images, use an image with Firefox 123 or earlier.

For example in GitHub Actions, use:

    runs-on: ubuntu-22.04
    container:
      image: cypress/browsers:node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1
      options: --user 1001

Note that specifying the user ID is essential when testing against Firefox otherwise Cypress is unable to access Firefox, due to Firefox security restrictions.
This was referenced Mar 20, 2024
Closed
Merged
Merged
@jennifer-shehane
Copy link
Member

I'm able to reproduce this issue. My logs:



────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  2-advanced-examples/actions.cy.js                                              (2 of 20)
Still waiting to connect to Firefox, retrying in 1 second (attempt 18/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 19/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 20/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 21/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 22/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 23/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 24/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 25/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 26/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 27/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 28/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 29/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 30/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 31/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 32/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 33/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 34/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 35/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 36/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 37/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 38/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 39/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 40/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 41/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 42/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 43/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 44/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 45/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 46/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 47/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 48/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 49/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 50/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 51/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 52/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 53/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 54/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 55/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 56/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 57/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 58/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 59/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 60/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 61/62)
Still waiting to connect to Firefox, retrying in 1 second (attempt 62/62)
Cypress failed to make a connection to the Chrome DevTools Protocol after retrying for 50 seconds.

This usually indicates there was a problem opening the Firefox browser.

The CDP port requested was 58731.

Error: Could not find url target in browser about:blank. Targets were []
    at <embedded>:4291:96620
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async n (<embedded>:4291:95767)
    at async Object.connectToNewSpec (<embedded>:4291:199323)
    at async Object.O (<embedded>:4297:7581)
    at async Object.connectToNewSpec (<embedded>:4297:27035)
    at async v.relaunchBrowser (<embedded>:4363:37912)

Debug logs

Still waiting to connect to Firefox, retrying in 1 second (attempt 35/62)
  cypress:server:browsers:browser-cri-client error finding browser target, maybe retrying { delay: 1000, err: Error: Could not find url target in browser about:blank. Targets were [] at <embedded>:4291:96620     at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async n (<embedded>:4291:95767) at async Object.connectToNewSpec (<embedded>:4291:199323) at async Object.O (<embedded>:4297:7581) at async Object.connectToNewSpec (<embedded>:4297:27035) at async v.relaunchBrowser (<embedded>:4363:37912) } +2ms
  cypress:proxy:http:util:prerequests Never received pre-request or url without pre-request for request GET-https://firefox.settings.services.mozilla.com/v1/buckets/monitor/collections/changes/changeset?_expected=0 after waiting 500ms. Continuing without one. +21s
  cypress:network:cors Parsed URL { port: '443', protocol: 'https:', subdomain: 'firefox.settings.services', domain: 'mozilla', tld: 'com' } +21s
  cypress:server:remote-states getting remote state: undefined for: https://firefox.settings.services.mozilla.com/v1/buckets/monitor/collections/changes/changeset?_expected=0 +21s
  cypress:network:cors Parsed URL { port: '59351', protocol: 'http:', subdomain: null, domain: '', tld: 'localhost' } +0ms
  cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } for: http://localhost:59351 +0ms
  cypress:server:stream_buffer stream buffer writeable final called +21s
  cypress:network:agent addRequest called { isHttps: true, href: 'https://firefox.settings.services.mozilla.com/v1/buckets/monitor/collections/changes/changeset?_expected=0' } +21s
  cypress:network:agent got family { family: 4, href: 'https://firefox.settings.services.mozilla.com/v1/buckets/monitor/collections/changes/changeset?_expected=0' } +0ms
  cypress:server:request received status code & headers on request { requestId: 'request129', statusCode: 200, headers: { 'content-type': 'application/json' } } +21s
  cypress:server:request successful response received { requestId: 'request129' } +1ms
  cypress:net-stubbing:server:intercept-response InterceptResponse { req: { url: '/v1/buckets/monitor/collections/changes/changeset?_expected=0' }, request: undefined } +21s
  cypress:network:cors Parsed URL { port: '59351', protocol: 'http:', subdomain: null, domain: '', tld: 'localhost' } +11ms
  cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } for: http://localhost:59351 +11ms
  cypress:server:remote-states getting primary remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } +0ms
  cypress:network:cors Parsed URL { port: '59351', protocol: 'http:', subdomain: null, domain: '', tld: 'localhost' } +1ms
  cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } for: http://localhost:59351 +1ms
  cypress:proxy:http:util:prerequests Never received pre-request or url without pre-request for request GET-https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/normandy-recipes-capabilities/changeset?_expected=1710892864179 after waiting 500ms. Continuing without one. +528ms
  cypress:network:cors Parsed URL { port: '443', protocol: 'https:', subdomain: 'firefox.settings.services', domain: 'mozilla', tld: 'com' } +516ms
  cypress:server:remote-states getting remote state: undefined for: https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/normandy-recipes-capabilities/changeset?_expected=1710892864179 +516ms
  cypress:network:cors Parsed URL { port: '59351', protocol: 'http:', subdomain: null, domain: '', tld: 'localhost' } +0ms
  cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } for: http://localhost:59351 +0ms
  cypress:server:stream_buffer stream buffer writeable final called +528ms
  cypress:network:agent addRequest called { isHttps: true, href: 'https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/normandy-recipes-capabilities/changeset?_expected=1710892864179' } +527ms
  cypress:network:agent got family { family: 4, href: 'https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/normandy-recipes-capabilities/changeset?_expected=1710892864179' } +0ms
  cypress:server:request received status code & headers on request { requestId: 'request130', statusCode: 200, headers: { 'content-type': 'application/json' } } +527ms
  cypress:server:request successful response received { requestId: 'request130' } +0ms
  cypress:net-stubbing:server:intercept-response InterceptResponse { req: { url: '/v1/buckets/main/collections/normandy-recipes-capabilities/changeset?_expected=1710892864179' }, request: undefined } +526ms
  cypress:network:cors Parsed URL { port: '59351', protocol: 'http:', subdomain: null, domain: '', tld: 'localhost' } +10ms
  cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } for: http://localhost:59351 +10ms
  cypress:server:remote-states getting primary remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } +0ms
  cypress:network:cors Parsed URL { port: '59351', protocol: 'http:', subdomain: null, domain: '', tld: 'localhost' } +1ms
  cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:59351', strategy: 'file', fileServer: 'http://localhost:59352', domainName: 'localhost', props: null } for: http://localhost:59351 +1ms
  cypress:server:browsers:browser-cri-client Attaching to target url about:blank +1s
Still waiting to connect to Firefox, retrying in 1 second (attempt 36/62)

@MikeMcC399
Copy link
Contributor Author

I could not see anything related in the Firefox 124.0 release notes.

@jennifer-shehane jennifer-shehane added the stage: investigating Someone from Cypress is looking into this label Mar 20, 2024
@jennifer-shehane
Copy link
Member

We're investigating this issue.

@huyenltnguyen huyenltnguyen mentioned this issue Mar 20, 2024
Merged
4 tasks
@jennifer-shehane
Copy link
Member

jennifer-shehane commented Mar 20, 2024 •
edited
Loading

As a workaround, you can use a docker-image with a lower version of Firefox, like Firefox 123 in node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1.

We've verified this issue is not occurring in Firefox 123.

@MikeMcC399
Copy link
Contributor Author

MikeMcC399 commented Mar 20, 2024 •
edited
Loading

@jennifer-shehane

I already added the Docker workaround to the original post.

I also tested with Firefox 125.0b2 (beta version) and unfortunately the issue is still there.

@jennifer-shehane
Copy link
Member

jennifer-shehane commented Mar 20, 2024 •
edited
Loading

We've identified the issue, where Firefox changed some of their behavior in an undocumented way in Firefox 124 and have a fix ready in a branch. f8fc8d2 We'll prioritize getting a patch fix out for this tomorrow assuming we can get tests, reviews, etc done.

@ml-evs ml-evs mentioned this issue Mar 21, 2024
Closed
@MikeMcC399 MikeMcC399 mentioned this issue Mar 21, 2024
Closed
@AtofStryker AtofStryker mentioned this issue Mar 21, 2024
Merged
3 tasks
@HarelM HarelM mentioned this issue Mar 21, 2024
Merged
5 tasks
@crossan007 crossan007 mentioned this issue Mar 21, 2024
Closed
@jennifer-shehane
Copy link
Member

We're working on getting a release of 13.7.1 out which would have this fixed.

@jennifer-shehane jennifer-shehane removed the stage: investigating Someone from Cypress is looking into this label Mar 21, 2024
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Mar 21, 2024

Released in 13.7.1.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v13.7.1, please open a new issue.

@cypress-bot cypress-bot bot locked as resolved and limited conversation to collaborators Mar 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@AtofStryker AtofStryker

Labels
browser: firefox type: bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: patch new tab creation for firefox 124 and up to fix issue where… cypress-io/cypress
3 participants
@jennifer-shehane @AtofStryker @MikeMcC399