[BUG] #20707

marian100521 · 2023-02-07T09:57:23Z

Context:

Playwright Version: "@percy/playwright": "^1.0.4",
"@playwright/test": "^1.27.1",
Operating System: Ubuntu 22.04.1
Node.js version: "npm": "^8.19.3",
"node": "^16.17.1"
Browser: Firefox
Extra: Running on Actions on Github

Describe the bug

I'm running a pipeline in Github Actions with a Percy - Playwright easy test to test my platform:


on:
  workflow_call:
    inputs:
      project_id:
        required: true
        type: string
    secrets:
      PERCY_TOKEN:
        required: true

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-latest
    env:
      PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}    
    steps:
      - name: Checkout Repo
        uses: actions/checkout@master
      - name: Install npm packages
        run: |
          npm install
      - name: Install playwright
        run: |
          npx playwright install --with-deps
      - name: Run tests
        run: |
          xvfb-run --auto-servernum -- npm run test

const percySnapshot = require('@percy/playwright');

test('homepage has title and links to intro page', async ({ page }) => {
  await page.goto('http://localhost:3000', { timeout: 100000 });

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/My title/);

  await percySnapshot(page, 'Loads the app');
});

This is the configuration:

const { devices } = require('@playwright/test');
/**
 * @see https://playwright.dev/docs/test-configuration
 * @type {import('@playwright/test').PlaywrightTestConfig}
 */
const config = {
  testDir: './tests',
  // the rest of the options
  webServer: {
    command: 'yarn dev',
    port: 3000,
    timeout: 120000,
  },
  /* Maximum time one test can run for. */
  timeout: 30 * 1000,
  expect: {
    /**
     * Maximum time expect() should wait for the condition to be met.
     * For example in `await expect(locator).toHaveText();`
     */
    timeout: 5000,
  },
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Maximum time each action such as `click()` can take. Defaults to 0 (no limit). */
    actionTimeout: 0,
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://localhost:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'firefox',
      use: {
        ...devices['Desktop Firefox'],
      },
    },
  ],
};

module.exports = config;

I have that error code:

> [email protected] test
> percy exec -- playwright test --reporter=dot

[percy] Downloading Chromium 929511...
[percy] Successfully downloaded Chromium 929511
[percy] Percy has started!
[percy] Running "playwright test --reporter=dot"

Running 1 test using 1 worker
××[percy] Snapshot taken: Loads the app
±

  1) [firefox] › test1.spec.js:11:1 › homepage has title and links to intro page ===================

    Test timeout of 30000ms exceeded.

    page.goto: Navigation failed because page was closed!
    =========================== logs ===========================
    navigating to "http://localhost:3000/", waiting until "load"
    ============================================================

      10 |
      11 | test('homepage has title and links to intro page', async ({ page }) => {
    > 12 |   await page.goto('http://localhost:3000', { timeout: 100000 });
         |              ^
      13 |
      14 |   // Expect a title "to contain" a substring.
      15 |   await expect(page).toHaveTitle(/My Page/);

        at /home/poc--web-deploy/tests/test1.spec.js:12:14
        at /home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:376:15
        at TestInfoImpl._runFn (/home/poc--web-deploy/node_modules/@playwright/test/lib/testInfo.js:146:7)
        at /home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:331:26
        at TimeoutManager.runWithTimeout (/home/poc--web-deploy/node_modules/@playwright/test/lib/timeoutManager.js:62:7)
        at TestInfoImpl._runWithTimeout (/home/poc--web-deploy/node_modules/@playwright/test/lib/testInfo.js:135:26)
        at WorkerRunner._runTest (/home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:312:5)
        at WorkerRunner.runTestGroup (/home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:192:11)
        at process.<anonymous> (/home/poc--web-deploy/node_modules/@playwright/test/lib/worker.js:74:5)

    Pending operations:
      - page.goto at tests/test1.spec.js:12:14


    Retry #1 ---------------------------------------------------------------------------------------

    Test timeout of 30000ms exceeded.

    page.goto: Navigation failed because page was closed!
    =========================== logs ===========================
    navigating to "http://localhost:3000/", waiting until "load"
    ============================================================

      10 |
      11 | test('homepage has title and links to intro page', async ({ page }) => {
    > 12 |   await page.goto('http://localhost:3000', { timeout: 100000 });
         |              ^
      13 |
      14 |   // Expect a title "to contain" a substring.
      15 |   await expect(page).toHaveTitle(/My Page/);

        at /home/poc--web-deploy/tests/test1.spec.js:12:14
        at /home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:376:15
        at TestInfoImpl._runFn (/home/poc--web-deploy/node_modules/@playwright/test/lib/testInfo.js:146:7)
        at /home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:331:26
        at TimeoutManager.runWithTimeout (/home/poc--web-deploy/node_modules/@playwright/test/lib/timeoutManager.js:62:7)
        at TestInfoImpl._runWithTimeout (/home/poc--web-deploy/node_modules/@playwright/test/lib/testInfo.js:135:26)
        at WorkerRunner._runTest (/home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:312:5)
        at WorkerRunner.runTestGroup (/home/poc--web-deploy/node_modules/@playwright/test/lib/workerRunner.js:192:11)
        at process.<anonymous> (/home/poc--web-deploy/node_modules/@playwright/test/lib/worker.js:74:5)

    Pending operations:
      - page.goto at tests/test1.spec.js:12:14


    attachment #1: trace (application/zip) ---------------------------------------------------------
    test-results/test1-homepage-has-title-and-links-to-intro-page-firefox-retry1/trace.zip
    Usage:

        npx playwright show-trace test-results/test1-homepage-has-title-and-links-to-intro-page-firefox-retry1/trace.zip

    ------------------------------------------------------------------------------------------------


  1 flaky
    [firefox] › test1.spec.js:11:1 › homepage has title and links to intro page ====================
[percy] Finalized build #25: https://percy.io/62bedad8/poc-e2e-test/builds/24902284

Maybe I have something wrong in the config.

Thanks in advance

The text was updated successfully, but these errors were encountered:

mxschmitt · 2023-02-07T10:16:24Z

Can you share a repro which we can run locally? Does it only happen with Firefox?

marian100521 · 2023-02-07T10:36:42Z

Well, this is only happening under Github Actions. It is working fine locally. Is my config right?

marian100521 · 2023-02-07T11:08:22Z

I've tried with Chrome

  shell: /usr/bin/bash -e {0}
  env:
    PERCY_TOKEN: 

> [email protected] test
> percy exec -- playwright test --reporter=dot

[percy] Downloading Chromium 929511...
[percy] Successfully downloaded Chromium 929511
[percy] Percy has started!
[percy] Running "playwright test --reporter=dot"

Running 1 test using 1 worker
××[percy] Snapshot taken: Loads the app
[percy] Snapshot taken: Loads the app
±

  1) [Google Chrome] › test1.spec.js:11:1 › homepage has title and links to intro page =============

    Test timeout of 30000ms exceeded.

    page.goto: net::ERR_ABORTED; maybe frame was detached?
    =========================== logs ===========================
    navigating to "http://localhost:3000/", waiting until "load"
    ============================================================

      10 |
      11 | test('homepage has title and links to intro page', async ({ page }) => {
    > 12 |   await page.goto('http://localhost:3000', { timeout: 100000 });
         |              ^
      13 |
      14 |   // Expect a title "to contain" a substring.
      15 |   await expect(page).toHaveTitle(/My Title/);

        at /home/runner/work/poc-deploy/poc-deploy/tests/test1.spec.js:12:14
        at /home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/workerRunner.js:376:15
        at TestInfoImpl._runFn (/home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/testInfo.js:146:7)
        at /home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/workerRunner.js:331:26
        at TimeoutManager.runWithTimeout (/home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/timeoutManager.js:62:7)
        at TestInfoImpl._runWithTimeout (/home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/testInfo.js:135:26)
        at WorkerRunner._runTest (/home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/workerRunner.js:312:5)
        at WorkerRunner.runTestGroup (/home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/workerRunner.js:192:11)
        at process.<anonymous> (/home/runner/work/poc-deploy/poc-deploy/node_modules/@playwright/test/lib/worker.js:74:5)

    Pending operations:
      - page.goto at tests/test1.spec.js:12:14


    Retry #1 ---------------------------------------------------------------------------------------

    Test timeout of 30000ms exceeded.

    attachment #1: trace (application/zip) ---------------------------------------------------------
    test-results/test1-homepage-has-title-and-links-to-intro-page-Google-Chrome-retry1/trace.zip
    Usage:

        npx playwright show-trace test-results/test1-homepage-has-title-and-links-to-intro-page-Google-Chrome-retry1/trace.zip

    ------------------------------------------------------------------------------------------------


  1 flaky
    [Google Chrome] › test1.spec.js:11:1 › homepage has title and links to intro page ==============
[percy] Encountered an error uploading snapshot: Loads the app
[percy] Error: The name of each snapshot must be unique, and this name already exists in the build: 'Loads the app' -- You can fix this by passing a 'name' param when creating the snapshot. See the docs for more info on identifying snapshots for your specific client: https://percy.io/docs
[percy] Finalized build #28: https://percy.io/62bedad8/poc-e2e-test/builds/24904276```

mxschmitt · 2023-02-08T00:42:51Z

Looks like this is not related to Percy, I'd recommend to use HTML reporter to understand better how your application was behaving. You can also try setting the DEBUG=pw:protocol environment variable which will give a lot of debug output which helps us to understand better what's actually happening.

aslushnikov · 2023-02-14T16:20:00Z

I'll close this as per the response above; if you think there's still a bug in Playwright, and you have a good repro, please do not hesitate to file a new issue!

mxschmitt added the triaging label Feb 7, 2023

aslushnikov closed this as completed Feb 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] #20707

[BUG] #20707

marian100521 commented Feb 7, 2023

mxschmitt commented Feb 7, 2023

marian100521 commented Feb 7, 2023 •

edited

Loading

marian100521 commented Feb 7, 2023 •

edited

Loading

mxschmitt commented Feb 8, 2023

aslushnikov commented Feb 14, 2023

[BUG] #20707

[BUG] #20707

Comments

marian100521 commented Feb 7, 2023

mxschmitt commented Feb 7, 2023

marian100521 commented Feb 7, 2023 • edited Loading

marian100521 commented Feb 7, 2023 • edited Loading

mxschmitt commented Feb 8, 2023

aslushnikov commented Feb 14, 2023

marian100521 commented Feb 7, 2023 •

edited

Loading

marian100521 commented Feb 7, 2023 •

edited

Loading