[BUG] HTTP authentication doesn't work

[ivan-freedomfi]

Context:

• Playwright Version: [email protected]
• Operating System: Mac
• Node.js version: 14.18.1
• Browser: chromium

npx envinfo --preset playwright --markdown

System:

• OS: macOS 12.3.1
• Memory: 359.18 MB / 16.00 GB

Binaries:

• Node: 14.18.1 - ~/.nvm/versions/node/v14.18.1/bin/node
• Yarn: 1.22.18 - /usr/local/bin/yarn
• npm: 6.14.15 - ~/.nvm/versions/node/v14.18.1/bin/npm

Languages:

• Bash: 3.2.57 - /bin/bash

npmPackages:

• playwright: ^1.21.1 => 1.21.1

Code Snippet

import { test, request } from '@playwright/test';

test('requestContext', async () => {
  const httpCredentials = {
    username: 'user',
    password: 'pass',
  };

  const requestContext = await request.newContext({
    baseURL: 'https://playwright.dev',
    httpCredentials,
  });

  await requestContext.get('/');
});

Output (no Authorization header):

> DEBUG='pw:api' playwright test


Running 1 test using 1 worker

  _  [google-pixel] _ src/reqeust-context.spec.ts:3:1 _ requestContext (194ms)
  pw:api => apiRequest.newContext started +0ms
  pw:api <= apiRequest.newContext succeeded +34ms
  pw:api => tracing.stop started +3ms
  pw:api <= tracing.stop succeeded +5ms
  pw:api => apiRequestContext.get started +2ms
  pw:api _ GET https://playwright.dev/ +24ms
  pw:api   user-agent: Playwright/1.21.1 (x64; macOS 12.3) node/14.18 +0ms
  pw:api   accept: */* +0ms
  pw:api   accept-encoding: gzip,deflate,br +0ms
  pw:api _ 200 OK +108ms
  pw:api   connection: close +0ms
  pw:api   content-length: 5429 +1ms
  pw:api   server: GitHub.com +0ms
  pw:api   content-type: text/html; charset=utf-8 +0ms
  pw:api   last-modified: Mon, 09 May 2022 11:23:51 GMT +0ms
  pw:api   access-control-allow-origin: * +0ms
  pw:api   etag: W/"6278f9c7-4548" +0ms
  pw:api   expires: Tue, 10 May 2022 01:54:44 GMT +0ms
  pw:api   cache-control: max-age=600 +0ms
  pw:api   content-encoding: gzip +0ms
  pw:api   x-proxy-cache: MISS +0ms
  pw:api   x-github-request-id: 6D34:38AE:26FFEA3:283D4A9:6279C38C +0ms
  pw:api   accept-ranges: bytes +0ms
  pw:api   date: Tue, 10 May 2022 16:02:47 GMT +0ms
  pw:api   via: 1.1 varnish +0ms
  pw:api   age: 210 +0ms
  pw:api   x-served-by: cache-hhn4035-HHN +0ms
  pw:api   x-cache: HIT +0ms
  pw:api   x-cache-hits: 1 +0ms
  pw:api   x-timer: S1652198568.925187,VS0,VE1 +0ms
  pw:api   vary: Accept-Encoding +0ms
  pw:api   x-fastly-request-id: caafc4ba29de9239b66f7b29184e932cb8a6c812 +0ms
  pw:api <= apiRequestContext.get succeeded +7ms


  1 passed (674ms)

Describe the bug

Passing httpCredentials to request.newContext() doesn't work: the requests are still not authorized (no Authentication header) - see output above.

But if I do it manually like below, it works:

import { test, request } from '@playwright/test';

test('requestContext', async () => {
  const httpCredentials = {
    username: 'user',
    password: 'pass',
  };

  const btoa = (str: string) => Buffer.from(str).toString('base64');
  const credentialsBase64 = btoa(`${httpCredentials.username}:${httpCredentials.password}`);

  const requestContext = await request.newContext({
    baseURL: 'https://playwright.dev',
    extraHTTPHeaders: {
      'Authorization': `Basic ${credentialsBase64}`,
    },
  });

  await requestContext.get('/');
});

Output of manual adding of Authorization: the header is present:

> DEBUG='pw:api' playwright test


Running 1 test using 1 worker

  _  [google-pixel] _ src/reqeust-context.spec.ts:3:1 _ requestContext (188ms)
  pw:api => apiRequest.newContext started +0ms
  pw:api <= apiRequest.newContext succeeded +32ms
  pw:api => tracing.stop started +2ms
  pw:api <= tracing.stop succeeded +4ms
  pw:api => apiRequestContext.get started +1ms
  pw:api _ GET https://playwright.dev/ +25ms
  pw:api   user-agent: Playwright/1.21.1 (x64; macOS 12.3) node/14.18 +0ms
  pw:api   accept: */* +1ms
  pw:api   accept-encoding: gzip,deflate,br +0ms
  pw:api   authorization: Basic dXNlcjpwYXNz +0ms
  pw:api _ 200 OK +103ms
  pw:api   connection: close +0ms
  pw:api   content-length: 5429 +0ms
  pw:api   server: GitHub.com +0ms
  pw:api   content-type: text/html; charset=utf-8 +0ms
  pw:api   last-modified: Mon, 09 May 2022 11:23:51 GMT +0ms
  pw:api   access-control-allow-origin: * +0ms
  pw:api   etag: W/"6278f9c7-4548" +1ms
  pw:api   expires: Tue, 10 May 2022 01:54:44 GMT +0ms
  pw:api   cache-control: max-age=600 +0ms
  pw:api   content-encoding: gzip +0ms
  pw:api   x-proxy-cache: MISS +0ms
  pw:api   x-github-request-id: 6D34:38AE:26FFEA3:283D4A9:6279C38C +0ms
  pw:api   accept-ranges: bytes +0ms
  pw:api   date: Tue, 10 May 2022 16:05:03 GMT +0ms
  pw:api   via: 1.1 varnish +0ms
  pw:api   age: 345 +0ms
  pw:api   x-served-by: cache-hhn4064-HHN +0ms
  pw:api   x-cache: HIT +0ms
  pw:api   x-cache-hits: 1 +0ms
  pw:api   x-timer: S1652198703.138463,VS0,VE1 +0ms
  pw:api   vary: Accept-Encoding +0ms
  pw:api   x-fastly-request-id: 57039b443b1418acb114dcfe9baf5b026644fd94 +0ms
  pw:api <= apiRequestContext.get succeeded +8ms


  1 passed (680ms)

[mxschmitt]

Playwright will only add the basic auth when the response of the initial request was a 401 and the response header www-authenticate starts with Basic. This is how the rfc7617 describes it. Did other http clients do it different or against which backend server are you testing?

[pt-haplo]

@mxschmitt I am just starting out with Playwright and have hit this issue. From what you are saying, Playwright is expecting a "challenge" response from the server, after which it will resend the request with the Authorization header.

In my situation, I don't get that. It seems that the server that I'm using for API testing expects the Authorization header straight off the bat (and, I would guess from your comment, does not conform to RFC7617). If I use the workaround shown above (adding the header manually), the server will respond as expected.

If I don't manually insert the header, when Playwright makes the request, the server responds with "403 Forbidden".

[pt-haplo]

I have just read the RFC7617 specification and, interestingly, it says that the server can respond with a 401 status code, not that it will or must respond with that status code:

Upon receipt of a request for a URI within the protection space that
lacks credentials, the server can reply with a challenge using the
401 (Unauthorized) status code ([RFC7235], Section 3.1) and the
WWW-Authenticate header field ([RFC7235], Section 4.1).

Additionally, it seems to be saying that the challenge response would be sent only if the server receives "a request for a URI within the protection space that lacks credentials". So the challenge should only be issued if the credentials weren't provided in the first place.

Given that the server is not required to send a 401 response if the credentials are missing (and indeed, the server that I'm testing against does not), would it make more sense for Playwright to simply include the credentials in the original request?

I think where I (and possibly @ivan-freedomfi) tripped up is that I assumed that adding the httpCredentials to the request would mean that the Authorization header would be explicitly added to the request. I was not expecting that Playwright would make a request without the Authorization, check for a 401 status code, then re-send the request with the Authorization header included.

This behaviour is also somewhat opaque to somebody reading the Playwright documentation.

What I have noted, since discovering the "workaround" above, is that the "workaround" is actually the "solution". It's alluded-to in the documentation here with this piece of code:

// playwright.config.ts
import type { PlaywrightTestConfig } from '@playwright/test';

const config: PlaywrightTestConfig = {
  use: {
    // All requests we send go to this API endpoint.
    baseURL: 'https://api.github.com',
    extraHTTPHeaders: {
      // We set this header per GitHub guidelines.
      'Accept': 'application/vnd.github.v3+json',
      // Add authorization token to all requests.
      // Assuming personal access token available in the environment.
      'Authorization': `token ${process.env.API_TOKEN}`,
    },
  }
};
export default config;

But in the "basic auth" case, we don't need the Accept header, and the Authorization header should be Basic as already shown in @ivan-freedomfi's code.

I struggled with all this because initially I was reading the guide for authentication, which had a note:

Note: This guide covers cookie/token-based authentication (logging in via the app UI). For HTTP authentication use browser.newContext([options]).

The trouble I had was that there was no context or examples given, so it wasn't all that helpful. Now that I've put all the pieces together, it finally makes sense!

[rmunn]

In #16005 I have a reproduction where the httpCredentials are not being sent even though the server is responding with a 401.

[rmunn]

Given that the server is not required to send a 401 response if the credentials are missing (and indeed, the server that I'm testing against does not), would it make more sense for Playwright to simply include the credentials in the original request?

I agree with this opinion as well. If I went to the bother of creating a new browser context so that I could specify httpCredentials, I'm expecting those credentials to be sent with that request. Having them not sent until after the server replies with a 401 seems like a waste of time.

[rmunn]

I think where I (and possibly @ivan-freedomfi) tripped up is that I assumed that adding the httpCredentials to the request would mean that the Authorization header would be explicitly added to the request. I was not expecting that Playwright would make a request without the Authorization, check for a 401 status code, then re-send the request with the Authorization header included.

Playwright 1.45 has just been released, with an httpCredentials.send option to request exactly this behavior. Set it to "always" and the Authorization header will be sent on first request. (Default value is "unauthorized", which doesn't send the header until after receiving an HTTP response of 401 Unauthorized from the server).