Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use HTTP/2 audit on pagespeed insights #12208

Open
SoftCreatR opened this issue Mar 5, 2021 · 6 comments
Open

Use HTTP/2 audit on pagespeed insights #12208

SoftCreatR opened this issue Mar 5, 2021 · 6 comments
Assignees
@connorjclark
Labels
needs-investigation needs-priority

Comments

@SoftCreatR
Copy link

As reported already (#12113), PSI reports a wrong HTTP version for several websites.

This seems to be a problem if the target page doesn't return a 200 response code.

Working: https://developers.google.com/speed/pagespeed/insights/?hl=de&url=https%3A%2F%2Fwww.softcreatr.com%2Fx.php
Not working: https://developers.google.com/speed/pagespeed/insights/?hl=de&url=https%3A%2F%2Fwww.softcreatr.com

Doing

curl -c /tmp/cookies -L -I -k https://www.softcreatr.com

shows, that there are 1-2 redirects, before returning 200, but every response is sent via HTTP/2, so PSI's result is a false positive.
@connorjclark
Copy link
Collaborator

Thank you for filing this! I've raised this issue with the right people internally.

@connorjclark
Copy link
Collaborator

connorjclark commented Mar 5, 2021 •
edited
Loading

This seems to be a problem if the target page doesn't return a 200 response code.

I tried other pages that redirect: https://paulirish.com -> https://www.paulirish.com ; that redirects as expected, although the first request is done over http/1.1 in PSI (even tho it says h2 in devtools network panel...), the rest are h2. I suspect this is another bug.

For your url, it seems our system is getting into an infinite redirect loop (altho Chrome itself has no problem...)

output of curl command you gave:

HTTP/2 302
date: Fri, 05 Mar 2021 23:25:15 GMT
content-type: text/html; charset=UTF-8
location: https://www.softcreatr.com/
set-cookie: wsc_a05070_cookieHash=db3459a9a91968fc6f18d88a0993af342af3eff2; path=/; domain=.softcreatr.com; secure; HttpOnly; SameSite=Lax
server: softcreatr-media
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
referrer-policy: no-referrer
feature-policy: accelerometer 'none'; camera 'none'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; payment 'none'; usb 'none'
expect-ct: max-age=604800
content-security-policy: default-src 'none'; base-uri 'self'; script-src 'self' softcreatr.com *.softcreatr.com https://js.stripe.com https://feedback.shopvote.de https://platform.twitter.com https://static-eu.payments-amazon.com https://cdnjs.cloudflare.com 'unsafe-inline' 'unsafe-eval' data:; img-src 'self' data: https: 'unsafe-inline'; style-src 'self' softcreatr.com *.softcreatr.com https://feedback.shopvote.de https://cdnjs.cloudflare.com 'unsafe-inline'; frame-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://js.stripe.com https://player.twitch.tv https://www.youtube-nocookie.com https://player.vimeo.com https://w.soundcloud.com https://embed.spotify.com https://open.spotify.com https://platform.twitter.com https://static-eu.payments-amazon.com https://payments.amazon.de; connect-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://feedback.shopvote.de https://payments-de.amazon.com https://payments.amazon.de; frame-ancestors 'self' softcreatr.com *.softcreatr.com *.1-2.dev; form-action 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://www.paypal.com/cgi-bin/webscr https://payments.amazon.de https://www.sofortueberweisung.de https://www.sofort.com data:; font-src 'self' softcreatr.com *.softcreatr.com https://cdnjs.cloudflare.com; manifest-src 'self' softcreatr.com *.softcreatr.com; worker-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:; object-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:; media-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:;
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-staple: max-age=31536000; includeSubDomains; preload
x-tls-cipher: ECDHE-RSA-CHACHA20-POLY1305
x-tls-protocol: TLSv1.2
x-tls-sni-host: www.softcreatr.com
host: www.softcreatr.com

HTTP/2 302
date: Fri, 05 Mar 2021 23:25:15 GMT
content-type: text/html; charset=UTF-8
location: https://www.softcreatr.com/login/?url=https%3A%2F%2Fwww.softcreatr.com%2F
link: <https://www.softcreatr.com/font/getFont.php?family=&filename=fontawesome-webfont.woff2&v=4.7.0>; rel=preload; as=font; crossorigin=anonymous,<https://www.softcreatr.com/font/getFont.php?font=sc-brands&type=woff2&v=1.1>; rel=preload; as=font; crossorigin=anonymous
set-cookie: wsc_a05070_h2pushes=%5B%22https%3A%5C%2F%5C%2Fwww.softcreatr.com%5C%2Ffont%5C%2FgetFont.php%3Ffamily%3D%26filename%3Dfontawesome-webfont.woff2%26v%3D4.7.0%22%2C%22https%3A%5C%2F%5C%2Fwww.softcreatr.com%5C%2Ffont%5C%2FgetFont.php%3Ffont%3Dsc-brands%26type%3Dwoff2%26v%3D1.1%22%5D; path=/; domain=.softcreatr.com; secure; HttpOnly; SameSite=Lax
server: softcreatr-media
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
referrer-policy: no-referrer
feature-policy: accelerometer 'none'; camera 'none'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; payment 'none'; usb 'none'
expect-ct: max-age=604800
content-security-policy: default-src 'none'; base-uri 'self'; script-src 'self' softcreatr.com *.softcreatr.com https://js.stripe.com https://feedback.shopvote.de https://platform.twitter.com https://static-eu.payments-amazon.com https://cdnjs.cloudflare.com 'unsafe-inline' 'unsafe-eval' data:; img-src 'self' data: https: 'unsafe-inline'; style-src 'self' softcreatr.com *.softcreatr.com https://feedback.shopvote.de https://cdnjs.cloudflare.com 'unsafe-inline'; frame-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://js.stripe.com https://player.twitch.tv https://www.youtube-nocookie.com https://player.vimeo.com https://w.soundcloud.com https://embed.spotify.com https://open.spotify.com https://platform.twitter.com https://static-eu.payments-amazon.com https://payments.amazon.de; connect-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://feedback.shopvote.de https://payments-de.amazon.com https://payments.amazon.de; frame-ancestors 'self' softcreatr.com *.softcreatr.com *.1-2.dev; form-action 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://www.paypal.com/cgi-bin/webscr https://payments.amazon.de https://www.sofortueberweisung.de https://www.sofort.com data:; font-src 'self' softcreatr.com *.softcreatr.com https://cdnjs.cloudflare.com; manifest-src 'self' softcreatr.com *.softcreatr.com; worker-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:; object-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:; media-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:;
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-staple: max-age=31536000; includeSubDomains; preload
x-tls-cipher: ECDHE-RSA-CHACHA20-POLY1305
x-tls-protocol: TLSv1.2
x-tls-sni-host: www.softcreatr.com
host: www.softcreatr.com

HTTP/2 200
date: Fri, 05 Mar 2021 23:25:15 GMT
content-type: text/html; charset=UTF-8
vary: Accept-Encoding
expires: Mon, 26 Jul 1997 05:00:00 GMT
last-modified: Fri, 05 Mar 2021 23:25:15 GMT
cache-control: max-age=0, no-cache, no-store, must-revalidate
pragma: no-cache
x-ua-compatible: IE=edge
x-frame-options: SAMEORIGIN
server: softcreatr-media
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
referrer-policy: no-referrer
feature-policy: accelerometer 'none'; camera 'none'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; payment 'none'; usb 'none'
expect-ct: max-age=604800
content-security-policy: default-src 'none'; base-uri 'self'; script-src 'self' softcreatr.com *.softcreatr.com https://js.stripe.com https://feedback.shopvote.de https://platform.twitter.com https://static-eu.payments-amazon.com https://cdnjs.cloudflare.com 'unsafe-inline' 'unsafe-eval' data:; img-src 'self' data: https: 'unsafe-inline'; style-src 'self' softcreatr.com *.softcreatr.com https://feedback.shopvote.de https://cdnjs.cloudflare.com 'unsafe-inline'; frame-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://js.stripe.com https://player.twitch.tv https://www.youtube-nocookie.com https://player.vimeo.com https://w.soundcloud.com https://embed.spotify.com https://open.spotify.com https://platform.twitter.com https://static-eu.payments-amazon.com https://payments.amazon.de; connect-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://feedback.shopvote.de https://payments-de.amazon.com https://payments.amazon.de; frame-ancestors 'self' softcreatr.com *.softcreatr.com *.1-2.dev; form-action 'self' softcreatr.com *.softcreatr.com *.1-2.dev https://www.paypal.com/cgi-bin/webscr https://payments.amazon.de https://www.sofortueberweisung.de https://www.sofort.com data:; font-src 'self' softcreatr.com *.softcreatr.com https://cdnjs.cloudflare.com; manifest-src 'self' softcreatr.com *.softcreatr.com; worker-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:; object-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:; media-src 'self' softcreatr.com *.softcreatr.com *.1-2.dev blob:;
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-staple: max-age=31536000; includeSubDomains; preload
x-tls-cipher: ECDHE-RSA-CHACHA20-POLY1305
x-tls-protocol: TLSv1.2
x-tls-sni-host: www.softcreatr.com
host: www.softcreatr.com

It seems the first request does https://www.softcreatr.com -> https://www.softcreatr.com/, then the second does https://www.softcreatr.com/ -> https://www.softcreatr.com/login/?url=https%3A%2F%2Fwww.softcreatr.com%2F . so.... That seems fine. Seems like our bug for sure.

@connorjclark
Copy link
Collaborator

connorjclark commented Mar 6, 2021 •
edited
Loading

More on the initial request not being h2 when redirected...

Here is the part of the devtools log I get from https://paulirish.com on LR/PSI . It redirects to https://www.paulirish.com

{
      "method": "Network.requestWillBeSent",
      "params": {
        "requestId": "D3EE1699B7C6CCB9A2E3751CC66DA572",
        "loaderId": "D3EE1699B7C6CCB9A2E3751CC66DA572",
        "documentURL": "https://paulirish.com/",
        "request": {
          "url": "https://paulirish.com/",
          "method": "GET",
          "headers": {
            "Upgrade-Insecure-Requests": "1",
            "User-Agent": "Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4143.7 Mobile Safari/537.36 Chrome-Lighthouse",
            "Accept-Language": "en-US"
          },
          "mixedContentType": "none",
          "initialPriority": "VeryHigh",
          "referrerPolicy": "no-referrer-when-downgrade"
        },
        "timestamp": 683681.556119,
        "wallTime": 1614986232.949958,
        "initiator": {
          "type": "other"
        },
        "type": "Document",
        "frameId": "0AF140F9EB36604229CEAA1D610DA467",
        "hasUserGesture": false
      }
    },
    {
      "method": "Network.requestWillBeSent",
      "params": {
        "requestId": "D3EE1699B7C6CCB9A2E3751CC66DA572",
        "loaderId": "D3EE1699B7C6CCB9A2E3751CC66DA572",
        "documentURL": "https://www.paulirish.com/",
        "request": {
          "url": "https://www.paulirish.com/",
          "method": "GET",
          "headers": {
            "Upgrade-Insecure-Requests": "1",
            "User-Agent": "Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4143.7 Mobile Safari/537.36 Chrome-Lighthouse",
            "Accept-Language": "en-US"
          },
          "mixedContentType": "none",
          "initialPriority": "VeryHigh",
          "referrerPolicy": "no-referrer-when-downgrade"
        },
        "timestamp": 683681.649908,
        "wallTime": 1614986233.044964,
        "initiator": {
          "type": "other"
        },
        "redirectResponse": {
          "url": "https://paulirish.com/",
          "status": 301,
          "statusText": "Moved Permanently",
          "headers": {
            "date": "Fri, 05 Mar 2021 23:17:13 GMT",
            "cache-control": "max-age=3600",
            "expires": "Sat, 06 Mar 2021 00:17:13 GMT",
            "location": "https://www.paulirish.com/",
            "cf-request-id": "08a6475cc30000303f2c14d000000001",
            "expect-ct": "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"",
            "report-to": "{\"group\":\"cf-nel\",\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report?s=xfH9YOwvp19x%2FwY3oCpMAHYv5Pev%2B9xSaiXXNJL3231%2BexwJXbtU9K9Onbwz0XHogQnICUhzuDNswluW2ulxmdWTeT7Lxw7%2FZXz4S5m%2F\"}],\"max_age\":604800}",
            "nel": "{\"report_to\":\"cf-nel\",\"max_age\":604800}",
            "vary": "Accept-Encoding",
            "server": "cloudflare",
            "cf-ray": "62b70e746e00303f-ORD",
            "X-TotalFetchedSize": "654",
            "X-ProtocolIsH2": "true",
            "X-TotalMs": "51",
            "X-TCPMs": "30",
            "X-RequestMs": "19",
            "X-ResponseMs": "2"
          },
          "mimeType": "",
          "connectionReused": false,
          "connectionId": 0,
          "remoteIPAddress": "",
          "remotePort": 0,
          "fromDiskCache": false,
          "fromServiceWorker": false,
          "fromPrefetchCache": false,
          "encodedDataLength": 747,
          "timing": {
            "requestTime": 683681.556978,
            "proxyStart": -1,
            "proxyEnd": -1,
            "dnsStart": -1,
            "dnsEnd": -1,
            "connectStart": -1,
            "connectEnd": -1,
            "sslStart": -1,
            "sslEnd": -1,
            "workerStart": -1,
            "workerReady": -1,
            "workerFetchStart": -1,
            "workerRespondWithSettled": -1,
            "sendStart": -1,
            "sendEnd": -1,
            "pushStart": 0,
            "pushEnd": 0,
            "receiveHeadersEnd": 92.257
          },
          "responseTime": 1614986233043.061,
          "protocol": "http/1.1",
          "securityState": "secure"
        },
        "type": "Document",
        "frameId": "0AF140F9EB36604229CEAA1D610DA467",
        "hasUserGesture": false
      }
    },
    {
      "method": "Network.responseReceived",
      "params": {
        "requestId": "D3EE1699B7C6CCB9A2E3751CC66DA572",
        "loaderId": "D3EE1699B7C6CCB9A2E3751CC66DA572",
        "timestamp": 683681.975976,
        "type": "Document",
        "response": {
          "url": "https://www.paulirish.com/",
          "status": 200,
          "statusText": "OK",
          "headers": {
            "date": "Fri, 05 Mar 2021 23:17:13 GMT",
            "content-type": "text/html",
            "last-modified": "Thu, 15 Oct 2020 04:50:24 GMT",
            "cache-control": "max-age=600",
            "expires": "Fri, 05 Mar 2021 23:27:13 GMT",
            "vary": "Accept-Encoding,User-Agent",
            "content-security-policy": "default-src 'self' https://c.disquscdn.com https://disqus.com; script-src 'self' 'unsafe-inline' https://ssl.google-analytics.com https://api.github.com https://disqus.com https://go.disqus.com https://*.disquscdn.com https://www.google-analytics.com https://paulirish.disqus.com https://platform.twitter.com https://cdn.syndication.twimg.com https://www.googletagmanager.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://platform.twitter.com; img-src * 'self' data:; font-src 'self' data: https://fonts.gstatic.com; connect-src 'self' https://paulirish.com https://fonts.gstatic.com https://firebaseinstallations.googleapis.com https://firebaselogging.googleapis.com https://firebaseremoteconfig.googleapis.com https://www.google-analytics.com https://firebaselogging-pa.googleapis.com; frame-src 'self' https://platform.twitter.com https://accounts.google.com https://jsfiddle.net https://vimeo.com https://player.vimeo.com https://embed.verite.co https://www.youtube.com https://apis.google.com https://disqus.com https://paulirish.wufoo.com; upgrade-insecure-requests; report-uri https://paulirish.report-uri.com/r/d/csp/enforce;",
            "cf-cache-status": "DYNAMIC",
            "cf-request-id": "08a6475d190000c51838221000000001",
            "expect-ct": "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"",
            "report-to": "{\"group\":\"cf-nel\",\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report?s=JBld3kM9CKdddDkrh3%2Fln%2Bf6e%2F853D9D%2B%2BkTGpbDRqpv8FRcEY9CwrJAdXVtgJf1AbJNPMqj47lhZehugCeTPbNojaTmbUQ4eaCB457yWJS1mg%3D%3D\"}],\"max_age\":604800}",
            "nel": "{\"max_age\":604800,\"report_to\":\"cf-nel\"}",
            "server": "cloudflare",
            "cf-ray": "62b70e74ff5ac518-ORD",
            "X-TotalFetchedSize": "23160",
            "X-ProtocolIsH2": "true",
            "X-TotalMs": "282",
            "X-TCPMs": "29",
            "X-RequestMs": "246",
            "X-ResponseMs": "7",
            "X-Original-Content-Encoding": "br"
          },
          "mimeType": "text/html",
          "connectionReused": false,
          "connectionId": 0,
          "remoteIPAddress": "",
          "remotePort": 0,
          "fromDiskCache": false,
          "fromServiceWorker": false,
          "fromPrefetchCache": false,
          "encodedDataLength": 2146,
          "timing": {
            "requestTime": 683681.651801,
            "proxyStart": -1,
            "proxyEnd": -1,
            "dnsStart": -1,
            "dnsEnd": -1,
            "connectStart": -1,
            "connectEnd": -1,
            "sslStart": -1,
            "sslEnd": -1,
            "workerStart": -1,
            "workerReady": -1,
            "workerFetchStart": -1,
            "workerRespondWithSettled": -1,
            "sendStart": -1,
            "sendEnd": -1,
            "pushStart": 0,
            "pushEnd": 0,
            "receiveHeadersEnd": 319.849
          },
          "responseTime": 1614986233365.476,
          "protocol": "http/1.1",
          "securityState": "secure"
        },
        "frameId": "0AF140F9EB36604229CEAA1D610DA467"
      }
    },

Here is the same section of the devtools log when running Lighthouse locally with chrome canary:

{
    "method": "Network.requestWillBeSentExtraInfo",
    "params": {
      "requestId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "associatedCookies": [],
      "headers": {
        ":method": "GET",
        ":authority": "paulirish.com",
        ":scheme": "https",
        ":path": "/",
        "upgrade-insecure-requests": "1",
        "user-agent": "Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4420.0 Mobile Safari/537.36 Chrome-Lighthouse",
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "sec-fetch-site": "none",
        "sec-fetch-mode": "navigate",
        "sec-fetch-user": "?1",
        "sec-fetch-dest": "document",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "en-US,en;q=0.9"
      }
    }
  },
  {
    "method": "Page.lifecycleEvent",
    "params": {
      "frameId": "F004E3FF02335E2B610F933D672EBECB",
      "loaderId": "EF61FDAF5AC9129665FE3A54C4F9D456",
      "name": "networkAlmostIdle",
      "timestamp": 37482.05489
    }
  },
  {
    "method": "Page.lifecycleEvent",
    "params": {
      "frameId": "F004E3FF02335E2B610F933D672EBECB",
      "loaderId": "EF61FDAF5AC9129665FE3A54C4F9D456",
      "name": "networkIdle",
      "timestamp": 37482.05489
    }
  },
  {
    "method": "Network.responseReceivedExtraInfo",
    "params": {
      "requestId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "blockedCookies": [],
      "headers": {
        "date": "Fri, 05 Mar 2021 23:37:03 GMT",
        "cache-control": "max-age=3600",
        "expires": "Sat, 06 Mar 2021 00:37:03 GMT",
        "location": "https://www.paulirish.com/",
        "cf-request-id": "08a6598736000027f466246000000001",
        "expect-ct": "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"",
        "report-to": "{\"max_age\":604800,\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report?s=Y41kf3chc3eTlqk1jpBHF4MThltyIDfsWP3C7jaoNz91%2FNYUV0JsKbG3gtrVBMgWzAFJs3awOQ7U7GOdZgbUldPFNsMk8ZPlt5MXAE8U\"}],\"group\":\"cf-nel\"}",
        "nel": "{\"report_to\":\"cf-nel\",\"max_age\":604800}",
        "vary": "Accept-Encoding",
        "server": "cloudflare",
        "cf-ray": "62b72b852b4727f4-SLC"
      },
      "resourceIPAddressSpace": "Public"
    }
  },
  {
    "method": "Network.requestWillBeSent",
    "params": {
      "requestId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "loaderId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "documentURL": "https://www.paulirish.com/",
      "request": {
        "url": "https://www.paulirish.com/",
        "method": "GET",
        "headers": {
          "Upgrade-Insecure-Requests": "1",
          "User-Agent": "Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4420.0 Mobile Safari/537.36 Chrome-Lighthouse"
        },
        "mixedContentType": "none",
        "initialPriority": "VeryHigh",
        "referrerPolicy": "strict-origin-when-cross-origin"
      },
      "timestamp": 37482.744821,
      "wallTime": 1614987423.429662,
      "initiator": {
        "type": "other"
      },
      "redirectResponse": {
        "url": "https://paulirish.com/",
        "status": 301,
        "statusText": "",
        "headers": {
          "date": "Fri, 05 Mar 2021 23:37:03 GMT",
          "cache-control": "max-age=3600",
          "expires": "Sat, 06 Mar 2021 00:37:03 GMT",
          "location": "https://www.paulirish.com/",
          "cf-request-id": "08a6598736000027f466246000000001",
          "expect-ct": "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"",
          "report-to": "{\"max_age\":604800,\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report?s=Y41kf3chc3eTlqk1jpBHF4MThltyIDfsWP3C7jaoNz91%2FNYUV0JsKbG3gtrVBMgWzAFJs3awOQ7U7GOdZgbUldPFNsMk8ZPlt5MXAE8U\"}],\"group\":\"cf-nel\"}",
          "nel": "{\"report_to\":\"cf-nel\",\"max_age\":604800}",
          "vary": "Accept-Encoding",
          "server": "cloudflare",
          "cf-ray": "62b72b852b4727f4-SLC"
        },
        "mimeType": "",
        "requestHeaders": {
          ":method": "GET",
          ":authority": "paulirish.com",
          ":scheme": "https",
          ":path": "/",
          "upgrade-insecure-requests": "1",
          "user-agent": "Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4420.0 Mobile Safari/537.36 Chrome-Lighthouse",
          "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
          "sec-fetch-site": "none",
          "sec-fetch-mode": "navigate",
          "sec-fetch-user": "?1",
          "sec-fetch-dest": "document",
          "accept-encoding": "gzip, deflate, br",
          "accept-language": "en-US,en;q=0.9"
        },
        "connectionReused": false,
        "connectionId": 28,
        "remoteIPAddress": "172.67.130.17",
        "remotePort": 443,
        "fromDiskCache": false,
        "fromServiceWorker": false,
        "fromPrefetchCache": false,
        "encodedDataLength": 476,
        "timing": {
          "requestTime": 37482.195017,
          "proxyStart": -1,
          "proxyEnd": -1,
          "dnsStart": 0.255,
          "dnsEnd": 56.17,
          "connectStart": 56.17,
          "connectEnd": 118.602,
          "sslStart": 85.07,
          "sslEnd": 118.596,
          "workerStart": -1,
          "workerReady": -1,
          "workerFetchStart": -1,
          "workerRespondWithSettled": -1,
          "sendStart": 118.754,
          "sendEnd": 118.868,
          "pushStart": 0,
          "pushEnd": 0,
          "receiveHeadersEnd": 548.109
        },
        "responseTime": 1614987423427.278,
        "protocol": "h2",
        "securityState": "secure",
        "securityDetails": {
          "protocol": "TLS 1.3",
          "keyExchange": "",
          "keyExchangeGroup": "X25519",
          "cipher": "AES_128_GCM",
          "certificateId": 0,
          "subjectName": "sni.cloudflaressl.com",
          "sanList": [
            "paulirish.com",
            "sni.cloudflaressl.com",
            "*.paulirish.com"
          ],
          "issuer": "Cloudflare Inc ECC CA-3",
          "validFrom": 1595548800,
          "validTo": 1627128000,
          "signedCertificateTimestampList": [
            {
              "status": "Verified",
              "origin": "Embedded in certificate",
              "logDescription": "Google 'Argon2021' log",
              "logId": "F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3",
              "timestamp": 1595567700006,
              "hashAlgorithm": "SHA-256",
              "signatureAlgorithm": "ECDSA",
              "signatureData": "304502201E5CE83AA7BAE618403970F57D84CA4A9C511EE06062322FB70F6CE12AC8832002210094815E1767254B1EA7DDD9AAB3618BF4F293315E744F1449B0D716B3E7A92848"
            },
            {
              "status": "Verified",
              "origin": "Embedded in certificate",
              "logDescription": "DigiCert Yeti2021 Log",
              "logId": "5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA",
              "timestamp": 1595567700057,
              "hashAlgorithm": "SHA-256",
              "signatureAlgorithm": "ECDSA",
              "signatureData": "3045022100FC2DCFCFB1EC2C64EC36A6E75938B8C49AD124BD0CC96F16B334E8FE7C9CB6400220609D434E61CD7A92413709FB2038950F368541DCD1BAD4118BB80B0528E85724"
            }
          ],
          "certificateTransparencyCompliance": "compliant"
        }
      },
      "type": "Document",
      "frameId": "F004E3FF02335E2B610F933D672EBECB",
      "hasUserGesture": false
    }
  },
  {
    "method": "Network.requestWillBeSentExtraInfo",
    "params": {
      "requestId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "associatedCookies": [],
      "headers": {
        ":method": "GET",
        ":authority": "www.paulirish.com",
        ":scheme": "https",
        ":path": "/",
        "upgrade-insecure-requests": "1",
        "user-agent": "Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4420.0 Mobile Safari/537.36 Chrome-Lighthouse",
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "sec-fetch-site": "none",
        "sec-fetch-mode": "navigate",
        "sec-fetch-user": "?1",
        "sec-fetch-dest": "document",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "en-US,en;q=0.9"
      }
    }
  },
  {
    "method": "Network.responseReceivedExtraInfo",
    "params": {
      "requestId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "blockedCookies": [],
      "headers": {
        "date": "Fri, 05 Mar 2021 23:37:04 GMT",
        "content-type": "text/html",
        "set-cookie": "__cfduid=d83583d8f5b34bd7eae780a4d89fb35411614987424; expires=Sun, 04-Apr-21 23:37:04 GMT; path=/; domain=.paulirish.com; HttpOnly; SameSite=Lax",
        "last-modified": "Thu, 15 Oct 2020 04:50:24 GMT",
        "cache-control": "max-age=600",
        "expires": "Fri, 05 Mar 2021 23:47:04 GMT",
        "vary": "Accept-Encoding,User-Agent",
        "content-security-policy": "default-src 'self' https://c.disquscdn.com https://disqus.com; script-src 'self' 'unsafe-inline' https://ssl.google-analytics.com https://api.github.com https://disqus.com https://go.disqus.com https://*.disquscdn.com https://www.google-analytics.com https://paulirish.disqus.com https://platform.twitter.com https://cdn.syndication.twimg.com https://www.googletagmanager.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://platform.twitter.com; img-src * 'self' data:; font-src 'self' data: https://fonts.gstatic.com; connect-src 'self' https://paulirish.com https://fonts.gstatic.com https://firebaseinstallations.googleapis.com https://firebaselogging.googleapis.com https://firebaseremoteconfig.googleapis.com https://www.google-analytics.com https://firebaselogging-pa.googleapis.com; frame-src 'self' https://platform.twitter.com https://accounts.google.com https://jsfiddle.net https://vimeo.com https://player.vimeo.com https://embed.verite.co https://www.youtube.com https://apis.google.com https://disqus.com https://paulirish.wufoo.com; upgrade-insecure-requests; report-uri https://paulirish.report-uri.com/r/d/csp/enforce;",
        "cf-cache-status": "DYNAMIC",
        "cf-request-id": "08a6598991000027f49e97f000000001",
        "expect-ct": "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"",
        "report-to": "{\"max_age\":604800,\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report?s=7mkW3UYZt2GAmiBzKTYWID2%2BM3yVhEUlobwRWp4I8MMaGf4VwXeI49DeS7p7XHMeHcvrNQe%2BtEP0ywXveSL1i1sVnwKrnSZSJDY1Plg1EKnNuQ%3D%3D\"}],\"group\":\"cf-nel\"}",
        "nel": "{\"report_to\":\"cf-nel\",\"max_age\":604800}",
        "server": "cloudflare",
        "cf-ray": "62b72b88e98d27f4-SLC",
        "content-encoding": "br"
      },
      "resourceIPAddressSpace": "Public"
    }
  },
  {
    "method": "Network.responseReceived",
    "params": {
      "requestId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "loaderId": "34017A3A2DC1CB0B4E3A2BDBC64F20B1",
      "timestamp": 37483.528308,
      "type": "Document",
      "response": {
        "url": "https://www.paulirish.com/",
        "status": 200,
        "statusText": "",
        "headers": {
          "date": "Fri, 05 Mar 2021 23:37:04 GMT",
          "content-type": "text/html",
          "set-cookie": "__cfduid=d83583d8f5b34bd7eae780a4d89fb35411614987424; expires=Sun, 04-Apr-21 23:37:04 GMT; path=/; domain=.paulirish.com; HttpOnly; SameSite=Lax",
          "last-modified": "Thu, 15 Oct 2020 04:50:24 GMT",
          "cache-control": "max-age=600",
          "expires": "Fri, 05 Mar 2021 23:47:04 GMT",
          "vary": "Accept-Encoding,User-Agent",
          "content-security-policy": "default-src 'self' https://c.disquscdn.com https://disqus.com; script-src 'self' 'unsafe-inline' https://ssl.google-analytics.com https://api.github.com https://disqus.com https://go.disqus.com https://*.disquscdn.com https://www.google-analytics.com https://paulirish.disqus.com https://platform.twitter.com https://cdn.syndication.twimg.com https://www.googletagmanager.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://platform.twitter.com; img-src * 'self' data:; font-src 'self' data: https://fonts.gstatic.com; connect-src 'self' https://paulirish.com https://fonts.gstatic.com https://firebaseinstallations.googleapis.com https://firebaselogging.googleapis.com https://firebaseremoteconfig.googleapis.com https://www.google-analytics.com https://firebaselogging-pa.googleapis.com; frame-src 'self' https://platform.twitter.com https://accounts.google.com https://jsfiddle.net https://vimeo.com https://player.vimeo.com https://embed.verite.co https://www.youtube.com https://apis.google.com https://disqus.com https://paulirish.wufoo.com; upgrade-insecure-requests; report-uri https://paulirish.report-uri.com/r/d/csp/enforce;",
          "cf-cache-status": "DYNAMIC",
          "cf-request-id": "08a6598991000027f49e97f000000001",
          "expect-ct": "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"",
          "report-to": "{\"max_age\":604800,\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report?s=7mkW3UYZt2GAmiBzKTYWID2%2BM3yVhEUlobwRWp4I8MMaGf4VwXeI49DeS7p7XHMeHcvrNQe%2BtEP0ywXveSL1i1sVnwKrnSZSJDY1Plg1EKnNuQ%3D%3D\"}],\"group\":\"cf-nel\"}",
          "nel": "{\"report_to\":\"cf-nel\",\"max_age\":604800}",
          "server": "cloudflare",
          "cf-ray": "62b72b88e98d27f4-SLC",
          "content-encoding": "br"
        },
        "mimeType": "text/html",
        "requestHeaders": {
          ":method": "GET",
          ":authority": "www.paulirish.com",
          ":scheme": "https",
          ":path": "/",
          "upgrade-insecure-requests": "1",
          "user-agent": "Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4420.0 Mobile Safari/537.36 Chrome-Lighthouse",
          "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
          "sec-fetch-site": "none",
          "sec-fetch-mode": "navigate",
          "sec-fetch-user": "?1",
          "sec-fetch-dest": "document",
          "accept-encoding": "gzip, deflate, br",
          "accept-language": "en-US,en;q=0.9"
        },
        "connectionReused": true,
        "connectionId": 28,
        "remoteIPAddress": "172.67.130.17",
        "remotePort": 443,
        "fromDiskCache": false,
        "fromServiceWorker": false,
        "fromPrefetchCache": false,
        "encodedDataLength": 1341,
        "timing": {
          "requestTime": 37482.745769,
          "proxyStart": -1,
          "proxyEnd": -1,
          "dnsStart": -1,
          "dnsEnd": -1,
          "connectStart": -1,
          "connectEnd": -1,
          "sslStart": -1,
          "sslEnd": -1,
          "workerStart": -1,
          "workerReady": -1,
          "workerFetchStart": -1,
          "workerRespondWithSettled": -1,
          "sendStart": 74.553,
          "sendEnd": 74.71,
          "pushStart": 0,
          "pushEnd": 0,
          "receiveHeadersEnd": 779.721
        },
        "responseTime": 1614987424209.761,
        "protocol": "h2",
        "securityState": "secure",
        "securityDetails": {
          "protocol": "TLS 1.3",
          "keyExchange": "",
          "keyExchangeGroup": "X25519",
          "cipher": "AES_128_GCM",
          "certificateId": 0,
          "subjectName": "sni.cloudflaressl.com",
          "sanList": [
            "paulirish.com",
            "sni.cloudflaressl.com",
            "*.paulirish.com"
          ],
          "issuer": "Cloudflare Inc ECC CA-3",
          "validFrom": 1595548800,
          "validTo": 1627128000,
          "signedCertificateTimestampList": [
            {
              "status": "Verified",
              "origin": "Embedded in certificate",
              "logDescription": "Google 'Argon2021' log",
              "logId": "F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3",
              "timestamp": 1595567700006,
              "hashAlgorithm": "SHA-256",
              "signatureAlgorithm": "ECDSA",
              "signatureData": "304502201E5CE83AA7BAE618403970F57D84CA4A9C511EE06062322FB70F6CE12AC8832002210094815E1767254B1EA7DDD9AAB3618BF4F293315E744F1449B0D716B3E7A92848"
            },
            {
              "status": "Verified",
              "origin": "Embedded in certificate",
              "logDescription": "DigiCert Yeti2021 Log",
              "logId": "5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA",
              "timestamp": 1595567700057,
              "hashAlgorithm": "SHA-256",
              "signatureAlgorithm": "ECDSA",
              "signatureData": "3045022100FC2DCFCFB1EC2C64EC36A6E75938B8C49AD124BD0CC96F16B334E8FE7C9CB6400220609D434E61CD7A92413709FB2038950F368541DCD1BAD4118BB80B0528E85724"
            }
          ],
          "certificateTransparencyCompliance": "compliant"
        }
      },
      "frameId": "F004E3FF02335E2B610F933D672EBECB"
    }
  },

Here's what I noticed:

  • Local log has request...ExtraInfo events. PSI log has none.
  • Local log has a Network.responseReceivedExtraInfo response for the original request that redirects (but no Network.responseReceived like other requests 🤔 ). Devtools log doesn't have that.
  • Of course, only PSI log has X-ProtocolIsH2 headers, as expected.
  • Both logs show a redirectResponse on the Network.requestWillBeSent event for https://www.paulirish.com. In PSI, the X-ProtocolIsH2 header set, so the data is there! But we don't try to read this data from there.

@connorjclark
Copy link
Collaborator

For your particular issue @SoftCreatR, it seems that the server for that URL is responding differently to PSI than to curl. Can you think of any reason that your server would respond with redirects like this:

https://www.softcreatr.com -> https://www.softcreatr.com/ -> https://www.softcreatr.com/ -> ... ?

@SoftCreatR
Copy link
Author

SoftCreatR commented Mar 6, 2021 •
edited
Loading

@connorjclark Thanks for the info regarding the endless loop. This happened because your client did not accept cookies. However, I've just fixed that, because that was unintended and just a side-effect of something that has been implemented recently. This also fixed the PSI response for

https://www.softcreatr.com/login/?url=https%3A%2F%2Fwww.softcreatr.com%2F (Click)

which is the redirect target for

https://www.softcreatr.com/ (Click)

But as you can see, it still reports HTTP/1.1 wrong for the main URL.

@connorjclark
Copy link
Collaborator

Yup, can verify that here too. We'll track that bug in this issue.

@paulirish paulirish mentioned this issue Mar 9, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@connorjclark connorjclark

Labels
needs-investigation needs-priority
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@SoftCreatR @connorjclark @adamraine @devtools-bot