You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Could not read from this url, got java.net.ProtocolException: Too many follow-up requests: 21
We wouldn't want Prout to maintain cookies across separate checkpoint snapshots (each snapshot should be isolated), but when taking a single snapshot, which might redirect (eg in order to acquire cookies), it could be that we do want to maintain cookies.
Prout currently uses OkHttp as the HTTP client used to take checkpoint snapshots:
Documentation on how to get OkHttp to store cookies between requests (with a CookieJar) is limited, but Java 9 introduced an alternative HTTP client, java.net.http.HttpClient:
Some Guardian endpoints like https://support.theguardian.com/uk/contribute now require an Okta cookie, and will redirect endlessly if the cookie can't be dropped (see eg guardian/support-frontend#5152 (comment)):
Giving us an error like this:
We wouldn't want Prout to maintain cookies across separate checkpoint snapshots (each snapshot should be isolated), but when taking a single snapshot, which might redirect (eg in order to acquire cookies), it could be that we do want to maintain cookies.
Prout currently uses OkHttp as the HTTP client used to take checkpoint snapshots:
prout/app/lib/CheckpointSnapshot.scala
Line 40 in 705f02a
Switch HTTP client to
java.net.http.HttpClient
?Documentation on how to get OkHttp to store cookies between requests (with a
CookieJar
) is limited, but Java 9 introduced an alternative HTTP client,java.net.http.HttpClient
:...which seems to supply all of Prout's requirements in a HTTP client, including this new one for cookies:
java.net.CookieManager
seems to offer a suitable level of cookie controlHttpClient.Redirect
lets us enable redirects - and by default, there is a redirect limit of 5, which should be fine.google-search-indexing-observatory
for an example ofjava.net.http.HttpClient
being used asynchronously with Scala code &Future
s.Of course, switching to the JDK HTTP client would also mean we'd get to drop a dependency, which would be nice!
The text was updated successfully, but these errors were encountered: