[🐛 Bug]: Remote Webdriver Hangs if any requests don't get a response

[CaptainCsaba]

What happened?

We have been using a Selenium Grid to automate some work processes. Some of our test websites contain outgoing requests that don't get a response while using the grid. These are going out to "google-analytics.com" and "visitor-track.com".

Even though the whole website has loaded, the automated Selenium process hangs for minutes then eventually shuts down and returns a 504 Gateway Timeout.. This happens while everything is loaded, except for 2 requests to the mentioned URL, which get no responses.

Is there a way to tell the webdriver to automatically close the browsers outgoing requests after X seconds? Ideally it should not give an error and stop the process, just close the requests and move onto the next step.

Command used to start Selenium Grid with Docker

We are using selenium-grid on kubernetes cluster (AKS) installed from the latest helm chart

Relevant log output

(not 100% percent sure if this is relevant)

16:02:33.172 INFO [LocalDistributor.newSession] - Session request received by the Distributor:

[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--disable-dev-shm-usage, --start-maximized], extensions: [], prefs: {plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1}}, pageLoadStrategy: normal}]

16:02:34.589 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 66de14f06c77d499f0ec366a0cbc2927

Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 107.0.5304.87, chrome: {chromedriverVersion: 107.0.5304.62 (1eec40d3a576..., userDataDir: /tmp/.com.google.Chrome.YkR7X3}, goog:chromeOptions: {debuggerAddress: localhost:42903}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: LINUX, proxy: {}, se:bidiEnabled: false, se:cdp: ws://172.22.0.102:4444/sess..., se:cdpVersion: 107.0.5304.87, se:vnc: ws://172.22.0.102:4444/sess..., se:vncEnabled: true, se:vncLocalAddress: ws://172.22.0.102:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}

16:04:14.892 WARN [DefaultChannelPipeline.onUnhandledInboundException] - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

org.openqa.selenium.WebDriverException: java.lang.IllegalArgumentException: statusCode

Build info: version: '4.6.0', revision: '79f1c02ae20'

System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-1078-azure', java.version: '11.0.16'

Driver info: driver.version: unknown

at org.openqa.selenium.remote.http.jdk.JdkHttpClient$5.send(JdkHttpClient.java:232)

at org.openqa.selenium.netty.server.WebSocketMessageHandler.lambda$channelRead0$0(WebSocketMessageHandler.java:47)

at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)

at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)

at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566)

at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)

at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: java.lang.IllegalArgumentException: statusCode

at java.net.http/jdk.internal.net.http.websocket.WebSocketImpl.sendClose(WebSocketImpl.java:301)

at org.openqa.selenium.remote.http.jdk.JdkHttpClient$5.lambda$send$2(JdkHttpClient.java:215)

at org.openqa.selenium.remote.http.jdk.JdkHttpClient$5.send(JdkHttpClient.java:222)

... 9 more

16:05:38.271 WARN [DefaultChannelPipeline.onUnhandledInboundException] - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

org.openqa.selenium.WebDriverException: java.lang.IllegalArgumentException: statusCode

Operating System

System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-1078-azure', java.version: '11.0.16', installed from helm chart

Docker Selenium version (tag)

version: '4.6.0', revision: '79f1c02ae20'

[CaptainCsaba]

Is there anything I can do/edit to start a discussion about this?

[joerg1985]

@CaptainCsaba did you try to use a different PageLoadStrategy?

[VietND96]

Based on your env selenium-grid on kubernetes cluster (AKS) and statement Even though the whole website has loaded, the automated Selenium process hangs for minutes then eventually shuts down and returns a 504 Gateway Timeout..
I guess the issue due to ingress, since default timeout nginx is 60 seconds (meanwhile Selenium Grid the default of SE_NODE_SESSION_TIMEOUT and SE_SESSION_REQUEST_TIMEOUT is 300 seconds).
From chart version 0.26.0 onwards, feat #2047 added to set annotations timeout for ingress. Checkout https://github.com/SeleniumHQ/docker-selenium/blob/trunk/charts/selenium-grid/README.md#ingress-configuration for details

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.