Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RemoteWebDriver cannot be cast to HasDevTools #9803

Closed
boris779 opened this issue Sep 8, 2021 · 37 comments
Closed

RemoteWebDriver cannot be cast to HasDevTools #9803

boris779 opened this issue Sep 8, 2021 · 37 comments
Labels

Comments

@boris779
Copy link
Contributor

boris779 commented Sep 8, 2021

🐛 Bug Report

It's not possible to use the DevTools with the RemoteDriver. I was told to use the HasDevTools Interface but it still not working when using a RemoteWebDriver (with a Grid)

java.lang.ClassCastException: class org.openqa.selenium.remote.RemoteWebDriver cannot be cast to class org.openqa.selenium.devtools.HasDevTools (org.openqa.selenium.remote.RemoteWebDriver and org.openqa.selenium.devtools.HasDevTools are in unnamed module of loader 'app')

Detailed steps to reproduce the behavior:

webDriver is RemoteWebDriver which is used in a SeleniumGrid Environment.

Java:
DevTools devTools = ((HasDevTools) webDriver).getDevTools();

Kotlin:
val devTools: DevTools = (webDriver as HasDevTools).devTools

Used Selenium Version: 4.0.0-rc-1 (also tested with beta-3 and beta-4)

Environment

OS: Ubuntu 21
Browser: Chrome
Browser version: > 87
Browser Driver version: >87
Language Bindings version: Java

@pujagani
Copy link
Contributor

Thank you for providing the details. I am sharing a working code snippet below that uses Augmenter. Essentially, Augmenter enhances the RemoteWebDriver to allow the use for DevTools apis.

public class ChromeDevToolsAugmenter {

  public static void main(String[] args) throws MalformedURLException {

    ImmutableCapabilities capabilities = new ImmutableCapabilities("browserName", "chrome");
    WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444"), capabilities);

    driver = new Augmenter().augment(driver);

    driver.get("https://www.duckduckgo.com");
    DevTools devTools = ((HasDevTools) driver).getDevTools();
    devTools.createSession();

    devTools.send(Log.enable());
    devTools.send(Browser.setWindowBounds(new WindowID(1), new Bounds(
      Optional.of(20),
      Optional.of(20),
      Optional.of(20),
      Optional.of(20),
      Optional.of(WindowState.NORMAL))));

    driver.quit();
  }
}

Please try it out with Augmenter and provide feedback if the error still persists. Will be happy to dig into it.

@boris779
Copy link
Contributor Author

Hi!.
No sorry, I tried some code examples but still get the same error.
2021-09-21_19h09_07

java.lang.ClassCastException: class org.openqa.selenium.remote.RemoteWebDriver cannot be cast to class org.openqa.selenium.devtools.HasDevTools (org.openqa.selenium.remote.RemoteWebDriver and org.openqa.selenium.devtools.HasDevTools are in unnamed module of loader 'app')

@boris779
Copy link
Contributor Author

I added my source code to GitHub.
https://github.com/boris779/selenium4_caps_bug/tree/devTools

Beware that you switch to the devTools Branch, I created the repo for another bug

@boris779
Copy link
Contributor Author

There is also a runconfig in the ressources folder called "Dev Tools Selenium 4 remote chrome" which maybe help you

@pujagani
Copy link
Contributor

pujagani commented Sep 22, 2021

Thank you for sharing the details and the code. Appreciate it. I will try it out on my machine and revert. However, based on the error message shared, I had a question. What Java version do you use? I am suspecting a Java version issue, since Java 9 introduced modules. Similarly, Java 16 has modules as well that causes a similar error as you shared.

@pujagani
Copy link
Contributor

pujagani commented Sep 22, 2021

I have run the Cucumber run-config as mentioned and I am unable to reproduce the issue. All tests pass as expected. The only thing I changed in the code is the platform since I am on MacOS. The tests are passing after multiple runs.
Screenshot 2021-09-22 at 1 12 33 PM

Grid is creating a CDP session and upgrading the connection to use WebSocket correctly.
Screenshot 2021-09-22 at 1 12 45 PM

OS: MacOS
Browser: Chrome
Browser version: 93
Browser Driver version: 93
Language Bindings version: rc-1, beta-3
Java version: 11

I see that the cucumber-java8 used in the project does not support Java 12+, so that eliminates Java 16 module issue. Is there anything else I should be considering? or anything else I can do to recreate the issue?

@boris779
Copy link
Contributor Author

boris779 commented Sep 27, 2021

Hi!

I tried it with JDK 17 and JDK 11. Both not working, but get different error messages/exceptions.

To get cucumber-java8 running with Java12+ you have to change the "Before lunch" actions in the run config. It's important to:

  • clean when changing the JDK Version
  • add a maven Goal: install

This is because of https://stackoverflow.com/questions/67787645/cucumber-kotlin-illegalstateexception-when-launching-test-via-intellij/67817085#67817085

With JDK-17:
Step failed org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://172.25.0.1:4444/session/3816a89326169d124b0b109dc4ad22fe/se/cdp Build info: version: '4.0.0-rc-1', revision: 'bc5511cbda' System info: host: 'enterprise', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.11.0-36-generic', java.version: '17-ea' Driver info: driver.version: unknown at org.openqa.selenium.remote.http.netty.NettyWebSocket.<init>(NettyWebSocket.java:104) at org.openqa.selenium.remote.http.netty.NettyWebSocket.lambda$create$3(NettyWebSocket.java:137) at org.openqa.selenium.remote.http.netty.NettyClient.openSocket(NettyClient.java:127) at org.openqa.selenium.devtools.Connection.<init>(Connection.java:73) at org.openqa.selenium.devtools.SeleniumCdpConnection.<init>(SeleniumCdpConnection.java:34) at org.openqa.selenium.devtools.SeleniumCdpConnection.lambda$create$0(SeleniumCdpConnection.java:56) at java.base/java.util.Optional.map(Optional.java:260) at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:54) at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:47) at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:48) at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:29) at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:186) at at.co.boris.webdriversession.webdriverfactory.RemoteChromeDriverFactory.createDriver(RemoteChromeDriverFactory.java:45) at at.co.boris.webdriversession.webdriverfactory.DriverFactory.createWebDriver(DriverFactory.java:43) at at.co.boris.webdriversession.WebDriverSession.<init>(WebDriverSession.java:22) at at.co.boris.webdriversession.WebDriverSessionStore.getOrCreate(WebDriverSessionStore.java:18) at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriverSession(AbstractStepDefs.java:62) at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriver(AbstractStepDefs.java:56) at at.co.boris.stepdefinitions.WikipediaSteps.lambda$new$0(WikipediaSteps.java:26) at ✽.the start page is loaded(file:///home/boris/repos/selenium4_caps_bug/src/test/resources/features/wikipedia.feature:6)

@boris779
Copy link
Contributor Author

boris779 commented Sep 27, 2021

With JDK-11 I get the same error:

`/usr/lib/jvm/java-11-openjdk-amd64/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:46279,suspend=y,server=n -Dbrowser=remote_chrome -DbaseURL=http://www.wikipedia.at -Ddriver.version=93.0 -Dbrowser.version=93.0 -Dselenium.grid=http://localhost:4444 -Dorg.jetbrains.run.directory=/home/boris/repos/selenium4_caps_bug/src/test/resources/features -javaagent:/home/boris/idea/plugins/java/lib/rt/debugger-agent.jar -Dfile.encoding=UTF-8 -classpath /home/boris/repos/selenium4_caps_bug/target/test-classes:/home/boris/.m2/repository/io/cucumber/cucumber-java8/6.10.4/cucumber-java8-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/cucumber-core/6.10.4/cucumber-core-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/cucumber-gherkin/6.10.4/cucumber-gherkin-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/cucumber-gherkin-messages/6.10.4/cucumber-gherkin-messages-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/messages/15.0.0/messages-15.0.0.jar:/home/boris/.m2/repository/io/cucumber/tag-expressions/3.0.1/tag-expressions-3.0.1.jar:/home/boris/.m2/repository/io/cucumber/cucumber-expressions/10.3.0/cucumber-expressions-10.3.0.jar:/home/boris/.m2/repository/io/cucumber/datatable/3.5.0/datatable-3.5.0.jar:/home/boris/.m2/repository/io/cucumber/cucumber-plugin/6.10.4/cucumber-plugin-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/docstring/6.10.4/docstring-6.10.4.jar:/home/boris/.m2/repository/io/cucumber/html-formatter/13.0.0/html-formatter-13.0.0.jar:/home/boris/.m2/repository/io/cucumber/create-meta/4.0.0/create-meta-
.
.
.

Testing started at 16:23 ...
Connected to the target VM, address: '127.0.0.1:46279', transport: 'socket'
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
2021-09-27 | 16:23:54.309 | main | INFO | a.c.b.s.WikipediaSteps | ID: | Constructor AbstractStepDefs
2021-09-27 | 16:23:54.462 | main | INFO | a.c.b.s.WikipediaSteps | ID: | Before All Once
Sep 27, 2021 4:23:56 PM org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer createTracer
INFO: Using OpenTelemetry for tracing
Sep 27, 2021 4:24:11 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Sep 27, 2021 4:24:15 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
INFO: Found exact CDP implementation for version 93
Sep 27, 2021 4:24:15 PM org.openqa.selenium.remote.http.WebSocket$Listener onError
WARNING: Invalid Status code=400 text=Bad Request
java.io.IOException: Invalid Status code=400 text=Bad Request
at org.asynchttpclient.netty.handler.WebSocketHandler.abort(WebSocketHandler.java:92)
at org.asynchttpclient.netty.handler.WebSocketHandler.handleRead(WebSocketHandler.java:118)
at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)

Step failed
org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://172.25.0.1:4444/session/49f231a2e40bb712aab1daddae4df952/se/cdp
Build info: version: '4.0.0-rc-1', revision: 'bc5511cbda'
System info: host: 'enterprise', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.11.0-36-generic', java.version: '11.0.11'
Driver info: driver.version: unknown
at org.openqa.selenium.remote.http.netty.NettyWebSocket.(NettyWebSocket.java:104)
at org.openqa.selenium.remote.http.netty.NettyWebSocket.lambda$create$3(NettyWebSocket.java:137)
at org.openqa.selenium.remote.http.netty.NettyClient.openSocket(NettyClient.java:127)
at org.openqa.selenium.devtools.Connection.(Connection.java:73)
at org.openqa.selenium.devtools.SeleniumCdpConnection.(SeleniumCdpConnection.java:34)
at org.openqa.selenium.devtools.SeleniumCdpConnection.lambda$create$0(SeleniumCdpConnection.java:56)
at java.base/java.util.Optional.map(Optional.java:265)
at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:54)
at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:47)
at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:48)
at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:29)
at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:186)
at at.co.boris.webdriversession.webdriverfactory.RemoteChromeDriverFactory.createDriver(RemoteChromeDriverFactory.java:45)
at at.co.boris.webdriversession.webdriverfactory.DriverFactory.createWebDriver(DriverFactory.java:43)
at at.co.boris.webdriversession.WebDriverSession.(WebDriverSession.java:22)
at at.co.boris.webdriversession.WebDriverSessionStore.getOrCreate(WebDriverSessionStore.java:18)
at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriverSession(AbstractStepDefs.java:62)
at at.co.boris.stepdefinitions.AbstractStepDefs.getCurrentWebDriver(AbstractStepDefs.java:56)
at at.co.boris.stepdefinitions.WikipediaSteps.lambda$new$0(WikipediaSteps.java:26)
at ✽.the start page is loaded(file:///home/boris/repos/selenium4_caps_bug/src/test/resources/features/wikipedia.feature:6)`

@boris779
Copy link
Contributor Author

To simplify I also tried your lightweight example and created a new repository

https://github.com/boris779/selenium4_devtools

@pujagani
Copy link
Contributor

To simplify I also tried your lightweight example and created a new repository

https://github.com/boris779/selenium4_devtools

Thank you for providing the details. However, still unable to reproduce the issue with the new repository as well. Which version of Grid are you running? I am running tests using the Java Selenium binding rc-1. The Grid runs in the Standalone mode: java -jar selenium-server-4.0.0-rc-1.jar standalone.

package at.co.boris.devtools;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v92.browser.Browser;
import org.openqa.selenium.devtools.v92.browser.model.Bounds;
import org.openqa.selenium.devtools.v92.browser.model.WindowID;
import org.openqa.selenium.devtools.v92.browser.model.WindowState;
import org.openqa.selenium.devtools.v92.log.Log;
import org.openqa.selenium.remote.Augmenter;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.MalformedURLException;
import java.net.URL;
import java.util.Optional;

import static org.openqa.selenium.remote.CapabilityType.BROWSER_VERSION;

public class Main {

    public static void main(String[] args) throws MalformedURLException {

        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability("browserName", "chrome");
      //  capabilities.setCapability(BROWSER_VERSION, "94.0");

        WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4441"), capabilities);

        driver = new Augmenter().augment(driver);

        driver.get("https://www.duckduckgo.com");
        DevTools devTools = ((HasDevTools) driver).getDevTools();
        devTools.createSession();

        devTools.send(Log.enable());
        devTools.send(Browser.setWindowBounds(new WindowID(1), new Bounds(
                Optional.of(20),
                Optional.of(20),
                Optional.of(20),
                Optional.of(20),
                Optional.of(WindowState.NORMAL))));

        driver.quit();


    }
}

I do not explicitly set the version on the client and the server-side and let the Grid figure out the best match.
OS: MacOS
Browser: Chrome
Browser version: 94.0.4606.61
Browser Driver version: 94.0.4606.61
Language Bindings version: rc-1
Java version: 9, 11

@boris779
Copy link
Contributor Author

boris779 commented Oct 5, 2021

That's really weird, I was also running the selenium grid rc1 but the docker version.

java -jar selenium-server-4.0.0-rc-1.jar standalone -D selenium/standalone-chrome:93.0 '{"browserName": "chrome", "browserVersion": "94.0", "platformName": "linux"}' -D selenium/standalone-firefox:latest '{"browserName": "firefox"}' --detect-drivers false

I will keep on searching

@boris779
Copy link
Contributor Author

boris779 commented Oct 5, 2021

Now we found the issue.

It works with the standalone version of selenium grid rc-1 but not with the docker-version.

@jantekb
Copy link

jantekb commented Oct 18, 2021

Essentially the same exception happens with OpenJDK 13.0.2+8, it depends on the JVM version number that you execute the remote driver in.

@pujagani
Copy link
Contributor

pujagani commented Nov 8, 2021

When I run using java -jar <path-to-jar-with-changes-in-trunk> standalone -D selenium/standalone-chrome:4.1.0-prerelease-20211105 '{"browserName": "chrome"}' -D selenium/standalone-firefox:4.1.0-prerelease-20211105 '{"browserName": "firefox"}' --detect-drivers false --port 4443, I see the following exception on client-side :

Exception in thread "main" org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://localhost:4444/session/d3c5facc7f2fe63e2272d89db4281438/se/cdp
Build info: version: '4.1.0', revision: 'Unknown'
System info: host: 'MacBook-Pro-2021.local', ip: '2405:201:c:1804:2037:3a2f:b4ee:ce26%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.16', java.version: '1.8.0_202'
Driver info: driver.version: unknown
	at org.openqa.selenium.remote.http.netty.NettyWebSocket.<init>(NettyWebSocket.java:104)
	at org.openqa.selenium.remote.http.netty.NettyWebSocket.lambda$create$3(NettyWebSocket.java:137)
	at org.openqa.selenium.remote.http.netty.NettyClient.openSocket(NettyClient.java:127)
	at org.openqa.selenium.devtools.Connection.<init>(Connection.java:77)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.<init>(SeleniumCdpConnection.java:34)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.lambda$create$0(SeleniumCdpConnection.java:56)
	at java.util.Optional.map(Optional.java:215)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:54)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:47)
	at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:48)
	at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:29)
	at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:186)
	at com.company.ChromeDevToolsAugmenter.main(ChromeDevToolsAugmenter.java:36)

After digging for a while, I suspect the issue is probably related to port mapping done here and the response URI rewritten here which return the URI for the client to make the websocket connection with. The rewrite works well for all Grid modes and CDP, except when used with Docker and CDP. Essentially, the Grid(Standalone/Hub/Router) port needs to ensure correct forwarding to the newly created container port.

Locally, if I hard code the container port and pass that in grid-url option while running the jar, the CDP connection is made properly and it does not throw an error. Though that is not the correct way of doing it and it will not work for multiple containers. It was for just locally figuring things out.

@boris779
Copy link
Contributor Author

So can you fix it or create concrete tasks how to fix it?

@pujagani
Copy link
Contributor

I have shared the issue comment above with Diego. He is the core contributor and expert for selenium-docker. He mentioned he is aware of the issue and will help on the same.

@korabel00
Copy link

korabel00 commented Dec 6, 2021

Hi, should I consider my problem as the same since I've got Selenoide and when I'm trying to get my DevTools I got
net/bytebuddy/matcher/ElementMatcher
java.lang.NoClassDefFoundError: net/bytebuddy/matcher/ElementMatcher

            WebDriver driver = new Augmenter().augment(getWebDriver());
            DevTools devTools = ((HasDevTools) driver).getDevTools();

@arsen-papoyan
Copy link

@korabel00 I have also run into that problem (not on Solenoid).
Just added bytebuddy maven dependency to the pom.xml and it worked fine.

@hellfish46
Copy link

hellfish46 commented Dec 9, 2021

Hello, I added bytebuddy maven dependency to the pom.xml but I got classCastException in line
DevTools devTools = ((HasDevTools) driver).getDevTools();:
java.lang.ClassCastException: class org.openqa.selenium.remote.RemoteWebDriver$ByteBuddy$kKiPAnjQ cannot be cast to class org.openqa.selenium.devtools.HasDevTools (org.openqa.selenium.remote.RemoteWebDriver$ByteBuddy$kKiPAnjQ is in unnamed module of loader net.bytebuddy.dynamic.loading.ByteArrayClassLoader @301da01d; org.openqa.selenium.devtools.HasDevTools is in unnamed module of loader 'app')
I use Selenoid (the latest version for today = v1.10.7) and can't understand what I do wrong.

@ixytiris
Copy link

ixytiris commented Dec 9, 2021

i have the same issue , running remote on selenoid

Augmenter augmenter = new Augmenter(); 
WebDriver driver;
	try {
	     driver = augmenter.augment(new RemoteWebDriver(new URL("http://" + RC_HOST) + ":" + (RC_PORT)+ "/wd/hub"), 
              browserOptions));
       }
DevTools devTools = ((HasDevTools) driver).getDevTools();


I am facing the error:
cannot cast 'org.openqa.selenium.remote.RemoteWebDriver to 'org.openqa.selenium.devtools.hasDevTools'

selenium.version: 4.0.0
JDK: 11.0.12

@korabel00
Copy link

Hello, I added bytebuddy maven dependency to the pom.xml but I got classCastException in line DevTools devTools = ((HasDevTools) driver).getDevTools();: java.lang.ClassCastException: class org.openqa.selenium.remote.RemoteWebDriver$ByteBuddy$kKiPAnjQ cannot be cast to class org.openqa.selenium.devtools.HasDevTools (org.openqa.selenium.remote.RemoteWebDriver$ByteBuddy$kKiPAnjQ is in unnamed module of loader net.bytebuddy.dynamic.loading.ByteArrayClassLoader @301da01d; org.openqa.selenium.devtools.HasDevTools is in unnamed module of loader 'app') I use Selenoid (the latest version for today = v1.10.7) and can't understand what I do wrong.

Absolutely the same

@arsen-papoyan
Copy link

I have spent two days on this problem.
These are the details that I used in my project.

  1. Add this after initialization:
    WebDriver driver = new Augmenter().augment(getWebDriver());
    DevTools devTools = ((HasDevTools) driver).getDevTools();
  2. Add bytebuddy maven dependency to pom.xml.
  3. Selenium Grid should also be 4th version.
    Finally, it worked fine.

@hellfish46
Copy link

hellfish46 commented Dec 9, 2021

@arsen-papoyan unfortunately it doesn't work for me. At least for Selenoid.

@diemol diemol closed this as completed in 9bf1af1 Dec 9, 2021
@diemol
Copy link
Member

diemol commented Dec 9, 2021

This issue actually refers to two different situations: "RemoteWebDriver cannot be cast to HasDevTools" and the "Unable to establish websocket connection to http...". The commit fixes the latter. For the earlier, if you are facing that, please open a new issue with all the information requested in the template.

The fix allows the websocket connection when interacting with WebDriver BiDi or CDP over Grid, which seems to be the actual issue being faced by the OP.

Here is some context for the fix...
In an conventional Grid setup, things work well because the Node receiving the WebDriver BiDi or CDP request, is running on the same host where the browser is running. This is not the case with Dynamic Grid (with Docker), because the Node is running in one container, and a new container is started to run the session. Therefore, the browser is running in one container, and the Node in a different one. That is why we need to forward one hop more the websocket request.

It is important to note, that doing WebDriver BiDi or CDP over Dynamic Grid is only supported when all components are running in Docker. A mix is not possible due to the network complexities of Docker.

This fix will be included in a bug release that will happen in the following days.

@jantekb
Copy link

jantekb commented Dec 10, 2021

I have opened #10132 for the casting part

@arsen-papoyan
Copy link

arsen-papoyan commented Jan 4, 2022

Hi all,
About three weeks ago it worked for me, I have no changes in my project and now I am getting this exception:

java.lang.ExceptionInInitializerError
Caused by: java.lang.IllegalArgumentException: Cannot subclass primitive, array or final types: class com.sun.proxy.$Proxy12
at net.bytebuddy.ByteBuddy.subclass(ByteBuddy.java:486)
at net.bytebuddy.ByteBuddy.subclass(ByteBuddy.java:459)
at net.bytebuddy.ByteBuddy.subclass(ByteBuddy.java:356)
at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:179)

@boris779
Copy link
Contributor Author

Hi @diemol !
Sorry for comming back so late.
Did I got it right? If I run everything in docker (I followed the guide on this website https://github.com/SeleniumHQ/docker-selenium) --> I used the docker-compose-v3 and still get an error

Exception in thread "main" org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://172.28.0.3:4444/session/18db31ce0841a3d40eaa8faa98d5778e/se/cdp

@bachhavdipak
Copy link

Any update on this I am also facing an issue while executing my test on experitest platform browser?

@diemol
Copy link
Member

diemol commented Jan 13, 2022

@boris779 please check https://github.com/SeleniumHQ/docker-selenium#grid-url-and-session-timeout

@bachhavdipak please create a separate issue if the problem is different to this one.

@a-tikhomirov
Copy link

Is it possible to get DevTools for Selenoid (not Grid)?

@boris779
Copy link
Contributor Author

@a-tikhomirov yes, it works if I start my grid locally, or use an local WebDriver, but it does not work if I run the docker-version of selenium 4.1.1 #
@diemol I do not get the connection to the timeouts, the connection will be refused immediately:

I created a github repo... it has two branches, main (with the working local grid method) and the docker branch. I only used the docker-compose up command and a lightweight toml config. Maybe I have done something wrong.

@MiVGit
Copy link

MiVGit commented Jan 18, 2022

I was able to get debugger address at Selenoid with
String sessionId = ((RemoteWebDriver) WebDriverRunner.getWebDriver()).getSessionId().toString();
String debuggerUrl = String.format("http://<selenoid_address>/devtools/%s/page", sessionId);
URI uri = new URI(debuggerUrl);

Then create a connection
HttpClient.Factory clientFactory= HttpClient.Factory.createDefault();
Capabilities originalCapabilities = driver.getCapabilities();
Optional connection = Optional.of(new Connection(
clientFactory.createClient(ClientConfig.defaultConfig().baseUri(uri)),
uri.toString()));

And get DevTools with it
CdpInfo cdpInfo = (new CdpVersionFinder()).match(originalCapabilities.getBrowserVersion()).orElseGet(NoOpCdpInfo::new);
Optional devTools = connection.map((conn) -> {
Objects.requireNonNull(cdpInfo);
return new DevTools(cdpInfo::getDomains, conn);
});

Had a problem with new CdpVersionFinder()).match(...) tho.
For my browser originalCapabilities.getBrowserVersion() returned "90.XXX" and matcher could not find suitable cdpInfo for that. I used v96 instead.

@JuanManuelCarames
Copy link

@a-tikhomirov yes, it works if I start my grid locally, or use an local WebDriver, but it does not work if I run the docker-version of selenium 4.1.1 # @diemol I do not get the connection to the timeouts, the connection will be refused immediately:

I created a github repo... it has two branches, main (with the working local grid method) and the docker branch. I only used the docker-compose up command and a lightweight toml config. Maybe I have done something wrong.

I have the same problem....connectiom redused.

@JuanManuelCarames
Copy link

@boris779 did you find a solution through a docker implementation?

@boris779
Copy link
Contributor Author

Sadly not. I tried a lot of things

I didn't made the dynamic grid get running (in linux/ubuntu) all the information getting the docker API accessible didn't worked.

The only version I got running is the standalone variant, but this solution is not acceptable for our project.

@diemol
Copy link
Member

diemol commented Jan 24, 2022

@JuanManuelCarames @boris779
We'd be happy to help through any of our community channels (Slack/IRC/Matrix) , please join. I am there most of the time and I can help.

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators Feb 24, 2022
elgatov pushed a commit to elgatov/selenium that referenced this issue Jun 27, 2022
When having a Dynamic Grid, and a user who wants to do
CDP over Grid, we have the situation that the browser
is not running on the same host where the Node receives
the original CDP request. In this case, we are having a
Node running on one host, and the browser in a different
host/container.

For this, a forward url is being added to identify this
situation and redirect the request to the place where the
browser is running. All this because it is not possible
to access to the browser's debugger address remotely.

Fixes SeleniumHQ#9803
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests