[🐛 Bug]: Out of memory errors in hub

[krimsonkla]

What happened?

After updating to the latest hub, we started receiving out-of-memory errors on the hub. Attempted setting SE_JAVA_OPTS and seccomp=unconfined. Neither of which helped. After some searching, I found this is a common and recent issue with docker on jammy as reported in the following repositories.
adoptium/temurin-build#2974
adoptium/containers#215

Docker Selenium recently updated to Jammy here: #1923

Command used to start Selenium Grid with Docker

Partial compose file
hub:
        container_name: hub
        image: selenium/hub:latest
        network_mode: "host"
        environment:
          - JAVA_OPTIONS='-Xmx2000M'

Relevant log output

# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.

Operating System

Ubuntu 16.04.7 LTS

Docker Selenium version (tag)

20.10.7, build f0df350

[github-actions]

@krimsonkla, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[diemol]

@krimsonkla do you know what the fix is? I did not see an obvious fix after reading the issues.

[Swapnillll]

Same issue i 'm facing as well after pulling latest images for hub and nodes. For hub i did upgrade OS to jammy on the VM i'm running and can verify it worked. As i have lot of VMs i decided to pull older images with ubuntu 20.

Will verify upgrading the docker engine as well on older ubuntu OS.
Error -
[0.018s][warning][os,thread] Failed to start thread "VM Thread" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
Error occurred during initialization of VM
Cannot create VM thread. Out of system resources.
2023-11-29 10:57:02,368 INFO exited: selenium-grid-hub (exit status 1; not expected)

[VietND96]

I have read on this thread https://stackoverflow.com/a/72841934/14532601
I saw @krimsonkla reported docker engine 20.10.7 is used, how about your engine version @Swapnillll?
Can we try an upgrade on docker engine and confirm?

[Swapnillll]

Ubuntu 16.04.2 LTS
Docker engine version when it failed due to memory issue : Version: 17.05.0-ce. (i know its ancient :) )
After update :
Version: 20.10.7

Verified now , no issue seen for version 20.10.7
07:18:42.048 INFO [NodeServer$1.start] - Starting registration process for Node http://
07:18:42.050 INFO [NodeServer.execute] - Started Selenium node 4.15.0 (revision 1d14b5521b): http://
07:18:42.105 INFO [NodeServer$1.lambda$start$1] - Sending registration event...

Instead of upgrading docker engine i'm going to upgrade all our VMs to jammy though as we have been sitting on older version anyways (16/18 LTS)

[krimsonkla]

@diemol I couldn't find a fix. I had to pin back to the pre jammy images. I spent several hours scouring the net and trying different fixes, none of which helped.

[diemol]

@krimsonkla what about the above comment from @VietND96?

[lszasz]

I am having the same issue as everyone else here:
2023-12-07 16:52:22,629 INFO success: selenium-grid-hub entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
[0.011s][warning][os,thread] Failed to start thread "VM Thread" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
Error occurred during initialization of VM
Cannot create VM thread. Out of system resources.
2023-12-07 16:52:22,652 INFO exited: selenium-grid-hub (exit status 1; not expected)

OS: CentOS Linux 7
Selenium hub: [4.16.0]
Docker version 18.06.3-ce

[ishtiyaq-td]

I am also getting same memory error There is insufficient memory for the Java Runtime Environment to continue..

I was using docker 18 then updated it to 20.10.7 but still not working.

$ docker -v
Docker version 20.10.7, build f0df350

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS"

[diemol]

Seems that an upgrade to latest docker and if you are using Ubuntu then a bump to Jammy is needed.

[liqupan]

I upgraded the version to 20.10.10 and the problem was solved

https://stackoverflow.com/questions/72841549/container-fails-to-start-insufficient-memory-for-the-java-runtime-environment-t/72841934#72841934

[diemol]

Thank you for letting us know. I will close this issue, upgrading solves it.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.