Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chart: proactive to set browser args via container env var #2308

Merged
merged 1 commit into from
Jul 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion NodeChrome/wrap_chrome_binary
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,16 @@ done
# Set language environment variable
export LANGUAGE="\$LANGUAGE"

# Capture the filtered environment variables start with "SE_BROWSER_ARGS_" into an array
mapfile -t BROWSER_ARGS_ARRAY < <(printenv | grep ^SE_BROWSER_ARGS_)
# Iterate over the array
for var in "\${BROWSER_ARGS_ARRAY[@]}"; do
# Split the variable into name and value
IFS='=' read -r name value <<< "\$var"
SE_BROWSER_ARGS="\$SE_BROWSER_ARGS \$value"
done

# Note: exec -a below is a bashism.
exec -a "\$0" "$BASE_PATH" --no-sandbox "\$@"
exec -a "\$0" "$BASE_PATH" --no-sandbox \$SE_BROWSER_ARGS "\$@"
_EOF
chmod +x "$WRAPPER_PATH"
26 changes: 25 additions & 1 deletion NodeChromium/wrap_chromium_binary
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,31 @@ mv "$WRAPPER_PATH" "$BASE_PATH"

cat > "$WRAPPER_PATH" <<_EOF
#!/bin/bash

# umask 002 ensures default permissions of files are 664 (rw-rw-r--) and directories are 775 (rwxrwxr-x).
umask 002

# Debian/Ubuntu seems to not respect --lang, it instead needs to be a LANGUAGE environment var
# See: https://stackoverflow.com/a/41893197/359999
for var in "\$@"; do
if [[ \$var == --lang=* ]]; then
LANGUAGE=\${var//--lang=}
fi
done

# Set language environment variable
export LANGUAGE="\$LANGUAGE"

# Capture the filtered environment variables start with "SE_BROWSER_ARGS_" into an array
mapfile -t BROWSER_ARGS_ARRAY < <(printenv | grep ^SE_BROWSER_ARGS_)
# Iterate over the array
for var in "\${BROWSER_ARGS_ARRAY[@]}"; do
# Split the variable into name and value
IFS='=' read -r name value <<< "\$var"
SE_BROWSER_ARGS="\$SE_BROWSER_ARGS \$value"
done

# Note: exec -a below is a bashism.
exec -a "\$0" "$BASE_PATH" --no-sandbox "\$@"
exec -a "\$0" "$BASE_PATH" --no-sandbox \$SE_BROWSER_ARGS "\$@"
_EOF
chmod +x "$WRAPPER_PATH"
11 changes: 10 additions & 1 deletion NodeEdge/wrap_edge_binary
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,16 @@ done
# Set language environment variable
export LANGUAGE="\$LANGUAGE"

# Capture the filtered environment variables start with "SE_BROWSER_ARGS_" into an array
mapfile -t BROWSER_ARGS_ARRAY < <(printenv | grep ^SE_BROWSER_ARGS_)
# Iterate over the array
for var in "\${BROWSER_ARGS_ARRAY[@]}"; do
# Split the variable into name and value
IFS='=' read -r name value <<< "\$var"
SE_BROWSER_ARGS="\$SE_BROWSER_ARGS \$value"
done

# Note: exec -a below is a bashism.
exec -a "\$0" "$BASE_PATH" --no-sandbox "\$@"
exec -a "\$0" "$BASE_PATH" --no-sandbox \$SE_BROWSER_ARGS "\$@"
_EOF
chmod +x "$WRAPPER_PATH"
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ Talk to us at https://www.selenium.dev/support/
* [Configuring the containers](#configuring-the-containers)
* [SE_OPTS Selenium Configuration Options](#se_opts-selenium-configuration-options)
* [SE_JAVA_OPTS Java Environment Options](#se_java_opts-java-environment-options)
* [SE_BROWSER_ARGS_* Add arguments for launching browser](#se_browser_args_-add-arguments-for-launching-browser)
* [Node configuration options](#node-configuration-options)
* [Node configuration relay commands](#node-configuration-relay-commands)
* [Setting Sub Path](#setting-sub-path)
Expand Down Expand Up @@ -978,6 +979,30 @@ You can pass `SE_JAVA_OPTS` environment variable to the Java process.
$ docker run -d -p 4444:4444 -e SE_JAVA_OPTS=-Xmx512m --name selenium-hub selenium/hub:4.22.0-20240621
```

### SE_BROWSER_ARGS_* Add arguments for launching browser

Instead of adding arguments via the browser options from language bindings, for example:

```python
options = ChromeOptions()
options.add_argument('--incognito')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Remote(options=options, command_executor="http://localhost:4444/wd/hub")
```

You also can proactive to force applying arguments directly from (node, standalone or node-docker) container environment variables. Define the environment variable with name starts with `SE_BROWSER_ARGS_` and following by config key is up to you (ensure those are unique when you define multiple arguments). For example:

```bash
docker run -d -p 4444:4444 \
-e SE_BROWSER_ARGS_INCOGNITO=--incognito \
-e SE_BROWSER_ARGS_DISABLE_DSHM=--disable-dev-shm-usage \
selenium/standalone-chrome:latest
```

[List chromium command-line arguments](https://peter.sh/experiments/chromium-command-line-switches/) for your reference.

Note: Currently, this is applicable for node browsers Chrome/Chromium, Edge.

### Node configuration options

The Nodes register themselves through the Event Bus. When the Grid is started in its typical Hub/Node
Expand Down
20 changes: 17 additions & 3 deletions charts/selenium-grid/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ This chart enables the creation of a Selenium Grid Server in Kubernetes.
* [Configuration `global.K8S_PUBLIC_IP`](#configuration-globalk8s_public_ip)
* [Configuration of Nodes](#configuration-of-nodes)
* [Container ports and Service ports](#container-ports-and-service-ports)
* [Configuration of shm size limit for browser nodes](#configuration-of-shm-size-limit-for-browser-nodes)
* [Configuration of Probes](#configuration-of-probes)
* [Node Probes](#node-probes)
* [Distributor Probes](#distributor-probes)
Expand Down Expand Up @@ -384,6 +385,19 @@ edgeNode:
protocol: TCP
```

#### Configuration of shm size limit for browser nodes

By default, node browsers (Chrome/Chromium, Edge) leave the config key `dshmVolumeSizeLimit` as empty. It means the `/dev/shm` volume mount is disabled, and argument `--disable-dev-shm-usage` is passed to the browser via container environment variable (get motivation from [this post](https://www.ministryoftesting.com/articles/navigating-chromedriver-crashes-in-kubernetes-a-tale-of-test-automation-resilience)). You can set another valid value to enable it back. For example:

```yaml
chromeNode:
dshmVolumeSizeLimit: "2Gi"
edgeNode:
dshmVolumeSizeLimit: "2Gi"
```

For Firefox node, the default value is kept as `2Gi`. You can override it via `firefoxNode.dshmVolumeSizeLimit`.

### Configuration of Probes

#### Node Probes
Expand Down Expand Up @@ -859,7 +873,7 @@ This table contains the configuration parameters of the chart and their default
| `chromeNode.service.loadBalancerIP` | `` | Set specific loadBalancerIP when serviceType is LoadBalancer |
| `chromeNode.service.ports` | `[]` | Extra ports exposed in node service |
| `chromeNode.service.annotations` | `{}` | Custom annotations for service |
| `chromeNode.dshmVolumeSizeLimit` | `1Gi` | Size limit for DSH volume mounted in container (if not set, default is "1Gi") |
| `chromeNode.dshmVolumeSizeLimit` | `` | Size limit for DSH volume mounted in container (if not set, default is disabled, e.g "1Gi") |
| `chromeNode.startupProbe.enabled` | `true` | Enable Probe to check pod is started successfully (the following configs see `values.yaml`) |
| `chromeNode.readinessProbe.enabled` | `false` | Enable Readiness probe settings (the following configs see `values.yaml`) |
| `chromeNode.livenessProbe.enabled` | `false` | Enable Liveness probe settings (the following configs see `values.yaml`) |
Expand Down Expand Up @@ -902,7 +916,7 @@ This table contains the configuration parameters of the chart and their default
| `firefoxNode.service.loadBalancerIP` | `` | Set specific loadBalancerIP when serviceType is LoadBalancer |
| `firefoxNode.service.ports` | `[]` | Extra ports exposed in node service |
| `firefoxNode.service.annotations` | `{}` | Custom annotations for service |
| `firefoxNode.dshmVolumeSizeLimit` | `1Gi` | Size limit for DSH volume mounted in container (if not set, default is "1Gi") |
| `firefoxNode.dshmVolumeSizeLimit` | `2Gi` | Size limit for DSH volume mounted in container (if not set, default is disabled, e.g "1Gi") |
| `firefoxNode.startupProbe.enabled` | `true` | Enable Probe to check pod is started successfully (the following configs see `values.yaml`) |
| `firefoxNode.readinessProbe.enabled` | `false` | Enable Readiness probe settings (the following configs see `values.yaml`) |
| `firefoxNode.livenessProbe.enabled` | `false` | Enable Liveness probe settings (the following configs see `values.yaml`) |
Expand Down Expand Up @@ -945,7 +959,7 @@ This table contains the configuration parameters of the chart and their default
| `edgeNode.service.loadBalancerIP` | `` | Set specific loadBalancerIP when serviceType is LoadBalancer |
| `edgeNode.service.ports` | `[]` | Extra ports exposed in node service |
| `edgeNode.service.annotations` | `{}` | Custom annotations for service |
| `edgeNode.dshmVolumeSizeLimit` | `1Gi` | Size limit for DSH volume mounted in container (if not set, default is "1Gi") |
| `edgeNode.dshmVolumeSizeLimit` | `` | Size limit for DSH volume mounted in container (if not set, default is disabled, e.g "1Gi") |
| `edgeNode.startupProbe.enabled` | `true` | Enable Probe to check pod is started successfully (the following configs see `values.yaml`) |
| `edgeNode.readinessProbe.enabled` | `false` | Enable Readiness probe settings (the following configs see `values.yaml`) |
| `edgeNode.livenessProbe.enabled` | `false` | Enable Liveness probe settings (the following configs see `values.yaml`) |
Expand Down
12 changes: 11 additions & 1 deletion charts/selenium-grid/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,10 @@ template:
image: {{ printf "%s/%s:%s" $imageRegistry .node.imageName $imageTag }}
imagePullPolicy: {{ .node.imagePullPolicy }}
env:
{{- if empty .node.dshmVolumeSizeLimit }}
- name: SE_BROWSER_ARGS_DISABLE_DSHM
value: "--disable-dev-shm-usage"
{{- end }}
- name: SE_OTEL_SERVICE_NAME
value: {{ .name | quote }}
- name: SE_NODE_PORT
Expand Down Expand Up @@ -294,8 +298,10 @@ template:
{{- end }}
{{- end }}
volumeMounts:
{{- if not (empty .node.dshmVolumeSizeLimit) }}
- name: dshm
mountPath: /dev/shm
{{- end }}
{{- range $fileName, $value := $.Values.nodeConfigMap.extraScripts }}
- name: {{ tpl (default (include "seleniumGrid.node.configmap.fullname" $) $.Values.nodeConfigMap.scriptVolumeMountName) $ }}
mountPath: {{ $.Values.nodeConfigMap.extraScriptsDirectory }}/{{ $fileName }}
Expand Down Expand Up @@ -413,8 +419,10 @@ template:
{{- end }}
{{- end }}
volumeMounts:
{{- if not (empty .node.dshmVolumeSizeLimit) }}
- name: dshm
mountPath: /dev/shm
{{- end }}
{{- tpl (include "seleniumGrid.video.volumeMounts" .) $ | nindent 8 }}
{{- with $.Values.videoRecorder.resources }}
resources: {{- toYaml . | nindent 10 }}
Expand Down Expand Up @@ -491,10 +499,12 @@ template:
configMap:
name: {{ template "seleniumGrid.node.configmap.fullname" $ }}
defaultMode: {{ $.Values.nodeConfigMap.defaultMode }}
{{- if not (empty .node.dshmVolumeSizeLimit) }}
- name: dshm
emptyDir:
medium: Memory
sizeLimit: {{ default "1Gi" .node.dshmVolumeSizeLimit }}
sizeLimit: {{ .node.dshmVolumeSizeLimit }}
{{- end }}
{{- if eq (include "seleniumGrid.server.secureConnection" $) "true" }}
- name: {{ include "seleniumGrid.tls.fullname" $ | quote }}
secret:
Expand Down
18 changes: 9 additions & 9 deletions charts/selenium-grid/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -724,7 +724,7 @@ chromeNode:
memory: "1Gi"
cpu: "1"
limits:
memory: "1Gi"
memory: "2Gi"
cpu: "1"
# SecurityContext for chrome-node container
securityContext: {}
Expand Down Expand Up @@ -771,8 +771,8 @@ chromeNode:
# targetPort: 5900
# Custom annotations for service
annotations: {}
# Size limit for DSH volume mounted in container (if not set, default is "1Gi")
dshmVolumeSizeLimit: 1Gi
# Size limit for DSH volume mounted in container (if not set, default is disabled, e.g "1Gi")
dshmVolumeSizeLimit: ""
# Priority class name for chrome-node pods
priorityClassName: ""

Expand Down Expand Up @@ -889,7 +889,7 @@ firefoxNode:
memory: "1Gi"
cpu: "1"
limits:
memory: "1Gi"
memory: "2Gi"
cpu: "1"
# SecurityContext for firefox-node container
securityContext: {}
Expand Down Expand Up @@ -932,8 +932,8 @@ firefoxNode:
# targetPort: 5900
# Custom annotations for service
annotations: {}
# Size limit for DSH volume mounted in container (if not set, default is "1Gi")
dshmVolumeSizeLimit: 1Gi
# Size limit for DSH volume mounted in container (if not set, default is disabled, e.g "1Gi")
dshmVolumeSizeLimit: "2Gi"
# Priority class name for firefox-node pods
priorityClassName: ""

Expand Down Expand Up @@ -1048,7 +1048,7 @@ edgeNode:
memory: "1Gi"
cpu: "1"
limits:
memory: "1Gi"
memory: "2Gi"
cpu: "1"
# SecurityContext for edge-node container
securityContext: {}
Expand Down Expand Up @@ -1091,8 +1091,8 @@ edgeNode:
# targetPort: 5900
# Custom annotations for service
annotations: {}
# Size limit for DSH volume mounted in container (if not set, default is "1Gi")
dshmVolumeSizeLimit: 1Gi
# Size limit for DSH volume mounted in container (if not set, default is disabled, e.g "1Gi")
dshmVolumeSizeLimit: ""
# Priority class name for edge-node pods
priorityClassName: ""

Expand Down
1 change: 1 addition & 0 deletions tests/docker-compose-v3-test-node-docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ services:
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_ENABLE_MANAGED_DOWNLOADS=${SELENIUM_ENABLE_MANAGED_DOWNLOADS}
- SE_BROWSER_ARGS_DISABLE_DSHM=--disable-dev-shm-usage
- SE_LOG_LEVEL=${LOG_LEVEL}

selenium-hub:
Expand Down
3 changes: 2 additions & 1 deletion tests/docker-compose-v3-test-parallel.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ services:
replicas: 10
image: selenium/node-${NODE_CHROME}:${TAG}
user: ${UID}
shm_size: 2gb
depends_on:
- selenium-hub
environment:
Expand All @@ -21,6 +20,8 @@ services:
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_ENABLE_MANAGED_DOWNLOADS=true
- SE_BROWSER_ARGS_DISABLE_DSHM=--disable-dev-shm-usage
- SE_BROWSER_ARGS_INCOGNITO=--incognito --incognito
- SE_LOG_LEVEL=${LOG_LEVEL}

firefox:
Expand Down