Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update server-base and server-slim-base to production-like images #3336

Merged

Conversation

devinrsmith
Copy link
Member

This was a longer-than-expected effort to close the gap between production images, and integration testing images. The big updates: the base images are now based on ubuntu:22.04 (instead of an old debian), and the python version is now 3.10 (instead of an old 3.7). These images are now being produced with Dockerfiles from https://github.com/deephaven/deephaven-server-docker that are as similar to the production images as possible.

Necessary testing fixes already done in #3331

This was a longer-than-expected effort to close the gap between production images, and integration testing images. The big updates: the base images are now based on ubuntu:22.04 (instead of an old debian), and the python version is now 3.10 (instead of an old 3.7). These images are now being produced with Dockerfiles from https://github.com/deephaven/deephaven-server-docker that are as similar to the production images as possible.

Necessary testing fixes already done in deephaven#3331
@devinrsmith
Copy link
Member Author

Looks like some sphinx typing/documentation issues w/ 3.10.

@jmao-denver
Copy link
Contributor

I was able to make the sphinx task pass locally with the following change.

diff --git a/sphinx/source/conf.py b/sphinx/source/conf.py
index 5f3ee1b4c9..e8116e9c9a 100644
--- a/sphinx/source/conf.py
+++ b/sphinx/source/conf.py
@@ -85,10 +85,26 @@ jvm_properties = {
             'deephaven.dataDir': os.path.realpath(workspace),
         }

+jvm_options = {
+    '-XX:InitialRAMPercentage=25.0',
+    '-XX:MinRAMPercentage=70.0',
+    '-XX:MaxRAMPercentage=80.0',
+
+    # Allow netty to (reflectively) access java.nio.Buffer fields
+    '--add-opens=java.base/java.nio=ALL-UNNAMED',
+
+    # Allow our hotspot-impl project to access internals
+    '--add-exports=java.management/sun.management=ALL-UNNAMED',
+
+    # Allow our clock-impl project to access internals
+    '--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED',
+}
+
 from deephaven_internal import jvm
 jvm.init_jvm(
     jvm_classpath=glob(os.environ.get('DEEPHAVEN_CLASSPATH')),
     jvm_properties=jvm_properties,
+    jvm_options=jvm_options,
 )

 import jpy
@@ -96,6 +112,7 @@ py_scope_jpy = jpy.get_type("io.deephaven.engine.util.PythonScopeJpyImpl").ofMai
 py_dh_session = jpy.get_type("io.deephaven.integrations.python.PythonDeephavenSession")(py_scope_jpy)
 py_dh_session.getExecutionContext().open()

+
 import deephaven
 docs_title = "Deephaven python modules."
 package_roots = [jpy, deephaven]

@devinrsmith
Copy link
Member Author

I also needed to update the location of grpc_health_probe to match the production location

jmao-denver
jmao-denver previously approved these changes Jan 23, 2023
Copy link
Contributor

@jmao-denver jmao-denver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and partially tested locally

'-XX:MinRAMPercentage=70.0',
'-XX:MaxRAMPercentage=80.0',

# Allow netty to (reflectively) access java.nio.Buffer fields
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this valid? Only netty? Does sphinx still use netty?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the netty part is too specific. It looks like it's also required by arrow, and potentially other low-level components. I've removed the explicit reference to "netty".

@devinrsmith devinrsmith merged commit c695b4c into deephaven:main Jan 24, 2023
@devinrsmith devinrsmith deleted the nightly/use-production-like-base-images branch January 24, 2023 17:28
@github-actions github-actions bot locked and limited conversation to collaborators Jan 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants