v0.17.0

Jetty transition

This release marks our transition from Netty to Jetty.

The image ghcr.io/deephaven/server:0.17.0 (and related images) are now Jetty based. This greatly simplifies the amount of orchestration necessary to get Deephaven up and running in a containerized environment, and enhances the feature set that the server itself can provide.

For example, a minimal setup via docker is now as easy as:

docker run --rm -p 10000:10000 ghcr.io/deephaven/server:0.17.0

For setups that involve docker-compose.yml, a minimal example looks like:

version: "3"

services:
  server:
    image: ghcr.io/deephaven/server:0.17.0
    ports:
      - "10000:10000"

Notice there is no need for the web, grpc-proxy, nor envoy services in this setup.

The notebook storage is provided by default via the /data/notebooks directory. To mount a local /data volume to serve as storage, you can add a volume mount via docker:

docker run --rm -p 10000:10000 -v /path/to/your/data:/data ghcr.io/deephaven/server:0.17.0

Please reach out via Deephaven Community Slack if you have any questions about this release or about the transition process.

Docker images

• ghcr.io/deephaven/server:0.17.0
• ghcr.io/deephaven/server-slim:0.17.0
• ghcr.io/deephaven/server-all-ai:0.17.0
• ghcr.io/deephaven/server-nltk:0.17.0
• ghcr.io/deephaven/server-pytorch:0.17.0
• ghcr.io/deephaven/server-sklearn:0.17.0
• ghcr.io/deephaven/server-tensorflow:0.17.0

These images are amd64 and arm64 compatible.

For those that need time to transition, we are still providing the Netty based images with the -netty suffix. For example ghcr.io/deephaven/server-netty:0.17.0. We expect this will be the last release for Netty-based images.

What's Changed

• Write Encoding stats with data. by @abaranec in #2555
• Post release bump from 0.16.0 to 0.17.0 by @devinrsmith in #2802
• Bump docker base images by @devinrsmith in #2804
• Bump node and python images by @devinrsmith in #2807
• Update py/embedded-server setup for PyPi by @jjbrosnan in #2739
• Allow multiple concurrent DynamicTableRow writers by @nbauernfeind in #2805
• When an ExportObject runner throws StatusRuntimeException forward to client by @nbauernfeind in #2794
• ConstructSnapshot's SharedContext needs to be reset per rowset by @nbauernfeind in #2809
• Refactor CompilerTools into QueryCompiler by @nbauernfeind in #2808
• Arrow Flight Auth 1 and 2 by @nbauernfeind in #2713
• Make DeephavenSessionTestBase use NoLanguageDeephavenSession by @nbauernfeind in #2811
• Use ugp ex-lock to control ticking table ticking by @jmao-denver in #2819
• Updated C++ client code, tests, and examples. by @kosak in #2818
• C++ client examples used in our demo videos by @kosak in #2821
• Fix improper initial magic number placement in Parquet writing, thereby also "fixing" dictionary offset 0 issue by @rcaudy in #2820
• C++ client: fix documentation cross-references, fill in some more docs. by @kosak in #2823
• Remove Python specific checks for parallel evaluation of formulas by @jmao-denver in #2813
• Write a unit test to verify that @FinalDefault is respected by @nbauernfeind in #2825
• Bump to slf4j 2.0.0 and logback 1.4.0 by @devinrsmith in #2826
• Bump jetty to 11.0.11 by @devinrsmith in #2827
• Use double hashing for HashMapBase by @devinrsmith in #2810
• Remove deprecated uses of java.security by @nbauernfeind in #2830
• Merge py integration tests into the main suite by @jmao-denver in #2834
• Support numpy.bool_ to Java Boolean in array() by @jmao-denver in #2839
• Correctly base64 encode binary gRPC trailers to websocket frames by @niloc132 in #2842
• Fixed C++ build instructions and some missing headers by @SuperTails in #2726
• If a TeeStream is gc'd, don't close underlying streams by @niloc132 in #2837
• Package proto files in the top level of the jar based on them by @niloc132 in #2844
• TicketResolver: return failed export object instead of throwing by @nbauernfeind in #2845
• Add version in deephaven_server by @jmao-denver in #2861
• Improve RowSetBuilderSequential performance when adding keys from a chunk by @lbooker42 in #2856
• Bump web version v0.17.0 by @mofojed in #2864
• Update PartitionTable to use RowSetBuilder instead of RowSet.insert() by @lbooker42 in #2852
• Fix the table used for state manager sizing in ticking left + static right NJ when building from right by @rcaudy in #2877
• Netty enable SO_REUSEADDR to fix bind to port 8080 races by @nbauernfeind in #2879
• Add a vectorized ternary if to the query library. by @chipkent in #2876
• Remove dependency on python-dev-base, follow up to #2583 by @devinrsmith in #2878
• Bump images, add helper script by @devinrsmith in #2880
• Issue #2870: make gradleVersion consistent with checked in wrapper by @JamesXNelson in #2871
• Remove java execution logic during table.py module init by @devinrsmith in #2872
• SessionState should tryManage as dependencies may no longer be live by @nbauernfeind in #2884
• Add query language functions to get data structure sizes. by @chipkent in #2885
• Subscribe to all columns for stream tables by @nbauernfeind in #2882
• Fix test_auto_locking_partitioned_table by @devinrsmith in #2892
• Make io.deephaven.app.GcApplication#enabled public by @devinrsmith in #2891
• Fix improper call to "manage" for static snapshot tables in Table.snapshot implementation by @rcaudy in #2894
• Fix preview columns failing on boolean[] columns by @nbauernfeind in #2895
• Wrap ExecutionContext in Python by @jmao-denver in #2867
• Fix eager formula initialization not seeing new columns by @nbauernfeind in #2896
• Wrap consumeToPartitionedTable in Python by @jmao-denver in #2904
• Address integer overflow in static NJ and AJ; clean up wider-than-necessary type usage and eliminate legacy chunked hashers in NJ and AJ by @rcaudy in #2906
• Use builders for GroupBy initialization and updates by @lbooker42 in #2886
• Bump node and protoc base images by @devinrsmith in #2915
• Try to Pool / re use compressors where possible. Fixes #2788 by @abaranec in #2792
• Interchange the vec and array names in the query library by @chipkent in #2917
• Remove unused commons- dependencies by @devinrsmith in #2922
• Bump to jpy 0.12.0 by @devinrsmith in #2918
• Wrap the partitionedAggBy convenience mtd in py by @jmao-denver in #2907
• Fix modified row set for QueryTable.snapshot when rows have been added, and correct row set cleanup when not notifying. Also port some DHE fixes. by @rcaudy in #2921
• Convert array to vec to fix failing tests by @chipkent in #2924
• Bring latest C++ client up to main by @kosak in #2928
• Ensure python session execution context is used for python tests by @devinrsmith in #2926
• Fix QueryScope uri resolution from python by @devinrsmith in #2927
• Compute targetUrl based on ServerConfig by @devinrsmith in #2925
• Always create an empty array, in case the provided columns are null by @niloc132 in #2933
• Remove anonymous authentication warning logging by @devinrsmith in #2935
• Set correct cache headers for js api/ui resources by @niloc132 in #2937
• Implement gRPC storage API for notebooks and layouts by @niloc132 in #2843
• Version bump web v0.19.0 by @mofojed in #2939
• Only set storage path to $rootDir/data when run from gradle by @niloc132 in #2940
• De-root nginx (and grpc-proxy) by @JamesXNelson in #2847
• Add all-ai server docker image by @JamesXNelson in #2849
• Add HealthStatusManager and correctly enterTerminalState by @JamesXNelson in #2848
• In the cpp client, fix a compilation issue with optional definitions missing from table.cc. by @jcferretti in #2914
• Switch over developer prepareCompose workflow to jetty by @devinrsmith in #2941

Full Changelog: v0.16.1...v0.17.0