Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add developer quickstart guide for starting a local server #3382

Merged
merged 3 commits into from
Jan 31, 2023
Merged

Add developer quickstart guide for starting a local server #3382

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
357f45e
Add quickstart guide for local groovy and python server startup
devinrsmith Jan 31, 2023
ffe2339
Add developer quickstart guide for starting a local server
devinrsmith Jan 31, 2023
543e724
Improvements
devinrsmith Jan 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 59 additions & 61 deletions server/jetty-app/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,90 +1,88 @@
# Native packaging for Deephaven Jetty server

### Setting up a Python virtual environment

This is an optional prerequisite, but lets the later commands to run the server work without
specifying groovy as the console language. If you skip this step, be sure to use groovy, or
there will be errors on startup indicating that the Python environment is not suitable.

See https://github.com/deephaven/deephaven-core/issues/1657 for more discussion

1. On MacOS there is an extra patch to apply at this time, to add an extra argument to clang,
see the first code snippet of the comment at
https://github.com/deephaven/deephaven-core/issues/1657#issuecomment-989040798 for specifics.
1. Make sure Python is installed with a shared library. For example, if using `pyenv install`,
be sure to first set `PYTHON_CONFIGURE_OPTS="--enabled-shared"`.
1. Make a new directory for a virtual environment, and set it up:
```shell
$ mkdir dh-py && cd dh-py
$ python -m venv local-jetty-build
$ source local-jetty-build/bin/activate # this must be re-run for each new shell
$ cd -
```
1. Build and install wheels for deephaven-jpy and deephaven:
```shell
$ python -m pip install --upgrade pip # First upgrade pip
$ pip install wheel
$ export DEEPHAVEN_VERSION=0.21.0 # this should match the current version of your git repo

$ cd py/jpy
$ export JAVA_HOME=/path/to/your/java/home # Customize this to fit your computer
$ python setup.py bdist_wheel
$ pip install dist/deephaven_jpy-0.21.0-cp39-cp39-linux_x86_64.whl # This will vary by version/platform
$ cd -
# server-jetty-app

$ cd Integrations/python
$ python setup.py bdist_wheel
$ pip install dist/deephaven-0.21.0-py2.py3-none-any.whl
$ cd -
```
## Native development

This document is oriented towards getting a server up for local development.
If you are interested in running a native production release, please see
[https://deephaven.io/core/docs/how-to-guides/configuration/native-application/](https://deephaven.io/core/docs/how-to-guides/configuration/native-application/).

### Build
### Groovy Quickstart

```shell
./gradlew server-jetty-app:build
./gradlew server-jetty-app:run -Pgroovy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should mention that these commands are run from the root of the repo, not from this folder.

```

produces
### Python Quickstart

* `server/jetty-app/build/distributions/server-jetty-<version>.tar`
* `server/jetty-app/build/distributions/server-jetty-<version>.zip`
1. Setup virtual environment:

### Run
```shell
python -m venv /tmp/my-dh-venv
source /tmp/my-dh-venv/bin/activate
```

The above artifacts can be uncompressed and their `bin/start` script can be executed:
1. Build and install wheel

```shell
START_OPTS="-Ddeephaven.console.type=groovy" bin/start
```
```shell
./gradlew py-server:assemble

Alternatively, the uncompressed installation can be built directly by gradle:
# replace with the appropriate <version>
pip install "py/server/build/wheel/deephaven_core-<version>-py3-none-any.whl[autocomplete]"

# To install without the optional `autocomplete` feature, run:
# pip install "py/server/build/wheel/deephaven_core-<version>-py3-none-any.whl"
```

1. Run

```shell
./gradlew server-jetty-app:run
```

**Note:**

* This is not an exhaustive guide to managing python environments
* Depending on your OS and how your PATH is setup, you may need to use `python3`, or a path to the explicit python version you want to use
* You may choose to setup a "permanent" virtual environment location
* You'll need to re-install the wheel anytime you are making python code changes that affect the wheel
* `pip` can be a pain if you are trying to (re-)install a wheel with the same version number as before
* A `pip install --force-reinstall --no-deps "py/server/build/wheel/deephaven_core-<version>-py3-none-any.whl[autocomplete]"` may do the trick
* You can install other python packages in your venv using `pip install <some-other-package>`
* You can setup multiple virtual environments, and switch between them as necessary using `source /path/to/other-venv/bin/activate`
* You can de-activate the virtual environment by running `deactivate`
* You can use the `VIRTUAL_ENV` environment variable instead of sourcing / activating virtual environments: `VIRTUAL_ENV=/my/venv ./gradlew server-jetty-app:run`

### Start script

To create a more production-like environment, you can create and invoke the start script instead of running via gradle:

```shell
./gradlew server-jetty-app:installDist
./server/jetty-app/build/install/server-jetty/bin/start
```

And then run via:

```shell
START_OPTS="-Ddeephaven.console.type=groovy" ./server/jetty-app/build/install/server-jetty/bin/start
```
See [https://deephaven.io/core/docs/how-to-guides/configuration/native-application/](https://deephaven.io/core/docs/how-to-guides/configuration/native-application/)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these options not apply in the ./gradlew scenario? Or should the link to this config info be under the Configuration section below?

Copy link
Member Author

@devinrsmith devinrsmith Jan 31, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's "mostly" equivalent, but there may be differences. The big difference right now is EXTRA_CLASSPATH is not supported.

The other environment variables, which are picked up at runtime, should work (DEEPHAVEN_APPLICATION, DEEPHAVEN_DATA_DIR, DEEPHAVEN_CACHE_DIR, DEEPHAVEN_CONFIG_DIR, and DEEPHAVEN_QUIET).

for options when invoking the start script.

### Configuration

Finally, Gradle can be used to update the build and run the application in a single step:
The `START_OPTS` environment variable is used to set JVM arguments. For example:

```shell
./gradlew server-jetty-app:run -Pgroovy
START_OPTS="-Xmx12g" ./gradlew server-jetty-app:run
```

### Internals
While configuration properties can be inherited via JVM system properties (`-Dmy.property=my.value`), you may prefer to
set persistent configuration properties in the `<configDir>/deephaven.prop` file.
On Linux, this file is `~/.config/deephaven/deephaven.prop`.
On Mac OS, this file is `~/Library/Application Support/io.Deephaven-Data-Labs.deephaven/deephaven.prop`.

`server-jetty-app` is configured by default to include code that depends on JVM internals via
`--add-exports java.management/sun.management=ALL-UNNAMED`. To disable this, set the gradle property `-PexcludeHotspotImpl`.
See [config-dir](https://deephaven.io/core/docs/how-to-guides/configuration/native-application/#config-directory) for more information on `<configDir>`.

`server-jetty-app` is configured by default to include code that depends on JVM internals via
`--add-exports java.base/jdk.internal.misc=ALL-UNNAMED`. To disable this, set the gradle property `-PexcludeClockImpl`.
See [config-file](https://deephaven.io/core/docs/how-to-guides/configuration/config-file/) for more information on the configuration file format.

### Configuration / SSL
### SSL

By default, the server starts up on all interfaces with plaintext port 10000 (port 443 when SSL is enabled), a token
expiration duration of 5 minutes, a scheduler pool size of 4, and a max inbound message size of 100 MiB.
Expand Down