Skip to content

Commit

Permalink
docs: Update readme to use new instructions for loading JS plugins (#113
Browse files Browse the repository at this point in the history
)

The process has changed, no more manifest.json is required
  • Loading branch information
mofojed authored Nov 16, 2023
1 parent 8ac2192 commit 41fedda
Showing 1 changed file with 51 additions and 49 deletions.
100 changes: 51 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# Plugin modules

The `plugins` folder contains internally developed Plugin modules.
The `plugins` folder contains internally developed Plugin modules.
Each folder contains either a python plugin, a javascript plugin, or both.
The python plugin is housed in the top level of each plugin folder, and the javascript plugin is housed in the `src/js`
The python plugin is housed in the top level of each plugin folder, and the javascript plugin is housed in the `src/js`
folder within each individual plugin folder.

## Python Plugins

Each python plugin should have code stored in a non-colliding path within the `deephaven` folder. Generally, this path
will be `deephaven/plugin/<pluginName>`, but it may be different depending on the plugin. For example, the `plotly-express`
will be `deephaven/plugin/<pluginName>`, but it may be different depending on the plugin. For example, the `plotly-express`
plugin is housed in `deephaven/plot/express` so that it has a structure that mirrors Plotly Express.

Each python plugin should also have at least the following:

- `pyproject.toml`
- `setup.cfg`
- `README.md`
Expand All @@ -19,41 +21,67 @@ Each python plugin should also have at least the following:
- Black formatting applied to all python files

## JS Plugins

Each js plugin should be the following:

- Based off the [JS Module Plugin template](https://github.com/deephaven/deephaven-js-plugin-template/)
- Package name `@deephaven/js-plugin-<folderName>`
- Independent versioning, `npm install`, `npm run build`
- Exported as a CJS bundle
- Externalize `react`, `react-dom`, `redux`, and `react-redux` if used
- Externalize `react`, `react-dom`, `redux`, `react-redux`, and any appropriate `@deephaven/*` packages if used
- Add it as a `rollupOptions.external` in `vite.config.ts`

## Development

Start by setting up the python venv and pre-commit hooks.

Black and blacken-docs formatting is setup through a pre-commit hook.
### Pre-commit hooks/Python formatting

Black and blacken-docs formatting is setup through a pre-commit hook.
To install the pre-commit hooks, run the following commands from the root directory of this repo:

```shell
python -m venv .venv
source .venv/bin/activate
pip install --upgrade -r requirements.txt
pre-commit install
```
```

This will setup a venv, activate it, and install the pre-commit hooks. The hooks will run on every commit.
You can verify that pre-commit is setup by testing with the following:

```shell
pre-commit run --all-files
```

All steps should pass.

The above steps will also set up `tox` to run tests for the python plugins that support it.
### Running Python tests

The above steps will also set up `tox` to run tests for the python plugins that support it.
You can run tests with the following command from the `plugins/<plugin>` directory:

```shell
tox -e py
```

Then, run `npm install` to install js dependencies.
### Running plugin against deephaven-core

#### Building Python plugins for development

Build and install the plugin wheels for the plugins, plotly and matplotlib in this example, into the deephaven-core venv.
See READMEs in the directories of the python plugins you're working with for specific packages to install for development with that plugin.

You can build the wheels using the following commands (or similar for other plugins) from the root directory of this repo:

```
python -m build --wheel plugins/matplotlib
python -m build --wheel plugins/plotly
```

#### Building JS plugins for development

Run `npm install` to install js dependencies.

You can build the js plugin(s) in watch mode from the root directory of this repo by using the following commands:

Expand All @@ -64,78 +92,52 @@ cd plugins/plugin
npm start # starts just the current directory plugin in watch mode
```

This will rebuild the plugin(s) any time the source changes. Changes can be reflected by refreshing the Deephaven web IDE page after the build finishes.

To build for publishing, run

```shell
npm run build
```

Next, create a `manifest.json` file in the `plugins` directory of this project or modify the existing one.
This will rebuild the plugin(s) any time the source changes. However, you still need to restart the deephaven-core server each time a change is made for the change to be picked up.

In it, there should be JSON containing a plugins object containing metadata about all plugins to use, e.g.:
```
{
"plugins": [
{
/** Name of the plugin, and relative path from this manifest to the plugin's root folder. */
"name": "...",
#### Running deephaven-core

/** Version of the plugin. */
"version": "...",
##### Build deephaven-core

/** Location of the primary entry point for the plugin. */
"main": "..."
},
...
]
}
```
Build [deephaven-core](https://github.com/deephaven/deephaven-core) using the directions [here](https://deephaven.io/core/docs/how-to-guides/launch-build/#build-and-run-deephaven).

For example, if using matplotlib and plotly plugins with version 0.1.0, the file looks like this:
##### Install python plugin wheels

```
{
"plugins": [
{ "name": "matplotlib", "version": "0.1.0", "main": "src/js/dist/index.js" },
{ "name": "plotly", "version": "0.1.0", "main": "src/js/dist/index.js" }
]
}
```

Then, build [deephaven-core](https://github.com/deephaven/deephaven-core) using the directions [here](https://deephaven.io/core/docs/how-to-guides/launch-build/#build-and-run-deephaven).

Finally, install the plugin wheels for the plugins, plotly and matplotlib in this example, into the deephaven-core venv.
Then, install the python plugin wheels for the plugins, plotly and matplotlib in this example, into the deephaven-core venv.
See READMEs in the directories of the python plugins you're working with for specific packages to install for development with that plugin.

You can build the wheels using the following commands (or similar for other plugins) from the root directory of this repo:

```
python -m build --wheel plugins/matplotlib
python -m build --wheel plugins/plotly
```

Substitute in your local wheel locations for the wheels in the following command.
Note that `<deephaven-plugins-path>` is the path to this repo.

```
pip install <deephaven-plugins-path>/plotly/plugins/dist/deephaven_plugin_plotly-0.0.1.dev2-py3-none-any.whl <deephaven-plugins-path>/plugins/matplotlib/dist/deephaven_plugin_matplotlib-0.1.1-py3-none-any.whl
```

If installing multiple wheels, you can use the following shorthand to install all built wheels:

```
pip install <deephaven-plugins-path>/plugins/*/dist/*.whl
```

If you're reinstalling the python wheels without a version bump (generally for the purpose of development), you'll want to add the `--force-reinstall` tag. The `--no-deps` tag is also recommended as `--force-reinstall` will update all the dependencies as well, which is generally unnecessary.
For example, on reinstalls the above command becomes
For example, on reinstalls the above command becomes

```
pip install --force-reinstall --no-deps <deephaven-plugins-path>/plugins/*/dist/*.whl
```

Using the path to your local deephaven-js-plugins repo where the manifest.json is contained (which will be in the `plugins` directory if you followed the steps above), start the server with the following command:
##### Start deephaven-core

Finally, start up Deephaven with the appropriate js-plugin flags using the path to your deephaven-plugins repo. For example, to start with the matplotlib and plotly plugins, start the server with the following command:

```
START_OPTS="-Ddeephaven.jsPlugins.resourceBase=<deephaven-plugins-path>deephaven-js-plugins/plugins" ./gradlew server-jetty-app:run
START_OPTS="-Ddeephaven.jsPlugins.@deephaven/js-plugin-matplotlib=<deephaven-plugins-path>/plugins/matploltib/src/js -Ddeephaven.jsPlugins.@deephaven/js-plugin-plotly=<deephaven-plugins-path>/plugins/plotly/src/js" ./gradlew server-jetty-app:run
```

The Deephaven IDE can then be opened at http://localhost:10000/ide/, with your plugins ready to use.

0 comments on commit 41fedda

Please sign in to comment.