Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add runtime and runtime-async packages #200

Merged
merged 20 commits into from
Jun 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
a9dbcce
feat: add dependencies, config, and package.json for new runtime and …
chrispcampbell Jun 9, 2022
1d22c83
feat: add runtime package sources
chrispcampbell Jun 9, 2022
59f7366
feat: add runtime-async package sources
chrispcampbell Jun 9, 2022
a0a50d6
build: add typedoc and a gen-docs script
chrispcampbell Jun 9, 2022
4009690
build: fail the build if there are any untracked/modified files
chrispcampbell Jun 9, 2022
b647696
docs: add typedoc-generated API docs for runtime package
chrispcampbell Jun 9, 2022
1eda016
docs: unhide WasmModule but keep its properties hidden for now
chrispcampbell Jun 10, 2022
1c170d2
build: fix expansion of sed args in gen-docs script
chrispcampbell Jun 10, 2022
eed4468
build: don't remove docs directory in clean script
chrispcampbell Jun 10, 2022
b948019
docs: use index.md in the breadcrumb links
chrispcampbell Jun 10, 2022
b51c099
docs: hide private members and preserve ordering of members from sour…
chrispcampbell Jun 10, 2022
779f0b7
build: convert gen-docs to a Node script that can apply custom substi…
chrispcampbell Jun 10, 2022
fbd6b66
docs: add API docs for runtime-async package
chrispcampbell Jun 10, 2022
50f3df2
fix: ignore ./docs directory so that only children are deleted
chrispcampbell Jun 10, 2022
9429dcb
fix: remove unnecessary InitWasmModel type alias
chrispcampbell Jun 10, 2022
37e348b
feat: add terminate function to ModelRunner interface
chrispcampbell Jun 10, 2022
7b342a3
test: add integration test for runtime-async package
chrispcampbell Jun 10, 2022
7134f11
test: add test for createWasmModelRunner
chrispcampbell Jun 10, 2022
eb046f2
test: add tests for ModelScheduler
chrispcampbell Jun 11, 2022
a8a9362
docs: link to the API docs from the README
chrispcampbell Jun 11, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .eslintrc-ts-common.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
module.exports = {
root: true,
env: {
browser: true,
node: true
},
parser: '@typescript-eslint/parser',
rules: {
'@typescript-eslint/consistent-type-imports': 'error'
},
plugins: ['@typescript-eslint'],
extends: [
'eslint:recommended',
'plugin:eslint-comments/recommended',
'plugin:@typescript-eslint/recommended',
'prettier'
]
}
13 changes: 11 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,24 @@
"prettier:fix": "run-s prettier-local:fix prettier-pkgs:fix",
"precommit": "pnpm -r precommit",
"build": "pnpm run -r --workspace-concurrency=1 build",
"test:pkgs": "pnpm -r test",
"test:e2e": "./tests/modeltests",
"test": "run-s test:e2e"
"test": "run-s test:pkgs test:e2e"
},
"devDependencies": {
"@typescript-eslint/eslint-plugin": "^5.27.1",
"@typescript-eslint/parser": "^5.27.1",
"eslint": "^8.14.0",
"eslint-config-prettier": "^8.5.0",
"eslint-plugin-eslint-comments": "^3.2.0",
"glob": "^8.0.3",
"npm-run-all": "^4.1.5",
"prettier": "^2.6.2"
"prettier": "^2.6.2",
"tsup": "^6.1.0",
"typedoc": "^0.22.17",
"typedoc-plugin-markdown": "^3.12.1",
"typescript": "^4.7.3",
"vitest": "^0.14.1"
},
"author": "Climate Interactive",
"license": "MIT",
Expand Down
2 changes: 2 additions & 0 deletions packages/runtime-async/.eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
dist
docs
3 changes: 3 additions & 0 deletions packages/runtime-async/.eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = {
extends: ['../../.eslintrc-ts-common.cjs']
}
2 changes: 2 additions & 0 deletions packages/runtime-async/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
dist
docs/entry.md
3 changes: 3 additions & 0 deletions packages/runtime-async/.prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dist
docs
CHANGELOG.md
4 changes: 4 additions & 0 deletions packages/runtime-async/.typedoc/replacements.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[
["`ModelRunner`", "[`ModelRunner`](../../../runtime/docs/interfaces/ModelRunner.md)"],
["`WasmModelInitResult`", "[`WasmModelInitResult`](../../../runtime/docs/interfaces/WasmModelInitResult.md)"]
]
21 changes: 21 additions & 0 deletions packages/runtime-async/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2020-2022 Climate Interactive / New Venture Fund

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
50 changes: 50 additions & 0 deletions packages/runtime-async/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# @sdeverywhere/runtime-async

This package provides an implementation of the `ModelRunner` interface from
the `@sdeverywhere/runtime` package that uses a Web Worker (in the browser)
or a worker thread (in a Node.js app) to run the model asynchronously off
the main JavaScript thread.

## Usage

### 1. Initialize the `WasmModel` in a Web Worker

In your app project, define a separate JavaScript file, called
`worker.js` for example, that initializes the model worker in the
context of the Web Worker:

```js
import { initWasmModelAndBuffers } from '@sdeverywhere/runtime'
import { exposeModelWorker } from '@sdeverywhere/runtime-async/worker'

async function initWasmModel() {
const wasmModules = loadWasm()
return initWasmModelAndBuffers(...)
}

exposeModelWorker(initWasmModel)
```

### 2. Spawn the worker

In your web app, call the `spawnAsyncModelRunner` function, which will
spawn the Web Worker and initialize the `ModelRunner` that communicates
with the worker:

```js
import { spawnAsyncModelRunner } from '@sdeverywhere/runtime-async/runner'

async function initApp() {
// ...
const runner = await spawnAsyncModelRunner({ path: './worker.js' })
// ...
}
```

## Documentation

API documentation is available in the [`docs`](./docs/index.md) directory.

## License

SDEverywhere is distributed under the MIT license. See `LICENSE` for more details.
20 changes: 20 additions & 0 deletions packages/runtime-async/docs/functions/exposeModelWorker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[@sdeverywhere/runtime-async](../index.md) / exposeModelWorker

# Function: exposeModelWorker

**exposeModelWorker**(`init`): `void`

Expose an object in the current worker thread that communicates with the
[`ModelRunner`](../../../runtime/docs/interfaces/ModelRunner.md) instance running in the main thread. The exposed worker
object will take care of running the `WasmModel` on the worker thread
and sending the outputs back to the main process.

#### Parameters

| Name | Type | Description |
| :------ | :------ | :------ |
| `init` | () => `Promise`<[`WasmModelInitResult`](../../../runtime/docs/interfaces/WasmModelInitResult.md)\> | The function that initializes the `WasmModel` instance that is used in the worker thread. |

#### Returns

`void`
46 changes: 46 additions & 0 deletions packages/runtime-async/docs/functions/spawnAsyncModelRunner.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
[@sdeverywhere/runtime-async](../index.md) / spawnAsyncModelRunner

# Function: spawnAsyncModelRunner

**spawnAsyncModelRunner**(`workerSpec`): `Promise`<[`ModelRunner`](../../../runtime/docs/interfaces/ModelRunner.md)\>

Initialize a [`ModelRunner`](../../../runtime/docs/interfaces/ModelRunner.md) that runs the model asynchronously in a worker thread.

In your app project, define a JavaScript file, called `worker.js` for example, that
initializes the model worker in the context of the Web Worker:

```js
import { initWasmModelAndBuffers } from '@sdeverywhere/runtime'
import { exposeModelWorker } from '@sdeverywhere/runtime-async/worker'

async function initWasmModel() {
const wasmModules = loadWasm()
return initWasmModelAndBuffers(...)
}

exposeModelWorker(initWasmModel)
```

Then, in your web app, call the `spawnAsyncModelRunner` function, which
will spawn the Web Worker and initialize the [`ModelRunner`](../../../runtime/docs/interfaces/ModelRunner.md) that communicates
with the worker:

```js
import { spawnAsyncModelRunner } from '@sdeverywhere/runtime-async/runner'

async function initApp() {
// ...
const runner = await spawnAsyncModelRunner({ path: './worker.js' })
// ...
}
```

#### Parameters

| Name | Type | Description |
| :------ | :------ | :------ |
| `workerSpec` | { `path`: `string` } \| { `source`: `string` } | Either a `path` to the worker JavaScript file, or the `source` containing the full JavaScript source of the worker. |

#### Returns

`Promise`<[`ModelRunner`](../../../runtime/docs/interfaces/ModelRunner.md)\>
9 changes: 9 additions & 0 deletions packages/runtime-async/docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# @sdeverywhere/runtime-async

## Main Thread

- [spawnAsyncModelRunner](functions/spawnAsyncModelRunner.md)

## Worker Thread

- [exposeModelWorker](functions/exposeModelWorker.md)
56 changes: 56 additions & 0 deletions packages/runtime-async/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
{
"name": "@sdeverywhere/runtime-async",
"version": "0.1.0",
"files": [
"dist/**"
],
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.js",
"require": "./dist/index.cjs"
},
"./runner": {
"types": "./dist/runner.d.ts",
"import": "./dist/runner.js",
"require": "./dist/runner.cjs"
},
"./worker": {
"types": "./dist/worker.d.ts",
"import": "./dist/worker.js",
"require": "./dist/worker.cjs"
}
},
"scripts": {
"clean": "rm -rf dist",
"lint": "eslint src --ext .ts --max-warnings 0",
"prettier:check": "prettier --check .",
"prettier:fix": "prettier --write .",
"precommit": "../../scripts/precommit",
"test": "vitest run",
"test:watch": "vitest",
"test:ci": "vitest run",
"build": "tsup",
"docs": "../../scripts/gen-docs.js",
"ci:build": "run-s clean lint prettier:check build test:ci docs"
},
"dependencies": {
"@sdeverywhere/runtime": "^0.1.0",
"threads": "1.7.0"
},
"author": "Climate Interactive",
"license": "MIT",
"homepage": "https://sdeverywhere.org",
"repository": {
"type": "git",
"url": "https://github.com/climateinteractive/SDEverywhere.git",
"directory": "packages/runtime-async"
},
"bugs": {
"url": "https://github.com/climateinteractive/SDEverywhere/issues"
}
}
5 changes: 5 additions & 0 deletions packages/runtime-async/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// Copyright (c) 2020-2022 Climate Interactive / New Venture Fund

export { spawnAsyncModelRunner } from './runner'

export { exposeModelWorker } from './worker'
Loading