Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor runner code into multiple runtimes to simplify the addition of new ones #62

Merged
merged 3 commits into from
Jan 19, 2023

Conversation

Angelmmiguel
Copy link
Contributor

@Angelmmiguel Angelmmiguel commented Jan 17, 2023

The Runner struct had logic to process requests using different runtimes. Every of them requires a specific configuration for the source code and the WASI environment. Since we only supported two inputs (javascript and wasm), conditionals were simple and the code was maneagable. However, our goal is to introduce more languages in the future so this specific architecture doesn't fit these requires.

Here, I refactored the Runner code into a separate Worker module. This module defines both the main Worker class and the N runtimes we support. Adding a new runner only requires to add a new struct that impl the Runtime trait. Then, you only need to configure it in the init_runtime function (pending to implement).

Apart from that, I introduced another major change. Now, interpreted languages will mount their source code into the module. Other language requires to mount lib folders to work, so it feels natural to use the same approach to all of them. Note that in Wasmtime we can preopen folders, but not files. It is an important statement as a folder may contain multiple source files and we want the runtime to have access only to a single one.

For this, I added a new .wws folder. This temporary folder contains folders with the source code that it's required for a certain worker. To avoid collisions, every language will have its own scope and every source file a folder with a name based on its context. The folder name is based on the blake3 hash of the file content. I chose this implementation based on the performance metrics they provide.

.wws
| - js(lang prefix)
    | - blake3(source-file)
        | - index.(extension)
| - ...

That approach opens new possibilities as we can add glue code in those folders and even third-party libraries. It will also provides transparency as users will be able to see the code that it's run and even debug it.

It closes #61

Pending

  • Implements the init_runtime function to get a <dyn Runtime + Send + Sync> instance based on the file extension

@Angelmmiguel Angelmmiguel added the 🚀 enhancement New feature or request label Jan 17, 2023
@Angelmmiguel Angelmmiguel self-assigned this Jan 17, 2023
Copy link
Contributor

@ereslibre ereslibre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great refactor @Angelmmiguel! LGTM, minor comments from my side.

src/workers/runtime.rs Show resolved Hide resolved
src/workers/runtime.rs Outdated Show resolved Hide resolved
src/workers/runtime.rs Outdated Show resolved Hide resolved
src/workers/runtimes/temp_utils.rs Outdated Show resolved Hide resolved
src/workers/runtimes/javascript.rs Outdated Show resolved Hide resolved
src/workers/runtimes/javascript.rs Show resolved Hide resolved
src/workers/runtimes/native.rs Show resolved Hide resolved
src/workers/runtimes/temp_utils.rs Outdated Show resolved Hide resolved
@ereslibre
Copy link
Contributor

Also minor typo in commit title simlpify

@Angelmmiguel Angelmmiguel changed the title feat: refactor runner code into multiple runtimes to simlpify the addition of new ones feat: refactor runner code into multiple runtimes to simplify the addition of new ones Jan 19, 2023
@Angelmmiguel Angelmmiguel merged commit 67ac7f0 into main Jan 19, 2023
@Angelmmiguel Angelmmiguel deleted the 61-split-runner branch May 8, 2023 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Decouple language specifics from the runner
3 participants