Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple language and configuration from the Topiary library #643

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
52090aa
WIP: Strip out specific language and config code from engine
Aug 24, 2023
3a8cd0a
We don't need serde-toml-merge any more
Xophmeister Oct 10, 2023
ba43921
We need serde + serde_json for GraphViz output in library
Xophmeister Oct 10, 2023
526198a
Update dependencies
Xophmeister Oct 10, 2023
a96637f
WIP: Config migration
Xophmeister Oct 10, 2023
e0d2bc1
WIP: Configuration migration/refactoring
Xophmeister Oct 11, 2023
2c12aa1
WIP: Break up config into submodules
Xophmeister Oct 11, 2023
569c790
WIP: Break up: Separate TOML collation and filesystem functions
Xophmeister Oct 11, 2023
2e3e80a
WIP: Break up: Separate (de)serialisation
Xophmeister Oct 12, 2023
85d3939
WIP: Break up: Don't use mod.rs and don't set module name to same as …
Xophmeister Oct 12, 2023
441a42b
Let's try fixing bugs until the LSP starts working...
Xophmeister Oct 12, 2023
aee6cb5
The CLI needs serde
Xophmeister Oct 12, 2023
76048e8
Correct crate::configuration::fs
Xophmeister Oct 12, 2023
dcaaa3c
Correct crate::configuration::format
Xophmeister Oct 12, 2023
70f6afb
Correct remainder of crate::configuration::*
Xophmeister Oct 12, 2023
381d99c
WIP: Improve the configuration API
Xophmeister Oct 13, 2023
03dbafe
First approximation refactor of configuration in CLI
Xophmeister Oct 13, 2023
f3ecaf5
Some refactor polish
Xophmeister Oct 13, 2023
6c14306
Use CLI configuration, rather than library's
Xophmeister Oct 13, 2023
16becc2
Tidy-up and FIXMEs in library code
Xophmeister Oct 13, 2023
f6c34ff
Most of the configuration moving is done
Xophmeister Oct 13, 2023
994a9c1
Make serde::Language available
Oct 23, 2023
60006c9
Move detect to Configuration
Oct 24, 2023
19b1add
Make Deserialized Language in LanguageDefinition non-referenced
Oct 26, 2023
27ab302
Resolve to_language_definition todo
Oct 26, 2023
8b64f86
Remove playground and related files
Jan 2, 2024
f3772bc
Resolve warnings
Jan 3, 2024
1df9807
Update bench to new `formatter` api
Jan 3, 2024
a161981
Remove vscode files
Jan 3, 2024
2a6f0c4
Move tests to new API
Jan 3, 2024
60462e1
Remove LanguageDefinition
Jan 4, 2024
779fea3
Update CI
Jan 4, 2024
eb002d7
Update README after separation
Jan 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 4 additions & 53 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- name: Checkout code
uses: actions/checkout@v3.1.0
uses: actions/checkout@v3

- name: Install Nix
uses: cachix/install-nix-action@v22
uses: cachix/install-nix-action@v24
with:
nix_path: nixpkgs=channel:nixpkgs-unstable

Expand All @@ -21,12 +21,6 @@ jobs:
name: tweag-topiary
authToken: "${{ secrets.CACHIX_TWEAG_TOPIARY_AUTH_TOKEN }}"

- name: Set up frontend cache
uses: actions/cache@v2
with:
path: '**/node_modules'
key: node_modules-${{ hashFiles('**/package-lock.json')}}

- name: Clippy, test, and benchmark
run: export GC_DONT_GC=1; nix -L flake check

Expand All @@ -36,51 +30,8 @@ jobs:
- name: Verify that usage in README.md matches CLI output
run: ./verify-documented-usage.sh

- name: Build web playground Wasm app
if: success() && matrix.os == 'ubuntu-latest'
run: nix build .#topiary-playground

- name: Copy web playground Wasm app into playground frontend
if: success() && matrix.os == 'ubuntu-latest'
run: |
mkdir -p web-playground/src/wasm-app
cp -r result/* web-playground/src/wasm-app/

- name: Move sample input and queries into playground frontend
if: success() && matrix.os == 'ubuntu-latest'
run: |
mkdir -p web-playground/src/samples
mv web-playground/src/wasm-app/languages_export.ts web-playground/src/samples/

- name: Install web playground frontend dependencies
if: success() && matrix.os == 'ubuntu-latest'
run: npm install --prefix web-playground

- name: Start web playground frontend
if: success() && matrix.os == 'ubuntu-latest'
run: |
npm run dev --prefix web-playground &
# Loop until there's a response
while [[ "$(curl --silent --output /dev/null --write-out "%{http_code}" http://localhost:5173/playground)" != "200" ]]; do
sleep 2
done

- name: Test web playground frontend
if: success() && matrix.os == 'ubuntu-latest'
run: npm run e2e --prefix web-playground

- name: Make web playground frontend release build
if: success() && matrix.os == 'ubuntu-latest'
run: npm run build --prefix web-playground

- name: Copy playground into website
if: success() && matrix.os == 'ubuntu-latest'
run: |
rm -rf website/playground
cp -r web-playground/dist website/playground

- name: Upload GitHub Pages artifact
uses: actions/upload-pages-artifact@v1.0.7
uses: actions/upload-pages-artifact@v3
if: success() && matrix.os == 'ubuntu-latest' && github.ref == 'refs/heads/main'
with:
path: 'website'
Expand All @@ -101,4 +52,4 @@ jobs:
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v1
uses: actions/deploy-pages@v4
87 changes: 0 additions & 87 deletions .vscode/launch.json

This file was deleted.

58 changes: 0 additions & 58 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,61 +54,3 @@ that is provided by Nix, if you have both installed. For convenience,
the `generate-coverage.sh` script can be run from the root of this
repository to avoid contaminating your environment, but note it will
download a full toolchain on each run.

## Web site and web playground

If you have [Deno](https://deno.land/) installed, you can start a local web
server like this:

```bash
deno run -A local-web-server.ts
```

The web site should then be running on http://localhost:8080.

In order to build or update the Wasm playground, you can run this:

```bash
./update-wasm-app.sh
```

If you need to add or update Tree-sitter grammar Wasm files, the easiest way would be using Nix.

Simply enter our `devShell` with `nix develop`, and then run `update-wasm-grammars`.
Alternatively, if you have `git`, `tree-sitter` and `emcc` in your `PATH`, you can run the `./update-wasm-grammars.sh` file.

To use docker instead, the legacy approach can still be used (using JSON as an example):

1. Make sure you have Docker running and that you are member of the `docker`
group so you can run it without being root.
2. `npm install tree-sitter-cli` (or some other way)
3. `npm install tree-sitter-json` (or by cloning the git repository)
- If you used npm, tree-sitter-json will be fetched under `node_modules/tree-sitter-json/`
- If you used git, it will be wherever you cloned the repository (most likely `tree-sitter-json/`)

Whichever of these options you pick, we will assume `JSON_GRAMMAR` is the directory where the `grammar.js` can be found.
4. Make sure you have a file at
`JSON_GRAMMAR/src/grammar.json`.
5. Run `npx tree-sitter build-wasm JSON_GRAMMAR`. If you get a Docker permission
error, you may have to add yourself to the docker group.
6. `mv tree-sitter-json.wasm web-playground/public/scripts/`

For OCaml, the process is slightly different because the tree-sitter-ocaml repository/package contains two grammars:

1. `npm install tree-sitter-cli`
2. `npm install tree-sitter-ocaml` (or git, like above)
3. Run `npx tree-sitter build-wasm OCAML_GRAMMAR/ocaml`.
4. Run `npx tree-sitter build-wasm OCAML_GRAMMAR/ocaml_interface`.
5. `mv tree-sitter-ocaml*.wasm web-playground/public/scripts/`

The playground frontend is a small React app. You can run a development server
for that like this:

```bash
cd web-playground
npm install
npm run dev
```

If you want to build the playground so it works with the full website running
with Deno as above, you can now just do `npm run build`.
Loading