Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Ensure that CLI and wasm compilers are sync in noir_wasm_testing #2079

Merged
merged 43 commits into from
Aug 4, 2023
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
20bc4a7
chore: install nargo from current commit
TomAFrench Jul 28, 2023
15b107e
chore: fully qualified path to noirup
TomAFrench Jul 28, 2023
b49b81c
chore: build nargo in nix
TomAFrench Jul 28, 2023
f9f7ec4
chore: build wasm and nargo binary in parallel
TomAFrench Jul 28, 2023
c1d6b1d
chore: use nix cache when building nargo
TomAFrench Jul 28, 2023
ce12ebe
chore: remove unnecessary nix install
TomAFrench Jul 28, 2023
2ebc9aa
chore: make nargo executable
TomAFrench Jul 28, 2023
4f67589
chore: bypass build.sh
TomAFrench Jul 28, 2023
3a6082d
chore: explicit path to nargo
TomAFrench Jul 28, 2023
548fdb2
chore: make nargo executable
TomAFrench Jul 28, 2023
4f351fa
chore: print out dir structure
TomAFrench Jul 28, 2023
943f156
chore: remove unnecessary checkout
TomAFrench Jul 28, 2023
8c0a61c
chore: be extra explicit about nargo location
TomAFrench Jul 28, 2023
053c1af
chore: download nargo into noir project dir
TomAFrench Jul 28, 2023
19a1679
Debug nargo run
jonybur Jul 28, 2023
b59f16b
Fix path
jonybur Jul 28, 2023
c0eb535
Debug nargo binary search
jonybur Jul 28, 2023
9a27ce3
Use realpath
jonybur Jul 28, 2023
d62175c
Construct path with pwd
jonybur Jul 28, 2023
806cc30
Change implementation of find
jonybur Jul 28, 2023
404bff6
Change chmod
jonybur Jul 28, 2023
a36c4dc
Move chomd
jonybur Jul 28, 2023
a17d166
Fix binary search
jonybur Aug 1, 2023
d80b43d
Fix chmod
jonybur Aug 1, 2023
9982570
Workflow fix
jonybur Aug 1, 2023
ef7086d
Add ldd
jonybur Aug 2, 2023
09e29e4
Installs missing dep for nargo
jonybur Aug 2, 2023
a05bb33
Locate libomp.so
jonybur Aug 2, 2023
7d25d03
Change locate for find
jonybur Aug 2, 2023
9ff2ad8
Merge workflow steps
jonybur Aug 2, 2023
c1843ec
Fix dynamic linker
jonybur Aug 2, 2023
1eede86
Update workflow to match build-nargo
jonybur Aug 3, 2023
b174657
Debug steps
jonybur Aug 3, 2023
293f301
Change to debug log
jonybur Aug 3, 2023
5545cbe
Fix paths
jonybur Aug 3, 2023
ef31162
Fix paths
jonybur Aug 3, 2023
9dd23c3
Add artifact upload/download
jonybur Aug 3, 2023
75867f2
Add upload artifact stage, add debug ls
jonybur Aug 3, 2023
161d100
Fix test build artifact
jonybur Aug 3, 2023
5e7bf62
Add rest of workflow
jonybur Aug 4, 2023
c9727ad
Remove test
jonybur Aug 4, 2023
bc5793a
Update .github/workflows/wasm.yml
TomAFrench Aug 4, 2023
431b791
Update .github/workflows/wasm.yml
TomAFrench Aug 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/Cross.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[build]
# pre-build = [
# "dpkg --add-architecture $CROSS_DEB_ARCH",
# "apt-get update && apt-get install --assume-yes xz-utils pkgconfig build-essential cmake openssl libssl-dev libomp-dev libssl-dev:$CROSS_DEB_ARCH libomp-dev:$CROSS_DEB_ARCH",
# ]

[build.env]
passthrough = [
"HOME",
"RUST_BACKTRACE",
"BARRETENBERG_BIN_DIR"
]
volumes = [
"HOME",
]
169 changes: 154 additions & 15 deletions .github/workflows/wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,125 @@ on:
push:
branches:
- jb/noir-testing-workflow
- tom/noir-wasm-testing

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref || github.run_id }}
cancel-in-progress: true

jobs:
build:
name: Build Wasm
build-barretenberg:
runs-on: ubuntu-latest
steps:
- name: Checkout Noir repo
uses: actions/checkout@v3

- name: Checkout Build Nargo repo
uses: actions/checkout@v3
with:
repository: noir-lang/build-nargo
path: build-nargo
ref: ${{ inputs.build-nargo-ref || 'master' }}

- name: Collect locked barretenberg rev
run: |
echo "BB_REV=$(jq -r .nodes.barretenberg.locked.rev ./flake.lock)" >> $GITHUB_ENV
echo "BB_REV is ${{ env.BB_REV }}"

- uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-22.11
github_access_token: ${{ secrets.GITHUB_TOKEN }}

- uses: cachix/cachix-action@v12
with:
name: barretenberg
authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"

# Upload does not work with symlinks, using this workaround:
# https://github.com/actions/upload-artifact/issues/92#issuecomment-1080347032
- name: Build barretenberg as libbarretenberg-wasm32
run: |
echo "BB_REV is ${{ env.BB_REV }}"
nix build "github:AztecProtocol/barretenberg/${{ env.BB_REV }}#wasm32"
echo "ARTIFACT_UPLOAD_PATH=$(readlink -f result)" >> $GITHUB_ENV

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ env.ARTIFACT_UPLOAD_PATH }}
retention-days: 3

build-nargo:
needs: [build-barretenberg]
runs-on: ubuntu-22.04
env:
CROSS_CONFIG: ${{ github.workspace }}/.github/Cross.toml
strategy:
fail-fast: false
matrix:
target: [x86_64-unknown-linux-gnu]

steps:
- name: Checkout Noir repo
uses: actions/checkout@v3

- name: Checkout Build Nargo repo
uses: actions/checkout@v3
with:
repository: noir-lang/build-nargo
path: build-nargo
ref: ${{ inputs.build-nargo-ref || 'master' }}

- uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ github.workspace }}/libbarretenberg-wasm32

- name: Setup toolchain
uses: dtolnay/[email protected]
with:
targets: ${{ matrix.target }}

- name: Build Nargo
env:
BARRETENBERG_BIN_DIR: ${{ github.workspace }}/libbarretenberg-wasm32/bin
run: |
cargo install cross --force --git https://github.com/cross-rs/cross
cross build --package nargo_cli --release --target=${{ matrix.target }} --no-default-features --features plonk_bn254_wasm

- name: Package artifacts
run: |
mkdir dist
cp ./target/${{ matrix.target }}/release/nargo ./dist/nargo
7z a -ttar -so -an ./dist/* | 7z a -si ./nargo-${{ matrix.target }}.tar.gz

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo
path: ./dist/*
retention-days: 3

build-wasm:
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3

- name: Setup Nix
uses: cachix/install-nix-action@v22
- uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-22.11
github_access_token: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -27,29 +131,64 @@ jobs:
run: |
nix build -L .#wasm

- name: Dereference symlink
run: echo "UPLOAD_PATH=$(readlink -f result)" >> $GITHUB_ENV

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: noir_wasm
path: ${{ env.UPLOAD_PATH }}
retention-days: 3

test:
needs: [build-wasm, build-nargo]
name: Test
runs-on: ubuntu-latest
steps:
- name: Checkout noir-wasm-testing
uses: actions/checkout@v3
with:
repository: noir-lang/noir-wasm-testing
path: noir-wasm-testing
ref: use-injected-nargo
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

- name: Download wasm package artifact
uses: actions/download-artifact@v3
with:
name: noir_wasm
path: ./@noir-lang_noir_wasm

- name: Inject built wasm into noir-wasm-testing
run: |
mkdir ./noir-wasm-testing/@noir-lang_noir_wasm
cp -r -L ./result/* ./noir-wasm-testing/@noir-lang_noir_wasm
jq '.dependencies["@noir-lang/noir_wasm"] = "file:./@noir-lang_noir_wasm"' ./noir-wasm-testing/package.json > ./noir-wasm-testing/package.json.tmp
mv ./noir-wasm-testing/package.json.tmp ./noir-wasm-testing/package.json
jq '.dependencies["@noir-lang/noir_wasm"] = "file:./@noir-lang_noir_wasm"' ./package.json > ./package.json.tmp
mv ./package.json.tmp ./package.json

- name: Download nargo binary
uses: actions/download-artifact@v3
with:
name: nargo
path: ./nargo

- name: Run build.sh in noir-wasm-testing
working-directory: ./noir-wasm-testing
- name: Install dependencies and Run noir-wasm-testing program
run: |
chmod +x build.sh
./build.sh
nargo_binary=$(find $(pwd) -name "nargo" -type f)
echo "nargo binary found at $nargo_binary."
ls -l $nargo_binary
echo "Attempting to make nargo binary executable:"
chmod +x $nargo_binary || echo "chmod failed with exit code $?"
echo "File permissions after chmod:"
ls -l $nargo_binary
echo "Library dependencies of nargo binary:"
ldd $nargo_binary
if [ -x "$nargo_binary" ]; then
$nargo_binary -p "${{ github.workspace }}/src/noir-script" compile noir-wasm-testing
else
echo "nargo binary NOT found or is not executable."
exit 1
fi

- name: Install dependencies
working-directory: ./noir-wasm-testing
run: npm install

- name: Run tests
working-directory: ./noir-wasm-testing
run: npm test