Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: decouple ypb #4749

Merged
merged 27 commits into from
Feb 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1e38125
Build nargo against Ubuntu 20 for better compatability
charlielye Feb 21, 2024
31cf1a9
retry a docker create thing.
charlielye Feb 21, 2024
60a38d3
bootstrap scripts try to use cache by default. l1-contracts will atte…
charlielye Feb 21, 2024
3af0393
Merge branch 'master' into cl/more_build_cache
charlielye Feb 21, 2024
42d1dea
Merge branch 'master' into cl/more_build_cache
charlielye Feb 22, 2024
97822b1
Remove old cache images.
charlielye Feb 23, 2024
f4d59db
Better retry.
charlielye Feb 23, 2024
6ac2000
Early out if can't use cache.
charlielye Feb 23, 2024
bd3a1bd
fix unbound
charlielye Feb 23, 2024
c52fec8
Remove codegen sol file.
charlielye Feb 23, 2024
ad2659c
Merge branch 'master' into cl/more_build_cache
charlielye Feb 23, 2024
332e1a3
fix
charlielye Feb 23, 2024
e40e51d
avm-transpiler to target ubuntu 20
charlielye Feb 23, 2024
6ece897
fix
charlielye Feb 23, 2024
2465a8d
fix
charlielye Feb 23, 2024
0b0efad
remove old cache images when doing bootstrap_docker.sh
charlielye Feb 24, 2024
5c56c48
Allow skipping use of cache with NO_CACHE
charlielye Feb 25, 2024
e8b55f4
Add ./bootstrap.sh full command.
charlielye Feb 25, 2024
4e7f7cb
Make noir-projects cli cache friendly.
charlielye Feb 25, 2024
18bfdbb
wip
charlielye Feb 25, 2024
40216be
fix
charlielye Feb 25, 2024
7c308e3
fix
charlielye Feb 25, 2024
df20fba
Merge branch 'cl/more_build_cache' into cl/decouple_ypb
charlielye Feb 25, 2024
b6468a1
fix
charlielye Feb 25, 2024
fefccb8
Gah. Cos of arm we actually do need net connection to productionify. …
charlielye Feb 25, 2024
a694413
Merge remote-tracking branch 'origin' into cl/decouple_ypb
charlielye Feb 26, 2024
1871570
Merge branch 'master' into cl/decouple_ypb
charlielye Feb 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1389,16 +1389,15 @@ workflows:
# Yarn Project
- yarn-project-base:
requires:
- avm-transpiler
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the transpiler is removed and not added in yarn-project. Is this intentional? (I think it's currently needed to transpile just after compiling the contracts).

- l1-contracts
- bb-js
- noir-packages
- noir-projects
- boxes-files
<<: *defaults
- yarn-project:
requires:
- yarn-project-base
- l1-contracts
- noir-projects
- boxes-files
<<: *defaults
- yarn-project-prod: *defaults_yarn_project
- yarn-project-formatting: *defaults_yarn_project
Expand Down
3 changes: 2 additions & 1 deletion boxes/.dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ node_modules
.tsbuildinfo
Dockerfile*
.dockerignore
docker-compose.yml
docker-compose.yml
**/artifacts
3 changes: 2 additions & 1 deletion boxes/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,12 @@ FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/noir-projects as noir-projects

# We need yarn. Start fresh container.
FROM node:18.19.0
RUN apt update && apt install netcat-openbsd
COPY --from=aztec /usr/src /usr/src
COPY --from=noir /usr/src/noir/target/release/nargo /usr/src/noir/target/release/nargo
COPY --from=noir-projects /usr/src/noir-projects/aztec-nr /usr/src/noir-projects/aztec-nr
WORKDIR /usr/src/boxes
ENV AZTEC_NARGO=/usr/src/noir/target/release/nargo
ENV AZTEC_CLI=/usr/src/yarn-project/cli/aztec-cli-dest
RUN yarn && yarn build
ENTRYPOINT ["yarn", "workspace"]
ENTRYPOINT ["/bin/sh", "-c"]
6 changes: 5 additions & 1 deletion boxes/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@ services:

boxes:
image: aztecprotocol/boxes
command: "@aztec/$BOX test"
entrypoint: >
sh -c '
while ! nc -z aztec 8080; do sleep 1; done;
yarn workspace @aztec/$BOX test
'
environment:
DEBUG: "aztec:*"
DEBUG_COLORS: "true"
Expand Down
2 changes: 1 addition & 1 deletion boxes/react/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"serve": "serve -p 3000 ./dist",
"formatting": "prettier --check ./src && eslint ./src",
"formatting:fix": "prettier -w ./src",
"test": "yarn prep && NODE_NO_WARNINGS=1 node --experimental-vm-modules $(yarn bin jest) --runInBand"
"test": "NODE_NO_WARNINGS=1 node --experimental-vm-modules $(yarn bin jest) --runInBand"
},
"jest": {
"preset": "ts-jest/presets/default-esm",
Expand Down
8 changes: 3 additions & 5 deletions build_manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -135,13 +135,8 @@ yarn-project-base:
- ^yarn-project/yarn.lock
- ^yarn-project/.*/package.json$
dependencies:
- l1-contracts
- bb.js
- noir
- noir-packages
- boxes-files
- avm-transpiler
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transpilation happens as part of noir-projects, not as part of yarn-project.
We moved noir code out of yarn-project and consume it, so i guess this was a stale dependency.

- noir-projects

yarn-project:
buildDir: yarn-project
Expand All @@ -151,6 +146,9 @@ yarn-project:
- ^yarn-project/cli/aztec-cli
dependencies:
- yarn-project-base
- l1-contracts
- boxes-files
- noir-projects

yarn-project-prod:
buildDir: yarn-project
Expand Down
15 changes: 10 additions & 5 deletions yarn-project/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,24 @@
# - Run the tests.
# - Run the formatter checks.
# Any subsequent build steps needed to support downstream containers should be done in those containers build files.
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/yarn-project-base as builder
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/l1-contracts as contracts
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/noir-projects as noir-projects
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/boxes-files as boxes-files

# Copy in the entire workspace.
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/yarn-project-base
COPY --from=contracts /usr/src/l1-contracts /usr/src/l1-contracts
COPY --from=noir-projects /usr/src/noir-projects /usr/src/noir-projects
COPY --from=boxes-files /usr/src/boxes /usr/src/boxes
COPY . .

# Generate L1 contract TypeScript artifacts.
RUN cd l1-artifacts && ./scripts/generate-artifacts.sh && rm -rf /usr/src/l1-contracts
# This is actually our code generation tool. Needed to build contract typescript wrappers.
RUN yarn workspace @aztec/noir-compiler build
# Builds noir contracts (TODO: move this stage pre yarn-project). Generates typescript wrappers.
# Generates typescript wrappers.
RUN yarn workspace @aztec/noir-contracts.js build:contracts
# We need to build accounts as it needs to copy in account contracts from noir-contracts.
RUN yarn workspace @aztec/accounts build:copy-contracts
RUN yarn workspace @aztec/protocol-contracts build:copy-contracts
RUN yarn workspace @aztec/noir-protocol-circuits-types build
RUN yarn tsc -b

ENTRYPOINT ["yarn"]
20 changes: 2 additions & 18 deletions yarn-project/yarn-project-base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -42,25 +42,15 @@
# RUN yarn workspaces focus --production && yarn cache clean && rm -rf ../**/src
# - Create final slim image by copying needed dirs into a fresh image.
#
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/l1-contracts as contracts
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/bb.js as bb.js
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/noir-projects as noir-projects
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/noir-packages as noir-packages
FROM 278380418400.dkr.ecr.eu-west-2.amazonaws.com/boxes-files as boxes-files

FROM node:18.19.0
RUN apt update && apt install -y jq curl perl && rm -rf /var/lib/apt/lists/* && apt-get clean

# Copy L1 contracts.
COPY --from=contracts /usr/src/l1-contracts /usr/src/l1-contracts
# Copy in bb.js
# Copy in portalled packages.
COPY --from=bb.js /usr/src/barretenberg/ts /usr/src/barretenberg/ts
# Copy in noir packages
COPY --from=noir-packages /usr/src/noir/packages /usr/src/noir/packages
# Copy in noir projects
COPY --from=noir-projects /usr/src/noir-projects /usr/src/noir-projects
# Copy in boxes
COPY --from=boxes-files /usr/src/boxes /usr/src/boxes

# We install a symlink to yarn-project's node_modules at a location that all portalled packages can find as they
# walk up the tree as part of module resolution. The supposedly idiomatic way of supporting module resolution
Expand Down Expand Up @@ -88,11 +78,5 @@ RUN yarn --immutable && rm -rf /root/.cache/puppeteer && /bin/bash -c '\
[[ $F =~ (.*-) ]] && ln $F /root/.yarn/berry/cache/${BASH_REMATCH[1]}8.zip; \
done'

# If everything's worked properly, we should no longer need access to the network.
RUN echo "enableNetwork: false" >> .yarnrc.yml

# Check package.json inheritance and tsconfig project references.
RUN yarn prepare:check

# Generate L1 contract TypeScript artifacts.
RUN cd l1-artifacts && ./scripts/generate-artifacts.sh && rm -rf /usr/src/l1-contracts
RUN yarn prepare:check
Loading