Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: parallel gtest #5498

Merged
merged 3 commits into from
Mar 28, 2024
Merged

feat: parallel gtest #5498

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e3cc3fe
feat: parallel gtest
IlyasRidhuan Mar 28, 2024
bbb32fb
fix: decompose into array for stdlib test
IlyasRidhuan Mar 28, 2024
d85ead9
Merge branch 'master' into parallel_g_test
ludamad Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion barretenberg/cpp/dockerfiles/Dockerfile.x86_64-linux-clang-assert
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,14 @@ COPY . .
# Build everything to ensure everything builds. All tests will be run from the result of this build.
RUN ./format.sh check && cmake --preset clang16 -DCMAKE_BUILD_TYPE=RelWithAssert -DCI=ON && cmake --build --preset clang16
RUN srs_db/download_grumpkin.sh
RUN git clone --depth=1 https://github.com/google/gtest-parallel.git
RUN cd gtest-parallel && git checkout 96f4f904922f9bf66689e749c40f314845baaac8

FROM ubuntu:lunar
RUN apt update && apt install curl libstdc++6 -y
RUN apt update && apt install curl libstdc++6 python3.11-full -y
COPY --from=builder /usr/src/barretenberg/cpp/srs_db /usr/src/barretenberg/cpp/srs_db
COPY --from=builder /usr/src/barretenberg/cpp/build/bin /usr/src/barretenberg/cpp/build/bin
# Copy gtest-parallel
COPY --from=builder /usr/src/barretenberg/cpp/gtest-parallel /usr/src/barretenberg/cpp/build/gtest-parallel
# Cache dir needed for gtest-parallel
RUN mkdir -p /root/.cache
2 changes: 1 addition & 1 deletion barretenberg/cpp/scripts/avm-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ docker run --rm -t $IMAGE_URI /bin/sh -c "\
cd /usr/src/barretenberg/cpp; \
srs_db/download_ignition.sh 1; \
cd build; \
./bin/vm_tests;"
./gtest-parallel/gtest-parallel ./bin/vm_tests --workers=32;"
4 changes: 2 additions & 2 deletions barretenberg/cpp/scripts/bb-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,12 @@ TESTS=(
polynomials_tests
srs_tests
)
TESTS_STR="${TESTS[@]}"
TESTS_STR="${TESTS[@]/#/./bin/}"

docker run --rm -t $IMAGE_URI /bin/sh -c "\
set -xe; \
cd /usr/src/barretenberg/cpp; \
srs_db/download_ignition.sh 1; \
srs_db/download_grumpkin.sh; \
cd build; \
for BIN in $TESTS_STR; do ./bin/\$BIN; done"
echo $TESTS_STR | xargs gtest-parallel/gtest-parallel --worker=32;"
9 changes: 7 additions & 2 deletions barretenberg/cpp/scripts/run_tests
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ shift
shift

$(aws ecr get-login --region us-east-2 --no-include-email) 2> /dev/null

# use the image rebuild patterns to compute a content hash, use this to get a URI
REPOSITORY=barretenberg-x86_64-linux-clang-assert

export PATH="$PATH:$(git rev-parse --show-toplevel)/build-system/scripts"
IMAGE_URI=$(calculate_image_uri $REPOSITORY)
retry docker pull $IMAGE_URI
Expand All @@ -25,9 +25,14 @@ if [ -f "$TESTS" ]; then
TESTS=$(cat $TESTS | tr '\n' ' ')
fi

# Convert the string it an array so we can process as args to gtest-parallel
IFS=' ' read -r -a TEST_ARRAY <<< $TESTS
# Prepend ./bin/ and append any args (e.g. gTest Filters)
TESTS_STR="${TEST_ARRAY[@]/#/./bin/}"
TESTS_STR="${TESTS_STR[@]/%/ ${@}}"
docker run --rm -t $IMAGE_URI /bin/sh -c "\
set -xe; \
/usr/src/barretenberg/cpp/srs_db/download_ignition.sh $NUM_TRANSCRIPTS; \
/usr/src/barretenberg/cpp/srs_db/download_grumpkin.sh; \
cd /usr/src/barretenberg/cpp/build; \
for BIN in $TESTS; do ./bin/\$BIN $@; done"
echo $TESTS_STR | xargs gtest-parallel/gtest-parallel --worker=32;"
Loading