Skip to content

Commit

Permalink
Fix CI s3 api command to fetch latest results (#15687)
Browse files Browse the repository at this point in the history
An `s3` fetch api seems to be returning incorrectly ordered output, this PR fixes the issue.

Authors:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: #15687
  • Loading branch information
galipremsagar authored May 7, 2024
1 parent bd96614 commit 2e81857
Showing 1 changed file with 2 additions and 4 deletions.
6 changes: 2 additions & 4 deletions ci/cudf_pandas_scripts/pandas-tests/diff.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,8 @@ MAIN_ARTIFACT=$(rapids-s3-path)cuda12_$(arch)_py${PY_VER}.main-${RAPIDS_FULL_VER
PR_ARTIFACT=$(rapids-s3-path)cuda12_$(arch)_py${PY_VER}.pr-${RAPIDS_FULL_VERSION}-results.json

rapids-logger "Fetching latest available results from nightly"
aws s3api list-objects-v2 --bucket rapids-downloads --prefix "nightly/" --query "sort_by(Contents[?ends_with(Key, '_py${PY_VER}.main-${RAPIDS_FULL_VERSION}-results.json')], &LastModified)[::-1].[Key]" --output text > s3_output.txt

read -r COMPARE_ENV < s3_output.txt
export COMPARE_ENV
aws s3api list-objects-v2 --bucket rapids-downloads --prefix "nightly/" --query "sort_by(Contents[?ends_with(Key, '_py${PY_VER}.main-${RAPIDS_FULL_VERSION}-results.json')], &LastModified)[::].[Key]" --output text | tee s3_output.txt
COMPARE_ENV=$(tail -n 1 s3_output.txt)
rapids-logger "Latest available results from nightly: ${COMPARE_ENV}"

aws s3 cp "s3://rapids-downloads/${COMPARE_ENV}" main-results.json
Expand Down

0 comments on commit 2e81857

Please sign in to comment.