Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make improvements in pandas-test reporting #15485

Merged
merged 13 commits into from
Apr 10, 2024
Merged

Make improvements in pandas-test reporting #15485

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
bfb492b
address reviews
galipremsagar Apr 8, 2024
4e61e1c
fix mismatching versions
galipremsagar Apr 8, 2024
67b9877
Merge branch 'branch-24.06' into pandas-reporting-2
galipremsagar Apr 8, 2024
336f8d3
Merge branch 'branch-24.06' into pandas-reporting-2
galipremsagar Apr 8, 2024
dace1bd
fetch same as current version
galipremsagar Apr 8, 2024
10f66c0
Merge branch 'pandas-reporting-2' of https://github.com/galipremsagar…
galipremsagar Apr 8, 2024
c383e78
Merge branch 'branch-24.06' into pandas-reporting-2
galipremsagar Apr 8, 2024
9c5dc90
simplify using github.paginate
galipremsagar Apr 9, 2024
4299743
Merge branch 'branch-24.06' into pandas-reporting-2
galipremsagar Apr 9, 2024
aca05cf
style
galipremsagar Apr 9, 2024
5f2c4a0
Merge branch 'pandas-reporting-2' of https://github.com/galipremsagar…
galipremsagar Apr 9, 2024
06e7b6c
Update .github/workflows/test.yaml
galipremsagar Apr 10, 2024
b0254f5
Merge branch 'branch-24.06' into pandas-reporting-2
galipremsagar Apr 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions .github/workflows/status.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,13 +85,18 @@ jobs:
state: CUSTOM_STATE = 'success'
} = contentJSON;

// Fetch the first job ID from the workflow run
const jobs = await github.rest.actions.listJobsForWorkflowRun({
// Fetch all jobs using pagination
const jobs = await github.paginate(
github.rest.actions.listJobsForWorkflowRun,
{
owner: context.repo.owner,
repo: context.repo.repo,
run_id: process.env.WORKFLOW_RUN_ID,
});
const job = jobs.data.jobs.find(job => job.name === JOB_NAME);
}
);

// Fetch the first job ID from the workflow run
const job = jobs.find(job => job.name === JOB_NAME);
const JOB_ID = job ? job.id : null;

// Set default target URL if not defined
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ jobs:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]))
matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(min_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]))
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
Expand Down
9 changes: 5 additions & 4 deletions ci/cudf_pandas_scripts/pandas-tests/diff.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,13 @@
GH_JOB_NAME="pandas-tests-diff / build"
rapids-logger "Github job name: ${GH_JOB_NAME}"

MAIN_ARTIFACT=$(rapids-s3-path)cuda12_$(arch)_py310.main-results.json
PR_ARTIFACT=$(rapids-s3-path)cuda12_$(arch)_py39.pr-results.json
PY_VER="39"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same reason as here: https://github.com/rapidsai/cudf/pull/15485/files#r1556345903

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don’t want to hardcode any values here. Our matrix will change over time and break this. Try using ${RAPIDS_PY_VERSION}.

Copy link
Contributor Author

@galipremsagar galipremsagar Apr 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I open a follow-up for this one? I will need to do some testing here after this PR merges and nightlies are uploaded.

MAIN_ARTIFACT=$(rapids-s3-path)cuda12_$(arch)_py${PY_VER}.main-results.json
PR_ARTIFACT=$(rapids-s3-path)cuda12_$(arch)_py${PY_VER}.pr-results.json

rapids-logger "Fetching latest available results from nightly"
aws s3api list-objects-v2 --bucket rapids-downloads --prefix "nightly/" --query "sort_by(Contents[?ends_with(Key, '.main-results.json')], &LastModified)[::-1].[Key]" --output text > s3_output.txt
cat s3_output.txt
aws s3api list-objects-v2 --bucket rapids-downloads --prefix "nightly/" --query "sort_by(Contents[?ends_with(Key, '_py${PY_VER}.main-results.json')], &LastModified)[::-1].[Key]" --output text > s3_output.txt

read -r COMPARE_ENV < s3_output.txt
export COMPARE_ENV
rapids-logger "Latest available results from nightly: ${COMPARE_ENV}"
Expand Down
Loading