-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: split windows x86 job #2943
Conversation
This PR causes the following changes to the html docs (ubuntu-latest-3.11-x64):
See CI logs for the full diff. |
Does splitting by tag also address the issue?
(splitting by other tags should work too) |
It's a good suggestion - running multiple pytest commands in the job immediately fails the entire job if one command fails, so the entire suite may not be run. Is there an additional benefit to splitting by
but just splitting by composition seems to be ok also |
8a6156e
to
6147f7e
Compare
This PR causes the following changes to the html docs (ubuntu-latest-3.11-x64):
See CI logs for the full diff. |
.github/workflows/pnl-ci.yml
Outdated
@@ -181,10 +188,14 @@ jobs: | |||
timeout-minutes: 180 | |||
run: pytest --junit-xml=tests_out.xml --verbosity=0 -n logical ${{ matrix.extra-args }} | |||
|
|||
- name: Set extra-args env for artifacts | |||
run: echo "EXTRA_ARGS_DESC=$(echo ${{ matrix.extra-args }} | tr ' ="' '_\-\0')" >> $GITHUB_ENV |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a comment explaining the not allowed characters in artifact names?
Using >> $GITHUB_OUTPUT
and steps.chosen-step-id.output
below would be more targetted and avoid setting an env var for all follow up steps
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will. It looks like only double quote is actually disallowed, so I can remove the other substitutions. But I also think space->underscore is nice although unnecessary
.github/workflows/pnl-ci.yml
Outdated
@@ -209,7 +220,7 @@ jobs: | |||
|
|||
- name: Upload dist packages | |||
uses: actions/upload-artifact@v4 | |||
if: matrix.version-restrict == '' | |||
if: matrix.version-restrict == '' && (! contains(matrix.python-architecture, 'x86') || ! contains(matrix.extra-args, 'not composition')) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does:
overwrite: matrix.python-architecture == 'x86'
work?
the dist packages should be identical and it would be a bit easier to read
can you also add one sentence about the necessary adjustments to upload to the git commit description
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like overwrite
won't allow expressions, only true/false, unless I can't figure how to do it correctly. overwrite: true
can still work and would currently only affect the x86. If we needed to add other jobs later without overwriting, we'd have to change the name anyway
Windows x86 ci runs can fail because the container runs out of memory during testing. This is caused by memory leaks in psyneulink and limited available memory on the containers. As a workaround, running the full set of tests over multiple jobs reduces peak memory usage. Three jobs replace the original Windows x86 build, running pytest with: 1. -m llvm 2. -m "not llvm and composition" 3. -m "not llvm and not composition" Jobs with the same OS, python version, and architecture overwrite uploaded dist packages.
6147f7e
to
5d388b1
Compare
This PR causes the following changes to the html docs (ubuntu-latest-3.11-x64):
See CI logs for the full diff. |
This reverts commit 427808e. No longer needed.
This reverts commit 427808e. No longer needed.
This reverts commit 427808e. No longer needed.
Windows x86 ci runs can fail because the container runs out of memory
during testing. This is caused by memory leaks in psyneulink and limited
available memory on the containers. As a workaround, running the full
set of tests over multiple jobs reduces peak memory usage.
Three jobs replace the original Windows x86 build, running pytest with:
Jobs with the same OS, python version, and architecture overwrite
uploaded dist packages.