-
Notifications
You must be signed in to change notification settings - Fork 199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set up GitHub Actions Workflow for Testing Parsl with Flux #3159
Merged
Merged
Changes from all commits
Commits
Show all changes
73 commits
Select commit
Hold shift + click to select a range
b3bcdef
Added flux ci
mercybassey 1b6e399
removed tests
mercybassey 5c7a6c0
configured the CI to install Parsl
mercybassey 2bf4de1
Merge branch 'master' into parsl+flux
mercybassey a5fbe78
Merge branch 'master' into parsl+flux
mercybassey b1eda08
Install checked-out version of Parsl in CI
mercybassey 9fa40ef
Merge remote-tracking branch 'origin/parsl+flux' into parsl+flux
mercybassey efacad0
Add basic Parsl verification to CI
mercybassey 177944c
Fixed indentation
mercybassey f714af6
Add step to install pytest
mercybassey a71e86c
Installed pytest and pytest-random-order for CI tests
mercybassey 8447148
Install dependencies from test-requirements.txt for CI tests
mercybassey 54d21e8
Adjust CI test command to resolve pytest failure
mercybassey d8b073b
Merge branch 'master' into parsl+flux
mercybassey 15d280e
Combining steps
mercybassey f1c8105
Merge branch 'parsl+flux' of github.com:mercybassey/parsl into parsl+…
mercybassey 961561c
Added --random-order
mercybassey a2f0923
Configured CI to install python3-dev
mercybassey fe5921e
Added a test for writing to non-writable directory
mercybassey c934f15
Merge branch 'master' into parsl+flux
mercybassey 89c294f
Added logging in the test
mercybassey 00741ba
Merge branch 'parsl+flux' of github.com:mercybassey/parsl into parsl+…
mercybassey 636a1ce
Merge branch 'master' into parsl+flux
mercybassey a9d8afa
Merge branch 'master' into parsl+flux
mercybassey e696cc0
commented out failing test
mercybassey c44adfc
Merge branch 'parsl+flux' of github.com:mercybassey/parsl into parsl+…
mercybassey bb03950
Merge branch 'master' into parsl+flux
mercybassey c79c0ec
commented out failing tests
mercybassey 48d4e9a
fixed flake8 issue
mercybassey d15ee18
Added a step to test Parsl with flux
mercybassey 778fc8f
Added local_threads.py config
mercybassey 542e74a
Configured the CI to start flux, before running tests
mercybassey e6db9b9
Streamlined parl+flux in CI
mercybassey 9916252
Reconfigured the 'Start Flux and Test Parsl with Flux'
mercybassey 15f0763
Added verbosity since the run doesn't exits
mercybassey 3bd14ad
removed concurrency for CI
mercybassey a1b9923
Added timeout of 5minutes and without any verbosity flag
mercybassey 4536aa9
Running CI with verbosity level -v
mercybassey 668caaa
Running CI with verbosity level -vvv
mercybassey 939f79f
Removed the std.out and test.memo.stdout.x files
mercybassey e66b069
Removed the std.out and test.memo.stdout.x files
mercybassey 627dccc
fixed flux test
mercybassey 8401356
Merge branch 'master' into parsl+flux
mercybassey 3a5628f
running ci again
mercybassey 448ae1f
fix makefile typo
mercybassey 0b8676f
Added flux test config
mercybassey c9bab6b
minor
mercybassey 45b4032
fixed flake8 error and seperated the flux+parsl test in ci
mercybassey ee3af80
Merge branch 'master' into parsl+flux
mercybassey d661fc1
running tests
mercybassey 791c268
Merge branch 'master' into parsl+flux
mercybassey 9df54fc
Running tests
mercybassey 8bc0bfd
Merge branch 'parsl+flux' of github.com:mercybassey/parsl into parsl+…
mercybassey 0903435
fixed flake8 error
mercybassey 2583075
remove a mistake in the makefile
mercybassey b514001
added a comment at 'flux_local_test'
mercybassey b2b96fd
Merge branch 'master' into parsl+flux
benclifford c7761aa
Merge branch 'master' into parsl+flux
mercybassey 91d1539
took test_stdout.py to how it was and corrected ci
mercybassey 215a035
Omitted tests marked as as well in 'Test Parsl with Flux Config'
mercybassey 3bb44d3
Merge branch 'master' into parsl+flux
benclifford 155d8e1
removed unwanted files
mercybassey d03cd52
Merge branch 'parsl+flux' of github.com:mercybassey/parsl into parsl+…
mercybassey f332bb1
Merge branch 'master' into parsl+flux
benclifford 0662487
Remove changes to .gitignore
benclifford f5d1630
Use 1 line between targets, like elsewhere
benclifford d7312d1
mark tests that need unix filesystem permissions enforced, which is n…
benclifford b4d5d85
Another bad file test that requires unix fs permission enforcement
benclifford 882f0e8
flake8
benclifford 4167a42
Increase workflow timeout - it was being reached in normal operation
benclifford 26c29e5
Increase timeout more - hit 10 minutes and i'd like to see what kind …
benclifford 49f210f
Upgrade base ubuntu to jammy
benclifford e990f08
Merge branch 'master' into parsl+flux
benclifford File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
name: Test Flux Scheduler | ||
on: | ||
pull_request: [] | ||
|
||
jobs: | ||
build: | ||
runs-on: ubuntu-22.04 | ||
permissions: | ||
packages: read | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
container: ['fluxrm/flux-sched:jammy'] | ||
timeout-minutes: 30 | ||
|
||
container: | ||
image: ${{ matrix.container }} | ||
options: "--platform=linux/amd64 --user root -it --init" | ||
|
||
name: ${{ matrix.container }} | ||
steps: | ||
- name: Make Space | ||
run: | | ||
rm -rf /usr/share/dotnet | ||
rm -rf /opt/ghc | ||
|
||
- name: Checkout | ||
uses: actions/checkout@v3 | ||
|
||
- name: Install Dependencies and Parsl | ||
run: | | ||
apt-get update && apt-get install -y python3-pip curl | ||
pip3 install . -r test-requirements.txt | ||
|
||
- name: Verify Parsl Installation | ||
run: | | ||
pytest parsl/tests/ -k "not cleannet and not unix_filesystem_permissions_required" --config parsl/tests/configs/local_threads.py --random-order --durations 10 | ||
|
||
- name: Start Flux and Test Parsl with Flux | ||
run: | | ||
flux start pytest parsl/tests/test_flux.py --config local --random-order | ||
|
||
- name: Test Parsl with Flux Config | ||
run: | | ||
flux start pytest parsl/tests/ -k "not cleannet and not unix_filesystem_permissions_required" --config parsl/tests/configs/flux_local.py --random-order --durations 10 | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
from parsl.config import Config | ||
from parsl.executors import FluxExecutor | ||
|
||
|
||
def fresh_config(): | ||
return Config( | ||
executors=[FluxExecutor()], | ||
) | ||
|
||
|
||
config = fresh_config() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In case you don't know, for single run lines you don't need the pipe, and can just do:
It's just visual, but makes the file a little more tidy.