Add flaky decorator for tests with remote resources to reduce cold start failure rate.

[charles-turner-1]

Change Summary

• Add flaky pytest plugin
• Add @pytest.mark.flaky(max_runs=3, min_passes=1) to test that request cloud resources. This should give cloud containers time to spin up if needed.
• Updated CI environments to match.

Related issue number

#699

Checklist

• Unit tests for the changes exist N/A
• Tests pass on CI
• Documentation reflects the changes where applicable N/A
  • Potentially could think about adding notes about this issue with cloud resources to documentation.

[charles-turner-1]

Test failures on CI are resulting from

FAILED tests/test_source.py::test_open_dataset_kerchunk - ValueError: Reference-FS's target filesystem must have same valueof asynchronous

Investigating the cause now.

[mgrover1]

Test failures on CI are resulting from

FAILED tests/test_source.py::test_open_dataset_kerchunk - ValueError: Reference-FS's target filesystem must have same valueof asynchronous

Investigating the cause now.

Interesting... let me know what you find! I have not seen that before!

[charles-turner-1]

Interesting... let me know what you find! I have not seen that before!

Weirdly enough, it consistently passes on 3.10, fails on 3.11 & 3.12. My local environment is 3.13 but on MacOS system.

I'm gonna mark this as draft whilst I work on it - tests all pass locally, so I'm pretty stumped as to whats going on...

[mgrover1]

Sounds good! I am working on Australian time this week and can review later... 👍 currently on travel to Tasmania...

[andersy005]

I'm gonna mark this as draft whilst I work on it - tests all pass locally, so I'm pretty stumped as to whats going on...

there is a chance this might be caused by zarr v3. when you get a chance, can you confirm if you are using zarr v2 or zarr v3?

it might be worth pinning the zarr requirements to versions earlier than v3 until we have a robust solution for dealing with the breaking changes in zarr v3.

[charles-turner-1]

I've checked and my local & CI 3.10 are both on zarr==2.18.3, CI 3.11+ are on zarr 3.

Pinning the deps now.

[charles-turner-1]

Sounds good! I am working on Australian time this week and can review later... 👍 currently on travel to Tasmania...

Looks like this is ready to review now - @andersy005's zarr suggestion fixed the issue.

I'm gonna see if I can use this same cold start trick to stop the documentation builds failing, so hopefully I'll drop that in too soon

[mgrover1]

Looks great! Thanks for the fix!