Increase timeout for Mac CI #1309
Conversation
There was a problem hiding this comment.
This looks reasonable to me. I also had to fix the TEST_TIMEOUT int issue in #1285. Did you want to increase the timeout for the terra-main run as well?
I wish we had more time to look into this. It would be nice to pull logs from a bunch of runs and do some data analysis on the variability of the test times. I think we might benefit from setting some variable to disable lower level parallelization which might be conflicting with stestr's parallelization, but I am not sure.
| run: tox -e terra-main | ||
| if: runner.os == 'macOS' | ||
| env: | ||
| TEST_TIMEOUT: 120 |
There was a problem hiding this comment.
Have the cron tests been more reliable than the main tests? I don't see the OMP_NUM_THREADS setting in the main file and that's the kind of parallelization disabling that might help with interfering parallel tests.
There was a problem hiding this comment.
Ah, just left a comment about the same thing.
There was a problem hiding this comment.
Nice! If we had time, it might be good to look at this kind of thing more closely. Even though Windows and Linux are not timing out, they might still benefit from disabling parallelization of math operations like in numpy since stestr is already trying to use one cpu per test. We would need to look up the right environment variables and compare the run times.
|
@wshanks good point, I added it to the terra-main run too. That workflow still has the |
### Summary Mac CI tests can take significantly longer than Windows and Linux. This PR doubles the timeout for Mac tests so they don't fail as frequently and adds back the `OMP_NUM_THREADS: 1` option, which seems to help reduce individual test times.
Summary
Mac CI tests can take significantly longer than Windows and Linux. This PR doubles the timeout for Mac tests so they don't fail as frequently and adds back the
OMP_NUM_THREADS: 1option, which seems to help reduce individual test times.