Run CI (tests) on MacOS #3528
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,7 +15,7 @@ jobs: | |
| fail-fast: false | ||
| matrix: | ||
| go-version: [1.20.x] | ||
| platform: [ubuntu-latest, windows-2019, macos-latest] | ||
| runs-on: ${{ matrix.platform }} | ||
| steps: | ||
| - name: Checkout code | ||
|
|
@@ -37,8 +37,8 @@ jobs: | |
| go version | ||
| export GOMAXPROCS=2 | ||
| args=("-p" "2" "-race") | ||
| # Run with less concurrency on Windows/MacOS to minimize flakiness. | ||
| if [[ "${{ matrix.platform }}" == windows* || "${{ matrix.platform }}" == macos* ]]; then | ||
| unset args[2] | ||
| args[1]="1" | ||
| export GOMAXPROCS=1 | ||
|
|
@@ -49,7 +49,7 @@ jobs: | |
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| platform: [ubuntu-latest, windows-2019, macos-latest] | ||
| runs-on: ${{ matrix.platform }} | ||
| continue-on-error: true | ||
| steps: | ||
|
|
@@ -75,8 +75,8 @@ jobs: | |
| go version | ||
| export GOMAXPROCS=2 | ||
| args=("-p" "2" "-race") | ||
| # Run with less concurrency on Windows/MacOS to minimize flakiness. | ||
| if [[ "${{ matrix.platform }}" == windows* || "${{ matrix.platform }}" == macos* ]]; then | ||
| unset args[2] | ||
| args[1]="1" | ||
| export GOMAXPROCS=1 | ||
|
|
@@ -88,7 +88,7 @@ jobs: | |
| fail-fast: false | ||
| matrix: | ||
| go-version: [1.21.x] | ||
| platform: [ubuntu-latest, windows-2019, macos-latest] | ||
| runs-on: ${{ matrix.platform }} | ||
| steps: | ||
| - name: Checkout code | ||
|
|
@@ -103,8 +103,8 @@ jobs: | |
| go version | ||
| export GOMAXPROCS=2 | ||
| args=("-p" "2" "-race") | ||
| # Run with less concurrency on Windows/MacOS to minimize flakiness. | ||
| if [[ "${{ matrix.platform }}" == windows* || "${{ matrix.platform }}" == macos* ]]; then | ||
| unset args[2] | ||
| args[1]="1" | ||
| export GOMAXPROCS=1 | ||
|
|
@@ -126,8 +126,12 @@ jobs: | |
| CODECOV_BASH_SHA512SUM: d075b412a362a9a2b7aedfec3b8b9a9a927b3b99e98c7c15a2b76ef09862aeb005e91d76a5fd71b511141496d0fd23d1b42095f722ebcd509d768fba030f159e | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
| run: | | ||
| if [[ "${{ matrix.platform }}" == macos* ]]; then | ||
| shopt -s expand_aliases | ||
| alias sha512sum='shasum -a 512' | ||
| fi | ||
| curl -fsSLO "https://raw.githubusercontent.com/codecov/codecov-bash/${CODECOV_BASH_VERSION}/codecov" | ||
| echo "$CODECOV_BASH_SHA512SUM codecov" | sha512sum -c - | ||
|
There was a problem hiding this comment. For some reason (I didn't fully get), without the extra space it doesn't work on MacOS, but with it, it works correctly in all platforms 🤷🏻 There was a problem hiding this comment. I remember that we had issues with There was a problem hiding this comment. Oh, nice! Thanks for letting me know, I could consider working on that right after, but I'd prefer to keep it for a separate PR, if that sounds good to you as well 🙏🏻 |
||
| platform="${{ matrix.platform }}" | ||
| bash ./codecov -F "${platform%%-*}" | ||
| - name: Generate coverage HTML report | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried luck without this, and there has been no way to get all tests passing. I guess next step will be to take a look at that flakiness and try to correct it, but for now I guess it's fine, as it has been for Windows so far.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue is that tests are flaky on all platforms, primarily GRPC-TLS related https://github.com/grafana/xk6-grpc/issues/39. Which ones do you constantly see?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, there's nothing new, as I think all (or most) of them are documented at #2144.
It's just that it seems that Windows and MacOS runners are way more likely to fail (it's probably a matter of their performance), so I decided to add MacOS into that exception we already had for Windows, to avoid adding more flakiness.