Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚇 Retarget PR benchmarks to main #831

Merged
merged 2 commits into from
Sep 19, 2021
Merged

🚇 Retarget PR benchmarks to main #831

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cfe2e7e
🚇 Use main as baseline for PR compare benchmarks
s-weigand Sep 18, 2021
5c5e7a2
🚇 Added comparison of main against last realease Tests
s-weigand Sep 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 28 additions & 5 deletions .github/workflows/pr_benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,17 @@ jobs:
machine_file.write_text(machine_file_content)

- name: Run PR compare benchmarks
id: benchmark_run
run: |
git remote add upstream https://github.com/glotaran/pyglotaran
git fetch upstream
pushd benchmark
asv run v0.4.0^..v0.4.0 --machine gh_action
last_tag=$(git describe --tags $(git rev-list --tags --max-count=1))
asv run $last_tag^..$last_tag --machine gh_action
asv run upstream/main^..upstream/main --machine gh_action
asv run HEAD^..HEAD --machine gh_action
asv publish
echo ::set-output name=last_tag::$last_tag

- name: Checkout benchmark result repo
uses: actions/checkout@v2
Expand All @@ -73,16 +77,35 @@ jobs:
import subprocess

pr_nr = os.environ.get("PR_NR")
diff_result = subprocess.run(
["asv", "--config=benchmark/asv.conf.json", "compare", "v0.4.0", "HEAD"],
diff_release_main = subprocess.run(
["asv", "--config=benchmark/asv.conf.json", "compare", "${{steps.benchmark_run.outputs.last_tag}}", "HEAD"],
capture_output=True,
)
diff_main_PR = subprocess.run(
["asv", "--config=benchmark/asv.conf.json", "compare", "upstream/main", "HEAD"],
capture_output=True,
)
comment = f"""\
Benchmark is done. Checkout the [benchmark result page](https://glotaran.github.io/pyglotaran-benchmarks/prs/pr-{pr_nr}).
Benchmark differences below 5% might be due to CI noise.
<details>
<summary>
Benchmark diff
Benchmark diff ${{steps.benchmark_run.outputs.last_tag}} vs. main
</summary>

Parametrized benchmark signatures:
```python
BenchmarkOptimize.time_optimize(index_dependent, grouped, weight)
```

```
{diff_release_main.stdout.decode()}
```
</details>

<details>
<summary>
Benchmark diff main vs. PR
</summary>

Parametrized benchmark signatures:
Expand All @@ -91,7 +114,7 @@ jobs:
```

```
{diff_result.stdout.decode()}
{diff_main_PR.stdout.decode()}
```
</details>
"""
Expand Down