Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better comparison commits for PR benchmarking #5431

Merged
merged 2 commits into from
Aug 18, 2023
Merged

Better comparison commits for PR benchmarking #5431

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
de8e3fc
Don't check out head_ref - benchmark the GH simulated merge commit in…
trexfeathers Aug 17, 2023
040ffc2
What's New entry.
trexfeathers Aug 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/benchmarks_run.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.number }}
run: |
git checkout ${{ github.head_ref }}
python benchmarks/bm_runner.py branch origin/${{ github.base_ref }}

- name: Run overnight benchmarks
Expand Down
3 changes: 2 additions & 1 deletion benchmarks/bm_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,8 @@ class Branch(_SubParserGenerator):
description = (
"Performs the same operations as ``overnight``, but always on two commits "
"only - ``HEAD``, and ``HEAD``'s merge-base with the input "
"**base_branch**. If running on GitHub Actions: performance "
"**base_branch**. If running on GitHub Actions: HEAD will be GitHub's "
"merge commit and merge-base will be the merge target. Performance "
"comparisons will be posted in a comment on the relevant pull request.\n"
"Designed "
"for testing if the active branch's changes cause performance shifts - "
Expand Down
6 changes: 6 additions & 0 deletions docs/src/whatsnew/latest.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,12 @@ This document explains the changes made to Iris for this release
benchmarking of pull requests. Results are now posted as a new comment.
(feature branch: :pull:`5430`)

#. `@trexfeathers`_ changed pull request benchmarking to compare: GitHub's
simulated merge-commit (which it uses for all PR CI by default) versus the
merge target (e.g. `main`). This should provide the most 'atomic' account
of how the pull request changes affect performance. (feature branch:
:pull:`5431`)


.. comment
Whatsnew author names (@github name) in alphabetical order. Note that,
Expand Down