-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increase binary size baseline for postgresql IT with Mandrel 23.0 #38647
Increase binary size baseline for postgresql IT with Mandrel 23.0 #38647
Conversation
CPU updates seem to have a small effect on the binary size (~0.6% increase). The new baseline is based on a build with the mandrel builder image with tag `23.0.3.0-Final-java17-2024-02-04`
Hey. Seems this is happening every other month... you might want to increase the tolerance? |
Status for workflow
|
The first changes were due to insufficient testing (this kind of testing was freshly introduced and didn't cover all corner cases). This particular one is related to a CPU release (such releases happen once every 3 months) and might indeed lead to a recurring thing. Right now the threshold is set to 3% I am OK with increasing it to 5% but I would prefer someone more involved in performance testing to take that decision. The largest the threshold the latest we will notice about a performance degradation. It is also worth noting that in this case we didn't need to bump the baseline because we had a 3% increase from the baseline, but because of graalvm/mandrel#654 which results in tests run with the Red Hat build of Mandrel to hit the threshold. Perhaps we should use a different baseline for Red Hat builds, but that's not trivial as Quarkus currently doesn't distinguish them. |
From QE perspective we testing it regularly with RH build of mandrel so we are probably only/first one who hitting this.
Probably it would be possible to distinguish it if Maybe proposal for this is also have these metric/performance test as standalone profile and run them standalone weekly or when new version of mandrel/graal is released. |
Let's merge this for now, improvements can be made in a follow-up PR. |
CPU updates seem to have a small effect on the binary size (~0.6%
increase).
The new baseline is based on a build with the mandrel builder image with
tag
23.0.3.0-Final-java17-2024-02-04
Closes #38563