Qute: introduce RenderedResults assert API for tests

[mkouba]

Related to quarkiverse/quarkus-renarde#198.

[github-actions]

🙈 The PR is closed and the preview is expired.

[FroMage]

LGTM, great!

[quarkus-bot]

Status for workflow Quarkus Documentation CI

This is the status report for running Quarkus Documentation CI on commit 07f6a3c.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

⚠️ There are other workflow runs running, you probably need to wait for their status before merging.

[quarkus-bot]

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 07f6a3c.

Failing Jobs

Status	Name	Step	Failures	Logs	Raw logs	Build scan
✖	Native Tests - Data5	Build	Failures	Logs	Raw logs	🚧

Full information is available in the Build summary check run.

Failures

⚙️ Native Tests - Data5 #

- Failing: integration-tests/jpa-postgresql integration-tests/jpa-postgresql-withxml 

📦 integration-tests/jpa-postgresql

✖ Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-jpa-postgresql: Failed to build quarkus application

📦 integration-tests/jpa-postgresql-withxml

✖ io.quarkus.it.jpa.postgresql.ImageMetricsITCase.verifyImageMetrics line 15 - History - More details - Source on GitHub

org.opentest4j.AssertionFailedError: Expected analysis_results.fields.reflection to be within range [163 +- 3%] but was 168 ==> expected: <true> but was: <false>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
	at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
	at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
	at io.quarkus.test.junit.nativeimage.NativeBuildOutputExtension.assertValueWithinRange(NativeBuildOutputExtension.java:90)
	at io.quarkus.test.junit.nativeimage.NativeBuildOutputExtension.lambda$verifyImageMetrics$0(NativeBuildOutputExtension.java:66)

---

Flaky tests - Develocity

⚙️ Maven Tests - JDK 17

📦 integration-tests/maven

✖ io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed - History

• io.quarkus.maven.it.DevMojoIT expected "58156cea-eafe-44f9-b020-6ecd132a8677" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: io.quarkus.maven.it.DevMojoIT expected "58156cea-eafe-44f9-b020-6ecd132a8677" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AbstractHamcrestCondition.await(AbstractHamcrestCondition.java:86)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:691)
	at io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed(DevMojoIT.java:967)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

• io.quarkus.maven.it.DevMojoIT expected "bebb11d3-fff7-44bd-987c-cd0b4587d22f" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: io.quarkus.maven.it.DevMojoIT expected "bebb11d3-fff7-44bd-987c-cd0b4587d22f" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AbstractHamcrestCondition.await(AbstractHamcrestCondition.java:86)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:691)
	at io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed(DevMojoIT.java:967)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

✖ io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed - History

• io.quarkus.maven.it.DevMojoIT expected "58156cea-eafe-44f9-b020-6ecd132a8677" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: io.quarkus.maven.it.DevMojoIT expected "58156cea-eafe-44f9-b020-6ecd132a8677" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AbstractHamcrestCondition.await(AbstractHamcrestCondition.java:86)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:691)
	at io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed(DevMojoIT.java:967)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

• io.quarkus.maven.it.DevMojoIT expected "bebb11d3-fff7-44bd-987c-cd0b4587d22f" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: io.quarkus.maven.it.DevMojoIT expected "bebb11d3-fff7-44bd-987c-cd0b4587d22f" but was "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." within 2 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AbstractHamcrestCondition.await(AbstractHamcrestCondition.java:86)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:691)
	at io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed(DevMojoIT.java:967)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

[mkouba]

@gsmet Do you happen to know if io.quarkus.it.jpa.postgresql.ImageMetricsITCase.verifyImageMetrics is a flaky test?

[mkouba]

@gsmet Do you happen to know if io.quarkus.it.jpa.postgresql.ImageMetricsITCase.verifyImageMetrics is a flaky test?

Hm, it seems that this test fails in other PRs as well.

I'm going to merge this one.