Introduce integration test source set for gradle based project #24064

glefloch · 2022-03-02T20:49:07Z

This introduce a new intTest source set (src/intTest/java) and a new quarkusIntTest task that execcute @QuarkusIntegrationTest.
This tasks depends on quarkusBuild to make sure the final artifact used by integration test is generated.

Next step will be:

Update code start to create the intTest sourceSet instead of native-test
Update nativeTest task to combine both sourceSet
Deprecate the native-test sourceSet

Close #23528
Close #22035

glefloch · 2022-03-02T20:52:21Z

cc @edeandrea

aloubyansky · 2022-03-02T23:32:04Z

test-framework/junit5/src/main/java/io/quarkus/test/junit/IntegrationTestUtil.java

@@ -498,7 +498,7 @@ private static Path determineBuildOutputDirectory(final URL url) {
            if (url.getPath().endsWith("test-classes/")) {
                // we have the maven test classes dir
                return toPath(url).getParent();
-            } else if (url.getPath().endsWith("test/")) {
+            } else if (url.getPath().endsWith("test/") || url.getPath().endsWith("intTest/")) {


We should check whether what we are doing if url.getPath().endsWith("-tests.jar") will work for all the other cases. That could be a separate PR though.

glefloch · 2022-03-04T10:07:08Z

@edeandrea is it ok for you? this introduce a new quarkusIntTest task which will run tests located in src/intTest/java.
The tasks depends on the artifact produced by quarkusBuild.
The quarkusIntTest task is not linked to any other task.

edeandrea · 2022-03-04T14:44:36Z

devtools/gradle/gradle-application-plugin/src/main/java/io/quarkus/gradle/QuarkusPlugin.java

+                        intTestTask.dependsOn(quarkusBuild);
+                        intTestTask.setClasspath(intTestSourceSet.getRuntimeClasspath());
+                        intTestTask.setTestClassesDirs(intTestSourceSet.getOutput().getClassesDirs());
+                    });


Do you need intTestTask.setShouldRunAfter(List.of(tasks.findByName(JavaPlugin.TEST_TASK_NAME))); as well?

edeandrea · 2022-03-04T14:46:05Z

docs/src/main/asciidoc/gradle-tooling.adoc

+----
+
+This task depends on `quarkusBuild`. The final artifact will be produced before running tests.
+


Does quarkusBuild also run the check task (or at a minimum the test task)? If so, maybe you should mention that the unit tests will also be executed?

Maybe not necessary, but just a thought.

Currently quarkusBuild is not linked to any test task

Should it (or the intTest task) be?

I will add a dependency between quarkusIntTest and check

edeandrea · 2022-03-04T14:47:29Z

.../gradle/src/main/resources/it-test-basic-project/src/main/java/org/acme/ExampleResource.java

+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.eclipse.microprofile.config.inject.ConfigProperty;


I don't think this import is necessary?

edeandrea · 2022-03-04T14:47:42Z

.../gradle/src/main/resources/it-test-basic-project/src/main/java/org/acme/ExampleResource.java

@@ -0,0 +1,20 @@
+package org.acme;
+
+import javax.inject.Inject;


I don't think this import is necessary?

edeandrea · 2022-03-04T14:48:55Z

...dle/src/main/resources/it-test-basic-project/src/test/java/org/acme/ExampleResourceTest.java

+          .when().get("/hello")
+          .then()
+             .statusCode(200)
+             .body(is("Hello from Test"));


I don't understand how this test passes? ExampleResource only has a single endpoint, /hello, which returns "Hello World"? Am I missing something obvious somewhere?

I see in src/test/resources/application.properties there are 2 properties, yet ExampleResource.java never references them.

This leads me to believe that simply running the intTest task does not trigger the check task (or at a minimum the test task). Should it? Should you ever have the case where your integration tests pass yet your unit tests fail?

edeandrea · 2022-03-04T14:50:55Z

...dle/src/main/resources/it-test-basic-project/src/test/java/org/acme/ExampleResourceTest.java

+          .when().get("test-only")
+          .then()
+              .statusCode(200)
+              .body(is("Test only"));


I don't understand how this test passes? Where is the test-only endpoint defined? Am I missing something obvious?

I removed it.

edeandrea · 2022-03-04T14:54:33Z

integration-tests/gradle/src/test/java/io/quarkus/gradle/IntegrationTestBuildTest.java

+    public void shouldRunIntegrationTestAsPartOfBuild() throws Exception {
+        File projectDir = getProjectDir("it-test-basic-project");
+
+        BuildResult buildResult = runGradleWrapper(projectDir, "clean", "quarkusIntTest");


See comment above re: should the intTest task trigger the check (or at a minimum test) task?

Right, I will add a dependency the check task.

edeandrea · 2022-03-04T14:59:18Z

@glefloch I added some comments.

Also, should this also close #22035?

gsmet · 2022-03-05T17:26:56Z

Is intTest used by other projects? It sounds very weird to me. Maybe it would be better? True question :).

edeandrea · 2022-03-05T17:32:35Z

Are you talking about the source set name? Personally I think it is too short. I think the test task name is quarkusIntTest.

glefloch · 2022-03-05T17:44:16Z

There isn't convention for this.
In documention they usually declare ˋsrc/integrationTest/java And use ˋintegrationTest task name. Maybe we should go with that to be as close as possible from docs.

edeandrea · 2022-03-05T17:55:39Z

integrationTest is just so long and wordy. My last 2 companies we used intTest and the test task name was also intTest, following the convention that the test task name matches the source set name.

gsmet · 2022-03-05T18:41:36Z

Mkay. Personally, I never saw intTest being used but I'll let you guys do what you prefer given I don't use Gradle :).

edeandrea · 2022-03-05T21:49:47Z

Actually I was just looking at one of my personal projects. I use intTest as the source set name (https://github.com/edeandrea/xjc-generation-gradle-plugin/blob/main/build.gradle.kts#L27) but integrationTest as the task name (https://github.com/edeandrea/xjc-generation-gradle-plugin/blob/main/build.gradle.kts#L100). I also make the integrationTest task run after the test task (https://github.com/edeandrea/xjc-generation-gradle-plugin/blob/main/build.gradle.kts#L105) & the check task depend on the integrationTest task (https://github.com/edeandrea/xjc-generation-gradle-plugin/blob/main/build.gradle.kts#L114).

@gsmet you should use it - it's a far better tool IMO :)

glefloch · 2022-03-06T13:51:23Z

I updated the task name to match the name of the source set.
Looking at gradle documentation, there are both intTest and integrationTest.

integrationTest is longer but may be more relevant that intTest. it works too..

@aloubyansky, @geoand do you have a preference ?

geoand · 2022-03-06T14:31:43Z

I vote for integrationTest

aloubyansky · 2022-03-07T05:42:01Z

Another thing is, so far, every command in the Quarkus plugin had quarkus prefix. Could integrationTest potentially conflict with other plugins? Of course quarkusInterationTest will make it even longer, which isn't great.

glefloch · 2022-03-07T06:28:36Z

Yes this is a risk.

We can make the source set name configurable with a default value of ˋintegrationTest and use ˋquarkusIntTest as task name ?

edeandrea · 2022-03-07T13:46:41Z

Another thing is, so far, every command in the Quarkus plugin had quarkus prefix. Could integrationTest potentially conflict with other plugins? Of course quarkusInterationTest will make it even longer, which isn't great.

I agree - keep it consistent. If every task the plugin introduces has the quarkus prefix then this probably should too. quarkusIntTest works for me.

Personally I like intTest over integrationTest as the source set name, but at the end of the day it doesn't really matter that much to me. If you all like integrationTest then so be it.

glefloch · 2022-03-07T21:21:13Z

I adressed all comments, updated the sourceSet to integrationTest and the task name to quarkusIntTest

edeandrea · 2022-03-07T21:26:29Z

Nice work!

edeandrea · 2022-03-15T15:37:22Z

Hey @glefloch should the Using @QuarkusIntegrationTest section in the getting started testing guide include something about this new quarkusIntTest task?

Currently it just says

As a test annotated with @QuarkusIntegrationTest tests the result of the build, it should be run as part of the integration test suite - i.e. via the maven-failsafe-plugin if using Maven or an additional task if using Gradle. These tests will not work if run in the same phase as @QuarkusTest as Quarkus has not yet created the final artifact.

Maybe it should say something like

As a test annotated with @QuarkusIntegrationTest tests the result of the build, it should be run as part of the integration test suite - i.e. via the maven-failsafe-plugin if using Maven or the quarkusIntTest task if using Gradle. These tests will not work if run in the same phase as @QuarkusTest as Quarkus has not yet created the final artifact.

?

glefloch · 2022-03-15T16:13:29Z

Right, I will add a note on this guide.

rinmalavi · 2024-03-15T13:12:13Z

Right, I will add a note on this guide.

Not sure if ' or the quarkusIntTest task if using Gradle.' is the note added.
Reading this discussion I would assume quarkusIntTest will run only @QuarkusIntegrationTest in integrationTest source set, but it runs everything.
I am guessing i should add something to gradle.build to make this work, or was this supposed to be out of the box.

quarkus-bot bot added area/devtools Issues/PR related to maven, gradle, platform and cli tooling/plugins area/documentation area/gradle Gradle area/testing labels Mar 2, 2022

glefloch requested review from aloubyansky and geoand March 2, 2022 20:50

aloubyansky reviewed Mar 2, 2022

View reviewed changes

aloubyansky approved these changes Mar 2, 2022

View reviewed changes

geoand approved these changes Mar 3, 2022

View reviewed changes

edeandrea reviewed Mar 4, 2022

View reviewed changes

Introduce integration test source set for gradle based project

2f53c5f

glefloch force-pushed the fix/23528 branch from 2b8306c to 2f53c5f Compare March 7, 2022 21:19

glefloch merged commit b718fba into quarkusio:main Mar 8, 2022

quarkus-bot bot added this to the 2.8 - main milestone Mar 8, 2022

glefloch deleted the fix/23528 branch March 8, 2022 09:42

quarkus-bot bot added the kind/enhancement New feature or request label Mar 8, 2022

edeandrea mentioned this pull request Mar 15, 2022

Move failsafe config out of native profile #21997

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce integration test source set for gradle based project #24064

Introduce integration test source set for gradle based project #24064

glefloch commented Mar 2, 2022 •

edited

Loading

glefloch commented Mar 2, 2022

aloubyansky Mar 2, 2022

glefloch commented Mar 4, 2022

edeandrea Mar 4, 2022 •

edited

Loading

edeandrea Mar 4, 2022 •

edited

Loading

glefloch Mar 5, 2022

edeandrea Mar 5, 2022

glefloch Mar 5, 2022

edeandrea Mar 4, 2022

edeandrea Mar 4, 2022

edeandrea Mar 4, 2022 •

edited

Loading

edeandrea Mar 4, 2022

glefloch Mar 5, 2022

edeandrea Mar 4, 2022 •

edited

Loading

glefloch Mar 5, 2022

edeandrea commented Mar 4, 2022

gsmet commented Mar 5, 2022

edeandrea commented Mar 5, 2022

glefloch commented Mar 5, 2022

edeandrea commented Mar 5, 2022 •

edited

Loading

gsmet commented Mar 5, 2022

edeandrea commented Mar 5, 2022

glefloch commented Mar 6, 2022

geoand commented Mar 6, 2022

aloubyansky commented Mar 7, 2022

glefloch commented Mar 7, 2022

edeandrea commented Mar 7, 2022

glefloch commented Mar 7, 2022

edeandrea commented Mar 7, 2022

edeandrea commented Mar 15, 2022

glefloch commented Mar 15, 2022

rinmalavi commented Mar 15, 2024

		----

		This task depends on `quarkusBuild`. The final artifact will be produced before running tests.

		@@ -0,0 +1,20 @@
		package org.acme;

		import javax.inject.Inject;

Introduce integration test source set for gradle based project #24064

Introduce integration test source set for gradle based project #24064

Conversation

glefloch commented Mar 2, 2022 • edited Loading

glefloch commented Mar 2, 2022

Choose a reason for hiding this comment

glefloch commented Mar 4, 2022

edeandrea Mar 4, 2022 • edited Loading

Choose a reason for hiding this comment

edeandrea Mar 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

edeandrea Mar 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

edeandrea Mar 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

edeandrea commented Mar 4, 2022

gsmet commented Mar 5, 2022

edeandrea commented Mar 5, 2022

glefloch commented Mar 5, 2022

edeandrea commented Mar 5, 2022 • edited Loading

gsmet commented Mar 5, 2022

edeandrea commented Mar 5, 2022

glefloch commented Mar 6, 2022

geoand commented Mar 6, 2022

aloubyansky commented Mar 7, 2022

glefloch commented Mar 7, 2022

edeandrea commented Mar 7, 2022

glefloch commented Mar 7, 2022

edeandrea commented Mar 7, 2022

edeandrea commented Mar 15, 2022

glefloch commented Mar 15, 2022

rinmalavi commented Mar 15, 2024

glefloch commented Mar 2, 2022 •

edited

Loading

edeandrea Mar 4, 2022 •

edited

Loading

edeandrea Mar 4, 2022 •

edited

Loading

edeandrea Mar 4, 2022 •

edited

Loading

edeandrea Mar 4, 2022 •

edited

Loading

edeandrea commented Mar 5, 2022 •

edited

Loading