QuarkusTestResourceLifecycleManager should be executed only when a class is annotated by @QuarkusTestResource

[survivant]

I have junit tests and I have integration tests (named xxxIT). I want to launch the IT test only when the maven profile "integration-test" is active. (took that my quarkus example)

I have this in my pom.xml

<profile>
            <id>integration-test</id>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>${surefire-plugin.version}</version>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>integration-test</goal>
                                    <goal>verify</goal>
                                </goals>
                                <configuration>
                                    <includes>
                                        <include>**/*IT.java</include>
                                    </includes>
                                    <excludes>
                                        <exclude>**/*Native*IT.java</exclude>
                                    </excludes>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>

in my IT test, I have to create a Kind cluster and also launch a DockerCompose with TestContainer.

I created 2 classes that implements QuarkusTestResourceLifecycleManager

and I added them into my IT test like this

@QuarkusTest
@QuarkusTestResource(TwinApiServiceDockerCompose.class)
@QuarkusTestResource(KindClusterForQuarkusTest.class)
public class MachineMainCodecControllerTestIT {
...

That works fine for IT tests, but I found out that the juni test will start the cluster and dockercompose also even if I didn't put

@QuarkusTestResource(TwinApiServiceDockerCompose.class)
@QuarkusTestResource(KindClusterForQuarkusTest.class)

in my junit tests.

my junit test look like this

@QuarkusTest
class MachineHelperTest {
    ````

so.  My point is when a **@QuarkusTest** should use only the @QuarkusTestResource when they are specify in the class.

[famod]

As @geoand pointed out on the mailing list, any manager that is registered via @QuarkusTestResource directly on a test class has a global effect.
This basically works as designed. You are of course free to dislike the design and in fact other users were irritated by this behaviour in the past.
That's why I think that allowing @QuarkusTestResource for packages (package-info.java) and kind of deprecating class level usage could make the scope of this registration more clear.

For your use case I'd suggest to introduce a Quarkus test profile: https://quarkus.io/guides/getting-started-testing#testing_different_profiles (same article @geoand linked to in his message).

/cc @stuartwdouglas @geoand

[survivant]

here what I have in other SpringBoot projects

@ActiveProfiles("test")
@SpringBootTest
@Testcontainers
class ServiceTestIT {

    @Container
    private static DockerComposeContainer container = new DockerComposeContainer(...);
....

I check for easy example how to do a QuarkusTest with profile. I found this
https://github.com/quarkusio/quarkus/blob/5bbf33e3c45691fd7d5bc0f65fc1dd41592ece77/integration-tests/spring-web/src/test/java/io/quarkus/it/spring/web/testprofile/Profiles.java

public static class QuarkusTestProfileMatchingTag implements QuarkusTestProfile {

        @Override
        public Set<String> tags() {
            return new HashSet<>(Arrays.asList("test2", "test1", "test3"));
        }

        @Override
        public List<TestResourceEntry> testResources() {
            return Collections.singletonList(new TestResourceEntry(DummyTestResource.class));
        }

        public static class DummyTestResource implements QuarkusTestResourceLifecycleManager {

            public static AtomicInteger COUNT = new AtomicInteger(0);

            @Override
            public Map<String, String> start() {
                COUNT.incrementAndGet();
                return Collections.emptyMap();
            }

            @Override
            public void stop() {

            }
        }
    }

look a little to complicated.

could it be possible to have something like this

basic junit

@QuarkusTest
class MachineHelperTest {
...

tests with @QuarkusTestResource

@QuarkusTest
@QuarkusProfile("integration-test")
public class MachineMainCodecControllerTestIT {
...
}

@QuarkusTestProfile("integration-test")
@QuarkusTestResource(TwinApiServiceDockerCompose.class)
@QuarkusTestResource(KindClusterForQuarkusTest.class)
public class ITQuarkusTest {
...
}

You have already @QuarkusTestResource available, you could use them instead of

@Override
        public List<TestResourceEntry> testResources() {
            return Collections.singletonList(new TestResourceEntry(DummyTestResource.class));
        }

[stuartwdouglas]

The reason for the current design is that @QuarkusTest only starts Quarkus once. The test resources have to be global, so the resources are started before Quarkus starts. If you use Quarkus then Quarkus can start and stop, which is why we only allow this facility through profiles. The next release of JUnit will allow us to specify a way to sort the tests so we should be able to always run tests with the same profile in a batch to make this even better.

This allows Quarkus tests to run really quickly, because we are not starting and stopping all the time, but it does impose some limitations like this. I guess we could allow ad-hoc profiles that always start/stop, however its not really something we want to encourage as it will slow things down a lot (even if you app starts in one second it does not really take many tests until all those 1s delays start adding up).

[famod]

@stuartwdouglas

The next release of JUnit will allow us to specify a way to sort the tests so we should be able to always run tests with the same profile in a batch to make this even better.

Interesting! Do you have a link to an issue or PR for that change?
IIRC, Maven Surefire passes JUnit4 tests one by one to the forked test JVM, so that the test framework couldn't do anything about the order.
I suppose this works differently for JUnit5?

[cmasantos]

I agree that the QuarkusTestResource scope can be really weird if one is not aware of it. We are annotating a class and suddenly it's working everywhere. You can just abstract the container's life cycle on a base class but be careful if you are running tests in parallel.

[stuartwdouglas]

junit-team/junit5@b3ce7ae is the new ordering support.

[stuartwdouglas]

@survivant it sounds like you want something like this:

public class MyTestProfile implements QuarkusTestProfile {

    @Override
    public List<TestResourceEntry> testResources() {
        return Arrays.asList(new TestResourceEntry(TwinApiServiceDockerCompose.class), new TestResourceEntry(KindClusterForQuarkusTest.class));
    }
}

@QuarkusTestProfile(MyTestProfile.class)
public class ITQuarkusTest {

This will tie the test resources to the profile, and the resources will only be started for those tests.

As I mentioned earlier the reason why we don't want to allow this to be per-test without a profile is that then we need to start/stop Quarkus each time, which significantly slows down testing (I guess we could try and group all tests with the same annotation, but I would not really want to do that until we have sorting support).

[survivant]

@stuartwdouglas I understand the purpose of that. I just that it wasn't expected. Maybe the guide about testing need to explain a little more that.

Can I suggest an ehancement. Instead we could have this

@TestResource(KindClusterForQuarkusTest.class)
@TestResource(TwinApiServiceDockerCompose.class)
public class MyTestProfile implements QuarkusTestProfile {

}

@QuarkusTestProfile(MyTestProfile.class)
public class ITQuarkusTest {
..
}

[famod]

@geoand Isn't this now solved with the awesome addition of restrictToAnnotatedClass by @FroMage?

[geoand]

If I am reading this correctly, yes

[famod]

In this case, given how often people have complained about the global nature of @QuarkusTestResource, IMO it should be mentioned explicitely in the 1.12.1 blog @gsmet.
It's way too easy to miss it, especially since it's "hidden" in #14821 ("Support KubernetesServer test resource for CRUD operations").

[Manfred73]

I had several integration tests using wiremock proxy classes as test resources. I didn't want to write one big wiremock proxy, since each integration test needed to stub different things.

I had two separate integration tests, let's say A and B and I annotated A with @QuarkusTestResource(MyFirstWireMockProxy.class) B with @QuarkusTestResource(MySecondWireMockProxy.class). Both wiremock proxy classes used port 6666 which caused an issue with port already in use. At first I didn't understand why, but after reading this and the global nature @QuarkusTestResource I do. Very confusing though ;-)

When running integration test A or B separately, triggered both MyFirstWireMockProxy and MySecondWireMockProxy.

Having separate test profiles which stuartwdouglas mentioned on Feb 9, 2021 and defining the testresources in the profile, solved my issue (after removing the @QuarkusTestResource from the integration test - don't forget that!).