examples/integrations/oci/objectstorage-cdi fails when run in IntelliJ IDEA (module-info)

[barchetta]

Checkout out the 2.5.2 tag and load integrations/oci/objectstorage-cdi example into IntelliJ. Open ObjectStorageCdiMain.java and click the green triangle to "Run ObjectStorageCdiMain.main()" it will fail with:

java.lang.reflect.InaccessibleObjectException: Unable to make field private io.helidon.security.integration.jersey.FeatureConfig 
io.helidon.security.integration.jersey.SecurityFilterCommon.featureConfig accessible: module 
io.helidon.security.integration.jersey does not "opens io.helidon.security.integration.jersey" 
to unnamed module @577f9109
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)

When run via the command line using java -jar it works as expected. If you remove module-info.java it works as expected in IntelliJ. Apparently the presence of the module-info.java file informs IntelliJ to run the application as a module and there is no simple way to prevent this. I say simple because there are some alternatives, but they add friction to users that are just trying to load and run our examples.

Some additional info concerning options in IntelliJ IDEA:

• The JUnit configuration template has a Do not use --module-path option (under Modify Options), but the Application configuration template does not.
• The JAR Application configuration template does work because the application jar uses classpath, but this has its own shortcomings (needs to be setup correctly and requires jar file to be rebuilt).

So the cleanest solution might be to remove module-info.java files from examples that don't need it.

Examples in 2.x that have module-info.java. We should consider removing these:

./dbclient/mongodb/src/main/java/module-info.java
./dbclient/jdbc/src/main/java/module-info.java
./dbclient/common/src/main/java/module-info.java
./dbclient/pokemons/src/main/java/module-info.java
./security/attribute-based-access-control/src/main/java/module-info.java
./integrations/oci/objectstorage-cdi/src/main/java/module-info.java
./integrations/oci/vault-cdi/src/main/java/module-info.java
./integrations/oci/atp-reactive/src/main/java/module-info.java
./integrations/oci/atp-cdi/src/main/java/module-info.java
./integrations/oci/vault-reactive/src/main/java/module-info.java
./integrations/oci/objectstorage-reactive/src/main/java/module-info.java
./integrations/oci/metrics-reactive/src/main/java/module-info.java

[andrejusc]

If module-info.java causes issues in IDE and expectation is still to run some example via classpath - then I'd say alternative set of examples supporting module-info should be present. That will then allow those, who already works in the space of Java modules to evaluate those example via Java module approach. And if IDE doesn't work for that - then via Maven CLI.

[romain-grecourt]

We are not going to duplicate all examples. Instead we are going to remove module-info from all examples (see #4893 #4894 #4895) and add a couple of examples that will provide a module-info (see #4872).

[andrejusc]

But what is driving that removal decision? In Ide profile one could add needed --add-opens/--add-exports switches while invoking underlying JDK and those should resolve similar to above issues with unnamed modules access. At the same time it could help identify what else left in overall ecosystem still unnamed and make that named to everyone's benefit.

[barchetta]

@andrejusc it is very important to us that our examples work with as little friction as possible. Having module-info in an example adds friction when used with IntelliJ. We have seen this repeatedly trip up our users. So, "by default", Helidon examples won't include module-info.java.

But, as @romain-grecourt said, we do plan on having some examples whose purpose is to demonstrate the use of module path so users that care about JPMS can reference those examples.

[andrejusc]

@barchetta - this is interesting topic in itself about Jigsaw (aka Java modules) adoption/friction/IDE and Helidon. In some sense I think we have here contradicting goals:

• Assuming that Helidon as a Framework progresses towards Jigsaw adoption (please correct me if that is not true?) - one would expect to see almost all things, including examples, aligned to that direction
• Falling back to classpath usage approach shows that friction here is about some potential limitations of either Jigsaw itself (at least for myself I have 1 initial outstanding request there, but got no support) or IDE or other tools/libs around to use that properly. Which could be the case having some 3rd party libs still on pre-Java9 modules levels.

Till recent 2022.1.x version Intellij IDE had issues itself to support Java modules in robust way. They did multiple gradual enhancements. There was long and elaborated discussion under this issue: https://youtrack.jetbrains.com/issue/IDEA-273159 and some other IDE issues were linked to that as well.

By removing module-info.java from some examples and leaving in another ones - overall message to Helidon users then becomes inconsistent. At least in my opinion.

I need to admit that I didn't yet try any of referenced examples inside IDE and see what is the root cause if module-info.java is present. But even looking at your short error output - who on the earth would be able decode fast what unnamed module @577f9109 means. And so if that is coming from Jigsaw layer/core/functionality - ideally we'd need to fill Enh request to them to make this error more clear, so overall end user's understanding is present and could be also reported back to Helidon project if observed.

Each adoption faces friction, but if there is common belief (see my question earlier) that adoption is good - then effort to address friction (and not just put it aside) will be around.

[andrejusc]

@barchetta - I downloaded now Helidon 2.5.3 release and examples and built such:
target/helidon-examples-integrations-oci-objectstorage-cdi.jar

and did initially very primitive Java modules validation test with GraalVM 22.2 and JDK17 like such:

java -p target/helidon-examples-integrations-oci-objectstorage-cdi.jar:target/libs --validate-modules

which to my surprise failed with such:

file:///dev/tmp/helidon-2.5.3/examples/integrations/oci/objectstorage-cdi/target/libs/jakarta.json-1.1.6-module.jar org.glassfish.java.json
    contains org.glassfish.json conflicts with module java.json
    contains org.glassfish.json.api conflicts with module java.json
file:///dev/tmp/helidon-2.5.3/examples/integrations/oci/objectstorage-cdi/target/libs/javax.annotation-api-1.3.2.jar java.annotation automatic
    contains javax.annotation conflicts with module jsr305
file:///dev/tmp/helidon-2.5.3/examples/integrations/oci/objectstorage-cdi/target/libs/jakarta.annotation-api-1.3.5.jar java.annotation automatic
    contains same module as javax.annotation-api-1.3.2.jar
Error occurred during initialization of boot layer
java.lang.RuntimeException: Validation of module path failed

Such initial failure at least indicates that collected dependencies have clash from Java modules angle and that should be resolved first before returning to friction in IDE aspect/discussion.

And no surprise that one of clashing dependencies is brought by referenced OCI SDK 2.41, i.e. this one:

javax.annotation:javax.annotation-api:jar:1.3.2:compile

while good one coming from Helidon:

jakarta.annotation:jakarta.annotation-api:jar:1.3.5:compile

And so Java modules in this regards play quite good guard role of saying that one or another should be used, but not potentially both.

In one of projects I'm dealing with - I've explicitly added invocation of above --validate-modules command into place where dependencies list is finalized, so it assures that later Java modules booting will work.

Looking at above issue makes me think that such enhancement could make sense to those examples, which will be presenting Java modules functionality through them.

[romain-grecourt]

This is a dependency convergence issue. Can you share the output of dependency:tree ?
We may need to add an exclusion around some oci-sdk dependency, if that's the case let's file a new issue for that.

[romain-grecourt]

Validation of java modules is challenging, this is an area where we can improve. Though doing it from the examples is not practical.

[andrejusc]

Please see dependency tree for this specific example under discussion
helidon-2.5.3-example-deps.txt

Validation is not challenging actually - doing that already for 3+ months at least. Could share my Maven approach for that if you wish.

I think it should be introduced for Helidon project regardless it's core or examples. Only then we'll be able to claim we support and use Java modules properly.

[romain-grecourt]

We have 200+ artifacts and a dependency graph that we ship for each of them, validating that is VERY challenging.

[andrejusc]

I assume you're building each separate component's POM in your pipelines to see if that building is not failing. For this example you could see already it copies all dependencies into target/libs folder (this approach is coming from old classpath approach, but helpful then).

But even without this aggregation into single target/libs folder for dependencies - you could produce dependencies list for each POM into 1 long line and pass that long line into exact same validation command.

i.e. something like this:

        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>exec-maven-plugin</artifactId>
          <executions>
            <execution>
              <id>validate-modules</id>
              <phase>prepare-package</phase>
              <goals>
                <goal>exec</goal>
              </goals>
              <configuration>
                <executable>java</executable>
                <arguments>
                  <argument>--module-path=<put_your_deps_list_here></argument>
                  <argument>--validate-modules</argument>
                </arguments>
              </configuration>
            </execution>
          </executions>
        </plugin>

[andrejusc]

And assuming that for each POM dependencies meant to be Java modules (maybe need to filter out of test scope, etc.) to use - they could all be collected into some Maven property via dependency plugin on the fly and that property then passed to above snippet:

  <plugin>
      <artifactId>maven-dependency-plugin</artifactId>
      <version>2.8</version>
      <executions>
          <execution>
              <phase>generate-sources</phase>
              <goals>
                  <goal>build-classpath</goal>
              </goals>
              <configuration>
                <outputProperty>validate.modules.list</outputProperty>
                <pathSeparator>:</pathSeparator>
              </configuration>
          </execution>
      </executions>
  </plugin>

So even with 200+ or more dependencies - having this in some common Maven profile and activating where desired - should solve Java modules validation in consistent fashion.

[andrejusc]

@romain-grecourt - for this one:

We may need to add an exclusion around some oci-sdk dependency, if that's the case let's file a new issue for that.

If we agree on value of module-info.java presence for examples (and this one of them) - then I'll file such.

[andrejusc]

Attached is modified additional pom.xml with addition of modules profile.
pom.xml.txt

which demonstrates explained above and combined technique of dependencies validation as modules. You could invoke it like such:

mvn clean -Dmodules package

and once it's done - you could launch example in fully modularized way like such:

java -p target/helidon-examples-integrations-oci-objectstorage-cdi.jar:target/libs -m io.helidon.examples.integrations.oci.objectstorage.cdi/io.helidon.examples.integrations.oci.objectstorage.cdi.ObjectStorageCdiMain

and it should get to the point of this:

2022.09.18 22:20:17 INFO io.helidon.common.HelidonFeatures Thread[features-thread,5,main]: Helidon MP 2.5.3 features: [CDI, Config, Fault Tolerance, Health, JAX-RS, Metrics, Open API, Security, Server, Tracing]

But I could also reproduce your original issue in IDE even after resolving modules conflicts for CLI case. Which is weird.
Note - I could see that we have a lot of Automatic/unnamed modules in picture here - OCI SDK, HK2, WELD, maybe more. Not yet sure that IDE is capable to deal with all them properly in the mix with named Helidon modules. But I think it would be good try and resolve this use case for IDE as well. I'll continue more during week.

[arjav-desai]

We are going tackel module-info related work as part of #4872.

[andrejusc]

@arjav-desai - thank yo for update. Though we didn't reach here proper working state within IDE for example under discussion - -hope you could then introduce approach based on my proposed validate-modules technique into Helidon build/pipeline.