Add module-info.java to checker-qual #6326

mtf90 · 2023-11-30T16:27:49Z

This PR adds a module-info.java to the checker-qual project so that the checker-framework annotations can be directly used in projects that want to use the jlink or jpackage tools for distributing software. The current automatic modules (via MANIFEST.MF's Automatic-Module-Names) are not supported by jlink.

The corresponding issue is #4559.

mernst · 2024-01-16T20:53:02Z

@mtf90
./gradlew javadoc is failing with this error (and others):

framework/src/main/java/org/checkerframework/common/basetype/BaseTypeChecker.java:3: error: package com.google.common.collect is not visible
import com.google.common.collect.ImmutableSet;
                        ^
  (package com.google.common.collect is declared in module com.google.common, which is not in the module graph)

Could you please fix that? I would love to get this pull request merged. Thanks!

this is in accordance with the declaration order of JDK module definitions

make sure to compile all files with Java 9 first so that the compile may detect missing "requires" statements

mtf90 · 2024-01-17T09:47:56Z

Dear @mernst, I disabled the module path for the other javadoc processes as well which seemed to do the trick.

However, I am still not that satisfied with the PR.

The code for disabling module semantics is copied across various task configurations. Is there a global plugin/task/way to set this at a central place? Unfortunately, I'm not very well-versed with Gradle.
The module dependencies are mainly specified for resolving links in the javadocs (hence the static modifiers). Since the javadoc generation does not use module semantics, you may as well delete them. Especially if the other modules may at one point support named modules as well, you wouldn't even be able to specify the static dependency to org.checkerframework.checker because modules don't allow for cyclic dependencies. Currently, I cannot see how this can be implemented cleanly without a major refactoring. You probably have to live with the fact that most IDEs do not properly resolve the {@link}s now.
I don't know anything about Android specifics, so I have just disabled the module definition for this artifact completely.
The one good thing is that you probably do not need extra integration-tests. Since the compile tasks (hopefully) pick up the module definition, they should respect the specified exports declarations as well. So if you add a new annotation package and forget to export it as well, you should directly run into compilation errors when referencing it elsewhere.

mtf90 · 2024-01-17T13:22:56Z

I managed to clean up the configs some more, given that slowly I get the hang of sourceSets.

Also, the two pipelines seem to have failed due to a timeout, not an actual error (as all other pipelines have passed).

sgammon · 2024-03-06T04:56:38Z

@mtf90 / @mernst friendly bump? we ran into this in a downstream project. i would be happy to test if helpful

mtf90 · 2024-03-06T17:13:58Z

@sgammon, as mentioned in #6326 (comment), I don't see a clean way of implementing this without some larger refactoring (which is not my decision to make). So I can understand the reluctance of the maintainers.

As mentioned in #4559 (comment), there is a way to workaround this in downstream projects. For example, I managed to successfully implement it in LearnLib/learnlib#122 including a passing jlink integration test.

mernst · 2024-03-06T17:20:24Z

My apologies; I lost track of this pull request. I'm going to assign it for review and hopefully we can get it merged.

sgammon · 2024-03-06T18:02:11Z

@mernst No problem, and thank you guys for merging this!

(Tagging google/guava#2970 as well)

## Summary Minimal set of changes to ship a `module-info.java`, and support JVM 1.8 for the `annotations` module only. ### Reasoning It costs almost nothing to support _only_ the annotations on JVM 1.8, for projects which transitively include `error-prone-annotations`, which can happen via a simple dependency on something like Guava. But, it would be ideal to [ship a `module-info.java`](#2649), so `annotations` is now a `Multi-Release` JAR, with a `module-info.class` in `META-INF/versions/9`. I am working downstream to [support JPMS in Guava](google/guava#2970), and this PR will be necessary for that to eventually land (without hackery). [Checker Framework recently merged their support](typetools/checker-framework#6326) which means Error Prone is one of the last things on the classpath for Guava without a `module-info.java` definition. Automatic Modules also aren't the answer because they cannot be used with `jlink`. Such dependencies must be processed by tools like Moditect before they can be used in fully modular Java builds. Because Error Prone Annotations is a dependency in Guava, and is itself very popular, many projects need Error Prone to adopt JPMS so they can ship their own code with modular Java support. There may be libraries out there that depend on the annotations _and not the compiler_, who wish to ship a MRJAR and retain JVM 1.8 support (Guava). ### Non-release version number change One wrinkle here is that the Maven project version [is used unconditionally](https://github.com/apache/maven-compiler-plugin/blob/74cfc72acae4f55708bca189b2170167e83df6b3/src/main/java/org/apache/maven/plugin/compiler/CompilerMojo.java#L304-L305) [as the `--module-version`][0]; however, [`HEAD-SNAPSHOT` is not a valid module identifier](https://lists.apache.org/thread/qtghq13qgjoc4pn6ovsdxgnjnm4ozv4d). This leaves only a few choices to support JPMS through recent (`3.8.x+`) Maven Compiler plugin versions: - `HEAD-SNAPSHOT` needs to become `1.0-HEAD-SNAPSHOT`, and then it is a valid `--module-version` - Or, Maven Compiler Plugin needs to ship a bugfix and Error Prone will need to wait for a release (if a bugfix ships at all) I understand this can be a breaking change somewhere within Google, but I don't see a way around this without resorting to severe build hacks. I've gotten it to build anyway by building the `module-info.java` only in a separate Maven project, and then copying it into the JAR at the last moment, but there are serious issues with that route so I suggest it be abandoned in favor of a version string change during dev, if possible. ## Changelog - feat: add `module-info` to `annotations` module - feat: ship `annotations` as a `Multi-Release` JAR - feat: support `1.8` through latest JDK for `annotations` module - fix: remove automatic module definition from `annotations` module - fix: `HEAD-SNAPSHOT` → `1.0-HEAD-SNAPSHOT`, because of a Maven Compiler Plugin issue [precluding use as a module version][0] Fixes and closes #2649 [0]: https://issues.apache.org/jira/browse/MCOMPILER-579 Fixes #4311 FUTURE_COPYBARA_INTEGRATE_REVIEW=#4311 from sgammon:feat/jpms d209b0c PiperOrigin-RevId: 614005681

## Summary Minimal set of changes to ship a `module-info.java`, and support JVM 1.8 for the `annotations` module only. ### Reasoning It costs almost nothing to support _only_ the annotations on JVM 1.8, for projects which transitively include `error-prone-annotations`, which can happen via a simple dependency on something like Guava. But, it would be ideal to [ship a `module-info.java`](#2649), so `annotations` is now a `Multi-Release` JAR, with a `module-info.class` in `META-INF/versions/9`. I am working downstream to [support JPMS in Guava](google/guava#2970), and this PR will be necessary for that to eventually land (without hackery). [Checker Framework recently merged their support](typetools/checker-framework#6326) which means Error Prone is one of the last things on the classpath for Guava without a `module-info.java` definition. Automatic Modules also aren't the answer because they cannot be used with `jlink`. Such dependencies must be processed by tools like Moditect before they can be used in fully modular Java builds. Because Error Prone Annotations is a dependency in Guava, and is itself very popular, many projects need Error Prone to adopt JPMS so they can ship their own code with modular Java support. There may be libraries out there that depend on the annotations _and not the compiler_, who wish to ship a MRJAR and retain JVM 1.8 support (Guava). ### Non-release version number change One wrinkle here is that the Maven project version [is used unconditionally](https://github.com/apache/maven-compiler-plugin/blob/74cfc72acae4f55708bca189b2170167e83df6b3/src/main/java/org/apache/maven/plugin/compiler/CompilerMojo.java#L304-L305) [as the `--module-version`][0]; however, [`HEAD-SNAPSHOT` is not a valid module identifier](https://lists.apache.org/thread/qtghq13qgjoc4pn6ovsdxgnjnm4ozv4d). This leaves only a few choices to support JPMS through recent (`3.8.x+`) Maven Compiler plugin versions: - `HEAD-SNAPSHOT` needs to become `1.0-HEAD-SNAPSHOT`, and then it is a valid `--module-version` - Or, Maven Compiler Plugin needs to ship a bugfix and Error Prone will need to wait for a release (if a bugfix ships at all) I understand this can be a breaking change somewhere within Google, but I don't see a way around this without resorting to severe build hacks. I've gotten it to build anyway by building the `module-info.java` only in a separate Maven project, and then copying it into the JAR at the last moment, but there are serious issues with that route so I suggest it be abandoned in favor of a version string change during dev, if possible. ## Changelog - feat: add `module-info` to `annotations` module - feat: ship `annotations` as a `Multi-Release` JAR - feat: support `1.8` through latest JDK for `annotations` module - fix: remove automatic module definition from `annotations` module - fix: `HEAD-SNAPSHOT` → `1.0-HEAD-SNAPSHOT`, because of a Maven Compiler Plugin issue [precluding use as a module version][0] Fixes and closes #2649 [0]: https://issues.apache.org/jira/browse/MCOMPILER-579 Fixes #4311 COPYBARA_INTEGRATE_REVIEW=#4311 from sgammon:feat/jpms d209b0c PiperOrigin-RevId: 614026439

desruisseaux · 2024-03-27T21:26:56Z

checker-qual/src/main/java/module-info.java

+ */
+module org.checkerframework.checker.qual {
+  // javadoc-only dependencies
+  requires static java.compiler;


If this is really for Javadoc only, wouldn't it be cleaner to remove those requires static statements and replace them by --add-reads options when invoking the javadoc tool?

Technically, you can even skip the --add-reads because the javadoc phase does not read the module descriptor (due to circular dependencies which are not allowed with JPMS).

I mainly added them to be as explicit as possible with the dependency declarations. For example, IntelliJ would flag even more references if these dependencies were not specified. I don't know how well IDEs are able to include phase-specific build-flags.

Do you have any issues with these declarations? As far as I know, requires static should not affect dependent modules.

No issue that I'm aware of for now. I thought that those declarations are a risk, because the code could accidentally use those dependencies without warning from the compiler. The argument about being explicit can also be turned the other way around, i.e. they are not describing the reality if the code has no dependency to those modules. Anyway, this is not a major issue.

add module-info.java to checker-qual

52ee5a7

mtf90 mentioned this pull request Nov 30, 2023

Add module descriptor #4559

Closed

mernst added 2 commits January 16, 2024 12:18

Merge branch 'master' into module-info

b48b340

Update package name

24fb8b6

mtf90 added 7 commits January 17, 2024 00:04

disable module-path in other javadoc invocations as well

5888b5d

reorder module-info statements

9698354

this is in accordance with the declaration order of JDK module definitions

build all files with Java 9

308ae37

make sure to compile all files with Java 9 first so that the compile may detect missing "requires" statements

reorder items

802539a

further exclusions

0968dee

use previous exclusion semantics again

21e8632

update contributors

97e77f2

mtf90 added 2 commits January 17, 2024 11:05

actually include module-info in java9 compilation

bf0ed60

drop redundant configs

2dfb910

Merge branch 'master' into module-info

70e7253

mtf90 marked this pull request as ready for review January 17, 2024 20:57

mernst assigned smillst Mar 6, 2024

smillst approved these changes Mar 6, 2024

View reviewed changes

smillst merged commit 2776c6d into typetools:master Mar 6, 2024
29 checks passed

This was referenced Mar 6, 2024

feat: add jpms definition for annotations google/error-prone#4311

Closed

Tracking: JPMS Ecosystem elide-dev/jpms#1

Open

copybara-service bot mentioned this pull request Mar 8, 2024

feat: add jpms definition for annotations google/error-prone#4313

Closed

This was referenced Mar 10, 2024

Guava + JPMS google/guava#7094

Draft

Add module-info.java google/guava#2970

Open

bowbahdoe mentioned this pull request Mar 25, 2024

There is no module-info.java pgjdbc/pgjdbc#2657

Open

desruisseaux reviewed Mar 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add module-info.java to checker-qual #6326

Add module-info.java to checker-qual #6326

mtf90 commented Nov 30, 2023

mernst commented Jan 16, 2024

mtf90 commented Jan 17, 2024

mtf90 commented Jan 17, 2024 •

edited

Loading

sgammon commented Mar 6, 2024

mtf90 commented Mar 6, 2024

mernst commented Mar 6, 2024

sgammon commented Mar 6, 2024 •

edited

Loading

desruisseaux Mar 27, 2024

mtf90 Mar 28, 2024

desruisseaux Mar 30, 2024

Add module-info.java to checker-qual #6326

Add module-info.java to checker-qual #6326

Conversation

mtf90 commented Nov 30, 2023

mernst commented Jan 16, 2024

mtf90 commented Jan 17, 2024

mtf90 commented Jan 17, 2024 • edited Loading

sgammon commented Mar 6, 2024

mtf90 commented Mar 6, 2024

mernst commented Mar 6, 2024

sgammon commented Mar 6, 2024 • edited Loading

desruisseaux Mar 27, 2024

Choose a reason for hiding this comment

mtf90 Mar 28, 2024

Choose a reason for hiding this comment

desruisseaux Mar 30, 2024

Choose a reason for hiding this comment

mtf90 commented Jan 17, 2024 •

edited

Loading

sgammon commented Mar 6, 2024 •

edited

Loading