Improve cache effectives for inputs to ShadowJar task

[chali]

ShadowJar has currently two important inputs. https://github.com/johnrengelman/shadow/blob/master/src/main/groovy/com/github/jengelman/gradle/plugins/shadow/tasks/ShadowJar.java#L333-L336 and https://github.com/johnrengelman/shadow/blob/master/src/main/groovy/com/github/jengelman/gradle/plugins/shadow/tasks/ShadowJar.java#L98-L107

The getIncludedDependencies is actually derived from the getConfigurations The derivation process is the application of filters. I'm working on a use case described in #443. I have a multi-module project. One module produces two shadow jar. One is including classes from all submodules the other is including dependencies. In a configuration of shadow jar, I'm doing to achieve that result:

shadowJar {
    configurations = [configurations.runtime]
    dependencies {
        exclude(project(':common-submodule'))
    }
}

There is a lot of dependencies and the creation of dependencies jar should not be done unless it is necessary. Unfortunately, when I change the code in common-submodule it generates a new jar which leads to modification of configurations.runtime and because getConfigurations is marked as @InputFiles it invalidates task cache ant retriggers the execution. It could be avoided because the changed jar is filtered out from the configuration and final result of getIncludedDependencies would be unchanged.

I would like to propose marking getConfigurations as @Internal leaving only the final result of filtering as cached. It would help avoid cache invalidation. I talked to Gradle team to double check that I'm not doing anything unexpected from Gradle point of view and I was told this is quite a common pattern even in Gradle itself.

[chali]

I'm looking into the failed test. Looks like removing @InputFiles caused that gradle is not building all submodule jars. There is a proper path return from configuration resolution but there is no jar created.

> Task :lib:compileJava
> Task :lib:processResources NO-SOURCE
> Task :lib:classes
> Task :lib:jar
> Task :api:compileJava
> Task :api:processResources NO-SOURCE
> Task :api:classes
> Task :api:jar
> Task :impl:compileJava
> Task :impl:processResources NO-SOURCE
> Task :impl:classes
> Task :impl:shadowJar

vs

> Task :lib:compileJava
> Task :lib:processResources NO-SOURCE
> Task :lib:classes
> Task :lib:jar
> Task :api:compileJava
> Task :impl:compileJava
> Task :impl:processResources NO-SOURCE
> Task :impl:classes
> Task :impl:shadowJar FAILED

I will try to explore what could be done to go unblock this PR.

[chali]

I asked Gradle folks and it looks like Shadow plugin needs to update its expectations about Jar being built or add explicit task dependencies.

java-library plugin will not build jar for submodule dependencies in a multimodule project. That is different from java plugin.

nebula-plugins/gradle-nebula-integration#59

[johnrengelman]

Caused by: java.lang.IllegalArgumentException: neither file nor directory
	at org.vafer.jdependency.Clazzpath.addClazzpathUnit(Clazzpath.java:122)
	at org.vafer.jdependency.Clazzpath.addClazzpathUnit(Clazzpath.java:95)
	at org.vafer.jdependency.Clazzpath.addClazzpathUnit(Clazzpath.java:86)
	at org.vafer.jdependency.Clazzpath$addClazzpathUnit.call(Unknown Source)
	at com.github.jengelman.gradle.plugins.shadow.internal.UnusedTracker$_closure2.doCall(UnusedTracker.groovy:23)
	at com.github.jengelman.gradle.plugins.shadow.internal.UnusedTracker.<init>(UnusedTracker.groovy:23)
	at com.github.jengelman.gradle.plugins.shadow.internal.UnusedTracker.forProject(UnusedTracker.groovy:50)
	at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar.createCopyAction(ShadowJar.java:78)
	at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:178)
	at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar.copy(ShadowJar.java:92)

[johnrengelman]

Superseded by #524