Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make shadowJar task cacheable #524

Merged
merged 5 commits into from
Nov 10, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-5.1.1-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-5.6.2-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import org.gradle.api.artifacts.Dependency
import org.gradle.api.artifacts.ProjectDependency
import org.gradle.api.artifacts.SelfResolvingDependency
import org.gradle.api.file.FileCollection
import org.gradle.api.tasks.InputFiles
import org.gradle.api.tasks.SourceSet
import org.vafer.jdependency.Clazz
import org.vafer.jdependency.Clazzpath
Expand Down Expand Up @@ -50,6 +51,11 @@ class UnusedTracker {
return new UnusedTracker(classDirs, apiJars, toMinimize)
}

@InputFiles
FileCollection getToMinimize() {
return toMinimize
}

private static boolean isProjectDependencyFile(File file, Dependency dep) {
def fileName = file.name
def dependencyName = dep.name
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package com.github.jengelman.gradle.plugins.shadow.relocation

import java.lang.annotation.ElementType
import java.lang.annotation.Retention
import java.lang.annotation.RetentionPolicy
import java.lang.annotation.Target

/**
* Marks that a given instance of {@link Relocator} is is compatible with the Gradle build cache.
* In other words, it has its appropriate inputs annotated so that Gradle can consider them when
* determining the cache key.
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
@interface CacheableRelocator {

}
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
package com.github.jengelman.gradle.plugins.shadow.relocation

import org.codehaus.plexus.util.SelectorUtils
import org.gradle.api.tasks.Input

import java.util.regex.Pattern

Expand All @@ -30,6 +31,7 @@ import java.util.regex.Pattern
* @author Mauro Talevi
* @author John Engelman
*/
@CacheableRelocator
class SimpleRelocator implements Relocator {

private final String pattern
Expand All @@ -43,7 +45,7 @@ class SimpleRelocator implements Relocator {
private final Set<String> includes

private final Set<String> excludes

private final boolean rawString

SimpleRelocator() {
Expand Down Expand Up @@ -188,4 +190,39 @@ class SimpleRelocator implements Relocator {
return sourceContent.replaceAll("\\b" + pattern, shadedPattern)
}
}

@Input
String getPattern() {
return pattern
}

@Input
String getPathPattern() {
return pathPattern
}

@Input
String getShadedPattern() {
return shadedPattern
}

@Input
String getShadedPathPattern() {
return shadedPathPattern
}
Comment on lines +194 to +212
Copy link
Member

@Goooler Goooler Dec 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ghale

These getters are immutable and unused, can we remove them? I'm planning to remove them and migrate includes and excludes to org.gradle.api.provider.SetProperty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, if these are being removed in favor of property objects, then yes, these can be removed. They were only added in order to expose the inputs from the relocator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, the new property objects should also be exposed as inputs (i.e. a getter with an @Input annotation) in order to maintain proper cacheability after these are removed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you take a review for #1079?


@Input
Set<String> getIncludes() {
return includes
}

@Input
Set<String> getExcludes() {
return excludes
}

@Input
boolean getRawString() {
return rawString
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,27 @@

import com.github.jengelman.gradle.plugins.shadow.ShadowStats;
import com.github.jengelman.gradle.plugins.shadow.internal.*;
import com.github.jengelman.gradle.plugins.shadow.relocation.CacheableRelocator;
import com.github.jengelman.gradle.plugins.shadow.relocation.Relocator;
import com.github.jengelman.gradle.plugins.shadow.relocation.SimpleRelocator;
import com.github.jengelman.gradle.plugins.shadow.transformers.AppendingTransformer;
import com.github.jengelman.gradle.plugins.shadow.transformers.GroovyExtensionModuleTransformer;
import com.github.jengelman.gradle.plugins.shadow.transformers.ServiceFileTransformer;
import com.github.jengelman.gradle.plugins.shadow.transformers.Transformer;
import com.github.jengelman.gradle.plugins.shadow.transformers.*;
import org.gradle.api.Action;
import org.gradle.api.Task;
import org.gradle.api.artifacts.Configuration;
import org.gradle.api.file.FileCollection;
import org.gradle.api.internal.DocumentationRegistry;
import org.gradle.api.internal.file.FileResolver;
import org.gradle.api.internal.file.copy.CopyAction;
import org.gradle.api.tasks.InputFiles;
import org.gradle.api.tasks.Internal;
import org.gradle.api.tasks.Optional;
import org.gradle.api.tasks.TaskAction;
import org.gradle.api.specs.Spec;
import org.gradle.api.tasks.*;
import org.gradle.api.tasks.bundling.Jar;
import org.gradle.api.tasks.util.PatternSet;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;

@CacheableTask
public class ShadowJar extends Jar implements ShadowSpec {

private List<Transformer> transformers;
Expand All @@ -46,6 +44,29 @@ public ShadowJar() {
transformers = new ArrayList<>();
relocators = new ArrayList<>();
configurations = new ArrayList<>();

this.getInputs().property("minimize", new Callable<Boolean>() {
@Override
public Boolean call() throws Exception {
return minimizeJar;
}
});
this.getOutputs().doNotCacheIf("Has one or more transforms or relocators that are not cacheable", new Spec<Task>() {
@Override
public boolean isSatisfiedBy(Task task) {
for (Transformer transformer : transformers) {
if (!isCacheableTransform(transformer.getClass())) {
return true;
}
}
for (Relocator relocator : relocators) {
if (!isCacheableRelocator(relocator.getClass())) {
return true;
}
}
return false;
}
});
}

public ShadowJar minimize() {
Expand Down Expand Up @@ -93,7 +114,7 @@ protected void copy() {
getLogger().info(shadowStats.toString());
}

@InputFiles
@Classpath
public FileCollection getIncludedDependencies() {
return getProject().files(new Callable<FileCollection>() {

Expand Down Expand Up @@ -146,24 +167,33 @@ public ShadowJar transform(Class<? extends Transformer> clazz) throws Instantiat
*/
public <T extends Transformer> ShadowJar transform(Class<T> clazz, Action<T> c) throws InstantiationException, IllegalAccessException {
T transformer = clazz.newInstance();
if (c != null) {
c.execute(transformer);
}
transformers.add(transformer);
addTransform(transformer, c);
return this;
}

private boolean isCacheableTransform(Class<? extends Transformer> clazz) {
return clazz.isAnnotationPresent(CacheableTransformer.class);
}

/**
* Add a preconfigured transformer instance.
*
* @param transformer the transformer instance to add
* @return this
*/
public ShadowJar transform(Transformer transformer) {
transformers.add(transformer);
addTransform(transformer, null);
return this;
}

private <T extends Transformer> void addTransform(T transformer, Action<T> c) {
if (c != null) {
c.execute(transformer);
}

transformers.add(transformer);
}

/**
* Syntactic sugar for merging service files in JARs.
*
Expand Down Expand Up @@ -266,10 +296,7 @@ public ShadowJar relocate(String pattern, String destination) {
*/
public ShadowJar relocate(String pattern, String destination, Action<SimpleRelocator> configure) {
SimpleRelocator relocator = new SimpleRelocator(pattern, destination, new ArrayList<String>(), new ArrayList<String>());
if (configure != null) {
configure.execute(relocator);
}
relocators.add(relocator);
addRelocator(relocator, configure);
return this;
}

Expand All @@ -280,7 +307,7 @@ public ShadowJar relocate(String pattern, String destination, Action<SimpleReloc
* @return this
*/
public ShadowJar relocate(Relocator relocator) {
relocators.add(relocator);
addRelocator(relocator, null);
return this;
}

Expand All @@ -294,6 +321,14 @@ public ShadowJar relocate(Class<? extends Relocator> relocatorClass) throws Inst
return relocate(relocatorClass, null);
}

private <R extends Relocator> void addRelocator(R relocator, Action<R> configure) {
if (configure != null) {
configure.execute(relocator);
}

relocators.add(relocator);
}

/**
* Add a relocator of the provided class and configure.
*
Expand All @@ -303,14 +338,15 @@ public ShadowJar relocate(Class<? extends Relocator> relocatorClass) throws Inst
*/
public <R extends Relocator> ShadowJar relocate(Class<R> relocatorClass, Action<R> configure) throws InstantiationException, IllegalAccessException {
R relocator = relocatorClass.newInstance();
if (configure != null) {
configure.execute(relocator);
}
relocators.add(relocator);
addRelocator(relocator, configure);
return this;
}

@Internal
private boolean isCacheableRelocator(Class<? extends Relocator> relocatorClass) {
return relocatorClass.isAnnotationPresent(CacheableRelocator.class);
}

@Nested
public List<Transformer> getTransformers() {
return this.transformers;
}
Expand All @@ -319,7 +355,7 @@ public void setTransformers(List<Transformer> transformers) {
this.transformers = transformers;
}

@Internal
@Nested
public List<Relocator> getRelocators() {
return this.relocators;
}
Expand All @@ -328,7 +364,7 @@ public void setRelocators(List<Relocator> relocators) {
this.relocators = relocators;
}

@InputFiles @Optional
@Classpath @Optional
public List<Configuration> getConfigurations() {
return this.configurations;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import org.apache.tools.zip.ZipEntry
import org.apache.tools.zip.ZipOutputStream
import org.codehaus.plexus.util.IOUtil
import org.gradle.api.file.FileTreeElement
import org.gradle.api.tasks.Input

/**
* A resource processor that appends content for a resource, separated by a newline.
Expand All @@ -32,6 +33,7 @@ import org.gradle.api.file.FileTreeElement
* Modifications
* @author John Engelman
*/
@CacheableTransformer
class AppendingTransformer implements Transformer {
String resource

Expand Down Expand Up @@ -65,4 +67,9 @@ class AppendingTransformer implements Transformer {
IOUtil.copy(new ByteArrayInputStream(data.toByteArray()), os)
data.reset()
}

@Input
String getResource() {
return resource
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package com.github.jengelman.gradle.plugins.shadow.transformers

import java.lang.annotation.ElementType
import java.lang.annotation.Retention
import java.lang.annotation.RetentionPolicy
import java.lang.annotation.Target

/**
* Marks that a given instance of {@link Transformer} is is compatible with the Gradle build cache.
* In other words, it has its appropriate inputs annotated so that Gradle can consider them when
* determining the cache key.
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
@interface CacheableTransformer {

}
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import org.codehaus.plexus.util.IOUtil
* entries will all be merged into a single META-INF/services/org.codehaus.groovy.runtime.ExtensionModule resource
* packaged into the resultant JAR produced by the shadowing process.
*/
@CacheableTransformer
class GroovyExtensionModuleTransformer implements Transformer {

private static final GROOVY_EXTENSION_MODULE_DESCRIPTOR_PATH =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import org.apache.tools.zip.ZipEntry
import org.apache.tools.zip.ZipOutputStream
import org.gradle.api.file.FileTreeElement
import org.gradle.api.specs.Spec
import org.gradle.api.tasks.Input
import org.gradle.api.tasks.util.PatternFilterable
import org.gradle.api.tasks.util.PatternSet
import org.codehaus.plexus.util.IOUtil
Expand All @@ -42,6 +43,7 @@ import org.codehaus.plexus.util.IOUtil
* @author Charlie Knudsen
* @author John Engelman
*/
@CacheableTransformer
class ServiceFileTransformer implements Transformer, PatternFilterable {

private static final String SERVICES_PATTERN = "META-INF/services/**"
Expand Down Expand Up @@ -193,6 +195,7 @@ class ServiceFileTransformer implements Transformer, PatternFilterable {
* {@inheritDoc}
*/
@Override
@Input
Set<String> getIncludes() {
return patternSet.includes
}
Expand All @@ -210,6 +213,7 @@ class ServiceFileTransformer implements Transformer, PatternFilterable {
* {@inheritDoc}
*/
@Override
@Input
Set<String> getExcludes() {
return patternSet.excludes
}
Expand All @@ -222,5 +226,4 @@ class ServiceFileTransformer implements Transformer, PatternFilterable {
patternSet.excludes = excludes
return this
}

}
Loading