Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ConcurrentJarCreator.createEntry leaks file descriptors #79

Closed
piotrp opened this issue Apr 5, 2018 · 3 comments
Closed

ConcurrentJarCreator.createEntry leaks file descriptors #79

piotrp opened this issue Apr 5, 2018 · 3 comments
Labels
bug
Milestone
3.6.0

Comments

@piotrp
Copy link

piotrp commented Apr 5, 2018 •
edited
Loading

On Windows a plugin running in package phase (reproducible-build-maven-plugin) fails because JAR in target has on open file descriptor, opened via:

	at java.util.zip.ZipFile.<init>(ZipFile.java:156)
	at java.util.jar.JarFile.<init>(JarFile.java:166)
	at java.util.jar.JarFile.<init>(JarFile.java:103)
	at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:930)
	at sun.misc.URLClassPath$JarLoader.access$800(URLClassPath.java:791)
	at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:876)
	at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:869)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:868)
	at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:841)
	at sun.misc.URLClassPath$3.run(URLClassPath.java:565)
	at sun.misc.URLClassPath$3.run(URLClassPath.java:555)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.misc.URLClassPath.getLoader(URLClassPath.java:554)
	at sun.misc.URLClassPath.getLoader(URLClassPath.java:519)
	at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:484)
	at sun.misc.URLClassPath.findResource(URLClassPath.java:214)
	at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
	at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
	at org.codehaus.plexus.archiver.zip.PlexusIoZipFileResourceCollection$1.getResource(PlexusIoZipFileResourceCollection.java:81)
	at org.codehaus.plexus.archiver.zip.PlexusIoZipFileResourceCollection$ZipFileResourceIterator$ZipFileResource.getURL(PlexusIoZipFileResourceCollection.java:122)
	at org.codehaus.plexus.components.io.resources.PlexusIoURLResource.getContents(PlexusIoURLResource.java:38)
	at org.codehaus.plexus.components.io.resources.Deferred.getContents(Deferred.java:60)
	at org.codehaus.plexus.components.io.resources.proxy.ResourceInvocationHandler.invoke(ResourceInvocationHandler.java:66)
	at com.sun.proxy.$Proxy24.getContents(Unknown Source)
	at org.codehaus.plexus.archiver.ArchiveEntry.getInputStream(ArchiveEntry.java:137)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver$1.get(AbstractZipArchiver.java:550)
	at org.codehaus.plexus.archiver.zip.ConcurrentJarCreator.createEntry(ConcurrentJarCreator.java:276)
	at org.codehaus.plexus.archiver.zip.ConcurrentJarCreator.addArchiveEntry(ConcurrentJarCreator.java:196)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.zipFile(AbstractZipArchiver.java:512)
	at org.codehaus.plexus.archiver.jar.JarArchiver.zipFile(JarArchiver.java:501)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.zipFile(AbstractZipArchiver.java:561)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.addResources(AbstractZipArchiver.java:421)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.createArchiveMain(AbstractZipArchiver.java:331)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.execute(AbstractZipArchiver.java:234)
	at org.codehaus.plexus.archiver.AbstractArchiver.createArchive(AbstractArchiver.java:987)
	at org.apache.maven.plugins.assembly.archive.archiver.AssemblyProxyArchiver.createArchive(AssemblyProxyArchiver.java:445)
	at org.apache.maven.plugins.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:184)
	at org.apache.maven.plugins.assembly.mojos.AbstractAssemblyMojo.execute(AbstractAssemblyMojo.java:478)
	at org.apache.maven.plugins.assembly.mojos.SingleAssemblyMojo.execute(SingleAssemblyMojo.java:61)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:290)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:194)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

I started looking for leaks and found that stream opened in ConcurrentJarCreator.createEntry is closed only when an exception occurs, but not after successful execution.
@plamentotev
Copy link
Member

Hi,

Thank you for reporting this.

Could you please provide a project that reproduces the issue.

While I don't exclude the possibility of leaks, createEntry does not close the stream on normal completion because the stream is used by the caller and it is the caller responsibility to make sure it is closed.

@piotrp
Copy link
Author

piotrp commented Apr 6, 2018

After looking at it a bit more it may be an issue with how maven-assembly-plugin uses plexus-archiver to create fat JARs. I managed to reproduce it with an empty project: https://gist.github.com/Crack/01cbbc4561f75616a63242f30677bf54
To fail it must be built on Windows.

To find the stacktrace I provided above:

  1. Download http://search.maven.org/remotecontent?filepath=org/kohsuke/file-leak-detector/1.11/file-leak-detector-1.11-jar-with-dependencies.jar , save as fld.jar next to pom.xml from my project.
  2. Download https://github.com/Zlika/reproducible-build-maven-plugin/archive/v0.5.2.tar.gz , introduce sleep on exception to let maven linger after failure by changing DefaultZipStripper.strip to:
    @Override
    public void strip(final File zip, final File stripped) throws IOException
    {
        this.stripper.strip(zip, stripped);
        if (this.overwrite)
        {
            try {
                Files.move(stripped.toPath(), zip.toPath(), StandardCopyOption.REPLACE_EXISTING);
            } catch (FileSystemException e) {
                System.out.println("CATCH");
                try {Thread.sleep(1000 * 60 * 5);} catch (InterruptedException e1) {}
            }
        }
    }

and install it locally.
3. Build my project:

export MAVEN_OPTS=-javaagent:fld.jar=http=19999
mvn install
  1. Go to http://localhost:19999 to get information on all open files

@plamentotev
Copy link
Member

Thanks for the project and the detailed explanation.

I've found two leaks:

I'll prepare fix tomorrow.

@plamentotev plamentotev added the bug label Apr 6, 2018
@plamentotev plamentotev added this to the 3.6 milestone Apr 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
3.6.0
Development

No branches or pull requests
2 participants
@piotrp @plamentotev