Maven Project randomly fails with "Failed to open file channel '[..]\.m2\repository\.locks\[..].resolverlock" #728

fsmherrm · 2022-10-30T09:47:21Z

Hello,

we have a quite huge maven multi project with approx. 150 modules. During project build with "mvnd clean install" it randomly fails with for example followings error (just as an example, as specific error dependency randomly changes):

Failed to open file channel for '[..]\.m2\repository\.locks\commons-codec~commons-codec~1.15.resolverlock'

Unfortunately, I am not able to directly reliablely reproduce it but from my understanding this should not happen as Maven Daemon is designed to support concurrent M2 repository access.

What could be the reason? Is there any configuration that I could adjust to solve this?

Thank you and best regards
Marc

The text was updated successfully, but these errors were encountered:

gnodet · 2022-10-30T14:10:13Z

@cstamas fyi

fsmherrm · 2022-10-30T16:12:22Z

Here is the full exception stack log:

Caused by: java.io.UncheckedIOException: Failed to open file channel for '[..]\.m2\repository\.locks\commons-codec~commons-codec~1.15.resolverlock'
	at org.eclipse.aether.named.providers.FileLockNamedLockFactory.lambda$createLock$0(FileLockNamedLockFactory.java:78)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
	at org.eclipse.aether.named.providers.FileLockNamedLockFactory.createLock(FileLockNamedLockFactory.java:65)
	at org.eclipse.aether.named.support.NamedLockFactorySupport.lambda$getLock$0(NamedLockFactorySupport.java:53)
	at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1908)
	at org.eclipse.aether.named.support.NamedLockFactorySupport.getLock(NamedLockFactorySupport.java:49)
	at org.eclipse.aether.named.support.NamedLockFactorySupport.getLock(NamedLockFactorySupport.java:34)
	at org.eclipse.aether.internal.impl.synccontext.named.NamedLockFactoryAdapter$AdaptedLockSyncContext.acquire(NamedLockFactoryAdapter.java:141)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:233)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:357)
	at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:207)
	at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:243)
	at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:147)
	at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:401)
	at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:347)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.mvndaemon.mvnd.builder.SmartBuilderImpl.buildProject(SmartBuilderImpl.java:178)
	... 6 common frames omitted
Caused by: java.nio.file.AccessDeniedException: [..]\.m2\repository\.locks\commons-codec~commons-codec~1.15.resolverlock
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:120)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
	at org.eclipse.aether.named.providers.FileLockNamedLockFactory.lambda$createLock$0(FileLockNamedLockFactory.java:70)
	... 25 common frames omitted

cstamas · 2022-11-01T11:44:17Z

What version of mvnd, what version/provider of Java, what OS and what FS?
As I see from stack, some sort of windows, any anti virus software?

cstamas · 2022-11-01T12:19:30Z

Without any information, am guessing something along these lines (as very same thing is done by resolver locking as well): https://bugs.openjdk.org/browse/JDK-8252883 Would be good to get information about Java version here.

fsmherrm · 2022-11-01T12:28:46Z

PS C:\Users\[...]> mvnd --version
mvnd 0.8.2 windows-amd64 native client (2bba2d6a4d3a5012ddf9f1f42a22784cad4011e3)
Terminal: org.jline.terminal.impl.jansi.win.JansiWinSysTerminal
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: C:\Tools\maven-mvnd-0.8.2-windows-amd64\mvn
Java version: 11.0.11, vendor: AdoptOpenJDK, runtime: C:\Tools\jdk-11.0.11+9
Default locale: de_DE, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

I will try to use an updated JDK version and let you know if the issue still occurs.

fsmherrm · 2022-11-01T13:59:32Z

The issue still occurs with Java version: 11.0.17, vendor: Eclipse Adoptium

cstamas · 2022-11-01T16:11:03Z

Ok, mvnd 0.8.2 uses resolver 1.7.3... @fsmherrm If I prep a patched JAR (of affected resolver bit), are you willing (or can you) to try it out?

michael-o · 2022-11-01T16:13:29Z

I can't stop laughing about crappy file locking on Windows. The "fix" done in the JDK is pointless because it addresses only the symptom in the JUL FileHandler, so upgrading won't help. I assume is that we need to apply the same hack either with a sleep or a retry count.

cstamas · 2022-11-01T16:14:29Z

I have something else on my mind (than the fix in Java), and as @fsmherrm reproduces the issue consistently, am eager to try out my assumption....

cstamas · 2022-11-01T16:17:31Z

The issue still occurs with Java version: 11.0.17, vendor: Eclipse Adoptium

You mean 17 Adoptiun? Or 11 Adoptium? Or?

michael-o · 2022-11-01T16:26:30Z

Similar issue: https://bugs.openjdk.org/browse/JDK-8150344

cstamas · 2022-11-01T16:26:56Z

This is the patch: https://github.com/apache/maven-resolver/pull/211/files
Patched JAR is in my drive: https://drive.google.com/drive/folders/1A1xx5CszdplDdNeURrKDCBCku0Z2Wgxp?usp=share_link

ORIGINAL JAR SHA1:
71bb6016d8d415836187b791a9df3bf5cb401c44 maven-resolver-named-locks-1.7.3.jar

PATCHED JAR SHA1:
7eea1f1d27abbf8bab68b95c315af7dd5076180f maven-resolver-named-locks-1.7.3.jar

To apply it:

overwrite/replace the original mvnd/mvn/lib/maven-resolver-named-locks-1.7.3.jar with patched jar.

michael-o · 2022-11-01T16:28:13Z

https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilea?redirectedfrom=MSDN

Could it be that our handling is incorrect?

fsmherrm · 2022-11-02T09:13:59Z

@cstamas: Are you able to upload the patched jar somewhere else e.g. onedrive?

cstamas · 2022-11-02T09:16:56Z

@cstamas: Are you able to upload the patched jar somewhere else e.g. onedrive?

Is in Google Drive, or is that not reachable to you? There is a link to it above....

fsmherrm · 2022-11-02T09:28:26Z

@cstamas: yes, sorry. Can you provide it via github?

cstamas · 2022-11-02T09:30:54Z

Sadly no, GH does not allow JAR uploads (was my initial intent and was suprised to see that)...

michael-o · 2022-11-02T09:31:28Z

You can rename to .txt and upload.

cstamas · 2022-11-02T09:33:15Z

maven-resolver-named-locks-1.7.3.jar.zip

Just remove the .zip suffix from file name. Don't forget to check sha1sum as well....

cstamas · 2022-11-02T12:48:22Z

@fsmherrm any results?

fsmherrm · 2022-11-02T15:23:39Z

@cstamas: It seems to work now :) but I need to run it multiple times to be sure as I am not able to always reproduce it before.

But as a result, the folder .locks is now populated with more than 2000 .resolverlock files, is this intended?

cstamas · 2022-11-02T15:41:44Z

Sadly, this is the "price" to make it work on Windows that has very quirky locking semantics, see the patch apache/maven-resolver#211

cstamas · 2022-11-02T15:51:24Z

If you are willing to test more, i can prepare a similar patch like the "symptom patch" is in java...

Also, let's keep the discussion here.

fsmherrm · 2022-11-02T16:00:24Z

@cstamas : I am willing to test more :)

cstamas · 2022-11-02T16:37:20Z

Roger that, later evening, or tomorrow will prep alternative patch.

cstamas · 2022-11-02T17:22:38Z

2nd experiment: do what Java folks did: retry and hope the best:
apache/maven-resolver#214

Patch JAR:
maven-resolver-named-locks-1.7.3.jar.zip

SHA1: 5add45f97a6e37053993652a25d618e134d0c5c7

"Tunables":
-Dresolver.file-lock.attempts=5 to set retry attempts, default 5 attempts
-Dresolver.file-lock.sleepMillis=50 to set sleep time between attempts, default 50ms

cstamas · 2022-11-04T08:07:58Z

@fsmherrm any results? Sorry for "pressing", but am akin to release resolver 1.9.0 soon, and would like to incorporate these 'tweaks" (probably configurable by some flags) for Windows users..... (or ultimate fix, if it works)

fsmherrm · 2022-11-04T08:16:49Z

Hi @cstamas, sorry for delay, but for now it seems to work fine! Thank you very much for your instant support and fix.

@cstamas : When will you release a new version of mvnd that will include your fix?

cstamas · 2022-11-04T09:06:57Z

Cool, thanks @fsmherrm for testing this!

Sadly, the order is release resolver 1.9.0 (soon) -> release maven 3.9.0 (that would use fixed resolver) -> release mvnd (that would use released maven that use fixed resolver)....

Stay tuned, will figure out something with @gnodet for sure.

The final (to be in 1.9.0) resolver change is here apache/maven-resolver#216

gnodet · 2023-01-06T16:02:44Z

Fixed in https://github.com/apache/maven-mvnd/milestone/31

cstamas mentioned this issue Nov 1, 2022

[EXPERIMENT] Drop DELETE_ON_CLOSE flag apache/maven-resolver#211

Closed

cstamas mentioned this issue Nov 2, 2022

[EXPERIMENT] Do what Java does: retry and hope the best apache/maven-resolver#214

Closed

cstamas mentioned this issue Nov 4, 2022

[MRESOLVER-285] File locking tweaks for Windows OS apache/maven-resolver#216

Merged

gnodet added this to the 1.0.0-m1 milestone Jan 6, 2023

gnodet closed this as completed Jan 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Maven Project randomly fails with "Failed to open file channel '[..]\.m2\repository\.locks\[..].resolverlock" #728

Maven Project randomly fails with "Failed to open file channel '[..]\.m2\repository\.locks\[..].resolverlock" #728

fsmherrm commented Oct 30, 2022

gnodet commented Oct 30, 2022

fsmherrm commented Oct 30, 2022

cstamas commented Nov 1, 2022 •

edited

Loading

cstamas commented Nov 1, 2022 •

edited

Loading

fsmherrm commented Nov 1, 2022

fsmherrm commented Nov 1, 2022

cstamas commented Nov 1, 2022

michael-o commented Nov 1, 2022

cstamas commented Nov 1, 2022 •

edited

Loading

cstamas commented Nov 1, 2022

michael-o commented Nov 1, 2022

cstamas commented Nov 1, 2022 •

edited

Loading

michael-o commented Nov 1, 2022

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022 •

edited

Loading

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022

michael-o commented Nov 2, 2022

cstamas commented Nov 2, 2022 •

edited

Loading

cstamas commented Nov 2, 2022

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022

cstamas commented Nov 2, 2022

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022

cstamas commented Nov 2, 2022

cstamas commented Nov 4, 2022

fsmherrm commented Nov 4, 2022 •

edited

Loading

cstamas commented Nov 4, 2022 •

edited

Loading

gnodet commented Jan 6, 2023

Maven Project randomly fails with "Failed to open file channel '[..]\.m2\repository\.locks\[..].resolverlock" #728

Maven Project randomly fails with "Failed to open file channel '[..]\.m2\repository\.locks\[..].resolverlock" #728

Comments

fsmherrm commented Oct 30, 2022

gnodet commented Oct 30, 2022

fsmherrm commented Oct 30, 2022

cstamas commented Nov 1, 2022 • edited Loading

cstamas commented Nov 1, 2022 • edited Loading

fsmherrm commented Nov 1, 2022

fsmherrm commented Nov 1, 2022

cstamas commented Nov 1, 2022

michael-o commented Nov 1, 2022

cstamas commented Nov 1, 2022 • edited Loading

cstamas commented Nov 1, 2022

michael-o commented Nov 1, 2022

cstamas commented Nov 1, 2022 • edited Loading

michael-o commented Nov 1, 2022

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022 • edited Loading

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022

michael-o commented Nov 2, 2022

cstamas commented Nov 2, 2022 • edited Loading

cstamas commented Nov 2, 2022

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022

cstamas commented Nov 2, 2022

fsmherrm commented Nov 2, 2022

cstamas commented Nov 2, 2022

cstamas commented Nov 2, 2022

cstamas commented Nov 4, 2022

fsmherrm commented Nov 4, 2022 • edited Loading

cstamas commented Nov 4, 2022 • edited Loading

gnodet commented Jan 6, 2023

cstamas commented Nov 1, 2022 •

edited

Loading

cstamas commented Nov 1, 2022 •

edited

Loading

cstamas commented Nov 1, 2022 •

edited

Loading

cstamas commented Nov 1, 2022 •

edited

Loading

cstamas commented Nov 2, 2022 •

edited

Loading

cstamas commented Nov 2, 2022 •

edited

Loading

fsmherrm commented Nov 4, 2022 •

edited

Loading

cstamas commented Nov 4, 2022 •

edited

Loading