Upgrade from Tycho 2.6.0 to 2.7.4 => Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:2.7.4:validate-classpath

[rubenporras]

Since I upgraded to 2.7.4 some jobs fails with the following pattern:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:2.7.4:validate-classpath (default-validate-classpath) on project com.avaloq.tools.dsl.typedef.ui.test: Execution default-validate-classpath of goal org.eclipse.tycho:tycho-compiler-plugin:2.7.4:validate-classpath failed: java.io.FileNotFoundException: D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky${env.WORKSPACE}.repository.meta\p2-artifacts.properties (Access is denied) -> [Help 1]

I have different jobs failing on different plugins in a similar way (just on different plugins). As far as I can see the only difference between this jobs which fail and the verify build which always succeed is that the ones falling are nightly builds and that the use different parameters that affect org.junit.runner.manipulation.Filter, but some of this jobs do not fail on test plugins, so I am not sure that is relevant.

Interestingly the goal org.eclipse.tycho:tycho-compiler-plugin:2.7.4:validate-classpath has no problems on plug-ins before and plug-ins after (See below). Any idea what the problem could be?

Thanks

[INFO] ------------------------------------------------------------------------
[INFO] Building com.avaloq.tools.dsl.typedef.core.test 16.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.2.0:clean (default-clean) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Deleting D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.core.test\xtend-gen (includes = [], excludes = [.gitignore])
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:build-qualifier (default-build-qualifier) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] The project's OSGi version is 16.4.0.v20230531-0635-J3462-XXX
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:validate-id (default-validate-id) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:validate-version (default-validate-version) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO]
[INFO] --- target-platform-configuration:2.7.4:target-platform (default-target-platform) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO]
[INFO] --- tycho-compiler-plugin:2.7.4:validate-classpath (default-validate-classpath) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Resolving class path of com.avaloq.tools.dsl.typedef.core.test...
[INFO]
[INFO] --- xtend-maven-plugin:2.30.0:compile (xtend-generate) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO]
[INFO] --- xtend-maven-plugin:2.30.0:testCompile (xtend-generate) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] skip compiling sources because the configured directory '[D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.core.test\src\test\java]' does not exist.
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.core.test\src\main\resources
[INFO]
[INFO] --- tycho-compiler-plugin:2.7.4:compile (default-compile) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Compiling 3 source files to D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.core.test\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.core.test\src\test\resources
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:update-consumer-pom (default-update-consumer-pom) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO]
[INFO] --- tycho-source-plugin:2.7.4:plugin-source (plugin-source-public) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Building jar: D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.core.test\target\com.avaloq.tools.dsl.typedef.core.test-16.4.0-SNAPSHOT-sources.jar
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:package-plugin (default-package-plugin) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Building jar: D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.core.test\target\com.avaloq.tools.dsl.typedef.core.test-16.4.0-SNAPSHOT.jar
[INFO]
[INFO] --- tycho-p2-plugin:2.7.4:p2-metadata-default (default-p2-metadata-default) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO]
[INFO] --- tycho-surefire-plugin:2.7.4:test (default-test) @ com.avaloq.tools.dsl.typedef.core.test ---
[INFO] Skipping tests
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building com.avaloq.tools.dsl.typedef.ui.test 16.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.2.0:clean (default-clean) @ com.avaloq.tools.dsl.typedef.ui.test ---
[INFO] Deleting D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.typedef.ui.test\xtend-gen (includes = [], excludes = [.gitignore])
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:build-qualifier (default-build-qualifier) @ com.avaloq.tools.dsl.typedef.ui.test ---
[INFO] The project's OSGi version is 16.4.0.v20230531-0635-J3462-XXX
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:validate-id (default-validate-id) @ com.avaloq.tools.dsl.typedef.ui.test ---
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:validate-version (default-validate-version) @ com.avaloq.tools.dsl.typedef.ui.test ---
[INFO]
[INFO] --- target-platform-configuration:2.7.4:target-platform (default-target-platform) @ com.avaloq.tools.dsl.typedef.ui.test ---
[INFO]
[INFO] --- tycho-compiler-plugin:2.7.4:validate-classpath (default-validate-classpath) @ com.avaloq.tools.dsl.typedef.ui.test ---
[INFO] Resolving class path of com.avaloq.tools.dsl.typedef.ui.test...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building com.avaloq.tools.dsl.user.core.test 16.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.2.0:clean (default-clean) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Deleting D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.user.core.test\xtend-gen (includes = [**], excludes = [.gitignore])
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:build-qualifier (default-build-qualifier) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] The project's OSGi version is 16.4.0.v20230531-0635-J3462-XXX
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:validate-id (default-validate-id) @ com.avaloq.tools.dsl.user.core.test ---
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:validate-version (default-validate-version) @ com.avaloq.tools.dsl.user.core.test ---
[INFO]
[INFO] --- target-platform-configuration:2.7.4:target-platform (default-target-platform) @ com.avaloq.tools.dsl.user.core.test ---
[INFO]
[INFO] --- tycho-compiler-plugin:2.7.4:validate-classpath (default-validate-classpath) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Resolving class path of com.avaloq.tools.dsl.user.core.test...
[INFO]
[INFO] --- xtend-maven-plugin:2.30.0:compile (xtend-generate) @ com.avaloq.tools.dsl.user.core.test ---
[INFO]
[INFO] --- xtend-maven-plugin:2.30.0:testCompile (xtend-generate) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] skip compiling sources because the configured directory '[D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.user.core.test\src\test\java]' does not exist.
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.user.core.test\src\main\resources
[INFO]
[INFO] --- tycho-compiler-plugin:2.7.4:compile (default-compile) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Compiling 6 source files to D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.user.core.test\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.user.core.test\src\test\resources
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:update-consumer-pom (default-update-consumer-pom) @ com.avaloq.tools.dsl.user.core.test ---
[INFO]
[INFO] --- tycho-source-plugin:2.7.4:plugin-source (plugin-source-public) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Building jar: D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.user.core.test\target\com.avaloq.tools.dsl.user.core.test-16.4.0-SNAPSHOT-sources.jar
[INFO]
[INFO] --- tycho-packaging-plugin:2.7.4:package-plugin (default-package-plugin) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Building jar: D:\work\jenkins-slave\workspace\asmd-master-nightly-test-flaky\com.avaloq.tools.dsl.user.core.test\target\com.avaloq.tools.dsl.user.core.test-16.4.0-SNAPSHOT.jar
[INFO]
[INFO] --- tycho-p2-plugin:2.7.4:p2-metadata-default (default-p2-metadata-default) @ com.avaloq.tools.dsl.user.core.test ---
[INFO]
[INFO] --- tycho-surefire-plugin:2.7.4:test (default-test) @ com.avaloq.tools.dsl.user.core.test ---
[INFO] Skipping tests
[INFO]

[laeubi]

Have you tried using dedicated maven repositories fro the jobs? The local m2 repository is not really meant to be shared across jobs, also please note:

https://github.com/eclipse-tycho/tycho/blob/master/RELEASE_NOTES.md#caution-when-switching-between-tycho-versions

[rubenporras]

I am running the jobs with -Dmaven.repo.local=${env.WORKSPACE}/.repository. Does the parameter look right to you?

[ava-fred]

I work with @rubenporras and I have continued investigating the problem. I found that the problem only occurs when the builds are run on Windows machines. I also found this stack trace for the problem:

Caused by: java.io.FileNotFoundException: <Jenkins workspace path>\.repository\.meta\p2-artifacts.properties (Access is denied)
    at java.io.FileInputStream.open0 (Native Method)
    at java.io.FileInputStream.open (FileInputStream.java:216)
    at java.io.FileInputStream.<init> (FileInputStream.java:157)
    at org.eclipse.tycho.repository.local.index.FileBasedTychoRepositoryIndex.reconcile (FileBasedTychoRepositoryIndex.java:138)
    at org.eclipse.tycho.repository.local.index.FileBasedTychoRepositoryIndex.save (FileBasedTychoRepositoryIndex.java:121)
    at org.eclipse.tycho.repository.local.LocalArtifactRepository.save (LocalArtifactRepository.java:146)
    at org.eclipse.tycho.p2.target.TargetPlatformBaseImpl.saveLocalMavenRepository (TargetPlatformBaseImpl.java:168)
    at org.eclipse.tycho.p2.resolver.DefaultP2ResolutionResult.lambda$1 (DefaultP2ResolutionResult.java:63)
    at org.eclipse.tycho.p2.resolver.DefaultP2ResolutionResultEntry.getLocation (DefaultP2ResolutionResultEntry.java:74)
    at org.eclipse.tycho.p2.resolver.P2DependencyResolver.lambda$newDefaultTargetPlatform$6 (P2DependencyResolver.java:490)
    at org.eclipse.tycho.core.osgitools.targetplatform.DefaultDependencyArtifacts.lambda$addFragment$0 (DefaultDependencyArtifacts.java:101)
    at org.eclipse.tycho.core.osgitools.DefaultArtifactDescriptor.getLocation (DefaultArtifactDescriptor.java:71)
    at org.eclipse.tycho.core.osgitools.OsgiBundleProject.addExtraClasspathEntries (OsgiBundleProject.java:622)
    at org.eclipse.tycho.core.osgitools.OsgiBundleProject.resolveClassPath (OsgiBundleProject.java:291)
    at org.eclipse.tycho.core.osgitools.OsgiBundleProject.getBundleClassPath (OsgiBundleProject.java:510)
    at org.eclipse.tycho.core.osgitools.OsgiBundleProject.getClasspath (OsgiBundleProject.java:497)
    at org.eclipse.tycho.compiler.ValidateClassPathMojo.execute (ValidateClassPathMojo.java:53)

(I have removed the part where the exception bubbles up and the Maven stack trace entries)

I suspect the problem might be that Windows Defender locks the p2-artifacts.properties file exclusively (especially because it is created as a new temp file and renamed when saving (afaics)). I have asked for the directories used by the builds to be excluded from scanning, but it hasn't been implemented yet. I'll update this discussion once I have a result, but in the meantime I though I'd document what I found so far in case anybody else has similar problems.

[rubenporras]

We have done several tests disabling Windows Defender or the Windows Indexer, but that did not help.

However, by upgrading our Windows from to Windows Server 2022, the problem is solved.

Thanks for your help.