Drop jprocesses in favor of java.lang.ProcessHandle #16529

gastaldi · 2021-04-14T20:39:01Z

quarkus-bot · 2021-04-14T20:42:13Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `0cb340c`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	⚠️ Check →	Logs	Raw logs
🚫	Devtools Tests - JDK ${{ matrix.java.name }}
🚫	Devtools Tests - JDK 11 Windows
🚫	Gradle Tests - JDK 11 ${{ matrix.os.family }}
🚫	JVM Tests - JDK ${{ matrix.java.name }}
🚫	JVM Tests - JDK 11 Windows
🚫	Maven Tests - JDK ${{ matrix.java.name }}
🚫	Maven Tests - JDK 11 Windows
🚫	MicroProfile TCKs Tests
🚫	Native Tests - ${{ matrix.category }}
🚫	Native Tests - Windows - ${{ matrix.category }}

quarkus-bot · 2021-04-14T20:48:19Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `7a6ffb6`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	⚠️ Check →	Logs	Raw logs
🚫	Devtools Tests - JDK ${{ matrix.java.name }}
🚫	Devtools Tests - JDK 11 Windows
🚫	Gradle Tests - JDK 11 ${{ matrix.os.family }}
🚫	JVM Tests - JDK ${{ matrix.java.name }}
🚫	JVM Tests - JDK 11 Windows
🚫	Maven Tests - JDK ${{ matrix.java.name }}
🚫	Maven Tests - JDK 11 Windows
🚫	MicroProfile TCKs Tests
🚫	Native Tests - ${{ matrix.category }}
🚫	Native Tests - Windows - ${{ matrix.category }}

famod

PS: I think the old as well as the new solution have the same nasty trait of killing unrelated process instances when working locally (e.g. you have quarkus:dev running in some other project and run a Quarkus test in this repo).
But this is nothing to block this PR.

...ramework/devmode-test-utils/src/main/java/io/quarkus/test/devmode/util/DevModeTestUtils.java

quarkus-bot · 2021-04-14T20:55:09Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `2ca29d9`

Status	Name	Step	Test failures	Logs	Raw logs
⌛	Initial JDK 11 Build	`Build`	⚠️ Check →	Logs	Raw logs
🚫	Devtools Tests - JDK ${{ matrix.java.name }}
🚫	Devtools Tests - JDK 11 Windows
🚫	Gradle Tests - JDK 11 ${{ matrix.os.family }}
🚫	JVM Tests - JDK ${{ matrix.java.name }}
🚫	JVM Tests - JDK 11 Windows
🚫	Maven Tests - JDK ${{ matrix.java.name }}
🚫	Maven Tests - JDK 11 Windows
🚫	MicroProfile TCKs Tests
🚫	Native Tests - ${{ matrix.category }}
🚫	Native Tests - Windows - ${{ matrix.category }}

quarkus-bot · 2021-04-14T20:59:34Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `8b5468a`

Status	Name	Step	Test failures	Logs	Raw logs
⌛	Initial JDK 11 Build	`Build`	⚠️ Check →	Logs	Raw logs
🚫	Devtools Tests - JDK ${{ matrix.java.name }}
🚫	Devtools Tests - JDK 11 Windows
🚫	Gradle Tests - JDK 11 ${{ matrix.os.family }}
🚫	JVM Tests - JDK ${{ matrix.java.name }}
🚫	JVM Tests - JDK 11 Windows
🚫	Maven Tests - JDK ${{ matrix.java.name }}
🚫	Maven Tests - JDK 11 Windows
🚫	MicroProfile TCKs Tests
🚫	Native Tests - ${{ matrix.category }}
🚫	Native Tests - Windows - ${{ matrix.category }}

quarkus-bot · 2021-04-15T00:00:17Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `f79fd38`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Verify dependencies`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 11
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 15
⌛	Maven Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

✖ io.quarkus.gradle.FastJarFormatWorksTest.testFastJarFormatWorks() line 30 - More details - Source on GitHub

✖ io.quarkus.gradle.GrpcMultiModuleQuarkusBuildTest.testProtocErrorOutput() line 39 - More details - Source on GitHub

✖ io.quarkus.gradle.GrpcMultiModuleQuarkusBuildTest.testGrpcMultiModuleBuild() line 22 - More details - Source on GitHub

✖ io.quarkus.gradle.ImplementationFilesQuarkusBuildTest.test() line 28 - More details - Source on GitHub

✖ io.quarkus.gradle.KotlinGRPCProjectBuildTest.testBasicMultiModuleBuild() line 16 - More details - Source on GitHub

✖ io.quarkus.gradle.LegacyJarFormatWorksTest.testLegacyJarFormatWorks() line 31 - More details - Source on GitHub

✖ io.quarkus.gradle.MultiModuleKotlinProjectBuildTest.testBasicMultiModuleBuild() line 15 - More details - Source on GitHub

✖ io.quarkus.gradle.MultiModuleUberJarTest.testUberJarForMultiModule() line 29 - More details - Source on GitHub

✖ io.quarkus.gradle.MultiSourceProjectTest.shouldRunTest() line 16 - More details - Source on GitHub

✖ io.quarkus.gradle.MutableJarFormatBootsInDevModeTest.testFastJarFormatWorks() line 31 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canDetectClasspathChangeWhenBuilding() line 104 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canDetectUpToDateBuild() line 64 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canDetectUpToDateTests() line 133 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canDetectSystemPropertyChangeWhenBuilding() line 146 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.Build JAVA project line 51 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.Build KOTLIN project line 51 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.Build SCALA project line 51 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canRunTest() line 161 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canDetectResourceChangeWhenBuilding() line 75 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canDetectClassChangeWhenBuilding() line 89 - More details - Source on GitHub

✖ io.quarkus.gradle.QuarkusPluginFunctionalTest.canDetectOutputChangeWhenBuilding() line 117 - More details - Source on GitHub

✖ io.quarkus.gradle.TestFixtureModuleTest.testTaskShouldUseTestFixtures() line 19 - More details - Source on GitHub

✖ io.quarkus.gradle.TestFixtureMultiModuleTest.testTaskShouldUseTestFixtures() line 17 - More details - Source on GitHub

✖ io.quarkus.gradle.TestResourcesInBuildStepsTest.testBasicMultiModuleBuild() line 23 - More details - Source on GitHub

✖ io.quarkus.gradle.UberJarFormatWorksTest.testUberJarFormatWorks() line 29 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.BasicJavaLibraryModuleDevModeTest.main() line 24 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.BasicJavaPlatformModuleDevModeTest.main() line 24 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.BasicMultiModuleProjectDevModeTest.main() line 22 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.CustomQuarkusDevModeConfigurationTest.main() line 13 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.DotEnvQuarkusDevModeConfigurationTest.main() line 13 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.ImplementationFilesDevModeTest.main() line 21 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.InjectQuarkusAppPropertiesDevModeTest.main() line 19 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.JandexMultiModuleProjectDevModeTest.main() line 21 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.ModuleWithParentDependencyDevModeTest.main() line 14 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.MultiModuleIncludedBuildTest.main() line 24 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.MultiModuleKotlinProjectDevModeTest.main() line 22 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.MultiModuleNamedInjectionDevModeTest.main() line 21 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.MultiModuleWithEmptyModuleDevModeTest.main() line 18 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.QuarkusDevDependencyDevModeTest.main() line 14 - More details - Source on GitHub

✖ io.quarkus.gradle.devmode.ResourcesInBuildStepsDevModeTest.main() line 24 - More details - Source on GitHub

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/devmode

✖ io.quarkus.test.no.src.main.NoSrcMainDevModeTest.validateConfigBean - More details - Source on GitHub

📦 integration-tests/ide-launcher

✖ io.quarkus.it.launcher.IDELauncherTestCase.testIdeLauncher line 27 - More details - Source on GitHub

📦 integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinDevModeIT.testThatTheApplicationIsReloadedOnKotlinChange line 23 - More details - Source on GitHub

✖ io.quarkus.kotlin.maven.it.KotlinRemoteDevModeIT.testThatTheApplicationIsReloadedOnKotlinChange line 25 - More details - Source on GitHub

📦 integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaDevModeIT.testThatTheApplicationIsReloadedOnScalaChange - More details - Source on GitHub

✖ io.quarkus.scala.maven.it.ScalaRemoteDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 25 - More details - Source on GitHub

gsmet · 2021-04-15T07:38:04Z

Looks like CI is unhappy. Thanks for testing the bot :).

famod · 2021-04-15T07:48:52Z

Gradle tests should be better with the memory tweaks I did in #16249.

Maven test job seems to show that killing does not work properly?

ERROR [io.qua.run.Application] (Quarkus Main Thread) Port 8080 seems to be in use by another process. Quarkus may already be running or the port is used by another application.

quarkus-bot · 2021-04-15T16:57:48Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `60c6b91`

Status	Name	Step	Test failures	Logs	Raw logs
⌛	Gradle Tests - JDK 11 Windows	`Verify dependencies`	⚠️ Check →	Logs	Raw logs
✔️	JVM Tests - JDK 11
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 15
⌛	Maven Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/devmode

✖ io.quarkus.test.no.src.main.NoSrcMainDevModeTest.validateConfigBean - More details - Source on GitHub

📦 integration-tests/ide-launcher

✖ io.quarkus.it.launcher.IDELauncherTestCase.testIdeLauncher line 27 - More details - Source on GitHub

📦 integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinDevModeIT.testThatTheApplicationIsReloadedOnKotlinChange line 23 - More details - Source on GitHub

✖ io.quarkus.kotlin.maven.it.KotlinRemoteDevModeIT.testThatTheApplicationIsReloadedOnKotlinChange line 25 - More details - Source on GitHub

📦 integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaDevModeIT.testThatTheApplicationIsReloadedOnScalaChange - More details - Source on GitHub

✖ io.quarkus.scala.maven.it.ScalaRemoteDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 25 - More details - Source on GitHub

gsmet · 2021-04-15T17:32:44Z

The test failures are very suspicious. These tests weren't flaky.

famod · 2021-04-15T17:48:46Z

Yes, looks very related, especially that port error log I commented on earlier.

In the worst case, ProcessHandle.allProcesses() sees fewer processes than jprocesses. This needs a deep dive.

quarkus-bot · 2021-05-19T02:14:01Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `7f9340d`

⚠️ Artifacts of the workflow run were not available thus the report misses some details.

Status	Name	Step	Test failures	Logs	Raw logs
⌛	Initial JDK 11 Build	`Run actions/checkout@v2`	⚠️ Check →	Logs	Raw logs
⌛	CI Sanity Check	`Build`	⚠️ Check →	Logs	Raw logs

quarkus-bot · 2021-05-19T05:30:37Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `dd87e7c`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
✔️	JVM Tests - JDK 11
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 16
⌛	Maven Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/devmode

✖ io.quarkus.test.no.src.main.NoSrcMainDevModeTest.validateConfigBean - More details - Source on GitHub

📦 integration-tests/ide-launcher

✖ io.quarkus.it.launcher.IDELauncherTestCase.testIdeLauncher line 27 - More details - Source on GitHub

📦 integration-tests/kotlin

✖ io.quarkus.kotlin.maven.it.KotlinDevModeIT.testThatTheApplicationIsReloadedOnKotlinChange line 25 - More details - Source on GitHub

✖ io.quarkus.kotlin.maven.it.KotlinRemoteDevModeIT.testThatTheApplicationIsReloadedOnKotlinChange line 27 - More details - Source on GitHub

📦 integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaDevModeIT.testThatTheApplicationIsReloadedOnScalaChange - More details - Source on GitHub

✖ io.quarkus.scala.maven.it.ScalaRemoteDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 25 - More details - Source on GitHub

famod · 2021-05-25T10:39:43Z

FYI, I'm having a look at this in my fork. It's remarkable that only Windows jobs are affected.
I'll remove the watiting-for-ci label for the time being.

famod · 2021-05-25T11:00:39Z

This doesn't look good: JDK-8176725 ProcessHandle arguments is always null

The command line arguments are not available via a non-privileged API for other processes and so the Optional is always empty. The API is explicit that the values are OS specific.
If in the future, the arguments are available by a Window APIs, the implementation can be updated.

Status: Resolved
Resolution: Future Project

This also affects the commandLine that is being checked in this PR.

Investigating workarounds...

famod · 2021-05-25T12:29:14Z

Ok, I think I found a good workaround. Will see how CI fares in my fork. If all goes to plan I'll create a fresh PR and close this one...

gastaldi · 2021-05-25T13:20:59Z

@famod you can also push to my branch if you want:

git push [email protected]:gastaldi/quarkus.git remove_jprocessescd -f

famod

Ok, pushed to your branch @gastaldi

famod · 2021-05-26T09:15:56Z

test-framework/maven/src/main/java/io/quarkus/maven/it/verifier/RunningInvoker.java

+        // Kill all processes that were (indirectly) spawned by the current process.
+        // It's important to do it this way (instead of calling result.destroy() first)
+        // because otherwise children of that process can become orphaned zombies.
+        DevModeTestUtils.killDescendingProcesses();


This invocation order problem took the most time to figure out. I was able to see (on Windows, via Process Explorer) that sometimes a sub-branch of processes survives result.destroy(), loosing that parent and thus also the transitive parent connection to the surefire test JVM process. That resulted in descendants() not finding those "orphaned" processes anymore.

famod · 2021-05-26T12:39:35Z

All green! @geoand @gastaldi @gsmet (tripple g 😉) do you want to have another look?

gastaldi requested a review from famod April 14, 2021 20:39

quarkus-bot bot added area/dependencies Pull requests that update a dependency file area/testing labels Apr 14, 2021

gastaldi mentioned this pull request Apr 14, 2021

Drop jprocesses in favor of java.lang.ProcessHandle #16521

Closed

gastaldi force-pushed the remove_jprocessescd branch from 0cb340c to 7a6ffb6 Compare April 14, 2021 20:45

quarkus-bot bot added area/devtools Issues/PR related to maven, gradle, platform and cli tooling/plugins area/gradle Gradle labels Apr 14, 2021

famod approved these changes Apr 14, 2021

View reviewed changes

...ramework/devmode-test-utils/src/main/java/io/quarkus/test/devmode/util/DevModeTestUtils.java Outdated Show resolved Hide resolved

gastaldi force-pushed the remove_jprocessescd branch from 7a6ffb6 to 2ca29d9 Compare April 14, 2021 20:50

quarkus-bot bot added the area/maven label Apr 14, 2021

gastaldi force-pushed the remove_jprocessescd branch from 2ca29d9 to 8b5468a Compare April 14, 2021 20:54

gastaldi force-pushed the remove_jprocessescd branch from 8b5468a to f79fd38 Compare April 14, 2021 20:58

gastaldi force-pushed the remove_jprocessescd branch from f79fd38 to 60c6b91 Compare April 15, 2021 13:48

geoand approved these changes Apr 15, 2021

View reviewed changes

geoand added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Apr 15, 2021

famod mentioned this pull request Apr 18, 2021

Job status summary: purposely skipped step is reported instead of actual failing one quarkusio/quarkus-github-bot#82

Closed

gastaldi force-pushed the remove_jprocessescd branch 2 times, most recently from 7f9340d to dd87e7c Compare May 19, 2021 02:13

famod removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label May 25, 2021

Drop jprocesses in favor of java.lang.ProcessHandle

9fbdd7b

famod force-pushed the remove_jprocessescd branch from dd87e7c to 9fbdd7b Compare May 26, 2021 09:08

famod reviewed May 26, 2021

View reviewed changes

famod requested a review from geoand May 26, 2021 09:11

famod reviewed May 26, 2021

View reviewed changes

geoand approved these changes May 26, 2021

View reviewed changes

gastaldi merged commit d2878d5 into quarkusio:main May 26, 2021

quarkus-bot bot added this to the 2.0 - main milestone May 26, 2021

gastaldi deleted the remove_jprocessescd branch May 26, 2021 14:00

Drop jprocesses in favor of java.lang.ProcessHandle #16529

Drop jprocesses in favor of java.lang.ProcessHandle #16529

Conversation

gastaldi commented Apr 14, 2021

quarkus-bot bot commented Apr 14, 2021 • edited Loading

Failing Jobs - Building 0cb340c

quarkus-bot bot commented Apr 14, 2021 • edited Loading

Failing Jobs - Building 7a6ffb6

famod left a comment • edited Loading

Choose a reason for hiding this comment

quarkus-bot bot commented Apr 14, 2021 • edited Loading

Failing Jobs - Building 2ca29d9

quarkus-bot bot commented Apr 14, 2021 • edited Loading

Failing Jobs - Building 8b5468a

quarkus-bot bot commented Apr 15, 2021 • edited Loading

Failing Jobs - Building f79fd38

Test Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/devmode

📦 integration-tests/ide-launcher

📦 integration-tests/kotlin

📦 integration-tests/scala

gsmet commented Apr 15, 2021

famod commented Apr 15, 2021

quarkus-bot bot commented Apr 15, 2021 • edited Loading

Failing Jobs - Building 60c6b91

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/devmode

📦 integration-tests/ide-launcher

📦 integration-tests/kotlin

📦 integration-tests/scala

gsmet commented Apr 15, 2021

famod commented Apr 15, 2021

quarkus-bot bot commented May 19, 2021 • edited Loading

Failing Jobs - Building 7f9340d

quarkus-bot bot commented May 19, 2021 • edited Loading

Failing Jobs - Building dd87e7c

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/devmode

📦 integration-tests/ide-launcher

📦 integration-tests/kotlin

📦 integration-tests/scala

famod commented May 25, 2021

famod commented May 25, 2021 • edited Loading

famod commented May 25, 2021

gastaldi commented May 25, 2021 • edited Loading

famod left a comment • edited Loading

Choose a reason for hiding this comment

famod May 26, 2021

Choose a reason for hiding this comment

famod commented May 26, 2021

quarkus-bot bot commented Apr 14, 2021 •

edited

Loading

Failing Jobs - Building `0cb340c`

quarkus-bot bot commented Apr 14, 2021 •

edited

Loading

Failing Jobs - Building `7a6ffb6`

famod left a comment •

edited

Loading

quarkus-bot bot commented Apr 14, 2021 •

edited

Loading

Failing Jobs - Building `2ca29d9`

quarkus-bot bot commented Apr 14, 2021 •

edited

Loading

Failing Jobs - Building `8b5468a`

quarkus-bot bot commented Apr 15, 2021 •

edited

Loading

Failing Jobs - Building `f79fd38`

quarkus-bot bot commented Apr 15, 2021 •

edited

Loading

Failing Jobs - Building `60c6b91`

quarkus-bot bot commented May 19, 2021 •

edited

Loading

Failing Jobs - Building `7f9340d`

quarkus-bot bot commented May 19, 2021 •

edited

Loading

Failing Jobs - Building `dd87e7c`

famod commented May 25, 2021 •

edited

Loading

gastaldi commented May 25, 2021 •

edited

Loading

famod left a comment •

edited

Loading