Associate standard output/error streams to a particular plugin execution to have it in the rolling windows

[rmannibucau]

Hi,

it seems logs are not pseudo real time with some plugins (like junit-platform-maven-plugin:1.1.0:launch or exec:java/exec:exec), is it because the plugin uses a handler to spy logs? Any way to capture stdout/stderr too and add them in the rolling window?

[ppalaga]

Maybe junit-platform-maven-plugin:launch and exec:java/exec:exec suffer from the same problem as quarkus:dev discussed in #241 Do they spawn a child process using inheritIO() on the ProcessBuilder? Maybe you see a better/easier solution as the ones proposed by @gnodet in #241 ?

[rmannibucau]

@ppalaga yes they do manipulate them and for 3 of the 2 cases I mentionned it is through inheritIO. I was thinking mvnd launcher (ie distro) could add a javaagent to switch System.setOut/System.setErr to a custom impl which would enable mvnd to control it and kind of stack it to track it properly.

[ppalaga]

mvnd launcher (ie distro) could add a javaagent to switch System.setOut/System.setErr to a custom impl which would enable mvnd to control it and kind of stack it to track it properly.

I wonder whether the effect would be any different from what we already do around here by setting System.out/err to our custom impl?

[gnodet]

@rmannibucau in order to support multithreaded builds and have a clean output, we need to correlate each log entry to the corresponding plugin execution. We do that with a few hacks in the logging system. In addition, we do override the System.out and System.err to capture the output and do the correlation. However, the whole thing works because we can link a given thread to a plugin execution (that's what the custom system streams do). When forking the JVM, there's no easy way to grab the output and correlate things anymore. Any attempt to capture the output without any threading information is bound to fail as the information about which plugin is writing to the stream is lost. So my initial thought of using to redirect the output to a know file using dup2 and have the inherited processes output there too won't fully work.

I think if the plugins that do fork a process (there aren't too many of them) could be updated to poll the output and write it to the JVM System.out and System.err instead, that would work in a much better way and would allow those plugins to be run concurrently too.

[rmannibucau]

@gnodet if I'm not mistaken mvnd can instrument Process/ProcessBuilder - very worse case with an agent on mvnd jvm - to have the context at start() call time so all the logs from this process can be redirected to mvnd system no? It is actually more and more common to have such fork in plugins so I don't think that updating them all is easy. Worse case a degraded mode where a new bucket "unknown module logs" is shown would be fine too.

[gnodet]

@rmannibucau ah, I see what you meant now with the agent. Yes, instrumenting ProcessBuilder class could work I suppose. I'll have a look.

[gnodet]

Actually, the problem with the exec-maven-plugin is slightly different from the one mentioned in #241.
In this case, the problem is that the output stream is not related to a given mojo execution while when forking with inherited IO, the problem is that the java system streams are bypassed completely.

[rmannibucau]

Hi @gnodet , seems it does not fix the issue for junit-platform-maven-plugin and friends but only exec plugin which does not use inheritIO :(.

[rmannibucau]

Pushed #299 to show the issue I hit ATM.
Indeed turning to false will workaround it but is not desired for other reasons in the real project.
Let's discuss how to solve it in the PR now :).