Add ability to print live output messages in Test Explorer/dotnet test output

[bradwilson]

This adds the ability to show output messages in real time, which may assist in debugging a test which is long-running.

[bradwilson]

Available in 2.8.1-pre.4 https://xunit.net/docs/using-ci-builds

Documentation has been updated on enabling this feature:

• Via xunit.runner.json file: https://xunit.net/docs/configuration-files#showLiveOutput
• Via RunSettings: https://xunit.net/docs/runsettings#ShowLiveOutput

Using this example test:

using Xunit;
using Xunit.Abstractions;

namespace Empty;

public class TestClass(ITestOutputHelper helper)
{
	[Fact]
	public void TestMethod()
	{
		helper.WriteLine("This is a line of output");
	}
}

Output for dotnet test requires passing --logger console;verbosity=normal to that VSTest does not hide the output from xUnit.net:

Output for Test Explorer is in the Output Window, under the Tests tab:

[Piedone]

That was quick, thank you!

[Piedone]

Is there any reason not to always enable this, at least in CI? Does it have a noticable impact on performance, for example?

[bradwilson]

There should be no significant performance impact. It's not enabled by default because it's really a debugging tool more than anything else, and live printing output messages makes the output significantly more cluttered (especially with tests running in parallel that are all outputting messages).

[Piedone]

I see, thanks.

[NicolasDorier]

So with this PR, console;verbosity=normal is showing the output whether the showLiveOutput is true or false.

showLiveOutput is only deciding whether the logs are showing in real time or not. It isn't deciding whether the logs should be shown at all.

This is problematic, the test output is now difficult to read.
I would expect console;verbosity=normal to, like before, only show whether test pass or not. With the stacktrace if it didn't pass.

Dropping verbosity one level down remove the Passed logs.

[bradwilson]

We don't have that level of control. The verbosity from dotnet test is under the control of VSTest, and we don't even know when/if you change that verbosity.

I don't particularly consider "the logs got harder to read" to be a huge problem, since this is intended to be temporarily enabled to help find tests which are running over time and potentially never ending. This is not intended to be permanently enabled.

[NicolasDorier]

@bradwilson This is not intended to be permanently enabled.. I understand this, but my point is that even if you disable it, it prints the logs (not in real time), when before, it didn't.

Basically I don't want to see logs if with console;verbosity=normal and showLiveOutput=false.

If I want to debug, turning showLiveOutput=true works as expected.

[bradwilson]

Basically I don't want to see logs if with console;verbosity=normal and showLiveOutput=false.

Disable diagnostic messages.

[NicolasDorier]

Damn, didn't noticed I had that on... this fixed it thanks. Sorry for this!