Set-TestInconclusive does not fail the test when running in -Strict mode

[nohwnd]

Setting test to inconclusive marks the test as inconclusive even when running in Strict mode, we are probably making the check first, and then running the test.

Related #529

[nohwnd]

Repro:

$path = "$env:TEMP\Strict.tests.ps1"
'describe "a" { it "b" { Set-TestInconclusive } }' | Set-Content $path

Invoke-Pester -Strict "$env:TEMP\Strict.tests.ps1"

# output: 
# Tests completed in 36ms
# Tests Passed: 0, Failed: 0, Skipped: 0, Pending: 0, Inconclusive: 1 

[bergmeister]

This one is going to be opinionated but the user should make a choice whether to mark inconclusive as failed or not. Therefore only making it fail when using -Strict looks ok to me as per your proposal but without the switch I would leave it as it is.
VSTS has this 'great' behaviour that it marks the build as red in this case but the UI hides inconclusive tests by default...

[nohwnd]

This one is still happening. The Set-TestInconclusive was recently changed to use this implementation #1026 That PR should serve as a good guide in which places and in which files the process of detecting test inconclusive is happening. And when we are running in strict mode we should fail the test after we receive the known exceptions, instead of marking it as inconculsive (or pending or skipped). Also don't confuse this with PowerShell's Strict-Mode, this strict mode refers to -Strict parameter on Invoke-Pester described here.

[renehernandez]

@nohwnd I am checking this as possible first issue to contribute to, but is this still relevant since in the docs Set-TestInconclusive is already deprecated in favor of Set-ItResult.

Or should the resolution for this should cover both Set-TestInconclusive and Set-ItResult? (in case that Set-ItResult has the same issue)

[nohwnd]

@renehernandez Yes this is still valid. The underlying mechanism of setting the test result is (I hope :)) the same for both Set-ItResult and Set-TestInconclusive. The former was deprecated because it does just a single thing and uses Test instead of It in the name. When this issue is solved the test in which we set a result, should be marked as failed in the result when running with the -Strict switch on Invoke-Pester, and not as inconclusive (or pending or skipped).

[renehernandez]

@nohwnd Sounds good, I am go to start working on it this weekend

[nohwnd]

Good, ping me if you need help:) twitter will get you faster response :)