Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to Invoke-Pester to filter output #647

Merged
merged 1 commit into from
Jan 12, 2017
Merged

Add option to Invoke-Pester to filter output #647

merged 1 commit into from
Jan 12, 2017

Conversation

nohwnd
Copy link
Member

@nohwnd nohwnd commented Nov 19, 2016

Adds Show parameter to Invoke-Pester that filters the output of Pester.

The parameter takes flags of a new type [Pester.OutputTypes]
with the following options: Default, Passed, Failed, Pending,
Skipped, Inconclusive, Describe, Context, Summary.

And three special options: None, All and Fails. The All option is
the default and prints all the output. The None option is replacement
for the -Quiet parameter (still present and not deprecated).
The Fails option shows everything except Passed.

The options can be arbitrarily mixed, for example to provide a very
verbose ouptut that includes only failed tests and summary use:
Invoke-Pester -Show Summary, Failed

image

The Summary, Failed option is deliberaterly not chosen as the Fails
option because I assume that the Describe and Context information
are valuable for the general user, and it's much harder to spell out
the definition of Fails than the definition of Summary, Failed.

The Summary, Failed option can be added to the OutputTypes as a
special case but I could not come up with a good name. Feel free to
suggest it.

Another option that I considered was skipping empty describes
and Context, (for example by posponing descibe till a test result is
written), but that would change the behavior for tests with long Before*
blocks, as well as the general behavior for empty Describes. And I don't
want to put much logic in writing output to keep it straightforward
for the users.

Fix #645
Fix #568

Adds Show parameter to Invoke-Pester that filters the output of Pester.

The parameter takes flags of a new type [Pester.OutputTypes]
with the following options: Default, Passed, Failed, Pending,
Skipped, Inconclusive, Describe, Context, Summary.

And three special options: None, All and Fails. The All option is
the default and prints all the output. The None option is replacement
for the -Quiet parameter (still present and not deprecated).
The Fails option shows everything except Passed.

The options can be arbitrarily mixed, for example to provide a very
verbose ouptut that includes only failed tests and summary use:
Invoke-Pester -Show Summary, Failed

The Summary, Failed option is deliberaterly not chosen as the Fails
option because I assume that the Describe and Context information
are valuable for the general user, and it's much harder to spell out
the definition of Fails than the definition of Summary, Failed.

The Summary, Failed option can be added to the OutputTypes as a
special case but I could not come up with a good name. Feel free to
suggest it.

Another option that I considered was skipping empty describes
and Context, (for example by posponing descibe till a test result is
written), but that would change the behavior for tests with long Before*
blocks, as well as the general behavior for empty Describes. And I don't
want to put much logic in writing output to keep it straightforward
for the users.

Fix #645
Fix #568
@it-praktyk
Copy link
Contributor

it-praktyk commented Nov 19, 2016

I don't understand why but "-Show" is not available when I use the 'Tab' key for the Invoke-Pester.

Yes, I cloned Pester repo with pull requests and switched to the correct branch.

@nohwnd
Copy link
Member Author

nohwnd commented Nov 19, 2016

Does it work when you type it in without using intellisense? Or the whole feature is missing? Have you unloaded the module and then explicitly loaded it from the branch?

@it-praktyk
Copy link
Contributor

I realized that some functions have not updated comment based help so I added to my fork of repository the set of tests to check that aspect.

What I found you can see on the screnshots below.

  1. If a file with Pester tests is inside Pester directory '-Show' doesn't work

pester-pr647-1

  1. even a summary is not displayed

pester-pr647-2

  1. with the parameter '-Passthru' summary is empty

pester-pr647-3

pester-pr647-4

I compared behaviour with Pester v. 3.4.3 and all looks fine - except incomplete help ;-)

pester-pr647-5

pester-pr647-6

@it-praktyk
Copy link
Contributor

BTW, I've found issue in tests what I used - means for functions without parameters (like e.g. 'AfterAll') results aren't correct - I just commented 'http://disq.us/p/1dsho6h' on the tests author blog @lazywinadmin

@nohwnd
Copy link
Member Author

nohwnd commented Nov 20, 2016

@it-praktyk Hello, thanks for the effort, but I think your setup is wrong. What I think happened is that you downloaded the correct code but then you run the help.tests.ps1 file from a different copy of Pester. The help.tests.ps1 file imports the version of Pester to which the tests file belongs and that either resulted in some weird state where two versions of Pester are overlapping, or your version of Pester in the repo is just broken. The first sign that it's not working at all is that you are specifying -Show Failed (show just failed) yet you are seeing describes, contexts, passed tests etc.

I am pretty sure the -PassThru should work correctly, because all I've done in the commit is adding few ifs around the screen writing stuff and renaming some identifiers.

But anything can happen, so I cloned my branch on a new installation of windows 10, and tried the -Show with both internal Pester tests and your tests file, and it seems to be working fine, or at least it's not as tragically broken as in your screenshots. I can filter down to failures, just summary and so on.

capture

Could you please remove the import-module from your help.tests.ps1 or find a better set of tests to test against and try again?

@it-praktyk
Copy link
Contributor

@nohwnd , yes you are right. After removing line from Help.Test.ps1 looks work OK.

BTW, what do you think about adding tests for comment based help completness to general tests (means to the file Pester.Tests.ps1) ?

@@ -6,7 +6,7 @@ function New-PesterState
[String[]]$TestNameFilter,
[System.Management.Automation.SessionState]$SessionState,
[Switch]$Strict,
[Switch]$Quiet,
[Pester.OutputTypes]$Show = 'All',

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe by default show only summary (and there add an information for user how get full output)? Or only failed?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iSazonov This is for compatibility reasons. Changing this from All to something else would be a breaking change for users.

@SteveL-MSFT
Copy link
Contributor

@nohwnd would prefer always showing Summary and having NoSummary option, seems like most of the time you'd want the summary

@nohwnd
Copy link
Member Author

nohwnd commented Nov 23, 2016

That's possible to do, but I'd rather not do it. It would be confusing to mix positive and negative options. The same goes for making all of the options negative. I believe that in the future you'll be able to define the config in some kind of config file, and then you'd have no problem tailoring the behavior to your needs.

@SteveL-MSFT
Copy link
Contributor

@nohwnd a config file would be great

@nohwnd nohwnd merged commit 04a8452 into pester:master Jan 12, 2017
@nohwnd nohwnd deleted the FilterPesterOutput branch January 11, 2019 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants