Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding ancestorTitles property to JSON test output #4293

Merged
merged 1 commit into from
Aug 24, 2017

Conversation

mikehedman
Copy link
Contributor

Summary
This addresses the need brought up in #2479

The motivation I have is that in order to properly report our test runs to regulatory agencies, we are planning on using the JSON output to feed reports. But our unit tests can be nested a couple of layers deep in 'describe' and 'in' blocks. Currently, the JSON output only includes the title of the nearest test block, thus, losing the context of its parents.
For example, we want to see:

  • passwords
    • size
      • too short
      • too long
        and not just:
  • too long

Test plan
I've altered the sum.test.js file to create tests with zero, one, and many ancestors, and added checks for the new ancestorTitles property.

@cpojer cpojer merged commit b6c68d3 into jestjs:master Aug 24, 2017
@cpojer
Copy link
Member

cpojer commented Aug 24, 2017

Fine with me. Thanks!

@cpojer
Copy link
Member

cpojer commented Aug 24, 2017

What kind of regulatory agencies do you report results from Jest to?

@mikehedman
Copy link
Contributor Author

@cpojer Thanks for merging this in.

To answer your question about regulatory agencies, I work primarily in the medical device field. So the agency that would ultimately get my results would be the FDA here in America, and the analogous agencies overseas.
Here's a link to a high level FDA guidance that applies to work I do: https://www.fda.gov/MedicalDevices/ucm085281.htm#_Toc517237962

In practical terms, generally raw unit test results are not required. But there are cases where unit testing (as opposed to what I would call "functional" testing, such as hitting a web page via Selenium) is really the only way to properly exercise certain functionality. In these instances, a company's Regulatory/QA group might supplement their standard testing with specific unit testing results. Companies can also augment their standard reporting with a higher level "oh yeah, and we also continually run 6000 unit tests on this product".

I've also worked a fair bit in the "payments" industry, which also cares a lot about software doing what it's supposed to ;) Here's a relevant guidance document: https://www.pcisecuritystandards.org/documents/terminal_security_best_practices.pdf
See section B8 where it expressly states: "Low-level unit testing of the software should be performed and recorded. This may require writing specific test software or modifying calling functions to specifically cover boundary checking and error handling."

I hope that helps.

@mikehedman mikehedman deleted the mikehedman branch August 24, 2017 17:12
@cpojer
Copy link
Member

cpojer commented Aug 24, 2017

Thanks for the added context, that's really exciting!

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants