doc: recommend writing tests in new files and including comments #57028
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The previous phrasing encouraged or did not discourage appending new test cases to existing files - a practice that can reduce the debuggability of the tests over time as they get bigger and bigger, some times thousands of lines long with hundreds of test cases, and make the CI output increasingly difficult to read when one of the test cases fail in a very long test. This patch updates the guideline to explicitly discourage appending test cases this way. Also recommend including an opening comment to describe what the test does to optimize the test towards the scenario when it fails.
|
Review requested:
|
|
cc @jasnell @nodejs/tsc since we discussed this in the TSC meeting before. In recent code reviews I noticed that many are still appending new test cases this way, which is probably guided by the doc, so I think we need to update the docs to have a ground to ask for changes when people do this in the PR. |
jasnell
approved these changes
Feb 13, 2025
richardlau
approved these changes
Feb 13, 2025
targos
approved these changes
Feb 13, 2025
UlisesGascon
approved these changes
Feb 13, 2025
lpinca
approved these changes
Feb 13, 2025
lpinca
added
the
commit-queue
jakecastelli
approved these changes
Feb 14, 2025
jakecastelli
added
the
author ready
|
I personally found naming a test (file name) is also hard, do we have any convention or guide for it? |
|
I usually start with writing the comment and then take keywords from my comment to create the name. Not sure if that counts as guides. |
legendecas
approved these changes
Feb 14, 2025
bnb
approved these changes
Feb 14, 2025
nodejs-github-bot
removed
the
commit-queue
|
Landed in ff51d83 |
targos
pushed a commit
that referenced
this pull request
Feb 17, 2025
The previous phrasing encouraged or did not discourage appending new test cases to existing files - a practice that can reduce the debuggability of the tests over time as they get bigger and bigger, some times thousands of lines long with hundreds of test cases, and make the CI output increasingly difficult to read when one of the test cases fail in a very long test. This patch updates the guideline to explicitly discourage appending test cases this way. Also recommend including an opening comment to describe what the test does to optimize the test towards the scenario when it fails. PR-URL: #57028 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Ulises Gascón <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Jake Yuesong Li <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]> Reviewed-By: Tierney Cyren <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The previous phrasing encouraged or did not discourage appending new test cases to existing files - a practice that can reduce the debuggability of the tests over time as they get bigger and bigger, some times thousands of lines long with hundreds of test cases, and make the CI output increasingly difficult to read when one of the test cases fail in a very long test.
This patch updates the guideline to explicitly discourage appending test cases this way. Also recommend including an opening comment to describe what the test does to optimize the test towards the scenario when it fails.
For some concrete examples:
test-crypto-dh.jsandtest-wasi.jsbecause they were timing out in the CI, some might be related to actual bugs but there is no way to figure out which test specifically is timing out in a long test, so it had to be split to isolate them.To avoid losing git history I would not recommend splitting existing tests unless there's a ongoing motivation to do so (e.g. it's flaking in the CI). But I think we should at least discourage new tests to be written in a way that exacerbate the problem.