-
Notifications
You must be signed in to change notification settings - Fork 110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
--hide-successes is displaying test headings and names but not the result ... sometimes #152
Comments
Tasty uses ansi-terminal features to display the headings and the name of the current test while it's running, and then erase them if it hasn't failed. So it's possible that this may break under extreme conditions — perhaps when the amount of text to erase overflows the terminal or something. It may be a while until I get around to looking into this myself, but patches are welcome. |
Thanks that gives me something to go on |
Did you manage to create a simple reproducible example? Just a deep test tree is not enough to trigger this. Here's what I tried: import Test.Tasty
import Test.Tasty.HUnit
import Data.Function
deepTree :: TestTree
deepTree = flip fix depth $ \rec n ->
if n == 0
then testCase "Test case" $ return ()
else testGroup ("Test group " ++ show (depth+1-n)) [rec (n-1)]
where depth = 100
main = defaultMain deepTree |
Closing until there's some additional info. |
👍 |
I can duplicate the result here at Digital Asset with our test harness. I get 800+ headers of various tests and then the bodies of the failing ones. This happens even when I set the terminal wide-enough that the text doesn't overflow. We do have tests layered several levels deep if that helps narrow the cause. I also don't have a nice contained example, however. |
Would it be feasible for you to replace all test bodies with |
The tests are spread across probably 2-dozen different projects aggregated into one large test harness, making that a fair bit more awkward than it sounds. |
This is under some sort of unix, right? Could you do the following:
|
@luc-tielen could you post a reproducible example? Feel free to include only testGroups and trivial testCases that replicate your structure. |
Sure, here you go: https://github.com/luc-tielen/hspec-terminal-bug You can test it by running |
Yes, I can finally reproduce this — thanks! |
So the cause of the problem is that, when line wrapping occurs, the ANSI API treats the wrapped line as two separate lines. This is a bit surprising to me because if I resize the terminal back to the full size, the two lines become one — so the terminal remains aware that the two rendered lines are a single logical line. Currently, tasty assumes that what is output as a single line really is a single line from ANSI point of view to be able to return on that line and to erase it. We cannot use saveCursor/restoreCursor because they only support a single position, whereas we need an unbounded stack of positions. We could use getCursorPosition, but it will be almost as fragile. If, while a long-running test is executing, you resize the terminal, then all saved coordinates become wrong. @mpilgrem maybe you know a way to store a stack of terminal positions and be able to go back up to any of those positions that is robust to text wrapping and resizing the terminal? |
I don't really know the implementation of the code, but are all results printed at end? Or as they come in? Might be a lot easier than ANSI terminal magic.. something worth looking into? |
Nothing occurs to me so far. For Windows users, I found this blog entry that discusses under 'word wrap' how the native Windows console handles things (behind the scenes, 'this line has been wrapped here' codes are recorded in the screen buffer). In those consoles, 'Wrap text output on resize' is a layout option that a user can disable/enable and - I think, based on some searches - that the API does not expose what the user has chosen. I think that alone makes it near impossible to have an approach that is robust to terminal resizing. |
@luc-tielen the results are printed as they come in, and then the successful ones are erased. This way you get immediate feedback about which test is executing (especially if it's taking too long). Here's how it looks. If you don't need this immediate feedback, then exporting @mpilgrem thanks for looking into this! |
Hm, the issue with |
For me Here is a tiny repo which reproduces the problem.
Note that I understand that dealing with narrow terminals is hard for the reasons described above in this thread, but |
All: there is now a fix/workaround in master: you can pass |
We are using the
--hide-successes
flag and after adding 976 test it stopped functioning correctly. Instead of only displaying failures, it is displaying the heading and test names. We see this on both OSX and FreeBSD. Unfortunately I don't have a test case I can easily show just yet.Setting
TERM=dumb
causes the correct output to display.The text was updated successfully, but these errors were encountered: