You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Run a test with deflaking enabled & a test case failing (triggering deflake runs)
Check JUnitReport.xml
Expected behavior
No negative test case times
Details (please complete the following information):
This was determined to be caused by overhead time calculation introduced in #684 to fix #557. While calculating overhead, all instances of test cases are added together (which is 5*T for 4 deflake reruns), and then subtracted from T of a single run (ignoring reruns), resulting in "negative overhead"
Additional context
Notably, all results from FTL are non-negative
The text was updated successfully, but these errors were encountered:
…ocessing deflake reruns & reporting negative test case times.
Overhead (as described in #557) no longer exists in FTL, so this feature is not useful anymore.
Fixes#2363
…ocessing deflake reruns & reporting negative test case times. (#2364)
Overhead (as described in #557) no longer exists in FTL, so this feature is not useful anymore.
Fixes#2363
Describe the bug
https://firebase-community.slack.com/messages/test-lab/thread/C1MTSQ5QT-1677888583.152469
Flank's junit xml files list some test cases like so:
To Reproduce
Expected behavior
No negative test case times
Details (please complete the following information):
This was determined to be caused by overhead time calculation introduced in #684 to fix #557. While calculating overhead, all instances of test cases are added together (which is 5*T for 4 deflake reruns), and then subtracted from T of a single run (ignoring reruns), resulting in "negative overhead"
Additional context
Notably, all results from FTL are non-negative
The text was updated successfully, but these errors were encountered: