jUnitReport.xml contains wrong data with smart sharding

[andreschab90]

I am using smart sharding and i recognized the the jUnitReport-xml file contains wrong data compared to real results of the matrixes. I can provide you some xml files for debugging if needed

[bootstraponline]

I'll need more info to identity, reproduce, and resolve the issue.

[andreschab90]

i had a look in the links to inside the junit file.
I have a test A which fails, but the junit reports successful, because the link for this test-case points to the wrong matrix, to a successful one.

I have a matrix with 6 testcases, so the url should be present 6 times in the file, but it's only 2 times.

i guess some testcases gets the wrong matrix assigned

If you need the files then i can send them over to you via slack for example

[bootstraponline]

Thanks! I'll take a look soon.

[bootstraponline]

Debugged. Root cause is FTL does not produce XML files for all failed matrices. Flank currently detects failures by parsing the XML. If a failed matrix doesn't produce an XML then they'll be missing from the Flank XML report.

Moving to API based results fetching may solve the issue. However the FTL API is currently incomplete which blocks this work.
#545