Missing execution steps statuses if same step is called multiple times in a test

[fahadi-henix]

Hello!
I am not sure I created the bug in the right project, please excuse me if I got it wrong, and please let me know where this issue should be created.

👓 What I see :

I have a cucumber test where I call the same step multiple times :

# language: en
Feature: Add two products to the cart

	Scenario: Add two products to the cart
		Given I am logged in
		And I am on the "Home" page
		When I navigate to category "art"
		And I navigate to product "Affiche encadrée The best is yet to come"
		And I add to cart
		And I navigate to category "art"
		And I navigate to product "Illustration vectorielle Renard"
		And I add to cart
		Then The cart should contain
			| Product | Number | Dimension |
			| Affiche encadrée The best is yet to come | 1 | 40x60cm |
			| Illustration vectorielle Renard | 1 |  |

When I launch the test, using mvn test with this option : -Dcucumber.plugin="html:target/html-report.html,junit:target/report.xml"

I noticed than the output in the xml report is incorrect.
The step "I add to cart" which is called two times in the test, is only displayed once (in this case, all steps are PASSED, and the last call is missing).

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="Cucumber" time="22.914" tests="1" skipped="0" failures="0" errors="0">
<testcase classname="Add two products to the cart" name="Add two products to the cart" time="22.339">
<system-out><![CDATA[
Given I am logged in........................................................passed
And I am on the "Home" page.................................................passed
When I navigate to category "art"...........................................passed
And I navigate to product "Affiche encadrée The best is yet to come"........passed
And I add to cart...........................................................passed
And I navigate to category "art"............................................passed
And I navigate to product "Illustration vectorielle Renard".................passed
Then The cart should contain................................................passed
]]></system-out>
</testcase>
</testsuite>

The HTML report is accurate and does not have this bug.

✅ What did you expect to see?

I expect to see a perfect match between the steps and the output in the xml report :

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="Cucumber" time="22.914" tests="1" skipped="0" failures="0" errors="0">
<testcase classname="Add two products to the cart" name="Add two products to the cart" time="22.339">
<system-out><![CDATA[
Given I am logged in........................................................passed
And I am on the "Home" page.................................................passed
When I navigate to category "art"...........................................passed
And I navigate to product "Affiche encadrée The best is yet to come"........passed
And I add to cart...........................................................passed
And I navigate to category "art"............................................passed
And I navigate to product "Illustration vectorielle Renard".................passed
And I add to cart...........................................................passed
Then The cart should contain................................................passed
]]></system-out>
</testcase>
</testsuite>

📦 Which tool/library version are you using?

I am using cucumber 7.14.0
Bug was seen when launching on both linux and windows environments.
Here is the pom :

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns="http://maven.apache.org/POM/4.0.0"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>prestashoptest</groupId>
    <artifactId>prestashoptest</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <junit.version>5.10.0</junit.version>
        <cucumber.version>7.14.0</cucumber.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-suite</artifactId>
            <version>1.10.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.15.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-text</artifactId>
            <version>1.10.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <encoding>UTF-8</encoding>
                    <source>11</source>
                    <target>11</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
            </plugin>
        </plugins>
    </build>
</project>

🔬 How could we reproduce it?

Simply create a test and call the same step several times, and launch the test using Dcucumber.plugin="html:target/html-report.html,junit:target/report.xml"

📚 Any additional context?

---

This text was originally generated from a template, then edited by hand. You can modify the template here.

[mpkorstanje]

Cheers!

On reflection this probably shouldn't be a LinkedHashMap<String,String> but instead a List<Entry<String, String>>.

https://github.com/cucumber/cucumber-junit-xml-formatter/blob/6422ba2e888a1043a3ca5c2da6a3f77995820ee2/java/src/main/java/io/cucumber/junitxmlformatter/XmlReportWriter.java#L128-L137

Unfortunately I have very little time available at present.

[fahadi-henix]

Hi!

Thanks for the answer and insight, I tried it on my local environment, it seems to work fine.
I wanted to create a PR and a branch to implement the fix, it seems like I don't have enough right to create a new branch.
Do you think you can give me rights so I can implement a PR ?

[mpkorstanje]

You'll have to fork and submit a pull request from that fork.

• https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo
• https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork

[fahadi-henix]

Ok thank you very much for the quick reply!

I added a PR with code
Not sure about the point "I've changed the behaviour of the code" and how to add tests covering this though..

Do you have any idea about the timeline of this PR validation / merging and the cucumber release process ?

[mpkorstanje]

Cheers.

Not sure about the point "I've changed the behaviour of the code" and how to add tests covering this though..

Typically it's good to add tests. In this case, we pull the tests from the CCK, so it would be better to add an example of it there. I'll make an issue for that.

We can skip it for this PR.

Do you have any idea about the timeline of this PR validation / merging and the cucumber release process ?

We've setup our CI to make releases whenever. So probably as soon as the PR is megeged it can be released. Inclusion in Cucumber JVM may take a bit longer (depending on how quickly cucumber/cucumber-jvm#2826 can be resolved).

The biggest bottleneck and most difficult factor to predict is my own availability. Unfortunately I can't make releases from my phone just yet. 😄