Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging of reports with same test scenario doesn't work when output is different #359

Closed
Shendor opened this issue Oct 21, 2024 · 4 comments
Assignees
Labels
Milestone

Comments

@Shendor
Copy link

Shendor commented Oct 21, 2024

Describe the bug
When I supply 2 Cucumber json report files to CluecumberEngine with CluecumberCore.Builder.setGroupPreviousScenarioRuns(true)
it doesn't merge the same test scenarios (rerun) which have test steps with different outputs. In my case, each test step has a log attached so, obviously, it will be different when I rerun, for example, a failed test scenario.

To Reproduce
Steps to reproduce the behavior:

  1. Have 2 Cucumber json reports for the same features but run different time and where test steps have different output
  2. Set the flag CluecumberCore.Builder.setGroupPreviousScenarioRuns(true)
  3. Call CluecumberEngine.generateReports
  4. Open html report and see it treats same test scenarios as different

Expected behavior
Rerun test scenarios with same name of the same features and same test data should be displayed as test scenarios which were rerun (in Rerun Scenarios page)

Attachments
Nothing is displayed in page Rerun Scenarios of the Cluecumber html report

Additional context
Based on the code, I see that ElementMultipleRunsPreProcessor.generateCombinedId method uses this:

 if (step.getOutput() != null) {
                outputs.add(step.getOutput());
            }

which, to my mind, should not be in the resulted ID generation, as you may have the same test scenarios but different log outputs after rerun.
Can we change this logic please?
Possible options:

  1. Remove this piece of code
  2. Wrap it around a configuration, supplied from the CluecumberEngine and its Builder
    What do you think?
@bischoffdev bischoffdev self-assigned this Oct 30, 2024
@bischoffdev
Copy link
Collaborator

Will look into this, thanks for reporting.

@bischoffdev bischoffdev added this to the 3.8.3 milestone Nov 4, 2024
@bischoffdev
Copy link
Collaborator

bischoffdev commented Nov 5, 2024

Ready for release in 3.9.0

@bischoffdev
Copy link
Collaborator

Released

@Shendor
Copy link
Author

Shendor commented Nov 6, 2024

Thanks @bischoffdev, I'll check that version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants