[feature request] use tags for example tables for scenarios with more example tables

[jimmy882]

When I have a scenario with more example tables with different tags:

Scenario Outline: Test example
  Given Step with <param1>
  When Step with <param2>
  Then Result is <result>

@smoke
Examples:
| param1 | param2 | result |
| foo    | bar    | foobar |
| 1      | 2      | 3      |

@regression
Examples:
| param1 | param2 | result |
| abc    | def    | abcdef |
| z      | y      | z      |

I would like to run only those examples that are tagged with specific tag, for example @smoke.
This is possible with pure cucumber-js, do they use different tag-expressions?
According to cucumber documentation:
Tags can be placed above the following Gherkin elements:

• Feature
• Scenario
• Scenario Outline
• Examples

I tried to run cucumber-js and it worked, but when running cypress with cypress-cucumber-preprocessor, I get an error: We could not detect any tests in the above file. Write some tests and re-run.

[lgandecki]

Hey @jimmy882 . I believe we use the same tag-expressions, but since we are running things through cypress as basically dynamically-created mocha tests our implementation of how we deal with tags is custom (and a reason for lack of features and problems). I don't have any issues with what you are suggesting, and might get around to it one day, but it's not high on my list of priorities, since none of our clients requested this so far.

I'd love to take a PR though!

[Mark-J-Lawrence]

Hi @lgandecki , I'm wanting to run exactly the same scenario as @jimmy882 to express different outcomes in different environments, but have found the same problem.

I tried to trick it by setting a @all tag on the Scenario Outline and @e2e on the examples, and then set my TAGS env to @all and @e2e, but that didn't work either.

At the moment I'll have to copy the scenario outline and give it a different tag as a workaround

[satyasrikanth]

Looking for the same feature, is there any workaround, please?

[askojoke]

Also need this functionality with tagged examples. any news?

[AntonioGargaro]

Also looking for this functionality, bump.

[asanchezgiraldo]

We are also looking for this solution. Is there a beta version we can use to help test?

[phpworker]

we need that in our company!

[siparsons]

This feature would be great. bump

[giselerosa]

This feature would be very interesting, does anyone have any workarounds?

[kakamorsch]

It would be awesome to have this feature, did anyone thought on a workaround?

[devsrihari4]

this is what exactly I needed for a scenario. Appreciate help on this feature

[frnc07]

Any update on this feature request. It would be very helpful if this feature is implemented

[DesaiHimani]

This will be really helpful. Kindly share if there is any workaround for now.

[dkusakina]

If at some point this feature will be implemented, that would be really great, thank you!!

[091sameer]

We would really appreciate if this feature can be implemented. Kindly share if there is any workaround for now.

[RitikaGhosh15]

Hello.. Any update when will this be merged? Thank you for adding this feature in first place

[lgandecki]

Hello @RitikaGhosh15 , this is an issue not a PR, do you mean any specific PR you would like to be merged? The #557 ?

[RitikaGhosh15]

@lgandecki ohk, i assumed it to be a new feature. Actually i am trying to run tags attached to examples in cypress-cucumber-preprocessor with command but its not considering the example tag and running both the examples

npx cypress open --env env=devint TAGS='@mobile and @reg'

[Sonaryr]

This is exactly the usecase we had for our porject, that's why I made #557 to solve this, and in the mean while live up to the gherkin specs that says it should be possible.

[manojbharadwajZava]

Would be great to have this feature, currently there is no workaround :( If someone has one, please share!

[rkrisztian]

This is quite a huge setback for my project too. :(

[lgandecki]

there is work started here if you guys would like to contribute and help pushing this through the finish line:
#557

[rkrisztian]

Opened PR #607 for the feature files search bug.

I can also work on the examples tag filtering bug if you want...

[dankopetrovic]

@rkrisztian Thanks for looking into this issue. I would really appreciate if you could work on examples tag filtering...

I've set-up an example project (clone from https://github.com/TheBrainFamily/cypress-cucumber-example, with addition of examples with tags):
https://github.com/dankopetrovic/cypress-cucumber-example (credits to @RitikaGhosh15)

@features
Feature: Feature Description

    @desktop
    Scenario Outline: Opens a link <link>
        Given Open page with "<link>"

        @tag1
        Examples:
            | link     |
            | facebook |
            | twitter  |

        @tag2
        Examples:
            | link     |
            | linkedin |

npx cypress-tags run -e TAGS='@tag2'
Using cypress.json configuration:
Spec files:  cypress/integration/**/*.feature
Ignored files:  *.js
No matching tags found

[RitikaGhosh15]

Thank you @rkrisztian @dankopetrovic for looking into this issue.

[jime2003]

@dankopetrovic , OK, I tried https://github.com/TheBrainFamily/cypress-cucumber-example, although it seems a bit too old. Regardless, after updating the preprocessor to the latest version, I tried to check if we can tag examples individually there and filter for those tags:

• CYPRESS_TAGS='@tagS and @tagE2' npm run test => Worked.
• npx cypress run -e TAGS='@tagS and @tagE2' => Worked.

What did not work:

$ DEBUG=true npx cypress-tags run -e TAGS='@tagS and @tagE2'
Using cypress.json configuration:
Spec files:  {cypress/integration/**/*.{feature,cypress/integration/features}}
Ignored files:  *.js
DEBUG: Found glob [ '{cypress/integration/**/*.{feature,cypress/integration/features}}' ]
DEBUG: Found tag expression [ '@tagS and @tagE2' ]
No matching tags found

=> I think the problem is with the cypress-tags script, which seems to fail to find tagged examples (it works for TAGS='@tagS), but your reported errors are different from mine. While I think it makes sense to look into the cypress-tags script and see why I got this error (which I will try), I think it would be still useful to see an example project that lets me reproduce the exact same issue you guys reported, just to be sure we are dealing with the same error.

Edit: I found the problem:

      let testFiles = !Array.isArray(cypressConf.testFiles)
        ? cypressConf.testFiles.split(",")
        : cypressConf.testFiles;

This is just plain wrong when cypressConf.testFiles is set to **/*.{feature,features}. After doing a workaround, I still get a similar error:

No matching tags found

And that is because the tag filtering script does not take examples into account. This one should be an easy fix.

@rkrisztian Thanks for looking into this issue. I would really appreciate if you could work on examples tag filtering...

I've set-up an example project (clone from https://github.com/TheBrainFamily/cypress-cucumber-example, with addition of examples with tags):
https://github.com/dankopetrovic/cypress-cucumber-example (credits to @RitikaGhosh15)

@features
Feature: Feature Description

    @desktop
    Scenario Outline: Opens a link <link>
        Given Open page with "<link>"

        @tag1
        Examples:
            | link     |
            | facebook |
            | twitter  |

        @tag2
        Examples:
            | link     |
            | linkedin |

npx cypress-tags run -e TAGS='@tag2'
Using cypress.json configuration:
Spec files:  cypress/integration/**/*.feature
Ignored files:  *.js
No matching tags found

Hopefully, below command would work where I am trying to switch environment at run time specify in cypress.json. It is not working right now.
npx cypress-tags run -e TAGS='@tag1 and @tag2' --env env=qa --browser chrome --headed"

[rkrisztian]

@RitikaGhosh15 , @dankopetrovic , PR #609 opened.

@jime2003 , I think the env problem should be a separate issue report to keep things focused.

[jime2003]

@RitikaGhosh15 , @dankopetrovic , PR #609 opened.

@jime2003 , I think the env problem should be a separate issue report to keep things focused.

thanks @rkrisztian . I have opened new issue #610

[lgandecki]

🎉 This issue has been resolved in version 4.3.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[majco333]

It's working but screenshots are generated for other scenarious in the same feature file (without tag)...

Cypress version: 8.5.0:
Preprocessor version: 4.3.0:

[badeball]

@majco333, possible fixed by #4. Can you try

$ npm install "git://github.com/badeball/cypress-cucumber-preprocessor#ensure-unique-outline-names"

... and see if it helps?

[rkrisztian]

@badeball , I've been tryint to fix that too, I was totally unaware of your fix, this is my PR: #599

Unfortunately, it's more than just a fix, it's also refactored to a more functional (and with that a cleaner) style.

Edit: Actually, it seems my PR was opened sooner.

[majco333]

@badeball

I try it, but without success...

Input:

Output:

[badeball]

I'm having a little trouble understanding your problem here. It appears that a screenshot is taken from a failing test (hence the "(failed)" in the name), but I can't see any failed tests. Can you provide me with a repository illustrating the issue?

[majco333]

I'm having a little trouble understanding your problem here. It appears that a screenshot is taken from a failing test (hence the "(failed)" in the name), but I can't see any failed tests. Can you provide me with a repository illustrating the issue?

I run it like this (only with critical tag): npx cypress-tags run -e TAGS=@critical

Examples with tag @other are pending - it's ok
but screenshots are generated for these pending examples - it's problem...

[colinrotherham]

Calling .skip() throws inside mocha which means you get a screenshot generated for your skipped tests.

The error could be could be caught inside https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/master/lib/createTestFromScenario.js#L70

But it's logged against this Cypress issue: cypress-io/cypress#14867

[badeball]

I see. It would appear to me as if the above-mentioned branch / fix would resolve the issue. Any further attempt to explain to me that it doesn't should be in the form of a repository I can clone, run easily and observe the problem at hand.

[amitsimu]

I am still not able to set tags at example level and it fails to pick up the particular tags

@tag1
Scenario Outline: Test objective
Given When user is on this page
Then ....
....
@sanity
Examples:
| Type |
| Helicopter |

@regression
Examples:
|Type|
|Aeroplane|

executing this command npx cypress --env TAGS='@sanity and not @regression' --spec 'feature file path'

results in executing both the tags. If there is any other way please let me know
Cypress package version: 10.6.0
Bundled Node Version: 16.14.2
Preprocessor version 4.3.1

Appreciate your kind response @badeball

[badeball]

You're using the old package (ref. #689).

[amitsimu]

@badeball thanks for your response. Is the latest cypress-cucumber-preprocessor package v11.0.0 ? Should i be upgrading to that ?

[badeball]

No, it’s not. You can read latest release on the front page of the repository. And yeah, you should upgrade if you want updates and bug fixes such as what’s referred to here.

[amitsimu]

I have upgraded my tests but doesnt seem to execute the sanity suite

[badeball]

Why would it given that you’re specifying sanity tag? Wasn’t this the goal to begin with?

[amitsimu]

Yes the goal is to execute sanity tests but its executing including regression tags

[badeball]

As I said earlier in the thread - if you think you've found an issue with this, you need to provide me with an example in the form of a repository I can clone, run easily and observe the problem at hand.

[leila-c908]

Thanks for implementing this feature! It's super helpful.