Ability to affect future testflow and structure

[yaronassa]

Split from #873 :

An alternate point of view: I don't know if it'll break a major design decision at the very core of cucumber, but it would be extremely useful if FeforeFeatures, BeforeFeature, BeforeScenario and BeforeStep could mutate their cucumber payloads and affect the behavior of the future testflow.

Among these mutations, a .skip() option would be a good first candidate to implement.

I imagine there're a lot of benefits to the testflow being immutable once it's loaded. Is it an intentional design feature in cucumber?

To clarify:
Once cucumber begins to run (i.e. BeforeFeatures hook fires), the entire future test-flow has already been loaded and sequenced in memory. The features are ordered, and within each of them, the scenario order has been determined (all of them passed the run parameters tag filtering, of course).

It could be very helpful to manipulate this structure in ways that will affect the future test flow. These manipulations include:

Skipping scenarios and features - For example, say my selenium connection timed out on one of the ui tests, and I know from here on I'll just be waisting time trying to setup any other ui test. I would rather be able to skip any future scenario marked with @ui for this test run.

Adding scenarios and features - Some test environments are extremely asynchronous. i.e. some business processes may take several minutes (or more) to complete. A test that initiates an operation can add a validation scenario to the end of the test flow and allow all the other tests to run in between. Current solutions require the tests to be "forced" into the correct order by naming conventions, and the end-of-the-stack tests are ran even if the initiating scenarios failed.

Reordering scenarios and features - There can be reasons for running tests in different order in different situations. If it were possible to manipulate the features payload in BeforeFeatures, the infrastructure implementer could account for environment / launch parameters and manipulate the stack accordingly.

These are just some examples of the top of my head.

[charlierudolph]

Skipping scenarios and features

this is now done via https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/step_definitions.md#skipped-steps

Adding scenarios and features ... A test that initiates an operation can add a validation scenario to the end of the test flow and allow all the other tests to run in between

I think this would be better solved by allowing cucumber to be run in parallel

Reordering scenarios and features

Scenarios should not be dependent on each other so I don't see what reordering is useful for.

[charlierudolph]

Closing due to lack of response. Please reopen if necessary

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.