Tests are way too slow when this preprocessor is used

[vinayakpatil]

Tried a simple button test

1. button.spec.ts - finishes in 4s
2. button.feature - finishes in 40s

button.spec.ts

describe('Button - vanilla', () => {
  it('should launch button page', () => {
    cy.visit('/components/ui/button');
    cy.contains('Examples').click();
    cy.contains('Secondary Buttons').click();
    cy.get('#button-secondary').should('be.visible');
  });
});

button.feature

Feature: Button

  TRQ-XXXXX

  Background:
    Given button page is open

  Scenario: Default Button
    When I click on examples tab
    And I click on secondary-buttons example
    Then I should see secondary button

button.steps.ts

import { Given, When, And, Then } from 'cypress-cucumber-preprocessor/steps';

Given(`button page is open`, () => {
  cy.visit('/components/ui/button');
});

When(`I click on examples tab`, () => {
  cy.contains('Examples').click();
});

And(`I click on secondary-buttons example`, () => {
  cy.contains('Secondary Buttons').click();
});

Then(`I should see secondary button`, () => {
  cy.get('#button-secondary').should('be.visible');
});

[lgandecki]

That's not what we see. Could you please make a reproduction so we could take a look at what's going on?

[vinayakpatil]

@lgandecki Sure, I will setup a repo with the issue today but in the meanwhile you can refer to below plugins setup which I have been using in order to run tests under @nrwl/cypress builder.

const wp = require('@cypress/webpack-preprocessor');
const { getWebpackConfig } = require('@nrwl/cypress/plugins/preprocessor');

function preprocessTypescript(config) {
  if (!config.env.tsConfig) {
    throw new Error('Please provide an absolute path to a tsconfig.json as cypressConfig.env.tsConfig');
  }

  const webpackConfig = getWebpackConfig(config);

  webpackConfig.node = { fs: 'empty', child_process: 'empty', readline: 'empty' };
  webpackConfig.module.rules.push(
    {
      test: /\.feature$/,
      use: [
        {
          loader: 'cypress-cucumber-preprocessor/loader'
        }
      ]
    },
    {
      test: /\.features$/,
      use: [
        {
          loader: 'cypress-cucumber-preprocessor/lib/featuresLoader'
        }
      ]
    }
  );

  return async (...args) =>
    wp({
      webpackOptions: webpackConfig
    })(...args);
}

module.exports = (on, config) => {
  on('file:preprocessor', preprocessTypescript(config));
};

[vinayakpatil]

@lgandecki Did you get a chance to look at this?

I even tried https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example and found the similar results.

One other observation: cypress run is relatively faster but the above numbers which I had posted were for cypress open.

[stahloss]

I'm also finding similar results with our use of cypress-cucumber-preprocessor.

[boredland]

its absurdly slow for us too. do webpack builds happen for each test?

[huantaoliu]

any updates on this? it is indeed very slow when using this plugin

[lgandecki]

Hey guys, a reproduction and more info (operating system, cypress version, is it performance of run/open, etc) would be great as this is not what we (and a significant amount of happy users) see.
Make it easy for us to help :)

[vinayakpatil]

@lgandecki As mentioned in my earlier comment, have found similar results with reference webpack example.
https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example.

Slowness is observed during open mode only.

"cypress": "3.8.3",
"cypress-cucumber-preprocessor": "^2.0.1"

[lgandecki]

Thanks.
I cloned https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example.

changed package.json:

    "cypress": "^4.8.0",
    "cypress-cucumber-preprocessor": "^2.5.0",

changed google.com to github.com in cypress/support/step_definitions/webpack.ts (google was taking forever to load, but that has nothing to do with this plugin)
run
npm install
npx cypress open

clicked WebPack.feature -

clicked All.features -

both worked in an instant. Took a few seconds.

Can you give me some more details? Can you try with the newest versions (for some reasons you had 2.0.1 and 3.8.3). When you saw "slow" how slow is it? 10 x as in the original comment?

[vinayakpatil]

Hi @lgandecki I will retest this with updated version and share their results. Thanks

[rodrigo-vazquez]

Hi! Are there any news on this? We recently migrated from stock Cypress Mocha to cucumber using the preprocessor and the Suite of 10 somewhat long tests we have currently is taking double the time it used to with Mocha.

We're using Cypress 7.5.0 and cypress-cucumber-preprocessor 4.1.2.

Things we already tried...

1. Using or not using Data Tables
2. Using global Steps or Feature Steps
3. Going back versions of both Cypress and the Cucumber Preprocessor
4. Turning the 'numTestsKeptInMemory' to 0

Thanks!!

[badeball]

I did some investigating myself and unfortunately found no performance overhead associated with using the preprocessor (note: using latest version, see #689). I took one of the tests that are auto-generated when opening Cypress for the first time, duplicated and re-wrote it using steps. This can be found at badeball/cypress-cucumber-performance-comparison. Then I ran everything ten times. Below are the results.

	todo.feature	todo.spec.js
1. run	03:04	03:00
2. run	03:02	03:00
3. run	03:06	03:01
4. run	03:03	02:59
5. run	03:05	03:01
6. run	03:04	02:58
7. run	03:02	02:58
8. run	03:02	02:58
9. run	03:02	03:01
10. run	03:02	03:01

As you can see, there was little to no difference.

Unless anyone can show me otherwise, I'm considering this a non-issue. That's not so say that you're not experiencing any issues, but I can't investigate and fix something that I can't observe myself.