Serialize console.log/info/warn/error to stdout on CLI runs

[brian-mann]

Currently there's no way to see browser messages in stdout.

We can likely serialize the console.* method calls and send them to the node process.

Users would need a way to opt into this. Perhaps just another namespace DEBUG command like.

DEBUG=cypress:console:* cypress run
DEBUG=cypress:console:log cypress run
DEBUG=cypress:console:info cypress run
DEBUG=cypress:console:warn cypress run
DEBUG=cypress:console:error cypress run

[brian-mann]

Related to #448

[nwshane]

@brian-mann Do you have an estimate of when this will be implemented? It's hard to debug test failures in CI without seeing the browser logs :/

[brian-mann]

Today you can spy on console.log with cy.spy and it will print to the Command Log as an event. Don't have a better option, but this is something we've been actively comping in the Dashboard and are working on the infra to capture + display these along with the command data, network traffic, logs, etc.

Cypress.on('window:before:load', (win) => {
  cy.spy(win.console, "log")
})

[nwshane]

@brian-mann Ah, interesting. But we don't have access to the cypress logs either, do we? Except for in the video, which doesn't allow us to view e.g. detailed JSON output.

[brian-mann]

You have access to screenshots + video.

[lgandecki]

@nwshane I think you misunderstood, you will have access to the video and the console.logs in your CI. It won't be as perfect as watching a video with real-time matched logs, but you should be able to figure out what's going on if you purposefully left the console.logs in to debug the problem.

[arsduo]

Just spent a bunch of time today fighting with a test bug where the ability to throw a console.log into the code and see the output in STDOUT on CI would have been invaluable. Glad to see that this is in progress!

[jeremyjh]

@lgandecki can you explain how to get access to console.logs in CI? Nothing logged by the browser or by cy.log is displayed in the CLI output that I can see.

[chrishenry]

Would love to see this implemented! So much so that if there's anything I can to help build or QA this, let me know!

[folmert]

Any update on this feature? Without it it's really difficult to tell why my tests which pass locally are failing in headless browser on gitlab. Being able to see browser's console errors/warnings in the terminal output would give me a great hint what's wrong in my CI environment.

@jennifer-shehane I tried https://github.com/bahmutov/cypress-failed-log but it's not relevant to this issue, it only prints cypress commands into terminal output, there is nothing about displaying browser's errors.

[flotwig]

I made a Cypress plugin that prints out all console messages from the browser, including both user-created (console.log) messages and browser-created (CORS errors, etc.) messages.

It only supports Chrome so far, as I can't figure out a way to connect to Electron's Chrome Debug Protocol instance from a plugin.

Plugin here: https://github.com/flotwig/cypress-log-to-output

[Pomax]

Chrome only isn't super great for people using Selenium/Phantom/etc., did you test any of those @flotwig?

[flotwig]

@Pomax I haven't tested that, but if your browser accepts --remote-debugging-port, that's all it needs to work.

[x-yuri]

@flotwig Electron apps can be told to produce verbose output in a terminal (stderr) with ELECTRON_ENABLE_LOGGING=true. That includes output from console.log() statements. But for one reason or another it seems like Cypress has it enabled by default (maybe @jennifer-shehane can elaborate on this?). The reason why you don't generally see console.log statements in the terminal is because Cypress cypresses :) suppresses them by default. But you can make them visible with DEBUG=cypress:electron npx cypress .... Although some messages would still be unavailable.

[MaaikeFox]

@brian-mann , could you explain how to implement this please? I added it to a test with no result. We really need to be able to see console errors when a test fails

Cypress.on('window:before:load', (win) => { cy.spy(win.console, "log") })

[MaaikeFox]

I would also love to finally see this implemented in Cypress. It is very important for debugging failed tests, and that seems to me one of the first things a test tool should enable.

[jacob-8]

I've been bouncing around the different Cypress issues all related to the desire to see the browser logs printed into the CI output (issue #448 and others) and I finally found a workable solution which will spy on the console logs and errors and then print all cypress :

Step 1: Spy on console logs and errors by adding this snippet to cypress/support/index.js:

Cypress.on('window:before:load', (win) => {
    cy.spy(win.console, "log")
    cy.spy(win.console, "error")
})

Step 2: Use https://github.com/bahmutov/cypress-failed-log to print to the terminal on failed tests.

1. Run npm install --save-dev cypress-failed-log
2. Add require('cypress-failed-log'); to cypress/support/index.js
3. Add to cypress/plugins/index.js:

module.exports = (on, config) => {
  on('task', {
    failed: require('cypress-failed-log/src/failed')(),
  })
};

@folmert @MaaikeFox - and others, I hope this answers your search for a solution until this is officially integrated.

[pshynin]

how many +1 we need to implement this ? Let all thumbs up ?

[flotwig]

@pshynin There are available workarounds detailed above. If this is a feature you'd like to see in Cypress very soon, please feel free to open a pull request. See our contributing docs for more info.

[archfz]

There is also this package for outputing verbose information in a good looking way to the terminal when tests fail. https://github.com/archfz/cypress-terminal-report It also shows data for request captured with cy.route. @flotwig could this package be added to the plugins list https://docs.cypress.io/plugins/ ?

[dananpambudi]

There is also this package for outputing verbose information in a good looking way to the terminal when tests fail. https://github.com/archfz/cypress-terminal-report It also shows data for request captured with cy.route. @flotwig could this package be added to the plugins list https://docs.cypress.io/plugins/ ?

hi may i know how do u use this plugin?

[archfz]

@dananpambudi I am not sure if I understand your question.

If your asking for installation and setup then you should read the readme of the repository. If your having issues using it create an issue in the repository issue tracker.

If your asking how I personally use it, well I use it quite well to get more clues when cypress tests fail during CI in gitlab.

[dananpambudi]

thanks it works well @archfz

[flotwig]

@flotwig could this package be added to the plugins list docs.cypress.io/plugins ?

@archfz Definitely, go ahead and open a PR in our documentation repo to add it: https://github.com/cypress-io/cypress-documentation/blob/develop/CONTRIBUTING.md#adding-plugins

[JohnnyFun]

@archfz thanks for writing cypress-terminal-report!

Fwiw, I set my defaultTrimLength to 2000, otherwise default of 200 cuts off stack property of logged errors--you might consider bumping up the default for that.

[jennifer-shehane]

Closing this issue as making browser console info available for review during the run is part of Cypress Test Replay. Test Replay allows you to see all console logs, including errors, info and warnings, as they displayed when your tests ran.

If you'd like to log console output to the terminal, there are some great open source plugins available to facilitate this also like https://github.com/flotwig/cypress-log-to-output.

Thanks for all of your feedback on this issue. 🙏🏻