Continuous output to console from collector

[HansFalkenberg-Visma]

When a test run hangs or is killed by the CI runner, it would be really nice to have the logs of what has happened. The printer, running after the test has succeeded or failed, cannot provide that. Neither can the built-in video generator.

Maybe it would be simple to add an option to the collector to immediately log to the console?

[archfz]

We could consider an option called printLogsToConsole: "always-continuous". From an initial thought this would require some good amount of refactors in the code.

Can you elaborate more on the use case? I haven't really seen cases of test run hanging (usually it's a bug in cypress), and not sure what you mean by the test being killed by the CI runner.

[HansFalkenberg-Visma]

Sorry, I meant CD runner.

We are running cypress tests as part of the build/deployment pipeline, to verify that an full user workflow is working in the staging environment before automatically deploying to production.

This test is performed with on the deployed frontend in staging, using the actual backend. Unfortunately, the backend is flaky, and sometimes get stuck somewhere in the workflow, never allowing the frontend to achieve the end state tested for. What's worse, it usually takes several minutes to complete, but can sometimes take a couple of hours.

Therefore we are doing retries and have increased cypress command timeouts from their defaults. But to avoid the pipeline getting blocked for whatever reason, the deployment pipeline also has a timeout. The aggregated cypress timeouts may run afoul of the deployment pipeline timeout and the cypress process will be killed.

I hadn't seen a quick way of timing out a spec, but I see now there are some possibilities mentioned at cypress-io/cypress#9009

A per spec timeout may not be good enough though, as we have multiple specs and would want longer timeouts for each one than could comfortably be summed into a super long pipeline timeout for the worst case.

Regardless, timeout configuration and reasoning isn't straight forward. Getting the logs out as the events happen, would allow us to ignore lots of complexity and still always have something to start investigating from, whenever cypress is killed.

[archfz]

Ok I see your situation. But this is an edge case for this plugin. As I said before this would require subtantial change in the plugin. If you want you can post a POC PR.

But I would rather recommend you work on the flakyness and performance of your tests.

[archfz]

I've been thinking and I have implemented a simple solution for this, due to the fact that is only needed for debugging. Please see changelog. Release in 4.1.0