Do a proper full test of CLI in CI

[ndelangen]

We've had some regressions in the @storybook/cli package lately.

This is a clear signal we need proper tests that cover this behavior.

I would love for someone with great testing skills to add a script that properly tests the CLI.

I envision we have some basic fixtures of possible projects. Much like how lerna has fixtures for their commands
And then we run the CLI on it, and expect the output to match a snapshot.

[ndelangen]

Looking for someone who wants to pick this up!

[Hypnosphi]

In case of CRA/CRNA templates, we should also check that things work with current version of the boilerplate (see your concern). In that case we can't use snapshots as they will change because of external reasons.

Those compatibility checks should run on master only, by the way. PRs shouldn't fail because of unrelated changes in external tools. So for PRs it should be only fixtures and snapshots.

[Hypnosphi]

Alternatively, we could add create-react[-native]-app as devDependencies and run them before each test on CI. Whenever those packages update, we'll get a PR from greenkeeper bot

[ndelangen]

I like that idea @Hypnosphi

[Hypnosphi]

I'm not sure though that greenkeeper can track non-root package.json

[Hypnosphi]

Unfortunately, not

So maybe we should consider alternatives

[ndelangen]

Greenkeeper may not be able to help us there, but it's not like we're without hope.

We can can implement our own helpers to stay up to date.

[Hypnosphi]

Or we can just use latest versions, but make cli build non-required

[stale]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. We do try to do some housekeeping every once in a while so inactive issues will get closed after 90 days. Thanks!

[Hypnosphi]

Fixed with #1767