Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: strict mode for warnings #367

Closed
wants to merge 1 commit into from
Closed

feat: strict mode for warnings #367

wants to merge 1 commit into from

Conversation

varl
Copy link
Contributor

@varl varl commented Mar 10, 2021

Allow d2-style to print warnings from ESLint and introduce a concept of
strict and non-strict modes.

Strict mode: Set a zero tolerance on warnings by making ESLint exit with
a non-zero code if there are any warnings. Warnings will be shown in
strict mode.

Non-strict mode: Allow ESLint to exit with code zero when there are
warnings. Warnings are essentially ignored in this mode which matches
the behavior today.

This allows gradual introduction of new rules, that may later become
errors.

In strict mode a developer is expected to either:

a) fix the warning
b) explicitly mark the code that triggered the warning with an
eslint-disable comment.

A good disable directive uses the comment syntax for ESLint:

/* eslint-disable [rule] -- [comment] */

E.g.

/* eslint-disable-next-line no-console -- we need to log here */

When rules graduate from "warning" to "error" the code base should be
ready for the change.

@varl varl requested review from ismay and amcgee March 10, 2021 20:48
@varl
Copy link
Contributor Author

varl commented Mar 10, 2021

Supersedes #366.

This is a middle-ground that:

  • Is not a breaking change, the default functionality is the same as today: no warnings are shown on e.g. d2-style js check.
  • Allows warnings to be displayed in strict mode: d2-style js check --strict.
  • Strict mode will exit with a non-zero code if there are warnings.
  • We can add rules as warnings, and they don't impact users until they are upgraded to errors.
  • We can run strict mode on our apps after adding new rules to see how much effort it is going to take if we upgrade a rule to errors and plan accordingly.

Plays nice with: #362, #363, #364, #365

docs/faq.md Show resolved Hide resolved
@HendrikThePendric
Copy link
Contributor

HendrikThePendric commented Mar 11, 2021

My perspective is this:

  • I was happy with how cli-style worked, I didn't really see any need for structural changes
  • I would be very much in favour of introducing plugins that help us catch problems with i18n and a11y
  • But I expect apps and libs are going to have to make a lot of changes when they upgrade to the version of cli-style containing these rules. I fear to the extend it could probably prevent apps/libs from upgrading the package.
  • So I think we need to offer a gentle upgrade path somehow. Showing warnings instead of errors would be one way to achieve this. But we could also choose a completely different route, for example:
    • Custom commands to lint the codebase for a11y or i18n
    • Possibility to switch the i18n and a11y rules on and of (obviously this is possible already, but we could offer some convenience methods).

I feel like with this PR we're trying to implement a generic solution for a specific problem.

Also, I don't really have an opinion about the broader discussion about warnings VS errors, so will refrain from commenting on that.

@varl
Copy link
Contributor Author

varl commented Mar 11, 2021

I was happy with how cli-style worked, I didn't really see any need for structural changes

Me too, so this PR doesn't change anything in terms of usage.

I feel like with this PR we're trying to implement a generic solution for a specific problem.

This PR only provides a way to see rules configured as warnings in the console. Today they are completely ignored with no way of seeing them at all. Maybe strict is a poor choice of name?

I fear to the extend it could probably prevent apps/libs from upgrading the package.

I share this fear, and would like to deal with it elsewhere, so this PR keeps things as they are.

Showing warnings instead of errors would be one way to achieve this. But we could also choose a completely different route, for example:

Nice examples of alternatives! Those could indeed be ways to go about it. Internally those options would require significant structural changes in how commands are organized, as well as how we configure and run the toolchains.

@Mohammer5
Copy link
Contributor

Mohammer5 commented Mar 11, 2021

In terms of what flags to parse to the command, I'd prefer more explicit, less ambiguous flags:

--fail-on-warning (implicitly enables the flag below)
--show-warnings

With the above flags, we would have the current behavior (no flags), the "strict" mode (--fail-on-warning) and a more informative but less consequential mode (--show-warnings), which is more or less the eslint standard

@varl
Copy link
Contributor Author

varl commented Mar 11, 2021

--fail-on-warning (implicitly enables the flag below)
--show-warnings

I guess these would be more acceptable to @HendrikThePendric as well (no structural/conceptual changes)?

@ismay
Copy link
Contributor

ismay commented Mar 11, 2021

This seems similar to an issue that was filed with eslint: eslint/eslint#2309 (--strict for counting warnings as errors). I think it's relevant to our discussion here.

What I was proposing was basically to let eslint do its thing, without letting warnings fail anything, and without swallowing them. The way I understand the proposed strict mode here (which is similar to the proposed strict mode in the issue), is that it would exit with a non-zero exit code on warnings. As the maintainer in the issue there mentioned I also feel that that logic blurs the line between errors and warnings (a warning by definition shouldn't fail anything, then we should just make it an error).

Basically what I meant is just for us to remove the --quiet flag. To me that seems more in line with the eslint ecosystem (and stylelint as well). But I'm not sure if that goes against a requirement that we have, let me know if it does or if I'm misunderstanding something.

edit: Allowing --show-warnings would basically do the same as what I'm talking about. Seems good as well. Though by making it a flag I guess then we'd have to have the discussion where to enable it. I'd just make it the default.

@varl
Copy link
Contributor Author

varl commented Mar 11, 2021

But I'm not sure if that goes against a requirement that we have, let me know if it does or if I'm misunderstanding something.

The way d2-style was designed was to be a gatekeeper. That was its reason for existing, it has no nuance. Either it is good or it is bad.

What it is evolving to become is a developer support tool. And guidance requires more nuance than enforcing, so I see where the need for both errors and warnings in that sense.

To me that seems more in line with the eslint ecosystem (and stylelint as well).

The tools that d2-style used to accomplish its agenda were intended to be implementation details that are completely obscured from the user. This was changed when developers requested to be able to see lint/formatting errors in their editors.

edit: Allowing --show-warnings would basically do the same as what I'm talking about. Seems good as well. Though by making it a flag I guess then we'd have to have the discussion where to enable it. I'd just make it the default.

My intent was for the user to use it manually to check if there are any "soon to be problems", not that it should be enabled anywhere.

@varl
Copy link
Contributor Author

varl commented Mar 11, 2021

I need to finish up my implementation of the d2-style ci command.

That would be my enforcer of good and bad. The other commands could be allowed more nuance to decide what is ugly.

This removes the `--quiet` flag from the ESLint invocation, and fixes
the broken output where ESLint complains about us trying to lint
dot-files and folders by un-ignoring their built-in ignore filters.

We agree with ignoring node_modules in terms of linting however, so that
is left in place.
@varl varl changed the title feat: strict mode for warnings feat: show eslint warnings Mar 12, 2021
@varl varl changed the title feat: show eslint warnings feat: strict mode for warnings Mar 12, 2021
@varl varl closed this Mar 12, 2021
@varl varl deleted the allow-warnings branch March 12, 2021 08:10
@varl varl restored the allow-warnings branch March 12, 2021 08:11
@varl varl mentioned this pull request Mar 12, 2021
@ismay
Copy link
Contributor

ismay commented Mar 15, 2021

But I'm not sure if that goes against a requirement that we have, let me know if it does or if I'm misunderstanding something.

The way d2-style was designed was to be a gatekeeper. That was its reason for existing, it has no nuance. Either it is good or it is bad.

What it is evolving to become is a developer support tool. And guidance requires more nuance than enforcing, so I see where the need for both errors and warnings in that sense.

To me that seems more in line with the eslint ecosystem (and stylelint as well).

The tools that d2-style used to accomplish its agenda were intended to be implementation details that are completely obscured from the user. This was changed when developers requested to be able to see lint/formatting errors in their editors.

edit: Allowing --show-warnings would basically do the same as what I'm talking about. Seems good as well. Though by making it a flag I guess then we'd have to have the discussion where to enable it. I'd just make it the default.

My intent was for the user to use it manually to check if there are any "soon to be problems", not that it should be enabled anywhere.

I see what you mean. Yeah it seems like we both had a slightly different perception of the design goals of cli-style, thanks for explaining!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants