-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade lint infra & add a few more conservative built-in checks #2286
Changes from all commits
2a5dbf2
cce31b8
28d2393
b884f77
d67d3b8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,15 @@ | ||
name: Build | ||
name: Docutils Build | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this churn is warranted -- the workflow will be retired soon (TM), and all that need to care about which is Docutils-only and which is Sphinx already know There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think its really fair to dismiss as mere "churn" a change that clarifies names that are actively confusing, at least to me as a PEP editor (and one quite familiar with the build system) and likely to others as well. In particular here, there's no obvious indication as to whether the I'm not sure how one could argue that being clearer and more descriptive is a bad thing, so given it isn't retired yet, while I welcome your suggestions on revised or alternative names, if you feel it isn't worthwhile to spend time on it the most logical course of action would be to just use it for now, rather than both of us wasting time debating (or reverting) it. |
||
|
||
on: [push, pull_request, workflow_dispatch] | ||
|
||
jobs: | ||
build: | ||
name: Build with Docutils | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
- name: Check out repo | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style -- I never name the checkout action step There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ...Except you did here on the other workflow in this repo, complete with an emoji even 😝 I added it there for consistency with the others, which all had names and had |
||
uses: actions/checkout@v2 | ||
|
||
- name: Set up Python | ||
uses: actions/setup-python@v2 | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,10 @@ | ||
name: Deploy to GitHub Pages | ||
name: Sphinx Build | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd prefer not to do this now. Fixing up everything on preview build + etc is on my to do list, but I want outcome of 676 & backend provider done first. We may entirely delete this if RTD is used, for example. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Again, I don't think we should let the perfect be the enemy of the good and actively reject a change that clarifies something causing active confusion right now (potentially not only for PEP editors, but authors as well), just because we might change it at some point in the future. At least in the context of running on each PR, which is likely how 95% of people are going to actually see it (especially those besides you), calling the whole workflow "Deploy to GitHub pages" when that is actively confusing as that is the one thing it doesn't do (PEP authors might assume, for instance, that it actually deploys their PEPs live somewhere), and also not being clear what build system is involved (one might assume the same one in the I find it difficult to argue that clarifying this, even imperfectly, is not an improvement over the status quo, and if you feel its not worth spending more effort on, the least-effort course of action for all involved is going with the above for now. |
||
|
||
on: [push, pull_request, workflow_dispatch] | ||
|
||
jobs: | ||
deploy-to-pages: | ||
name: Build & deploy to GitHub Pages | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does this need a name? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is clearer, cleaner and more descriptive in the UI, so I don't see how it is a net harm. If you'd like to suggest a different name, feel free of course. |
||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
|
@@ -15,13 +16,13 @@ jobs: | |
- name: 🐍 Set up Python 3 | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: 3 | ||
python-version: '3.x' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "3" is cleaner and works There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I recommend following GitHub's docs and use For example, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The docs you linked to refer to SemVer ranges -- whilst not linked, I believe it means
Sorry, appreciate I didn't mention this originally. A There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Taking a step back, I'm having trouble understanding what justifies spending a bunch of our time and churn researching, debating and reverting this instead of simply using what is in GitHub's official documentation and leaving it at that, rather than taking a risk just to fulfill one person's personal stylistic preference on a single key value. If I'd known it would become a huge deal I wouldn't have dreamed of touching it, and I certainly won't make that mistake again—and I should have known better, from past experience. |
||
cache: "pip" | ||
|
||
- name: 👷 Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip | ||
python -m pip install -r requirements.txt | ||
python -m pip install --upgrade -r requirements.txt | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure I understand this one -- why do we need There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The issue here is we're not installing in to a fresh venv, but rather the existing Python, so some deps could already exist either from being system-installed, and/or injected by default into the env. Including As a sidenote, for repos of Python packages, rather than docs, websites, etc like this one, I keep everything clean by using venvs/conda envs. To note, even when creating a fresh venv with |
||
|
||
- name: 🔧 Build PEPs | ||
run: make pages -j$(nproc) | ||
|
@@ -33,7 +34,7 @@ jobs: | |
- name: 🚀 Deploy to GitHub pages | ||
# This allows CI to build branches for testing | ||
if: github.ref == 'refs/heads/main' | ||
uses: JamesIves/github-pages-deploy-action@4.1.1 | ||
uses: JamesIves/github-pages-deploy-action@v4.2.2 | ||
with: | ||
branch: gh-pages # The branch to deploy to. | ||
folder: build # Synchronise with build.py -> build_directory | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,8 +4,17 @@ on: [push, pull_request, workflow_dispatch] | |
|
||
jobs: | ||
pre-commit: | ||
name: Run pre-commit | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
- uses: actions/setup-python@v2 | ||
- uses: pre-commit/[email protected] | ||
- name: Check out repo | ||
Comment on lines
+7
to
+11
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. as above on we don't need to give everything a descriptive name There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe I missed it somewhere (please do point it out if so), but I can't seem to find where you've explained why it is actively harmful to do so, when it adds descriptiveness and clarity (as well as looks more polished) in the GitHub UI. The only thing I noticed you mention was that simply that you personally don't add it in your own work, which I don't see how has any bearing on blocking me from adding it here in mine, particularly when there are concrete (clarity, descriptiveness and clean UI) reasons to do otherwise and on a workflow I'm responsible for maintaining. I don't personally care for the use of emojis in step names (and more practically, they don't show up at all in my editor), but it would be equally inappropriate for me to propose or request removing them from the workflow you added due that personal preference. |
||
uses: actions/checkout@v2 | ||
|
||
- name: Set up Python 3 | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: '3.x' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Keep this as "3" for consistency with the other one There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry, but I would prefer to follow GitHub's official documentation and avoid any risk of the workflow breaking simply to conform to someone else's personal stylistic preference. See that discussion for more. |
||
|
||
- name: Run pre-commit hooks | ||
uses: pre-commit/[email protected] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,33 +1,77 @@ | ||
# See https://pre-commit.com for more information | ||
# See https://pre-commit.com/hooks.html for more hooks | ||
Comment on lines
+1
to
+2
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These can be removed There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I retained this it is helpful to explicitly link to the documentation of the syntax (first line) and the hooks used (second line), so that those coming across the file can quickly navigate to the canonical source explaining these two aspects. |
||
|
||
minimum_pre_commit_version: '2.8.2' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this needed? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes; it is a recommended best practice that ensures users (and the CI) has a sufficiently up to date version of pre-commit that supports the syntax and semantics used by the config and hooks used, as well as doesn't run into any edge-case bugs and compat issues from older versions. The most recent widely used major addition in this vein was |
||
|
||
default_language_version: | ||
python: python3 | ||
|
||
default_stages: [commit] | ||
Comment on lines
+6
to
+9
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why are these new defaults needed? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The The |
||
|
||
|
||
repos: | ||
# General file checks and fixers | ||
- repo: https://github.com/pre-commit/pre-commit-hooks | ||
rev: v3.4.0 | ||
rev: v4.1.0 | ||
hooks: | ||
- id: mixed-line-ending | ||
name: Normalize mixed line endings | ||
name: "Normalize mixed line endings" | ||
args: [--fix=lf] | ||
- id: fix-byte-order-marker | ||
name: "Remove Unicode BOM" | ||
- id: file-contents-sorter | ||
name: "Sort codespell ignore list" | ||
files: '.codespell/ignore-words.txt' | ||
|
||
- id: check-case-conflict | ||
name: "Check for case conflicts" | ||
- id: check-merge-conflict | ||
name: "Check for merge conflict markers" | ||
- id: check-executables-have-shebangs | ||
name: "Check that executables have shebangs" | ||
- id: check-shebang-scripts-are-executable | ||
name: "Check that shebangs are executable" | ||
|
||
- id: check-vcs-permalinks | ||
name: "Check that VCS links are permalinks" | ||
CAM-Gerlach marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- id: check-ast | ||
name: "Check Python AST" | ||
- id: check-json | ||
name: "Check JSON" | ||
- id: check-toml | ||
name: "Check TOML" | ||
CAM-Gerlach marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- id: check-yaml | ||
name: "Check YAML" | ||
Comment on lines
+20
to
+45
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What justifies all these new checks? The PEPs repo is fundamentally a text repo -- I don't want to see us loosing the wood for the trees. (Personally I'd also e.g. remove the line ending check as There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I took pains to be pretty conservative and only enabled a limited number of built-in checks after careful investigation and testing that were not just style-related and had an articulatable benefit for this repo, not enabling any that required fixing a few existing files in the repo, none that required dealing with any existing false positives throughout all past PEPs (except for the new PEP link check you specifically requested), and not adding any code checkers, linters or formatters (beyond simply verifying valid AST).
The extra checks cost users nothing unless they explicitly choose to install our pre-commit hooks and edit a file the checks validate, and even in that case only add a couple extra seconds total once per commit (and on the CI lint run, which is still the fasted CI of the three by several times over). False positives seem rather unlikely, given none of them I ended up adding produced any on the repo's entire history (I rejected those that did). |
||
|
||
# RST checks | ||
- repo: https://github.com/pre-commit/pygrep-hooks | ||
rev: v1.8.0 | ||
rev: v1.9.0 | ||
hooks: | ||
- id: rst-backticks | ||
name: "Check RST: No single backticks" | ||
- id: rst-inline-touching-normal | ||
name: "Check RST: No backticks touching text" | ||
files: '^pep-\d+\.txt|\.rst$' | ||
types: [text] | ||
- id: rst-directive-colons | ||
name: "Check RST: 2 colons after directives" | ||
files: '^pep-\d+\.txt|\.rst$' | ||
types: [text] | ||
|
||
# Manual codespell check | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we're going to change all the "name" fields from here down, please can we say "Validate the PEP 'X' field" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These names need to be as short as practical while still conveying the key information, since they get printed to the often very limited width of the user's terminal in quick succession, and by general convention, are framed as titles/fragments rather than complete sentences. Therefore, including extraneous and redundant words like "the" is not really necessary here. |
||
- repo: https://github.com/codespell-project/codespell | ||
rev: v2.1.0 | ||
hooks: | ||
- id: codespell | ||
name: Check for common misspellings in text files | ||
name: "Check for common misspellings in text files" | ||
stages: [manual] | ||
|
||
# Local checks for PEP headers and more | ||
- repo: local | ||
hooks: | ||
- id: check-required-fields | ||
name: "Check all PEPs have required fields" | ||
name: "Check PEPs have all required fields" | ||
language: pygrep | ||
entry: '(?-m:^PEP:(?=[\s\S]*\nTitle:)(?=[\s\S]*\nAuthor:)(?=[\s\S]*\nStatus:)(?=[\s\S]*\nType:)(?=[\s\S]*\nContent-Type:)(?=[\s\S]*\nCreated:))' | ||
args: ['--negate', '--multiline'] | ||
|
@@ -41,19 +85,19 @@ repos: | |
files: '^pep-\d+\.(rst|txt)$' | ||
types: [text] | ||
- id: validate-status | ||
name: "Validate PEP Status field" | ||
name: "Validate PEP 'Status' field" | ||
language: pygrep | ||
entry: '^Status:(?:(?! +(Draft|Withdrawn|Rejected|Accepted|Final|Active|Provisional|Deferred|Superseded|April Fool!)$))' | ||
files: '^pep-\d+\.(rst|txt)$' | ||
types: [text] | ||
- id: validate-type | ||
name: "Validate PEP Type field" | ||
name: "Validate PEP 'Type' field" | ||
language: pygrep | ||
entry: '^Type:(?:(?! +(Standards Track|Informational|Process)$))' | ||
files: '^pep-\d+\.(rst|txt)$' | ||
types: [text] | ||
- id: validate-content-type | ||
name: "Validate PEP Content-Type field" | ||
name: "Validate PEP 'Content-Type' field" | ||
language: pygrep | ||
entry: '^Content-Type:(?:(?! +text\/x-rst$))' | ||
files: '^pep-\d+\.(rst|txt)$' | ||
|
@@ -65,19 +109,19 @@ repos: | |
files: '^pep-\d+\.(rst|txt)$' | ||
types: [text] | ||
- id: validate-created | ||
name: "Validate created dates" | ||
name: "Validate PEP 'Created' field" | ||
language: pygrep | ||
entry: '^Created:(?:(?! +([0-2][0-9]|(3[01]))-(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)-(199[0-9]|20[0-9][0-9])( \([^()]+\))?$))' | ||
files: '^pep-\d+\.(rst|txt)$' | ||
types: [text] | ||
- id: validate-python-version | ||
name: "Validate PEP Python-Version field" | ||
name: "Validate PEP 'Python-Version' field" | ||
language: pygrep | ||
entry: '^Python-Version:(?:(?! +( ?[1-9]\.([0-9][0-9]?|x)(\.[1-9][0-9]?)?\??,?)+( \([^()]+\))?$))' | ||
files: '^pep-\d+\.(rst|txt)$' | ||
types: [text] | ||
- id: validate-resolution | ||
name: "Validate PEP Resolution field" | ||
name: "Validate PEP 'Resolution' field" | ||
language: pygrep | ||
entry: '(?<!\n\n)^Resolution: (?:(?!(https:\/\/\S*|:pep:`[a-zA-Z\d \-<>#]*?`)\n))' | ||
args: ['--multiline'] | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
#!/usr/bin/env python3 | ||
# http://legacy.python.org/dev/peps/pep-0465/ | ||
# https://www.python.org/dev/peps/pep-0465/ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Was "legacy" legacy at the point this was written? If so I think we should use a wayback machine link. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nope; I looked up the history on this and it was added during a period when the main site was hosted on the new system but some things, like the PEPs, were still on the old one and in the process of transitioning. This is simply a link to the PEP, so since I was moving the file anyway, it makes sense to use the most up to date link as of now. |
||
# https://gist.github.com/njsmith/9157645 | ||
|
||
# usage: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -308,7 +308,7 @@ References | |
.. [2] tp_call/PyObject_Call calling convention | ||
https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_call | ||
.. [3] Using PY_VECTORCALL_ARGUMENTS_OFFSET in callee | ||
https://github.com/markshannon/cpython/blob/vectorcall-minimal/Objects/classobject.c#L53 | ||
https://github.com/markshannon/cpython/blob/815cc1a30d85cdf2e3d77d21224db7055a1f07cb/Objects/classobject.c#L53 | ||
CAM-Gerlach marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this guaranteed to exist forevermore? I'd prefer to use a wayback machine link over this at the time it was written. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, at least pretty forevermore. GitHub describes them as permanent links and permalinks: A big downside of the Wayback Machine is it's much slower to load than GitHub. And there doesn't appear to be any Wayback Machine link for that page, let alone for when the PEP was written. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the link, sounds reasonable. (I'd probably prefer that authors used tags or similar going forwards, but for "fixing" previous this may be the least worst solution). A There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Tags are mutable and can change or be deleted, and require the user to either find a suitable tag (if one exists), or (if they control the repo) create one, which is far more effort than simply pressing the |
||
.. [4] Argument Clinic | ||
https://docs.python.org/3/howto/clinic.html | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -97,8 +97,8 @@ but also on the value of some class attributes, like the ``BinOp`` | |
example above. The Visitor pattern is insufficiently flexible for | ||
this: it can only select based on the class. | ||
|
||
For a complete example, see | ||
https://github.com/gvanrossum/patma/blob/master/examples/expr.py#L231 | ||
See a `complete example | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you only change the link here. See preference for wayback above. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I normally would have, but it makes the link much longer, "messier" and harder to read if as plain text, both in the source and in the rendered HTML, so I made it a standard inline link to perhaps better preserve the spirit, if not the literal text of the author's intent. I don't think Guido would object to that, but we can ask him if you really think its needed. |
||
<https://github.com/gvanrossum/patma/blob/be5969442d0584005492134c3b24eea408709db2/examples/expr.py#L231>`_. | ||
|
||
Like the Visitor pattern, pattern matching allows for a strict separation | ||
of concerns: specific actions or data processing is independent of the | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not going to comment on each of these individually -- I don't think any of these are required, though? If a PEP author cares enough about ancilliary files (as I believe you intend to do with your metadata one), he can add himself here manually.
I think I'd also prefer these were in an "ancilliary files" section instead of inline if we did do this, but as above I'd prefer we didn't
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only changes (other than adding myself as co-maintainer of the GitHub actions subdir, since I maintain the lint action, and have background experience in this area) were fixing two minor syntax issues, updating it to reflect the move of the confusingly-named/located
scan_ops.py
file to follow PEP 1 like all the others (and make clear its not infra), and adding Brett as owner of the PEP 12 template, as creator and PEP co-owner (which appeared to be a simple oversight).Other than that, all of the other subdirs and various ancillary files were assigned to their PEP author/sponsor—and per the intent, if not the explict letter of PEP 1, PEP authors/sponsors should be responsible for and notified about the files specific to their PEPs (as is their designated role per PEP 1), and that should be clear to others from the GitHub UI when one is modified, unless there is good cause in a particular circumstance to explicitly opt out.
@brettcannon is this okay with you?
Changing the current organization scheme is a much bigger change than this PR and is out of scope here, which just makes a few tweaks to fix a couple apparent oversights of auxiliary files getting out of sync with PEPs—a problem which not keeping the two together would tend to exacerbate.
Just FYI, I can't speak for BrE, but using "he" to refer to persons of unspecified gender (not all PEP authors are men) is considered pretty objectionable (and rather archaic/uncommon) in modern, idiomatic AmE and on this repo; see #855 , #2214 , #2258