Feature request: support Cython and Jupyter magic?

[9r0x]

Hi,

I was wondering if we could consider Cython support and/or Jupiter magic because

1. Cython code sometimes exists alongside Python code in a project, and a unified linter would be very useful?
2. Many code and lint inside Jupyter notebook, and % in the code cell could break the lining, would be great if it can be ignored/supported.

Thanks

[charliermarsh]

Cython I don't have much experience with so can't commit to anything on that front right now.

Jupyter integration though is very interesting to me and something I'd like to support!

[MarcoGorelli]

Hey,

I don't know how to use rust, but I've done some relevant work here on the Python side

For Cython, you might be interested in https://github.com/MarcoGorelli/cython-lint (which does everything flake8 used to, plus more) and seeing if that's portable to rust?

For Jupyter, I contributed its support to black, there's this file that deals with masking/unmasking magic symbols (%, %%, !, ?) - if possible, I'd strongly suggest something like that over regular expressions

[charliermarsh]

Amazing, thank you @MarcoGorelli -- that's super helpful. I'll probably just port that file to Rust then.

[charliermarsh]

Ruff was actually integrated into nbQA in the latest release, so you can now run (e.g.):

❯ nbqa ruff Untitled.ipynb
Untitled.ipynb:cell_1:2:5: F841 Local variable `x` is assigned to but never used
Untitled.ipynb:cell_2:1:1: E402 Module level import not at top of file
Untitled.ipynb:cell_2:1:8: F401 `os` imported but unused
Found 3 error(s).
1 potentially fixable with the --fix option.

Super cool :)

(Would still like to have a first-party integration at some point.)

[dhruvmanila]

(2) is being worked on (see #5030)

[vyasr]

@charliermarsh On the Cython side it's worth noting that isort supports Cython as of isort 5.0.0 so you may also be able to port those features over to rust (in the same way that you're hoping to port @MarcoGorelli's excellent cython-lint project). Between cython-lint and isort support I would probably consider that enough to claim Cython support, especially since flake8's support has always been patchy and incidental (prior to the change last year that disabled it entirely, necessitating flake8-force) and I believe the same is true for pylint. Just in case you were curious where to set the goalposts for this issue 😃

This might also be a strong argument for #283, though, and in particular supporting Python plugins alongside Rust plugins. Cython is a prime example of a case where the ruff developers may not have the expertise or interest in maintaining their own solution, but could happily delegate to a plugin author who doesn't have Rust experience but is willing to write one in Python.

Great work on ruff, by the way! Definitely a fan of ruff.

[MichaReiser]

@dhruvmanila do we support this now that we support ipython magics?

[dhruvmanila]

@dhruvmanila do we support this now that we support ipython magics?

@MichaReiser Yes, we support (2) but not (1). I'm not familiar with Cython so can't say what will be required to support it.

[MichaReiser]

@9r0x could you help us understand what features you would expect from a linter supporting cpython? Is it Cython specific lint rules or is it something else?

[charliermarsh]

Cython is a superset of Python. So at minimum it would require a different parser. Lets close this and create a new issue to track Cython, since there are really two independent requests here.

[MichaReiser]

Done, #10250