Nested namespace packages support

[dizzy57]

Summary

PEP 420 says nested namespace packages are allowed, i.e. marking a directory as a namespace package marks all subdirectories in the subtree as namespace packages.

is_package is modified to use Path::starts_with and the order of checks is reversed to do in-memory checks first before hitting the disk.

Test Plan

Added unit tests. Previously all tests were run with namespace_packages == &[]. Verified that one of the tests was failing before changing the implementation.

Future Improvements

The is_package_with_cache can probably be rewritten to avoid repeated calls to Path::starts_with, by caching all directories up to the namespace_root:

let namespace_root = namespace_packages
    .iter()
    .filter(|namespace_package| path.starts_with(namespace_package))
    .min();

[dizzy57]

Related: #6114 #5149

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

[charliermarsh]

Did you run into an issue with this in practice that motivated this change?

[dizzy57]

Did you run into an issue with this in practice that motivated this change?

We have a monorepo where all the files are located in an organization/project/subproject/ structure and both organization/ and organization/project/ directories are meant to be namespace packages, so no __init__.py in them.

We wanted to use banned-api (TID251), but for python sources it detected subproject as python package name. We didn't want to mark all the organization/project/ directories as namespace packages (it would be brittle, as more projects might be added without changing the ruff config), but marking just organization/ as a namespace package didn't work.

[dizzy57]

This test illustrates our situation pretty well. Currently ruff assigns package name core to this directory, but we want it to be recognized as python_modules.core.core with only python_modules being marked as namespace package.

https://github.com/dizzy57/ruff/blob/7607b2d64083c797aa334b3f2e767a2a12bf191b/crates/ruff_linter/src/packaging.rs#L111-L117

[charliermarsh]

Okay thanks. I think this is correct.