docs: use lychee to check link URLs (#3941)

* guide: install `mdbook-linkcheck`

* use `shutil` to copy license files

* move from `mdbook-linkcheck` to `lychee`

* clean guide & doc build products before build

* fix more broken links

* review: mejrs

.github/workflows/gh-pages.yml

@@ -26,21 +26,21 @@ jobs:
       - uses: dtolnay/rust-toolchain@nightly
 
       - name: Setup mdBook
-        uses: peaceiris/actions-mdbook@v1
+        uses: taiki-e/install-action@v2
         with:
-          mdbook-version: "0.4.19"
+          tool: mdbook,lychee
 
       - name: Prepare tag
         id: prepare_tag
         run: |
           TAG_NAME="${GITHUB_REF##*/}"
           echo "::set-output name=tag_name::${TAG_NAME}"
 
-      # This builds the book in target/guide.
+      # This builds the book in target/guide/.
       - name: Build the guide
         run: |
           python -m pip install --upgrade pip && pip install nox
-          nox -s build-guide
+          nox -s check-guide
         env:
           PYO3_VERSION_TAG: ${{ steps.prepare_tag.outputs.tag_name }}
 

.gitignore

@@ -14,6 +14,8 @@ dist/
 .eggs/
 venv*
 guide/book/
+guide/src/LICENSE-APACHE
+guide/src/LICENSE-MIT
 *.so
 *.out
 *.egg-info

.netlify/build.sh

@@ -71,9 +71,17 @@ if [ "${INSTALLED_MDBOOK_VERSION}" != "mdbook v${MDBOOK_VERSION}" ]; then
     cargo install mdbook@${MDBOOK_VERSION} --force
 fi
 
+# Install latest mdbook-linkcheck. Netlify will cache the cargo bin dir, so this will
+# only build mdbook-linkcheck if needed.
+MDBOOK_LINKCHECK_VERSION=$(cargo search mdbook-linkcheck --limit 1 | head -1 | tr -s ' ' | cut -d ' ' -f 3 | tr -d '"')
+INSTALLED_MDBOOK_LINKCHECK_VERSION=$(mdbook-linkcheck --version || echo "none")
+if [ "${INSTALLED_MDBOOK_LINKCHECK_VERSION}" != "mdbook v${MDBOOK_LINKCHECK_VERSION}" ]; then
+    cargo install mdbook-linkcheck@${MDBOOK_LINKCHECK_VERSION} --force
+fi
+
 pip install nox
 nox -s build-guide
-mv target/guide netlify_build/main/
+mv target/guide/ netlify_build/main/
 
 ## Build public docs
 

CHANGELOG.md

@@ -1687,7 +1687,7 @@ Yanked
 [0.9.2]: https://github.com/pyo3/pyo3/compare/v0.9.1...v0.9.2
 [0.9.1]: https://github.com/pyo3/pyo3/compare/v0.9.0...v0.9.1
 [0.9.0]: https://github.com/pyo3/pyo3/compare/v0.8.5...v0.9.0
-[0.8.4]: https://github.com/pyo3/pyo3/compare/v0.8.4...v0.8.5
+[0.8.5]: https://github.com/pyo3/pyo3/compare/v0.8.4...v0.8.5
 [0.8.4]: https://github.com/pyo3/pyo3/compare/v0.8.3...v0.8.4
 [0.8.3]: https://github.com/pyo3/pyo3/compare/v0.8.2...v0.8.3
 [0.8.2]: https://github.com/pyo3/pyo3/compare/v0.8.1...v0.8.2
@@ -1696,7 +1696,8 @@ Yanked
 [0.7.0]: https://github.com/pyo3/pyo3/compare/v0.6.0...v0.7.0
 [0.6.0]: https://github.com/pyo3/pyo3/compare/v0.5.3...v0.6.0
 [0.5.3]: https://github.com/pyo3/pyo3/compare/v0.5.2...v0.5.3
-[0.5.2]: https://github.com/pyo3/pyo3/compare/v0.5.0...v0.5.2
+[0.5.2]: https://github.com/pyo3/pyo3/compare/v0.5.1...v0.5.2
+[0.5.1]: https://github.com/pyo3/pyo3/compare/v0.5.0...v0.5.1
 [0.5.0]: https://github.com/pyo3/pyo3/compare/v0.4.1...v0.5.0
 [0.4.1]: https://github.com/pyo3/pyo3/compare/v0.4.0...v0.4.1
 [0.4.0]: https://github.com/pyo3/pyo3/compare/v0.3.2...v0.4.0

Contributing.md

@@ -70,6 +70,12 @@ First, install [`mdbook`][mdbook] and [`nox`][nox]. Then, run
 nox -s build-guide -- --open
 ```
 
+To check all links in the guide are valid, also install [`lychee`][lychee] and use the `check-guide` session instead:
+
+```shell
+nox -s check-guide
+```
+
 ### Help design the next PyO3
 
 Issues which don't yet have a clear solution use the [needs-design](https://github.com/PyO3/pyo3/issues?q=is%3Aissue+is%3Aopen+label%3Aneeds-design) label.
@@ -171,7 +177,7 @@ First, there are Rust-based benchmarks located in the `pyo3-benches` subdirector
 
     nox -s bench
 
-Second, there is a Python-based benchmark contained in the `pytests` subdirectory. You can read more about it [here](pytests).
+Second, there is a Python-based benchmark contained in the `pytests` subdirectory. You can read more about it [here](https://github.com/PyO3/pyo3/tree/main/pytests).
 
 ## Code coverage
 
@@ -211,4 +217,5 @@ In the meanwhile, some of our maintainers have personal GitHub sponsorship pages
 - [messense](https://github.com/sponsors/messense)
 
 [mdbook]: https://rust-lang.github.io/mdBook/cli/index.html
+[lychee]: https://github.com/lycheeverse/lychee
 [nox]: https://github.com/theacodes/nox

guide/book.toml

@@ -9,4 +9,4 @@ command = "python3 guide/pyo3_version.py"
 [output.html]
 git-repository-url = "https://github.com/PyO3/pyo3/tree/main/guide"
 edit-url-template = "https://github.com/PyO3/pyo3/edit/main/guide/{path}"
-playground.runnable = false
+playground.runnable = false

guide/pyclass_parameters.md

@@ -33,7 +33,7 @@ struct MyClass {}
 struct MyClass {}
 ```
 
-[params-1]: https://docs.rs/pyo3/latest/pyo3/struct.PyAny.html
+[params-1]: https://docs.rs/pyo3/latest/pyo3/types/struct.PyAny.html
 [params-2]: https://en.wikipedia.org/wiki/Free_list
 [params-3]: https://doc.rust-lang.org/std/marker/trait.Send.html
 [params-4]: https://doc.rust-lang.org/std/rc/struct.Rc.html

guide/src/async-await.md

@@ -38,7 +38,7 @@ However, there is an exception for method receiver, so async methods can accept
 
 Even if it is not possible to pass a `py: Python<'_>` parameter to `async fn`, the GIL is still held during the execution of the future – it's also the case for regular `fn` without `Python<'_>`/`&PyAny` parameter, yet the GIL is held.
 
-It is still possible to get a `Python` marker using [`Python::with_gil`]({{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.with_gil); because `with_gil` is reentrant and optimized, the cost will be negligible.
+It is still possible to get a `Python` marker using [`Python::with_gil`]({{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.with_gil); because `with_gil` is reentrant and optimized, the cost will be negligible.
 
 ## Release the GIL across `.await`
 
@@ -70,7 +70,7 @@ where
 
 ## Cancellation
 
-Cancellation on the Python side can be caught using [`CancelHandle`]({{#PYO3_DOCS_URL}}/pyo3/coroutine/struct.CancelHandle.html) type, by annotating a function parameter with `#[pyo3(cancel_handle)].
+Cancellation on the Python side can be caught using [`CancelHandle`]({{#PYO3_DOCS_URL}}/pyo3/coroutine/struct.CancelHandle.html) type, by annotating a function parameter with `#[pyo3(cancel_handle)]`.
 
 ```rust
 # #![allow(dead_code)]

guide/src/building_and_distribution.md

@@ -4,7 +4,7 @@ This chapter of the guide goes into detail on how to build and distribute projec
 
 The material in this chapter is intended for users who have already read the PyO3 [README](./index.md). It covers in turn the choices that can be made for Python modules and for Rust binaries. There is also a section at the end about cross-compiling projects using PyO3.
 
-There is an additional sub-chapter dedicated to [supporting multiple Python versions](./building_and_distribution/multiple_python_versions.html).
+There is an additional sub-chapter dedicated to [supporting multiple Python versions](./building_and_distribution/multiple_python_versions.md).
 
 ## Configuring the Python version
 
@@ -177,7 +177,7 @@ The downside of not linking to `libpython` is that binaries, tests, and examples
 
 By default, Python extension modules can only be used with the same Python version they were compiled against. For example, an extension module built for Python 3.5 can't be imported in Python 3.8. [PEP 384](https://www.python.org/dev/peps/pep-0384/) introduced the idea of the limited Python API, which would have a stable ABI enabling extension modules built with it to be used against multiple Python versions. This is also known as `abi3`.
 
-The advantage of building extension modules using the limited Python API is that package vendors only need to build and distribute a single copy (for each OS / architecture), and users can install it on all Python versions from the [minimum version](#minimum-python-version-for-abi3) and up. The downside of this is that PyO3 can't use optimizations which rely on being compiled against a known exact Python version. It's up to you to decide whether this matters for your extension module. It's also possible to design your extension module such that you can distribute `abi3` wheels but allow users compiling from source to benefit from additional optimizations - see the [support for multiple python versions](./building_and_distribution/multiple_python_versions.html) section of this guide, in particular the `#[cfg(Py_LIMITED_API)]` flag.
+The advantage of building extension modules using the limited Python API is that package vendors only need to build and distribute a single copy (for each OS / architecture), and users can install it on all Python versions from the [minimum version](#minimum-python-version-for-abi3) and up. The downside of this is that PyO3 can't use optimizations which rely on being compiled against a known exact Python version. It's up to you to decide whether this matters for your extension module. It's also possible to design your extension module such that you can distribute `abi3` wheels but allow users compiling from source to benefit from additional optimizations - see the [support for multiple python versions](./building_and_distribution/multiple_python_versions.md) section of this guide, in particular the `#[cfg(Py_LIMITED_API)]` flag.
 
 There are three steps involved in making use of `abi3` when building Python packages as wheels:
 
@@ -198,7 +198,7 @@ See the [corresponding](https://github.com/PyO3/maturin/pull/353) [PRs](https://
 Because a single `abi3` wheel can be used with many different Python versions, PyO3 has feature flags `abi3-py37`, `abi3-py38`, `abi3-py39` etc. to set the minimum required Python version for your `abi3` wheel.
 For example, if you set the `abi3-py37` feature, your extension wheel can be used on all Python 3 versions from Python 3.7 and up. `maturin` and `setuptools-rust` will give the wheel a name like `my-extension-1.0-cp37-abi3-manylinux2020_x86_64.whl`.
 
-As your extension module may be run with multiple different Python versions you may occasionally find you need to check the Python version at runtime to customize behavior. See [the relevant section of this guide](./building_and_distribution/multiple_python_versions.html#checking-the-python-version-at-runtime) on supporting multiple Python versions at runtime.
+As your extension module may be run with multiple different Python versions you may occasionally find you need to check the Python version at runtime to customize behavior. See [the relevant section of this guide](./building_and_distribution/multiple_python_versions.md#checking-the-python-version-at-runtime) on supporting multiple Python versions at runtime.
 
 PyO3 is only able to link your extension module to abi3 version up to and including your host Python version. E.g., if you set `abi3-py38` and try to compile the crate with a host of Python 3.7, the build will fail.
 
@@ -232,7 +232,7 @@ not work when compiling for `abi3`. These are:
 
 If you want to embed the Python interpreter inside a Rust program, there are two modes in which this can be done: dynamically and statically. We'll cover each of these modes in the following sections. Each of them affect how you must distribute your program. Instead of learning how to do this yourself, you might want to consider using a project like [PyOxidizer] to ship your application and all of its dependencies in a single file.
 
-PyO3 automatically switches between the two linking modes depending on whether the Python distribution you have configured PyO3 to use ([see above](#python-version)) contains a shared library or a static library. The static library is most often seen in Python distributions compiled from source without the `--enable-shared` configuration option. For example, this is the default for `pyenv` on macOS.
+PyO3 automatically switches between the two linking modes depending on whether the Python distribution you have configured PyO3 to use ([see above](#configuring-the-python-version)) contains a shared library or a static library. The static library is most often seen in Python distributions compiled from source without the `--enable-shared` configuration option. For example, this is the default for `pyenv` on macOS.
 
 ### Dynamically embedding the Python interpreter
 
@@ -242,7 +242,7 @@ This mode of embedding works well for Rust tests which need access to the Python
 
 For distributing your program to non-technical users, you will have to consider including the Python shared library in your distribution as well as setting up wrapper scripts to set the right environment variables (such as `LD_LIBRARY_PATH` on UNIX, or `PATH` on Windows).
 
-Note that PyPy cannot be embedded in Rust (or any other software). Support for this is tracked on the [PyPy issue tracker](https://foss.heptapod.net/pypy/pypy/-/issues/3286).
+Note that PyPy cannot be embedded in Rust (or any other software). Support for this is tracked on the [PyPy issue tracker](https://github.com/pypy/pypy/issues/3836).
 
 ### Statically embedding the Python interpreter
 
@@ -285,7 +285,7 @@ Thanks to Rust's great cross-compilation support, cross-compiling using PyO3 is
 * A toolchain for your target.
 * The appropriate options in your Cargo `.config` for the platform you're targeting and the toolchain you are using.
 * A Python interpreter that's already been compiled for your target (optional when building "abi3" extension modules).
-* A Python interpreter that is built for your host and available through the `PATH` or setting the [`PYO3_PYTHON`](#python-version) variable (optional when building "abi3" extension modules).
+* A Python interpreter that is built for your host and available through the `PATH` or setting the [`PYO3_PYTHON`](#configuring-the-python-version) variable (optional when building "abi3" extension modules).
 
 After you've obtained the above, you can build a cross-compiled PyO3 module by using Cargo's `--target` flag. PyO3's build script will detect that you are attempting a cross-compile based on your host machine and the desired target.
 

guide/src/building_and_distribution/multiple_python_versions.md

@@ -85,7 +85,7 @@ This `#[cfg]` marks code which is running on PyPy.
 
 ## Checking the Python version at runtime
 
-When building with PyO3's `abi3` feature, your extension module will be compiled against a specific [minimum version](../building_and_distribution.html#minimum-python-version-for-abi3) of Python, but may be running on newer Python versions.
+When building with PyO3's `abi3` feature, your extension module will be compiled against a specific [minimum version](../building_and_distribution.md#minimum-python-version-for-abi3) of Python, but may be running on newer Python versions.
 
 For example with PyO3's `abi3-py38` feature, your extension will be compiled as if it were for Python 3.8. If you were using `pyo3-build-config`, `#[cfg(Py_3_8)]` would be present. Your user could freely install and run your abi3 extension on Python 3.9.
 
@@ -104,5 +104,5 @@ Python::with_gil(|py| {
 });
 ```
 
-[`Python::version()`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.version
-[`Python::version_info()`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.version_info
+[`Python::version()`]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.version
+[`Python::version_info()`]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.version_info

guide/src/class.md

@@ -16,7 +16,7 @@ This chapter will discuss the functionality and configuration these attributes o
   - [`#[classmethod]`](#class-methods)
   - [`#[classattr]`](#class-attributes)
   - [`#[args]`](#method-arguments)
-- [Magic methods and slots](class/protocols.html)
+- [Magic methods and slots](class/protocols.md)
 - [Classes as function arguments](#classes-as-function-arguments)
 
 ## Defining a new class
@@ -82,7 +82,7 @@ When you need to share ownership of data between Python and Rust, instead of usi
 
 A Rust `struct Foo<T>` with a generic parameter `T` generates new compiled implementations each time it is used with a different concrete type for `T`. These new implementations are generated by the compiler at each usage site. This is incompatible with wrapping `Foo` in Python, where there needs to be a single compiled implementation of `Foo` which is integrated with the Python interpreter.
 
-Currently, the best alternative is to write a macro which expands to a new #[pyclass] for each instantiation you want:
+Currently, the best alternative is to write a macro which expands to a new `#[pyclass]` for each instantiation you want:
 
 ```rust
 # #![allow(dead_code)]
@@ -898,9 +898,7 @@ py_args=('World', 666), py_kwargs=Some({'x': 44, 'y': 55}), name=Hello, num=44,
 py_args=(), py_kwargs=None, name=World, num=-1, num_before=44
 ```
 
-## Making class method signatures available to Python
-
-The [`text_signature = "..."`](./function.md#text_signature) option for `#[pyfunction]` also works for `#[pymethods]`:
+The [`#[pyo3(text_signature = "...")`](./function/signature.md#overriding-the-generated-signature) option for `#[pyfunction]` also works for `#[pymethods]`.
 
 ```rust
 # #![allow(dead_code)]
@@ -1002,7 +1000,7 @@ impl MyClass {
 Note that `text_signature` on `#[new]` is not compatible with compilation in
 `abi3` mode until Python 3.10 or greater.
 
-## #[pyclass] enums
+## `#[pyclass]` enums
 
 Enum support in PyO3 comes in two flavors, depending on what kind of variants the enum has: simple and complex.
 

guide/src/class/numeric.md

@@ -440,6 +440,6 @@ fn wrap(obj: &Bound<'_, PyAny>) -> Result<i32, PyErr> {
 ```
 
 [`PyErr::take`]: {{#PYO3_DOCS_URL}}/pyo3/prelude/struct.PyErr.html#method.take
-[`Python`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html
+[`Python`]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html
 [`FromPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.FromPyObject.html
 [`pyo3::ffi::PyLong_AsUnsignedLongMask`]: {{#PYO3_DOCS_URL}}/pyo3/ffi/fn.PyLong_AsUnsignedLongMask.html

guide/src/class/protocols.md

@@ -450,7 +450,7 @@ Usually, an implementation of `__traverse__` should do nothing but calls to `vis
 Most importantly, safe access to the GIL is prohibited inside implementations of `__traverse__`,
 i.e. `Python::with_gil` will panic.
 
-> Note: these methods are part of the C API, PyPy does not necessarily honor them. If you are building for PyPy you should measure memory consumption to make sure you do not have runaway memory growth. See [this issue on the PyPy bug tracker](https://foss.heptapod.net/pypy/pypy/-/issues/3899).
+> Note: these methods are part of the C API, PyPy does not necessarily honor them. If you are building for PyPy you should measure memory consumption to make sure you do not have runaway memory growth. See [this issue on the PyPy bug tracker](https://github.com/pypy/pypy/issues/3848).
 
 [`IterNextOutput`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass/enum.IterNextOutput.html
 [`PySequence`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PySequence.html

guide/src/conversions/tables.md

@@ -43,7 +43,7 @@ The table below contains the Python type and the corresponding function argument
 | `typing.Sequence[T]` | `Vec<T>`                 | `&PySequence`        |
 | `typing.Mapping[K, V]` | `HashMap<K, V>`, `BTreeMap<K, V>`, `hashbrown::HashMap<K, V>`[^3], `indexmap::IndexMap<K, V>`[^4] | `&PyMapping` |
 | `typing.Iterator[Any]` | -                      | `&PyIterator`        |
-| `typing.Union[...]` | See [`#[derive(FromPyObject)]`](traits.html#deriving-a-hrefhttpsdocsrspyo3latestpyo3conversiontraitfrompyobjecthtmlfrompyobjecta-for-enums) | - |
+| `typing.Union[...]` | See [`#[derive(FromPyObject)]`](traits.md#deriving-frompyobject-for-enums) | - |
 
 There are also a few special types related to the GIL and Rust-defined `#[pyclass]`es which may come in useful: