Update PandasCompat.py to resolve references

[raybellwaves]

Description

This PR allows the PandasCompat sphinx ext to contain resolved references. For example, you can now add intersphinx mapping to the content of the admonition.

Motivation

I enjoy connecting the PyData communities and this PR allows for more opportunities to use intersphinx mapping to link back to the pandas docs.

History

I first tried this in a previous PR (#15383 (comment)) and commented here (#15383 (comment)) that I may get around to investigating this further. I finally had to time to work on this and made a bit of progress.

Testing

I created a separate repo for this at https://github.com/raybellwaves/compatsphinxext which deploys straight to https://raybellwaves.github.io/compatsphinxext you can see it's working as expected here: https://raybellwaves.github.io/compatsphinxext/compat.html. You should be able to fork that and tinker pretty quickly.

Further work

This could be cleaned up (for example I couldn't get the [source] to display in the admonition as I worked from the latest sphinx todo extension (https://github.com/sphinx-doc/sphinx/blob/master/sphinx/ext/todo.py)). The existing pandas-compat Admonition's could be switched to this if agreed. In addition, the documentation around how to write pandas-compat entries going forward (https://github.com/rapidsai/cudf/blob/branch-24.06/docs/cudf/source/developer_guide/documentation.md#comparing-to-pandas) will also have to be updated.

Longer term the extension could be published and used across RAPIDS libraries where there are differences in compatibility with PyData libraries e.g. pandas, network, scikit-learn to simplify linking to those dos. I'm not sure if I'll have time to work on this though.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[bdice]

A few initial comments. I don't know the internals of Sphinx that well (though I've dug through them before...), so feel free to let me know if I'm tweaking anything unusual.

[vyasr]

/ok to test

[vyasr]

@raybellwaves sorry this got a bit lost in the shuffle. @bdice is off for a couple of days so this may not get done until next week, but I've kicked off tests so that we can at least get a doc build going and make sure it works as expected.

[raybellwaves]

@raybellwaves sorry this got a bit lost in the shuffle. @bdice is off for a couple of days so this may not get done until next week, but I've kicked off tests so that we can at least get a doc build going and make sure it works as expected.

Np. Sorry i'm slow to fix as well. I'll ping when ready for another review.

[raybellwaves]

Ready for another round of reviews. Thanks for the code simplification suggestions @bdice. It still works as expected as shown here: https://raybellwaves.github.io/compatsphinxext/compat.html

A minor difference is the [source] link is now below the admonition (text box). I couldn't work out how to add it into the admonition box. I asked on the sphinx discussion page https://github.com/orgs/sphinx-doc/discussions/12366 but it hasn't been answered. I would say it's minor and only aesthetic and doesn't need to block the PR but of course you can disagree.

[vyasr]

@raybellwaves thanks for coming back to this! Given the timing (we're about to freeze the 24.06 release), I'm going to retarget at 24.08 and merge the latest. Hope you don't mind.

[vyasr]

/ok to test

[vyasr]

@raybellwaves could you fix the style check here? Thanks!

[vyasr]

/ok to test

[bdice]

It looks like docs builds failed in CI. https://github.com/rapidsai/cudf/actions/runs/9291509987/job/25571688171?pr=15704

We'll need to get those fixed before we can merge this. I'm not sure why it failed, but there is a warning:

WARNING: Domain 'PandasCompat::pandascompat' has not implemented a `resolve_any_xref` method [myst.domains]

[raybellwaves]

It looks like docs builds failed in CI. https://github.com/rapidsai/cudf/actions/runs/9291509987/job/25571688171?pr=15704

We'll need to get those fixed before we can merge this. I'm not sure why it failed, but there is a warning:

WARNING: Domain 'PandasCompat::pandascompat' has not implemented a `resolve_any_xref` method [myst.domains]

Not obvious why this appeared. I'll got a minimal pipeline i've been testing this on https://github.com/raybellwaves/compatsphinxext/actions/runs/9342573682/job/25710886035 at it runs fine there.

Could be a sphinx version? I have 7.3.7 in my version (https://github.com/raybellwaves/compatsphinxext/actions/runs/9342573682/job/25710886035#step:5:86) whereas the docs env has 6.2.1 (https://github.com/rapidsai/cudf/actions/runs/9291509987/job/25571688171?pr=15704#step:9:483)

May not be this. Also get 6.2.1 in a fresh install using https://github.com/rapidsai/cudf/blob/branch-24.08/conda/environments/all_cuda-122_arch-x86_64.yaml#L86. Latest conda is at 7.3.7 https://github.com/conda-forge/sphinx-feedstock not sure if another package is causing the sphinx pin.

LibMambaUnsatisfiableError: Encountered problems while solving:
  - package breathe-4.35.0-pyhd8ed1ab_0 requires sphinx >=4.0,<6.0.0a, but none of the providers can be installed

Could not solve for environment specs
The following packages are incompatible
├─ breathe >=4.35.0  is installable with the potential options
│  ├─ breathe 4.35.0 would require
│  │  └─ sphinx >=4.0,<6.0.0a , which can be installed;
│  └─ breathe 4.35.0 would require
│     └─ sphinx >=4.0,<7.0.0a , which can be installed;
└─ sphinx >=7.3.7  is not installable because it conflicts with any installable versions previously reported.

I created conda-forge/breathe-feedstock#60

The warning is similar to jupyter-book/jupyter-book#1226. Following https://myst-parser.readthedocs.io/en/latest/configuration.html#build-warnings I also tried add `suppress_warnings = ["myst. domains"]. I haven't tried building with this yet.

[vyasr]

/ok to test

[bdice]

@raybellwaves I tried to review this but I don't see any changes in the docs. Does this PR affect anything yet, or is it just setup for some future change?

[bdice]

The only difference I see is that the [source] link was moved out of the box:

Before

After

[raybellwaves]

The only difference I see is that the [source] link was moved out of the box:

Before

After

Correct. Follow on PR is #15846. Sorry if I forgot to link it before.

I assume the docs built ok this time. Let me know if I need to follow up on conda-forge/breathe-feedstock#60

[bdice]

Great! That's fine. It looks like the docs built so I'll merge this and we can try it with #15846.

[bdice]

/merge