Investigate autosummary integration / interoperation

[byrdie]

So I actually had everything working perfectly with sphinx-codeautolink on my local machine. The problems started as soon as I tried to build the documentation on readthedocs. Unfortunately, I didn't note the versions of all my pip packages before messing with my local machine, and now I can't get sphinx-codeautolink to work on my local machine or readthedocs. I've been messing around with the versions of Sphinx, beautifulsoup, pygments, etc. for hours and I can't seem to get it working again.

Issue

The issue I'm experiencing is that none of the code links are working. I'm using the pydata-sphinx-theme and the jupyter-sphinx extension to run my Python code, and I've set codeautolink_custom_blocks to recognize the jupyter-execute directive provided by jupyter-sphinx.

Since none of the links were working, I set codeautolink_warn_on_missing_inventory and codeautolink_warn_on_failed_resolve to True, and it printed a ton of Inventory missing ... warnings, Could not resolve ... warnings, and Cannot locate modules warnings. Intersphinx is working correctly for all of the normal documentation links.

Steps to reproduce

My conf.py file has the following relevant lines

extensions = [
    'sphinx.ext.napoleon',
    'sphinx.ext.autodoc',
    'sphinx.ext.autosummary',
    'sphinx.ext.intersphinx',
    'sphinx.ext.inheritance_diagram',
    'sphinx.ext.viewcode',
    'sphinxcontrib.bibtex',
    'jupyter_sphinx',
    'sphinx_codeautolink',
]

codeautolink_custom_blocks = {"jupyter-execute": None}
codeautolink_warn_on_missing_inventory = True
codeautolink_warn_on_failed_resolve = True

Here is the log for the readthedocs build with a full description of the installed versions of everything and all the warnings.

[felix-hilden]

Ouh, thanks for submitting! I hope this is not Sphinx messing up their internal APIs :P I'll have a look in the next few days. In the meantime you can try downgrading your build requirements 🙏

[byrdie]

Thanks for your help @felix-hilden, I have been trying to downgrade the versions of all my packages, but I can't yet seem to find a version that works. Can you recommend what versions of Sphinx/beautifulsoup/pygments/etc. to try? I tried looking at your logs from the last CI build, but they were expired.

[byrdie]

Currently, I've downgraded to Sphinx 7.1.0, beautifulsoup 4.8.1, and pygments 2.13

[felix-hilden]

I've last worked on this on Sphinx 5 😄 But I'm back at it just today, so I'll have a look for myself. If you get on before that, you can see our docs build requirements here.

[felix-hilden]

Okay I upgraded our requirements and tried both locally and on RTD, our build is fine. So then we'll have to dig in a bit. I had a look through the logs, and I can see the inventory being read correctly.

• Astropy units seem to not have their dedicated sphinx entries here (although I did not see the inventory directly), and they are in the units.si module instead. Either of these will break links. We have configuration for inventory_map (see here) to relocate inventory entries by hand, if that helps.
• Your Optika gives the resolution errors as you've no doubt seen, I would first check if all the type hints are correct, and that everything is documented in the correct place. For one of my own projects, I've done this to make sure all the docs and inventory line up for imported members.

[byrdie]

Thanks for looking into this, those suggestions are very helpful.

I've been playing around with it more, and it appears that the problem is with autosummary / autodoc, all code examples in .rst files seem to work. Do you know why that would be? I can see from your documentation that you use autodoc to generate code examples, so it seems like you support autodoc right?

[felix-hilden]

Yeah, I've tried to integrate autodoc seamlessly! See the docs (especially the example library) for more. But autosummary - that I don't know, and it's been a long time since I've messed with it. I prefer to write even API docs by hand.

Let me know if you find anything, but in the meantime I'll convert this issue into an autosummary investigation.

[byrdie]

Sounds good, thanks!

It's weird though, autosummary really doesn't do all that much, and it's obvious that codeautolink is finding the examples since it's complaining about them. I wonder what could be going wrong?

[byrdie]

I should also mention that I'm willing to open a PR to fix this if you point me in the right direction. I've looked through the code a bit, but I'm not sure where to start yet.

[felix-hilden]

Oh sure, that'd be nice 👌 I think the first thing to do is to test it. If you're up for it, you could try adding a test that uses autosummary to generate docs and then assert that the links are generated. Here's an example of a test that has expected links, Sphinx setup code and the rST. You can read the test code too to understand what happens better.

If that's too much right away, even building a dummy project that has a little bit of python code and then the documentation and an example would be very helpful to determine if anything works.