PyPDF2 2.3.0 from PyPi cannot be imported, missing _codecs directory

[nyanpasu64]

I tried installing https://pypi.org/project/pdfCropMargins/ which depends on PyPDF2. It reports an incorrect error:

Error in pdfCropMargins: No system PyPDF2 Python package
was found.  Reinstall pdfCropMargins via pip or install that
dependency ('pip install pypdf2').

I think the problem is that #1005 introduced files in a _codecs directory, but the packaging script fails to package subdirectory contents into the .tar.gz and .whl published to PyPI (https://pypi.org/project/PyPDF2/2.3.0/).

Another compounding problem is that pdfCropMargins misinterprets the nested ImportError as a direct ImportError from missing PyPDF2, and prints the misleading error that PyPDF2 is missing. I think this is a language flaw of Python's automatic exception propagation, but we're not here to remove exceptions from Python.

Solution: If I understand the problem correctly, you need to properly package the _codecs subdirectory and make a 2.3.1 release.

Environment

Which environment were you using when you encountered the problem?

$ python -m platform
# Linux-5.18.3-zen1-1-zen-x86_64-with-glibc2.35

$ python -c "import PyPDF2;print(PyPDF2.__version__)"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nyanpasu64/code/pdfCropMargins/venv/lib/python3.10/site-packages/PyPDF2/__init__.py", line 10, in <module>
    from ._merger import PdfFileMerger, PdfMerger
  File "/home/nyanpasu64/code/pdfCropMargins/venv/lib/python3.10/site-packages/PyPDF2/_merger.py", line 31, in <module>
    from ._encryption import Encryption
  File "/home/nyanpasu64/code/pdfCropMargins/venv/lib/python3.10/site-packages/PyPDF2/_encryption.py", line 34, in <module>
    from PyPDF2.generic import (
  File "/home/nyanpasu64/code/pdfCropMargins/venv/lib/python3.10/site-packages/PyPDF2/generic.py", line 43, in <module>
    from ._codecs import (  # noqa: rev_encoding
ModuleNotFoundError: No module named 'PyPDF2._codecs'

Code

This is a minimal, complete example that shows the issue:

import PyPDF2

[MartinThoma]

Oh, damn ... give me 10 minutes; I think I know what happened.

[MartinThoma]

I've forgot to add the package. 2.3.1 is now on PyPI

[n1ngu]

@MartinThoma thanks for the superfast fix!

From this issue I understand the release of PyPDF2 is NOT automated and is therefore prone to this kind of human mistakes?

Also the release process does neither involve testing the produced artifact to be published?

Is there any roadmap/issue for discussing/tracking the automation of the release process? Should I open an issue? Is help wanted/needed for this?

[MartinThoma]

the release of PyPDF2 is NOT automated

Correct

therefore prone to this kind of human mistakes

Wrong. That would have happened with an automated release as well.

Also the release process does neither involve testing the produced artifact to be published?

That is correct. Any suggestions on how to do this (in an automated way) are welcome. I could imagine:

1. Build the distribution files ( wheels + source dist)
2. Install it in an venv from those files (in two separate runs)
3. Running the unit tests

That would have prevented this issue... I guess I could add this to my makefile 🤔

Is there any roadmap/issue for discussing/tracking the automation of the release process?

No

Should I open an issue?

Sounds like a good idea 👍

Is help wanted/needed for this?

Yes, that would be nice :-) I don't want to put my PyPI credentials on Github, though.

I think the best option would be to adjust the github test workflow. Instead of running the tests with the local installation, they should run with an installation after installing PyPI. I think the overhead should not be big and I'd rather have longer test runs than such mistakes.

[MartinThoma]

@n1ngu In case you're curious, this was the issue: 6b9f472#diff-fa602a8a75dc9dcc92261bac5f533c2a85e34fcceaff63b3a3a81d9acde2fc52 (in setup.cfg).

We don't have changes there often/regularly, hence it's unlikely that this will happen soon again. Also it's super obvious to detect in most cases and trivial to fix. For this reason it's not high priority for me. But I'd like to approach this :-) (not necessarily by myself - a PR is welcome 😄 )