Infinite Recursion With PyPDF 4.1.0 in xml2rfc

[kitterma]

Replace this: What happened? What were you trying to achieve?

Environment

$ python -m platform
Linux-6.1.0-18-amd64-x86_64-with-glibc2.37

$ python -c "import pypdf;print(pypdf._debug_versions)"
4.1.0

Code + PDF

I don't have one. It's in the xml2rfc tests. It's not clear if this is an xml2rfc issue that was highlighted by a change in 4.1.0 or a pypdf regression. If needed, I can try to be more specific.

Traceback

This is an extract from the traceback I see (the line 21, line 21, line 29 patter repeats hundreds of times):

115s autopkgtest [09:16:38]: test run-pytest: [-----------------------
116s Testing with python3.12:
135s ..E..............................................
135s ======================================================================
135s ERROR: setUpClass (__main__.PdfWriterTests)
135s ----------------------------------------------------------------------
135s Traceback (most recent call last):
135s   File "/tmp/autopkgtest-lxc.gvxbgwgc/downtmp/build.QrX/src/xxx/test.py", line 496, in setUpClass
135s     cls.elements_pdfxml = xmldoc(None, bytes=elements_pdfdoc)
135s                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
135s   File "/usr/lib/python3/dist-packages/xml2rfc/walkpdf.py", line 96, in xmldoc
135s     text = xmltext(filename=filename, bytes=bytes)
135s            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
135s   File "/usr/lib/python3/dist-packages/xml2rfc/walkpdf.py", line 89, in xmltext
135s     obj = pyobj(filename=filename, bytes=bytes)
135s           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
135s   File "/usr/lib/python3/dist-packages/xml2rfc/walkpdf.py", line 77, in pyobj
135s     d, i = walk(obj, seen)
135s            ^^^^^^^^^^^^^^^
135s   File "/usr/lib/python3/dist-packages/xml2rfc/walkpdf.py", line 21, in walk
135s     d, i = walk(obj[key], seen)
135s            ^^^^^^^^^^^^^^^^^^^^
135s   File "/usr/lib/python3/dist-packages/xml2rfc/walkpdf.py", line 21, in walk
135s     d, i = walk(obj[key], seen)
135s            ^^^^^^^^^^^^^^^^^^^^
135s   File "/usr/lib/python3/dist-packages/xml2rfc/walkpdf.py", line 29, in walk
The last three lines repeat until you get to the recursion limit:
135s   File "/usr/lib/python3/dist-packages/pypdf/generic/_base.py", line 301, in __getattr__
135s     return getattr(self._get_object_with_check(), name)
135s                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
135s   File "/usr/lib/python3/dist-packages/pypdf/generic/_base.py", line 290, in _get_object_with_check
135s     o = self.get_object()
135s         ^^^^^^^^^^^^^^^^^
135s RecursionError: maximum recursion depth exceeded
135s 
135s ----------------------------------------------------------------------
135s Ran 48 tests in 18.098s

Related xml2rfc issue:

ietf-tools/xml2rfc#1111

[stefan6419846]

This is most likely related to #2464, although I am not sure why. The actual recursion issue seems to be on the xml2rfc side, PDF generation is done by WeasyPrint.

For future reference, I have attached a corresponding PDF file: file.pdf

[pubpub-zz]

the xml2rfc code uses a "bad trick" to identify DictionnaryObject using hasattr(obj, 'keys') : with #2464, functions of referenced object by IndirectObject are directly available in it.
proposed fix in walkpdf.py:

def walk(obj, seen):
    dobj = {}                            # Direct objects
    iobj = []                            # Indirect objects
    if isinstance(obj, pypdf.generic.DictionaryObject):    #<-----
    (...)

This is not an issue in pypdf

[kitterma]

On March 8, 2024 8:27:52 PM UTC, pubpub-zz ***@***.***> wrote: the xml2rfc code uses a "bad trick" to identify DictionnaryObject using `hasattr(obj, 'keys')` : with #2464, functions of referenced object by IndirectObject are directly available in it. proposed fix in walkpdf.py: ``` def walk(obj, seen): dobj = {} # Direct objects iobj = [] # Indirect objects if isinstance(obj, pypdf.generic.DictionaryObject): #<----- (...) ``` This is not an issue in pypdf

Thanks for checking. I have a related issue open with xml2rfc and I'll follow up with them. Scott K

[kitterma]

Now tested and this does resolve the issue.