Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GenConverter's unstructure RecursionError with class with forwardRef #159

Closed
johnwlockwood opened this issue Jul 20, 2021 · 3 comments
Closed

GenConverter's unstructure RecursionError with class with forwardRef #159

johnwlockwood opened this issue Jul 20, 2021 · 3 comments

Comments

@johnwlockwood
Copy link

johnwlockwood commented Jul 20, 2021 •
edited
Loading

  • cattrs version: 1.4.0
  • Python version: 3.9.1
  • Operating System: Mac OS Big Sur

Description

After something that worked with cattrs 1.3.0, raises a RecursionError with 1.4.0.
With an attrs class defined with forward referencing annotations, GenConverter's unstructure exceeds it's recursion depth.
I expected unstructure to turn an instance of a class into a dict.

What I Did

from __future__ import annotations
from typing import Optional, List
import attr
from cattr import GenConverter
unstructure = GenConverter().unstructure

@attr.s
class Node(object):
  name: str = attr.ib()
  nodes: List[Node] = attr.ib(default=attr.Factory(list))
 

n = Node("Bob", nodes=[Node("Jim")])

>>> d = unstructure(n)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/converters.py", line 153, in unstructure
    return self._unstructure_func.dispatch(
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/dispatch.py", line 47, in _dispatch
    return self._function_dispatch.dispatch(cl)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/dispatch.py", line 121, in dispatch
    return handler(typ)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/converters.py", line 645, in gen_unstructure_attrs_fromdict
    h = make_dict_unstructure_fn(
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/gen.py", line 46, in make_dict_unstructure_fn
    handler = converter._unstructure_func.dispatch(a.type)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/dispatch.py", line 47, in _dispatch
    return self._function_dispatch.dispatch(cl)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/dispatch.py", line 121, in dispatch
    return handler(typ)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/converters.py", line 675, in gen_unstructure_iterable
    h = make_iterable_unstructure_fn(
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/gen.py", line 214, in make_iterable_unstructure_fn
    handler = converter._unstructure_func.dispatch(type_arg)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/dispatch.py", line 47, in _dispatch
    return self._function_dispatch.dispatch(cl)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/dispatch.py", line 121, in dispatch
    return handler(typ)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/converters.py", line 645, in gen_unstructure_attrs_fromdict
    h = make_dict_unstructure_fn(
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/gen.py", line 32, in make_dict_unstructure_fn
    attrs = adapted_fields(cl)  # type: ignore
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/cattr/_compat.py", line 80, in adapted_fields
    return attrs_fields(type)
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/attr/_make.py", line 1763, in fields
    if not isclass(cls):
  File "/Users/john/.pyenv/versions/br3/lib/python3.9/site-packages/attr/_compat.py", line 118, in isclass
    return isinstance(klass, type)
RecursionError: maximum recursion depth exceeded while calling a Python object
@Tinche
Copy link
Member

Tinche commented Jul 20, 2021

Hm, confirmed. Will see about a fix.

@Tinche
Copy link
Member

Tinche commented Jul 24, 2021

Hello, this is fixed on master. Could you try your use case there and see if it works?

@johnwlockwood
Copy link
Author

Yes, it works! Thank you.

mergify bot pushed a commit to aws/jsii that referenced this issue Jan 6, 2022
@dependabot
chore(deps): update cattrs requirement from <1.10,>=1.8 to >=1.8,<1.1…
e37272d 
…1 in /packages/@jsii/python-runtime (#3315)

Updates the requirements on [cattrs](https://github.com/python-attrs/cattrs) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/cattrs/blob/main/HISTORY.rst">cattrs's changelog</a>.</em></p>
<blockquote>
<h2>1.10.0 (2022-01-04)</h2>
<ul>
<li>Add PEP 563 (string annotations) support for dataclasses.
(<code>[#195](python-attrs/cattrs#195) &lt;https://github.com/python-attrs/cattrs/issues/195&gt;</code>_)</li>
<li>Fix handling of dictionaries with string Enum keys for bson, orjson, and tomlkit.</li>
<li>Rename the <code>cattr.gen.make_dict_unstructure_fn.omit_if_default</code> parameter to <code>_cattrs_omit_if_default</code>, for consistency. The <code>omit_if_default</code> parameters to <code>GenConverter</code> and <code>override</code> are unchanged.</li>
<li>Following the changes in <code>attrs</code> 21.3.0, add a <code>cattrs</code> package mirroring the existing <code>cattr</code> package. Both package names may be used as desired, and the <code>cattr</code> package isn't going away.</li>
</ul>
<h2>1.9.0 (2021-12-06)</h2>
<ul>
<li>Python 3.10 support, including support for the new union syntax (<code>A | B</code> vs <code>Union[A, B]</code>).</li>
<li>The <code>GenConverter</code> can now properly structure generic classes with generic collection fields.
(<code>[#149](python-attrs/cattrs#149) &lt;https://github.com/python-attrs/cattrs/issues/149&gt;</code>_)</li>
<li><code>omit=True</code> now also affects generated structuring functions.
(<code>[#166](python-attrs/cattrs#166) &lt;https://github.com/python-attrs/cattrs/issues/166&gt;</code>_)</li>
<li><code>cattr.gen.{make_dict_structure_fn, make_dict_unstructure_fn}</code> now resolve type annotations automatically when PEP 563 is used.
(<code>[#169](python-attrs/cattrs#169) &lt;https://github.com/python-attrs/cattrs/issues/169&gt;</code>_)</li>
<li>Protocols are now unstructured as their runtime types.
(<code>[#177](python-attrs/cattrs#177) &lt;https://github.com/python-attrs/cattrs/pull/177&gt;</code>_)</li>
<li>Fix an issue generating structuring functions with renaming and <code>_cattrs_forbid_extra_keys=True</code>.
(<code>[#190](python-attrs/cattrs#190) &lt;https://github.com/python-attrs/cattrs/issues/190&gt;</code>_)</li>
</ul>
<h2>1.8.0 (2021-08-13)</h2>
<ul>
<li>Fix <code>GenConverter</code> mapping structuring for unannotated dicts on Python 3.8.
(<code>[#151](python-attrs/cattrs#151) &lt;https://github.com/python-attrs/cattrs/issues/151&gt;</code>_)</li>
<li>The source code for generated un/structuring functions is stored in the <code>linecache</code> cache, which enables more informative stack traces when un/structuring errors happen using the <code>GenConverter</code>. This behavior can optionally be disabled to save memory.</li>
<li>Support using the attr converter callback during structure.
By default, this is a method of last resort, but it can be elevated to the default by setting <code>prefer_attrib_converters=True</code> on <code>Converter</code> or <code>GenConverter</code>.
(<code>[#138](python-attrs/cattrs#138) &lt;https://github.com/python-attrs/cattrs/issues/138&gt;</code>_)</li>
<li>Fix structuring recursive classes.
(<code>[#159](python-attrs/cattrs#159) &lt;https://github.com/python-attrs/cattrs/issues/159&gt;</code>_)</li>
<li>Converters now support un/structuring hook factories. This is the most powerful and complex venue for customizing un/structuring. This had previously been an internal feature.</li>
<li>The <code>Common Usage Examples &lt;https://cattrs.readthedocs.io/en/latest/usage.html#using-factory-hooks&gt;</code>_ documentation page now has a section on advanced hook factory usage.</li>
<li><code>cattr.override</code> now supports the <code>omit</code> parameter, which makes <code>cattrs</code> skip the atribute entirely when unstructuring.</li>
<li>The <code>cattr.preconf.bson</code> module is now tested against the <code>bson</code> module bundled with the <code>pymongo</code> package, because that package is much more popular than the standalone PyPI <code>bson</code> package.</li>
</ul>
<h2>1.7.1 (2021-05-28)</h2>
<ul>
<li><code>Literal</code> s are not supported on Python 3.9.0 (supported on 3.9.1 and later), so we skip importing them there.
(<code>[#150](python-attrs/cattrs#150) &lt;https://github.com/python-attrs/cattrs/issues/150&gt;</code>_)</li>
</ul>
<h2>1.7.0 (2021-05-26)</h2>
<ul>
<li><code>cattr.global_converter</code> (which provides <code>cattr.unstructure</code>, <code>cattr.structure</code> etc.) is now an instance of <code>cattr.GenConverter</code>.</li>
<li><code>Literal</code> s are now supported and validated when structuring.</li>
<li>Fix dependency metadata information for <code>attrs</code>.
(<code>[#147](python-attrs/cattrs#147) &lt;https://github.com/python-attrs/cattrs/issues/147&gt;</code>_)</li>
<li>Fix <code>GenConverter</code> mapping structuring for unannotated dicts.
(<code>[#148](python-attrs/cattrs#148) &lt;https://github.com/python-attrs/cattrs/issues/148&gt;</code>_)</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/python-attrs/cattrs/commit/7d3a6ba5e0df942391349e332cb87f6871088015"><code>7d3a6ba</code></a> Bump to 1.10.0</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/22b24c28fbeb2b8ca90d568dc4939bdc98ec5902"><code>22b24c2</code></a> Tin/import cattrs (<a href="https://github-redirect.dependabot.com/python-attrs/cattrs/issues/203">#203</a>)</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/a0e56f43f061c43814d6f938833d1c325ed61525"><code>a0e56f4</code></a> Fix test with 32-bit time_t</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/bc9432e606177fe10aa3d2d11e715970c92526be"><code>bc9432e</code></a> Documentation tweaks</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/6260c58aa185200a08c76ecc99a941a26a93eeb8"><code>6260c58</code></a> Rename gen.make_dict_unstructure_fn.omit_if_default</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/bb4383c2d97aae0e8a01db64f142d07350861a17"><code>bb4383c</code></a> Remove walrus</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/81d7756541129a73b82f076b860d61c550296666"><code>81d7756</code></a> Clean up test</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/de16200c3d02d259d04960ab466f49fcf8fa47a4"><code>de16200</code></a> Fix preconf string Enum keys</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/077c9ea8521372f706346f1f39101db716ca3089"><code>077c9ea</code></a> CI tweak</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/3ad74d4456c19598e7278deef62e3af85eabc99d"><code>3ad74d4</code></a> setup.cfg B gone</li>
<li>Additional commits viewable in <a href="https://github.com/python-attrs/cattrs/compare/v1.8.0...v1.10.0">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johnwlockwood @Tinche