super() and property inheritance behavior

[ghost]

BPO	14965
Nosy	@rhettinger, @ncoghlan, @tiran, @merwok, @alex, @asvetlov, @durban, @Bluehorn, @ethanfurman, @kynan, @RonnyPfannschmidt, @habnabit, @kenodegard, @willrazen, @SwooshyCueb
PRs	bpo-14965: Proxy super().x = y and del super().x #26194 bpo-14965: Proxy super().x = y and del super().x (updated) #29950
Files	super_setattr.patch: setattr and delattr for super 44560_44559.diff: Alternate patch, targeted at Python 2.7 superprop.py duper.py: Python workaround

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-05-31.03:04:00.947>
labels = ['interpreter-core', 'type-bug', '3.11']
title = 'super() and property inheritance behavior'
updated_at = <Date 2021-12-09.19:32:05.455>
user = None

bugs.python.org fields:

activity = <Date 2021-12-09.19:32:05.455>
actor = 'Ronny.Pfannschmidt'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2012-05-31.03:04:00.947>
creator = '\xe7\x8c\xab.\xe9\xbb\x92'
dependencies = []
files = ['25820', '26966', '37546', '50057']
hgrepos = []
issue_num = 14965
keywords = ['patch']
message_count = 21.0
messages = ['161980', '162050', '162283', '168895', '174404', '174863', '174911', '179217', '232438', '233127', '275855', '391838', '393838', '394071', '394074', '394094', '394657', '408070', '408071', '408072', '408155']
nosy_count = 25.0
nosy_names = ['rhettinger', 'ncoghlan', 'christian.heimes', 'habnabit', 'eric.araujo', 'alex', 'cvrebert', 'asvetlov', 'THRlWiTi', 'daniel.urban', 'dabeaz', 'torsten', 'ethan.furman', 'kynan', 'josmiley', 'Ronny.Pfannschmidt', 'piotr.dobrogost', '\xe7\x8c\xab.\xe9\xbb\x92', 'simonzack', 'jcasale', 'Aaron Gallagher', 'kenodegard', 'Victor Milovanov', 'willrazen', 'SwooshyCueb']
pr_nums = ['26194', '29950']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14965'
versions = ['Python 3.11']

[ghost]

super() objects allow access to inherited properties fget() but not fset() or fdel(), resulting in unexpected behavior.

Today on pydev thread 'Property inheritance in Python' GvR said "I
don't see the need for a Python-Ideas detour. It seems worth fixing"

>>> class BaseProp(object):
...     @property
...     def p(self):
...         return self._p
...     @p.setter
...     def p(self, value):
...         self._p = value
>>> class DerivedProp(BaseProp):
...     @property
...     def p(self):
...         return super(DerivedProp, self).p * 2
...     @p.setter
...     def p(self, value):
...         super(DerivedProp, self).p = value / 2
>>> d = DerivedProp()
>>> d._p = 21
>>> d.p
42
>>> d.p = 50
Traceback (most recent call last):
   ...
AttributeError: 'super' object has no attribute 'p'

[josmiley]

>>> class DerivedProp(BaseProp):
...     @property
...     def p(self):
...         return super(DerivedProp, self).p * 2
...     @p.setter
...     def p(self, value):
...         BaseProp.p.__set__(self,value / 2)

[durban]

I'm attaching a patch implementing super.__setattr__ (and __delattr__).

The implementation in the patch only works, if super can find a data descriptor in the MRO, otherwise it throws an AttributeError. As it can be seen in the tests, in some cases this may result in counter-intuitive behaviour. But I wasn't able to find another behaviour, that is consistent with both super.__getattr__ and normal __setattr__ semantics.

[Bluehorn]

I stumbled across this omission as well in 2010 and brought this up on python-dev:

http://mail.python.org/pipermail/python-dev/2010-April/099672.html

There were no replies, but perhaps my post adds a bit of information and also there is another patch linked from there. I attached my patch from 2010 for reference.

[asvetlov]

I'm -0 for proposed changes, these changes reduce code readability from my perspective.
I think better to use existing approach: explicitly specify what do you want to do with overloaded properties.

[ghost]

I'm not a python dev, but would you say

super(self.__class__, self.__class__).x.fset(self, value)

is more readable than

super().x = value

[asvetlov]

I would say

@x.deleter
def x(self):
    del super().x

confuses me a bit.
But I'm only -0, let's see other developers for their opinions.

[dabeaz]

Just as a note, there is a distinct possibility that a "property" in a superclass could be some other kind of descriptor object that's not a property. To handle that case, the solution of

super(self.__class__, self.__class__).x.fset(self, value)

would actually have to be rewritten as

super(self.__class__, self.__class__).x.__set__(self, value)

That said, I agree it would be nice to have a simplified means of accomplishing this.

[simonzack]

+1 to this feature, this will dramatically simplify property setting code.

[simonzack]

For those who want to use this right away, I've added a python implementation of the patch, which passes the unit tests. There's a slight difference in usage, where instead of using super() directly, super_prop(super()) needs to be used, so we can still use super without arguments.

[tiran]

I had to add a workaround to ssl.SSLContext and would appreciate a better solution.

[VictorMilovanov]

There's a patch attached to this bug. Why is its stage "needs patch"?

[habnabit]

@daniel.urban I'm attempting to move this patch along, but since the contributing process has changed in the years since your patch, you'll need to sign the CLA. Are you interested in picking this back up at all? I haven't been given any indication of how to proceed if I'm doing this on your behalf, but hopefully the core team will enlighten us.

[willrazen]

@simonzack Your superprop.py doesn't work for multiple inheritance, because you're using __thisclass__.__mro__ in each step instead of the initial object mro

[durban]

@habnabit I believe I've already signed some contributor form some years ago. If there is a new one, I can sign that one too.

[willrazen]

Fixed superprop.py workaround, now works with multiple inheritance and follows mro adequately. Renamed to duper.py as inspired by Torsten. Uploading here and also to https://gist.github.com/willrazen/bef3fcb26a83dffb6692e5e10d3e67ac

[habnabit]

@daniel.urban would you kindly resubmit your patch as a PR to the cpython repo? I've learned out-of-band from someone else that putting patches on bpo is considered obsolete. you can use the PR I've submitted (#26194) and reset the author.

I'd be happy to do it myself (giving you a branch that's all set up, so all you need to do is click the 'new PR' button) if you tell me what to set the author to.

[rhettinger]

-0 from me as well. I don't think this is common or something that should be encouraged. As Andrew points out, "del super().x" doesn't have an obvious meaning and it could be regarded as a code smell.

The OP's first example would be an unpleasant API to debug -- it exhibits tight coupling between the parent and child class, it has Liskov issues, and it has implicit forwarding and indirection through descriptors. The tight coupling is especially problematic because Python's super() isn't guaranteed to call the parent class; rather, it can call a sibling class as determined by the MRO which cannot be known at the time the class is written.

Another thought is that super() is intentionally not a completely transparent proxy. While an explicit call super().__getitem__(k) works, we've denied support for super()[k]. To me, "super().x = 10" and "del super().x" fall in the same category.

Looking at the OP's

Fortunately, it doesn't seem to be a common need to use super() in a property setter or deleter to bypass the current class and call setter or deleter in a parent class property. Arguably, this kind of tight coupling isn't good design. The OP's first example would be an unpleasant API to debug.

FWIW,

[rhettinger]

Another thought: Given that this tracker issue has been open for a decade without resolution, we have evidence that this isn't an important problem in practice.

Arguably, people have been better off being nudged in another direction toward better design or having been forced to be explicit about what method is called and when.

[habnabit]

I will note, Raymond, that I’ve wanted this for years before discovering
this bpo issue, and I found it because you linked it on Twitter. ;)

On Wed, Dec 8, 2021 at 19:08 Raymond Hettinger <[email protected]>
wrote:

Raymond Hettinger <[email protected]> added the comment:

Another thought: Given that this tracker issue has been open for a decade
without resolution, we have evidence that this isn't an important problem
in practice.

Arguably, people have been better off being nudged in another direction
toward better design or having been forced to be explicit about what method
is called and when.

----------

---

Python tracker <[email protected]>
<https://bugs.python.org/issue14965\>

---

[RonnyPfannschmidt]

im on the noisy list because i faced this first in 2012

a key problem where i ran into this was mixins, - depending on whether a mixin was added or not one would get errors or not

from my pov a super object should look like the "next class"

so properties of the next base should behave as such, special methods as well

[JulienCochuyt]

@dabeaz wrote:

would actually have to be rewritten as

super(self.class, self.class).x.set(self, value)

To avoid infinite recursion in Python 3, it would have to be rewritten as:
super(DerivedClass, type(self)).x.__set__(self, value)

I agree del super().x would look quite odd, but IMHO super().x = value would look quite obvious.