Inconsistent behaviour in wildcard import

[vepadulano]

• Checked for duplicates

Describe the bug

Some classes are imported in the Python session namespace when wildcard import is used, while others aren't:

vpadulan@fedorathinkpad-T550 [~]: python
Python 3.8.7 (default, Jan 20 2021, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ROOT import *
>>> RDataFrame
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'RDataFrame' is not defined
>>> TPyDispatcher
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'TPyDispatcher' is not defined
>>> TGHorizontalFrame
<class cppyy.gbl.TGHorizontalFrame at 0x55f59c990900>
>>> RDF
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'RDF' is not defined
>>> TTree
<class cppyy.gbl.TTree at 0x55f59cac9650>
>>> TTreeReader
<class cppyy.gbl.TTreeReader at 0x55f59ce32910>
>>> 

Expected behavior

We should be able to tell more consistenty which names are available after wildcard import, it is handled here .

To Reproduce

from ROOT import * in a python session

Setup

ROOT master@34fc1dc
Fedora 32
built from source

Additional context

This issue stems from #7436

[guitargeek]

Hi @vepadulano, sorry for the very late reply 😛

I think doing from ROOT import * is quite a bad idea because of the dynamic nature of our Python bindings.

In fact, it doesn't even work when I try it with ROOT master (it just takes forever, the execution never finishes).

So in that sense, we also don't have any inconsistent behavior anymore :)

In fact, maybe we can just make it explicit that this is not supported by raising an exception here?
https://github.com/root-project/root/blob/master/bindings/pyroot/pythonizations/python/ROOT/_facade.py#L182

What do you think?

[vepadulano]

I think doing from ROOT import * is quite a bad idea because of the dynamic nature of our Python bindings.

Indeed it is quite a bad idea in general and I would really like to avoid it 😄 https://docs.python.org/3/tutorial/modules.html#importing-from-a-package

In fact, it doesn't even work when I try it with ROOT master (it just takes forever, the execution never finishes).

This is a bug unfortunately 🥲

What do you think?

Raising the exception is ok, but will break (probably many) existing applications. I am not saying it is a bad thing per se, but maybe we want to soft-break it first with a warning and then break it in a future ROOT release. Need to think about it a bit more, because this will also change whenever we get rid of the facade. But in principle I agree it is the correct way forward

[dpiparo]

We should discuss this matter further. Perhas a soft-break first, à la Python2 deprecation, might be a good starting point.

[guitargeek]

A soft-break would require the effort of fixing the wild card import just to drop it afterwards....

There is a structural problem with Python 3.11 that prevents the wildcard import with the lazy lookup to work. Upstream CPyCppyy dropped that feature even:
wlav/CPyCppyy@64fd890
Quote:

As of py3.11, there is no longer a lookup function pointer in the dict object to replace. Since this feature is not widely advertised, it's simply droped

The dropped SetCppLazyLookup() function is exactly the one we used for from ROOT import *.
https://github.com/root-project/root/blob/master/bindings/pyroot/pythonizations/python/ROOT/_facade.py#L162

@vepadulano, given this new information that it will never work with Python 3.11 onward, maybe we should remove the code path for import ROOT *?

[guitargeek]

A soft-break with deprecation warning could still be useful for users of Python 3.8, 3.9 or 3.10, but I doubt that it's worth it, given that no user even noticed that from ROOT import * stopped working with Python 3.11.

[vepadulano]

I am always wary about dropping support for a feature completely without a deprecation notice, even if that feature was buggy and incomplete. This might be one occasion where we simply can't do otherwise