IPython TAB-completion now shows deprecation warnings (pandas)

[jorisvandenbossche]

I know this might be the consequence of fixing #1299, but reporting anyway: while trying out the latest jedi 0.15.1, I see a bunch of deprecation warnings when tab-completing on a pandas Series in IPython (with latest pandas release 0.25.0):

In [1]: import jedi                                                                                                                                                                                                

In [2]: jedi.__version__                                                                                                                                                                                           
Out[2]: '0.15.1'

In [3]: import pandas as pd                                                                                                                                                                                        

In [4]: s = pd.Series([1, 2, 3])                                                                                                                                                                                   

/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: Series.base is deprecated and will be removed in a future version
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: Series.data is deprecated and will be removed in a future version
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: Series.flags is deprecated and will be removed in a future version
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: Series.ftype is deprecated and will be removed in a future version. Use Series.dtype instead.
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: Series.ftypes is deprecated and will be removed in a future version. Use Series.dtype instead.
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: `imag` has be deprecated and will be removed in a future verison
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: Series.itemsize is deprecated and will be removed in a future version
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: 
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#ix-indexer-is-deprecated
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: `real` has be deprecated and will be removed in a future verison
  return_obj = getattr(self._obj, name)
/home/joris/miniconda3/lib/python3.7/site-packages/jedi/evaluate/compiled/access.py:347: FutureWarning: Series.strides is deprecated and will be removed in a future version
  return_obj = getattr(self._obj, name)
In [5]: s.<TAB>

(the warnings end up before the In [5] block in the output, but they are from triggering the TAB)

[jorisvandenbossche]

We had this once before (when IPython started to use Jedi): pandas-dev/pandas#16409 (which links to ipython/ipython#9094, which seems to indicate that this was then fixed in Jedi), and then we also did some change in pandas to workaround it: pandas-dev/pandas#16414

[davidhalter]

This is probably fixable. We should probably catch all warnings when properties are queried.

I mean essentially we catch all exceptions, so why not catch warnings as well. :)

[takluyver]

At the library level, we could really use a way to prevent jedi from trying to access properties, not just silencing warnings. In h5py, there's a .value attribute which loads an entire dataset into memory. This might take a long time, perform a lot of IO, and fill up your memory with data.

[takluyver]

In this particular case, it's a deprecated attribute anyway, so I've just hidden it from completions with a custom __dir__ method. But it would be useful in general to have some way to indicate that a property is too expensive to load speculatively.

[davidhalter]

@takluyver Do you have an idea how we could do that? I assume jedi.Interpreter(...).completions(execute_properties=False) is not what you want, you would probably want something more fine grained that libraries could control?

[takluyver]

Yes, I'd imagine libraries could do something like this:

class Dataset:
    properties_no_auto = ('value',)

    @property
    def value(self):
        ...

This is no longer pressing for our use case - the problematic property is gone in master, and we have a workaround for stable branches. But it might come up for other libraries.

[jorisvandenbossche]

I don't know anything about the implemenation of jedi, so this might be dummy question.
But, why is it needed to execute the properties?

As I understood from the other issues that I linked, the main problem was with "chained" completions no longer working. So eg

object.property.<TAB>

in case the property returns a new object that has properties/methods to be tab completed. And for me the above case is really useful (I also opened one of those issues). But while for the above it is needed to execute object.property to know how to further tab complete it, with a naive view it does not seem to be necessary when tab completing the first property:

object.pro<TAB>

Or is it difficult for Jedi to distinguish those cases?

[davidhalter]

@jorisvandenbossche It would be possible to distinguish those cases, it would just be a bit of work, because it's currently not implemented like that. So if anyone volunteers to do it, I'm happy to help :)

Thanks for this idea! I didn't even think about that, because it doesn't really solve the problem. It will however help a lot of users to not run into this issue anymore, which might just be enough.