Incomplete completion with PyQt5

[jumper047]

I have a trouble with Jedi 0.13.2 and PyQt5 - it does not shows completion for parent's methods. Jedi 0.12 works fine. There is a code to illustrate the bug:
Jedi 0.12:

import jedi

source = '''
   ...: from PyQt5.QtWidgets import QWidget
   ...:
   ...: class MyWidget(QWidget):
   ...:
   ...:     def __init__(self):
   ...:         self.'''
script = jedi.Script(source, 7, len("        self."), "example.py")

There are 355 candidates in completion list, i upload whole list on pastebin - here.

Jedi 0.13 with same code gives only 27 candidates - here

[davidhalter]

Can you run this with jedi.set_debug_function() in both versions?

[jumper047]

Jedi 0.12:
`In [1]: import jedi

In [2]: jedi.set_debug_function()

In [3]: source = """
   ...: from PyQt5.QtWidgets import QWidget
   ...:
   ...: class MyWidget(QWidget):
   ...:
   ...:     def __init__(self):
   ...:         self."""

In [4]: script = jedi.Script(source, 7, len("        self."), "bug.py")
speed: init 102.50275778770447
speed: parsed 102.50375866889954

In [5]: len(script.completions())
speed: completions start 20.64350152015686
 dbg: eval_node <Name: self@7,8>@(7, 8)
  dbg: finder.filter_name 'self' in (<AnonymousInstanceFunctionExecution: <Function: __init__@6-7>>): [<AnonymousInstanceParamName: self@(6, 17)>]@(7, 8)
  dbg: finder._names_to_types: [<AnonymousInstanceParamName: self@(6, 17)>] -> ContextSet(<AnonymousInstance of <ClassContext: <Class: MyWidget@4-7>>(<jedi.evaluate.arguments.AnonymousArguments object at 0x0000000005F8E908>)>) dbg: trailer completion contexts: ContextSet(<AnonymousInstance of <ClassContext: <Class: MyWidget@4-7>>(<jedi.evaluate.arguments.AnonymousArguments object at 0x0000000005F8E908>)>)
 dbg: eval_node <Name: QWidget@4,15>@(4, 15)
  dbg: finder.filter_name 'QWidget' in (<ModuleContext: bug@2-7>): [<TreeNameDefinition: QWidget@(2, 28)>]@(4, 15)
  speed: import (<Name: PyQt5@2,5>, <Name: QtWidgets@2,11>) 21.10286855697632
  dbg: search_module 'PyQt5' in 'C:\\Users\\jumper047\\bug.py'
  dbg: search_module 'PyQt5.QtWidgets' in paths ['C:\\Users\\jumper047\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\PyQt5']
   dbg: finder.filter_name 'QWidget' in (<CompiledObject: <module 'PyQt5.QtWidgets' from 'C:\\Users\\jumper047\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\PyQt5\\QtWidgets.pyd'>>): [<CompiledName: (<CompiledContextName: PyQt5.QtWidgets>).QWidget>]@None
   dbg: finder._names_to_types: [<CompiledName: (<CompiledContextName: PyQt5.QtWidgets>).QWidget>] -> ContextSet(<CompiledObject: <sip.wrappertype object at 0x000000000320FF58>>)
  dbg: after import: ContextSet(<CompiledObject: <sip.wrappertype object at 0x000000000320FF58>>)
  dbg: finder._names_to_types: [<TreeNameDefinition: QWidget@(2, 28)>] -> ContextSet(<CompiledObject: <sip.wrappertype object at 0x000000000320FF58>>)
speed: completions end 24.809831857681274
Out[5]: 355`

Jedi 0.13:
In [1]: import jedi

In [2]: jedi.set_debug_function()

In [3]: source = """
   ...: from PyQt5.QtWidgets import QWidget
   ...:
   ...: class MyWidget(QWidget):
   ...:
   ...:     def __init__(self):
   ...:         self."""

In [4]: script = jedi.Script(source, 7, len("        self."), "bug.py")
dbg: Start environment subprocess 'c:\\users\\jumper047\\appdata\\local\\programs\\python\\python36\\python.exe'
sspeed: init 238.09089708328247
speed: parsed 238.09289717674255

In [5]: script.completions()
speed: completions start 14.469446897506714
 dbg: eval_node <Name: self@7,8>@(7, 8)
  dbg: finder.filter_name 'self' in (<FunctionExecutionContext: <Function: __init__@6-7>>): [<ParamName: self@(6, 17)>]@(7, 8)
  dbg: finder._names_to_types: [<ParamName: self@(6, 17)>] -> ContextSet(<AnonymousInstance of <ClassContext: <Class: MyWidget@4-7>>(AnonymousInstanceArguments())>)
dbg: trailer completion contexts: ContextSet(<AnonymousInstance of <ClassContext: <Class: MyWidget@4-7>>(AnonymousInstanceArguments())>)
 dbg: eval_node <Name: QWidget@4,15>@(4, 15)
  dbg: finder.filter_name 'QWidget' in (<ModuleContext: bug@2-7>): [<TreeNameDefinition: QWidget@(2, 28)>]@(4, 15)
  speed: import (<Name: PyQt5@2,5>, <Name: QtWidgets@2,11>) 14.477447748184204
  dbg: global search_module 'PyQt5' in 'C:\\Users\\Jumper047\\bug.py'
  dbg: search_module 'PyQt5.QtWidgets' in paths ['c:\\users\\jumper047\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\PyQt5']
   dbg: finder.filter_name 'QWidget' in (<CompiledObject: <module 'PyQt5.QtWidgets' from 'c:\\users\\jumper047\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\PyQt5\\QtWidgets.pyd'>>): [<CompiledName: (<CompiledContextName: PyQt5.QtWidgets>).QWidget>]@None
   dbg: finder._names_to_types: [<CompiledName: (<CompiledContextName: PyQt5.QtWidgets>).QWidget>] -> ContextSet(<CompiledObject: <sip.wrappertype object at 0x000000000320F698>>)
  dbg: after import: ContextSet(<CompiledObject: <sip.wrappertype object at 0x000000000320F698>>)
  dbg: finder._names_to_types: [<TreeNameDefinition: QWidget@(2, 28)>] -> ContextSet(<CompiledObject: <sip.wrappertype object at 0x000000000320F698>>)
speed: completions end 14.614461421966553
Out[5]:
[<Completion: DrawChildren>,
 <Completion: DrawWindowBackground>,
 <Completion: IgnoreMask>,
 <Completion: PaintDeviceMetric>,
 <Completion: PdmDepth>,
 <Completion: PdmDevicePixelRatio>,
 <Completion: PdmDevicePixelRatioScaled>,
 <Completion: PdmDpiX>,
 <Completion: PdmDpiY>,
 <Completion: PdmHeight>,
 <Completion: PdmHeightMM>,
 <Completion: PdmNumColors>,
 <Completion: PdmPhysicalDpiX>,
 <Completion: PdmPhysicalDpiY>,
 <Completion: PdmWidth>,
 <Completion: PdmWidthMM>,
 <Completion: RenderFlag>,
 <Completion: RenderFlags>,
 <Completion: __class__>,
 <Completion: __delattr__>,
 <Completion: __dict__>,
 <Completion: __dir__>,
 <Completion: __doc__>,
 <Completion: __eq__>,
 <Completion: __format__>,
 <Completion: __ge__>,
 <Completion: __init__>]

[davidhalter]

Sorry, this is probably an issue with how PyQT5 describes its objects. Extension module completion can only be so good, because we don't have all the information. If you want to make this better, please try. But I won't. It's just not worth it. The answer for this is creating good stubs.

So please wait for the issue #839 to be fixed.

And then you can use https://github.com/stlehmann/PyQt5-stubs.

[davidhalter]

Until this has been fixed, adding PyQT5 to `jedi.settings.auto_import_modules might help. Not sure if it does.

[jumper047]

Still no completion after adding. But if this is PyQt's issue why it working fine with previous version of jedi?

[davidhalter]

I have no idea :) There might be issues with a lot of different things. If you didn't try for a while PyQT5 might have changed. Also auto_import_modules might not work properly. There might have been changes between 0.12 and 0.13 that made loading extensions more secure. There might have been changes that broke something.

I really don't know and I'd be very grateful if you started to investigate. The point of investigation is usually evaluate.compiled/access.py and if you want to print things (because it runs in a subprocess now), use Script(..., environment=jedi.api.environment.jedi.api.environment.InterpreterEnvironment(). (This is what IPython uses. Or use jedi.Interpreter()).

[davidhalter]

BTW I just thought a bit about this and realized that we introduced getattr_static (in jedi.evaluate.compiled.getattr_static), which might have had an effect - but I feel like that was commited before 0.12.0 was released.

Also does completion on QWidget. work? Because if that doesn't work we have to work on that.

[jumper047]

With this script in jedi 0.13 completion contains all methods :/.
In [2]: script = '''
...: from PyQt5.QtWidgets import QWidget
...:
...: QWidget.'''

[davidhalter]

Hmm it might have something to do with loading get_filters, but this is really not high on my priority list.

[jumper047]

Tested it again on 14.1 and it works - now completion is same as in 12.1 version.
I also tries to install/uninstall PyQt5-stubs package in pypi, and there was no effect on completion - it remains full.

[davidhalter]

So I guess we're done here. :) I feel like I know now which bug caused this.