Support stub files semantics (fixes #55)

[tristanlatr]

Fixes #55

The first thing this PR does is to greatly improves our understanding of imports, including relative imports, if beniget is given the module name (or if it can deduce the correct module name from the given filename).

This is required not to confuse any TypeVar Name with the actual imported name from typing.py.

Then… why do we need this after all ? Because some feature of the language in stubs alters the name resolution logic in order to special case some expressions. This special cased expressions should then be treated as deferred annotations (so they can contain forward references without the need to be quoted).

See: https://typing.readthedocs.io/en/latest/guides/writing_stubs.html#language-features

More specifically, I'm using this feature to:

• link all builtins usage from several modules' chains to a single builtins dictionary analyzed from the builtins stubs provided by typeshed
• use the stubs to retrieve type information and provide some level of type inference

[tristanlatr]

Salut Serge,

Cette pull request est prête depuis un bout de temps maintenant. J’aimerai savoir si je peux faire quelque chose pour faire avancer ?
Peut être soumettre le support des imports dans une PR différente pour ça soit plus simple à revoir ?

Merci

[serge-sans-paille]

This PR history is mixing so many things I'm not really encouraged to have a look at it :-)

More important: the whole stub file support is not something I need for pythran, so I'm a bit reluctant to integrate it. Beniget is getting more and more complex and it's not something I enjoy.
Have you tried subclassing it to implement your behavior downstream? What would be needed to do so?

[tristanlatr]

Merci pour ton honnêteté, Serge

More important: the whole stub file support is not something I need for pythran, so I'm a bit reluctant to integrate it.

Je comprends ta réticence.
On the other hand, in the REAME, beniget claims to be a "building block to write static analyzer [...] for Python". In order to accomplish this mission, beniget should support the features the Python language has to offer, this includes stub files. It might even turn out to be useful for pythran as well (because numpy has a bunch of stub files that can be used to retrieve type info).

Have you tried subclassing it to implement your behavior downstream? What would be needed to do so?

It's probably possible to subclass DefUseChains... it will require to override the following methods: __init__, visit_FunctionDef, visit_ClassDef, visit_AnnAssign, visit_Import, visit_ImportFrom, visit_Call. But it's going to be totally coupled with internals and might break at any release. Also I'll need to include and sync all other beniget tests to ensure follow-up updates won't break unexpectedly the core of the def-use chains. So all in all this will be a lot of work...

Beniget is getting more and more complex and it's not something I enjoy.

It is Python that turned out to be more complex that we though in my opinion, beniget complexity is just the reflect of the complexity of the Python language. Look at all the bugs we fixed together in the past two years!
Higher is the complexity, stricter should be the testing process. We currently don't have code coverage in place, this would give us some reassurance. I can submit a PR for that if you like.
Also I'e like to run beniget on a lot of open source code and report the diff of the warnings on each PR, like we do for pydoctor.

This PR history is mixing so many things I'm not really encouraged to have a look at it :-)

Yes very sorry about that. As I suggested, splitting the PR up in two parts would help. Also I should really learn to do proper git-rebases ^^'.

Bonne soirée,