Skip to content
This repository has been archived by the owner on Apr 14, 2022. It is now read-only.

Improve handling of circular dependencies #1686

Merged
merged 55 commits into from
Nov 4, 2019
Merged

Improve handling of circular dependencies #1686

merged 55 commits into from
Nov 4, 2019

Conversation

MikhailArkhipov
Copy link

@MikhailArkhipov MikhailArkhipov commented Oct 14, 2019 •
edited
Loading

Continuation of #1584 by @AlexanderSher
Requires #1739

AlexanderSher and others added 30 commits September 22, 2019 00:05
@AlexanderSher
Refactoring
c9ff4e1
@AlexanderSher
WIP
6ab46ad
@AlexanderSher
Merge commit '88762ef1a11cfddd51d382c920359c36e95f8d4d' into Numpy
76887fc 
# Conflicts:
#	src/Analysis/Ast/Impl/Analyzer/PythonAnalyzerSession.cs
#	src/Analysis/Ast/Impl/Dependencies/DependencyResolver.cs
@AlexanderSher
Bug fixes
1068b18
@AlexanderSher
Fix part of the test
03f004d
@AlexanderSher
Fully restore tests
43c7766
Remove stale reference
c6a9c22
Initial
cbfd3e2
Merge branch 'master' of https://github.com/microsoft/python-language…
1360827 
…-server
Merge issues
e5f0110
Merge branch 'master' of https://github.com/microsoft/python-language…
87f280a 
…-server into numpy
Merge fixes
782ca9a
Fix null ref
ee0a213
Merge branch 'master' of https://github.com/microsoft/python-language…
ccaaa02 
…-server
Merge branch 'master' of https://github.com/microsoft/python-language…
51463f3 
…-server into mkeys2
Fix creation of entries for missing keys
a1b238b
Merge branch 'master' of https://github.com/microsoft/python-language…
2a1821d 
…-server into numpy
Increase cancellation time in debug
a9ae53f
Merge branch 'mkeys2' into numpy
f329e8f
Merge branch 'mkeys2' of https://github.com/MikhailArkhipov/python-la…
263ac4a 
…nguage-server into numpy
Merge branch 'numpy' of https://github.com/MikhailArkhipov/python-lan…
1523067 
…guage-server into numpy
Cache walker + add failing test
0ce2ef6
@AlexanderSher
Merge remote-tracking branch 'Microsoft/master' into Numpy
6bed88a 
# Conflicts:
#	src/Analysis/Ast/Impl/Analyzer/Handlers/FromImportHandler.cs
#	src/Analysis/Ast/Impl/Analyzer/Handlers/ImportHandler.cs
@AlexanderSher
Fix GetMemberNames
695dd85
@AlexanderSher
Remove InvalidateAnalysis call from constructor
d01cb08
@AlexanderSher
Build fix
152a4a2
@AlexanderSher
Merge branch 'master' into Numpy
bc80e57 
# Conflicts:
#	src/Analysis/Ast/Impl/Analyzer/Evaluation/ExpressionEval.cs
#	src/Analysis/Ast/Impl/Modules/PythonVariableModule.cs
#	src/Analysis/Ast/Impl/Modules/Resolution/ModuleResolutionBase.cs
Merge recent
50ca658
Merge issues
83d1c65
@AlexanderSher
Fix FindReferences
7ac9f14
@jakebailey
Copy link
Member

Interesting thing with goto def (maybe this is okay?): On master, gotodef on numpy.array tries to go to the pyd file, but with this PR, it goes to where numpy imported the value and doesn't go further. Not sure if it's that we're losing location info, or just choosing to stop earlier now once we hit a non-py file.

@MikhailArkhipov
Copy link
Author

Probably 9b5c74f

@MikhailArkhipov MikhailArkhipov changed the title [WIP] Improve handling of circular dependencies Improve handling of circular dependencies Oct 30, 2019
@jakebailey
Copy link
Member

Conflicts with master, all superficial. I can resolve them with GitHub to merge this.

@MikhailArkhipov
Copy link
Author

Up to you I can take care of this later today.

@jakebailey jakebailey mentioned this pull request Nov 1, 2019
Merged
@jakebailey
Copy link
Member

I went and did the merge, verifying it now.

@jakebailey
Copy link
Member

Merge is fine, though on my giant test project I get:

Analysis of pyparsing(Library) failed. System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Python.Analysis.Types.PythonType.get_UnknownType() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Types\PythonType.cs:line 162
   at Microsoft.Python.Analysis.Types.PythonType.Call(IPythonInstance instance, String memberName, IArgumentSet argSet) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Types\PythonType.cs:line 82
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.CallOperator(PythonOperator op, IMember left, IPythonType leftType, IMember right, IPythonType rightType, Expression expr, Boolean tryLeft, Boolean tryRight) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.Operators.cs:line 203
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromBinaryOp(Expression expr, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.Operators.cs:line 157
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromExpression(Expression expr, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 192
   at Microsoft.Python.Analysis.Analyzer.Symbols.FunctionEvaluator.Walk(ReturnStatement node) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\FunctionEvaluator.cs:line 204
   at Microsoft.Python.Parsing.Ast.ReturnStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\ReturnStatement.cs:line 34
   at Microsoft.Python.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\SuiteStatement.cs:line 40
   at Microsoft.Python.Parsing.Ast.IfStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\IfStatement.cs:line 49
   at Microsoft.Python.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\SuiteStatement.cs:line 40
   at Microsoft.Python.Analysis.Analyzer.Symbols.FunctionEvaluator.Evaluate() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\FunctionEvaluator.cs:line 67
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(MemberEvaluator e) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 99
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.EvaluateScope(ScopeStatement target) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 81
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.ProcessClassBody() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 107
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.EvaluateClass() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 59
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.Evaluate() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 36
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(MemberEvaluator e) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 99
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(ScopeStatement target) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 87
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromName(NameExpression expr, LookupOptions options) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 219
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromExpression(Expression expr, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 137
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.IsValidBase(Arg a, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 128
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.ProcessBases() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 116
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.EvaluateClass() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 55
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.Evaluate() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 36
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(MemberEvaluator e) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 99
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(ScopeStatement target) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 87
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromName(NameExpression expr, LookupOptions options) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 219
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromExpression(Expression expr, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 137
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.IsValidBase(Arg a, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 128
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.ProcessBases() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 116
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.EvaluateClass() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 55
   at Microsoft.Python.Analysis.Analyzer.Symbols.ClassEvaluator.Evaluate() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ClassEvaluator.cs:line 36
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(MemberEvaluator e) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 99
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(ScopeStatement target) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 87
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromName(NameExpression expr, LookupOptions options) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 219
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromExpression(Expression expr, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 137
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromCallable(CallExpression expr, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.Callables.cs:line 35
   at Microsoft.Python.Analysis.Analyzer.Evaluation.ExpressionEval.GetValueFromExpression(Expression expr, LookupOptions lookupOptions) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Evaluation\ExpressionEval.cs:line 143
   at Microsoft.Python.Analysis.Analyzer.Symbols.FunctionEvaluator.Walk(AssignmentStatement node) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\FunctionEvaluator.cs:line 184
   at Microsoft.Python.Parsing.Ast.AssignmentStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\AssignmentStatement.cs:line 47
   at Microsoft.Python.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\SuiteStatement.cs:line 40
   at Microsoft.Python.Analysis.Analyzer.Handlers.TryExceptHandler.HandleTryExcept(TryStatement node) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Handlers\TryExceptHandler.cs:line 31
   at Microsoft.Python.Analysis.Analyzer.AnalysisWalker.Walk(TryStatement node) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\AnalysisWalker.cs:line 89
   at Microsoft.Python.Parsing.Ast.TryStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\TryStatement.cs:line 71
   at Microsoft.Python.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\SuiteStatement.cs:line 40
   at Microsoft.Python.Analysis.Analyzer.Symbols.FunctionEvaluator.Evaluate() in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\FunctionEvaluator.cs:line 67
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(MemberEvaluator e) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 99
   at Microsoft.Python.Analysis.Analyzer.Symbols.ModuleSymbolTable.Evaluate(ScopeStatement target) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\Symbols\ModuleSymbolTable.cs:line 87
   at Microsoft.Python.Analysis.Analyzer.ModuleWalker.Walk(FunctionDefinition node) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\ModuleWalker.cs:line 197
   at Microsoft.Python.Parsing.Ast.FunctionDefinition.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\FunctionDefinition.cs:line 197
   at Microsoft.Python.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\SuiteStatement.cs:line 40
   at Microsoft.Python.Parsing.Ast.IfStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\IfStatement.cs:line 49
   at Microsoft.Python.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\SuiteStatement.cs:line 40
   at Microsoft.Python.Parsing.Ast.PythonAst.Walk(PythonWalker walker) in C:\Users\jabaile\mikhail-language-server\src\Parsing\Impl\Ast\PythonAst.cs:line 83
   at Microsoft.Python.Analysis.Analyzer.PythonAnalyzerSession.RestoreOrAnalyzeModule(IDependencyChainSingleNode`1 node, IPythonModule module, PythonAst ast, Int32 version) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\PythonAnalyzerSession.cs:line 531
   at Microsoft.Python.Analysis.Analyzer.PythonAnalyzerSession.AnalyzeEntry(IDependencyChainSingleNode`1 node, PythonAnalyzerEntry entry, IPythonModule module, PythonAst ast, Int32 version) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\PythonAnalyzerSession.cs:line 501
   at Microsoft.Python.Analysis.Analyzer.PythonAnalyzerSession.Analyze(IDependencyChainSingleNode`1 node, Stopwatch stopWatch) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\PythonAnalyzerSession.cs:line 325
   at Microsoft.Python.Analysis.Analyzer.PythonAnalyzerSession.Analyze(IDependencyChainNode node, AsyncCountdownEvent ace, Stopwatch stopWatch) in C:\Users\jabaile\mikhail-language-server\src\Analysis\Ast\Impl\Analyzer\PythonAnalyzerSession.cs:line 270

@MikhailArkhipov
Copy link
Author

This means DeclaringModule is null on a type. Might be worth adding assert in Location constructor. Alternatively, it is OK to return null from Call. Can I clone the project somewhere?

@jakebailey
Copy link
Member

Here: https://gist.github.com/jakebailey/701cdb7fa47a4da2596c48160058a121

Needs anaconda, I was using 3.7. Requirements file says how to use it with conda.

@MikhailArkhipov
Copy link
Author

MikhailArkhipov commented Nov 2, 2019 •
edited
Loading

Looks like #1702
SuperType does not have declaring module.

image

@MikhailArkhipov
Copy link
Author

#1739

@MikhailArkhipov MikhailArkhipov mentioned this pull request Nov 3, 2019
Closed
jakebailey
jakebailey approved these changes Nov 4, 2019
View reviewed changes
Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All tests are passing (other than odd scraper errors and existing datetime oddity). Tracking works, and as far as I can tell, my big projects work, numpy is consistent, etc.

I'll merge this quick so we can start fixing up the other PRs. Thanks.

@jakebailey jakebailey merged commit 3751b7a into microsoft:master Nov 4, 2019
This was referenced Nov 4, 2019
Open
Merged
Merged
Closed
Closed
@MikhailArkhipov MikhailArkhipov deleted the numpy branch November 16, 2019 19:44
@MikhailArkhipov MikhailArkhipov mentioned this pull request Apr 7, 2020
Closed
@MikhailArkhipov MikhailArkhipov mentioned this pull request Jun 16, 2020
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jakebailey jakebailey jakebailey approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@MikhailArkhipov @jakebailey @AlexanderSher