-
Notifications
You must be signed in to change notification settings - Fork 789
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parallel type checking #11550
Comments
@kerams, would you mind paying a look at the recent comments I made in the broader/fuzzy discussion for incremental recompile (#8044)? What I touch on my comments in #8044 is targeted at identifying if a file and a diff of signatures before it should engender a recompile of the file. The same approach could be taken earlier, it could be on diff of AST that we could already identify opportunities to skip recompile of a file, or more likely, proceed in checking the next stage. Why is it related to incremental recompilation? Because incremental recompilation will require serialisation and heuristics on the serialized graphs, and making things like a graph and serialised, means they can conceptually be parallelised for read only usage 🙂 (I know I'm stretching everything a little bit). Now, regarding your more precise issue, I have few things coming to mind: Are you familiar with the If That would be the beginning of a
It means invalidation of the serialised One big pitfall, at least, nest of edge cases, is shadowing and aliasing features of the language, and also stuff like It sounds like "after assembly reference resolution" maybe the next stage if bare AST can't help much in sorting out what you are pointing at. edit:
Don't worry, you are covered at least 10x by all mine :) |
We have a PR that could enable this: #11152 The restriction is that it will only parallelize type checking impl files that have a backing signature file. Our hope in the future is that we will add better tooling to create these signature files automatically through a gesture in an editor. Trying to parallelize type checking for impl files without sig files is a much more difficult undertaking. Sure, you could add heuristics like you suggested, but we would have to have a dependency graph of types and functions; which may be more expensive in of itself. |
Yes, I'm specifically talking about no sig scenarios.
Yes, that would be expensive. I was thinking keeping track of namespaces and modules alone could theoretically suffice. For instance, if we collect those for file @smoothdeveloper, I'm afraid I don't know enough about those topics, or whether your approach, if viable, would supersede mine (perhaps they could even complement each other). My evening thoughts were very narrowly concerned about full compilation performance. |
Here's a really silly idea. If I extract all namespaces and top level modules (a very shallow AST traversal would be enough) from file I'll answer myself. The aforementioned auto opens would cause a problem, but that can be easily checked separately. Partially qualified opens are worse though. |
It seems to me like a great bang for buck first pass at splitting out stuff just at the AST level, like having both a genius and captain obvious being one and single person bringing this :D I'm curious what roadblocks would be hit by this heuristics to turn faulty in the wild. @vasily-kirichenko, any opinion? |
Will move this to a discussion |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
The recent awesome work on parallel parsing and IL gen got me thinking whether it would be feasible to parallelize type checking as well.
Given F#'s significant file order, if the compiler can quickly establish that file
B
can't possibly reference anything in fileA
declared directly above it, can'tA
andB
be typechecked (and optimized) simultaneously? This would require an initial traversal of the AST in every file in order to create a file dependency graph. Specifically, things like this need to be checked:B
opens a module or class defined inA
,B
depends onA
A
contains anAutoOpen
that has immediate effect inB
(because of overlapping namespaces, regardless of actual contents),B
depends onA
B
references something fromA
using a qualified name,B
depends onA
I think these can be gleaned from the untyped syntax tree alone, the question is whether it would be fast enough to offset the overhead and whether typechecking has a significant cost. Apologies in advance if there's a gaping flaw in here somewhere :).
The text was updated successfully, but these errors were encountered: