-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use a better hashing algorithm for classpath hashing #32
Comments
We need benchmarks to back up the need for this. Some numbers are there, but what we need to answer more concretely is: how is this change going to affect big projects? Is the price high only in batch (and clean) compilation, or does it affect incremental compilation too, and to which extent? (Note: classpath hashing happens on every incremental compile, so it's no joke. Some of the questions above need more clarification, but the motivation for this change is clear.) One thing to consider is to associate hashes with classpath entries, and allow any compilation process to reuse those hashes. But if we do that, we need to think about proper invalidation. All options are:
|
This is fixed in master, we cache with xxHash and in parallel now |
Update nailgun to 1.0.2
We're currently using SHA-1. I believe a cryptographic hash is not necessary for hashing and detecting changes in jars, so we should use something faster and more lightweight like
xxHash
, see my previous attempt to merge this into Zinc here: sbt/zinc#371. We could in theory implement it in bloop since we have the classpath hooks in latest Zinc 1.x for this.However, this is only useful if we happen to find out that either of the following hypothesis are true:
I believe that these assumptions are most likely false, so this ticket is only for documenting the possibility of improving the hash. In practice, the cost of hashing the classpath happens only once (the first time you run the compiler) and after that the classpath should be the same, but this claim requires investigation.
I have no idea how bearable this process is for projects with gigantic classpaths that are likely to change (in essence, huge multi-module builds). So maybe it's worth it for them after all.
The text was updated successfully, but these errors were encountered: