patch a perf bottleneck #79
Conversation
| @@ -52,7 +53,7 @@ object Hash { | |||
| def apply(as: Array[Byte]): Array[Byte] = apply(new ByteArrayInputStream(as)) | |||
|
|
|||
| /** Calculates the SHA-1 hash of the given file.*/ | |||
| def apply(file: File): Array[Byte] = Using.fileInputStream(file)(apply) | |||
| def apply(file: File): Array[Byte] = Files.readAllBytes(file.toPath) | |||
There was a problem hiding this comment.
Wat. This reads all the bytes in the file. It doesn't calculate the SHA-1 hash for it..
There was a problem hiding this comment.
well then apply the hash to it afterwards... the slow bit is the FileFilterStream
like I said, I have absolutely no way of validating anything about this PR.
There was a problem hiding this comment.
def apply(file: File): Array[Byte] = apply(Files.readAllBytes(file.toPath))
|
using sbt/launcher#43 (and including the and get a CPU profile in yourkit. Unfortunately, it looks like the bottleneck is still there in So the actual bottleneck is the digest calculation. I don't understand why we calculate this again for every file, even if the size and timestamp has not changed. |
I did some analysis that looked the same as sbt/zinc#371 then I came up with this patch.
I've spent the last few hours trying to apply this as a monkey patch, but I can't get it to work, so I have absolutely no idea if it improves anything or not.
UPDATE this needs to actually calculate the hash. My point is that using NIO should be a lot faster than
FileFilterStream, which is using 99% of my CPU in the profile (not the calculation of the hash).