Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change replaces the CRC32 update code with a slicing-by-16 implementation that can process 16 bytes at a time. It uses the original byte-by-byte code as a fallback, but also marks that method for aggressive inlining to get rid of overhead from function calls. This reduces the runtime of the BZip and ZLib CRC checksums by about 75%, which is also good for overall I/O.
The performance can be improved even further by using unsafe code to eliminate array bounds checks. While that code isn't included in this PR, the benchmarks for the unsafe code are included below. The code can be found here: nsgomez@6a2e954
CLA
While this is my personal GitHub account, I'm submitting this PR in my capacity as a Microsoft employee.
The method CrcUtilities.GenerateSlicingLookupTable is derived from Crc32.NET's table generation code. We do not own that source code, but because it's MIT Licensed we have sufficient rights to use, copy, and distribute it. Otherwise, I certify that I own, and have sufficient rights to contribute, all source code and related material intended to be compiled or integrated with the source code for the SharpZipLib open source product (the "Contribution"). My Contribution is licensed under the MIT License.
Benchmarks
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.1248 (2004/?/20H1)
Intel Xeon W-2133 CPU 3.60GHz, 1 CPU, 12 logical and 6 physical cores
[Host] : .NET Framework 4.8 (4.8.4200.0), X64 RyuJIT
Job-KMVHYW : .NET Core 2.1.12 (CoreCLR 4.6.27817.01, CoreFX 4.6.27818.01), X64 RyuJIT
Job-DCFJWB : .NET Framework 4.8 (4.8.4200.0), X64 RyuJIT
net461
netcoreapp2.1