Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

utf8: AVX2 implementation of Valid #58

Merged
merged 67 commits into from
Jan 11, 2022
Merged

utf8: AVX2 implementation of Valid #58

merged 67 commits into from
Jan 11, 2022

Commits on Jan 3, 2022

  1. utf8: AVX2 implementation of valid

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    8781351 View commit details
    Browse the repository at this point in the history

  2. Rewrite with Avo 

    This is a direct shift and lift operation. Lots of opportunities to
refactor.
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    1bc60e4 View commit details
    Browse the repository at this point in the history

  3. Add missing build file

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    60ebe9c View commit details
    Browse the repository at this point in the history

  4. Scratch space should be 32 bytes

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    e6101cf View commit details
    Browse the repository at this point in the history

  5. Remove setting the Unroll field 

    Not used.
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    b43dff7 View commit details
    Browse the repository at this point in the history

  6. Slow implementation passes all the tests

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    4a869a0 View commit details
    Browse the repository at this point in the history

  7. Do some benchmarking

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    4025b79 View commit details
    Browse the repository at this point in the history

  8. Set cutoff to 128 bytes

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    70ac67a View commit details
    Browse the repository at this point in the history

  9. Generate tables from human description 

    Also fix errors in some of the tables.
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    f23287e View commit details
    Browse the repository at this point in the history

  10. Check for AVX2 support

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    0841d55 View commit details
    Browse the repository at this point in the history

  11. Add missing file

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    d3f76ab View commit details
    Browse the repository at this point in the history

  12. Default to stdlib for non-amd64 platforms

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    41fa615 View commit details
    Browse the repository at this point in the history

  13. Remove copy for remaining bytes

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    710631a View commit details
    Browse the repository at this point in the history

  14. Add +build to default

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    740f972 View commit details
    Browse the repository at this point in the history

  15. More tests

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    fc4dbc6 View commit details
    Browse the repository at this point in the history

  16. Revert table generation

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    83df272 View commit details
    Browse the repository at this point in the history

  17. Fix table generation

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    ba68474 View commit details
    Browse the repository at this point in the history

  18. Found some bugs!

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    c08cbf9 View commit details
    Browse the repository at this point in the history

  19. Add fuzzing harness for go1.18

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    5ffccde View commit details
    Browse the repository at this point in the history

  20. VZEROUPPER before returning 

    https://stackoverflow.com/questions/41303780/why-is-this-sse-code-6-times-slower-without-vzeroupper-on-skylake/41349852#41349852

Co-authored-by: Achille <[email protected]>
    @pelletier
    pelletier and Achille committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    350ebaa View commit details
    Browse the repository at this point in the history

  21. Regen

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    5bab350 View commit details
    Browse the repository at this point in the history

  22. Fix continuationMaskData comment

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    3ce819f View commit details
    Browse the repository at this point in the history

  23. Fix condition before jumping to the stdlib implm

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    cc2e8be View commit details
    Browse the repository at this point in the history

  24. Rename nibbleMask to lowerNibbleMask

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    23f1e31 View commit details
    Browse the repository at this point in the history

  25. Try skipping the exhaustive test 

    See if it makes CI happier. This test passes on my machine.
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    c9a092c View commit details
    Browse the repository at this point in the history

  26. Fix tags of utf8/valid_amd64.go

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    d38306f View commit details
    Browse the repository at this point in the history

  27. Revert "Fix tags of utf8/valid_amd64.go" 

    This reverts commit d38306f.
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    7a5d877 View commit details
    Browse the repository at this point in the history

  28. Fix constraint expression for utf8/valid

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    635952b View commit details
    Browse the repository at this point in the history

  29. Revert "Try skipping the exhaustive test" 

    This reverts commit c9a092c.
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    3932872 View commit details
    Browse the repository at this point in the history

  30. Revert "Revert "Try skipping the exhaustive test"" 

    This reverts commit 3932872.
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    a71eb92 View commit details
    Browse the repository at this point in the history

  31. Exit at 32B boundary any error already occurred

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    3e2a4d8 View commit details
    Browse the repository at this point in the history

  32. Add previous bug found by fuzzing

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    6d8d533 View commit details
    Browse the repository at this point in the history

  33. Trying to make the small inputs version faster 

    name                  old time/op    new time/op    delta
Valid/small16/AVX-32    14.8ns ± 0%    13.7ns ± 0%  -7.51%  (p=0.029 n=4+4)

name                  old speed      new speed      delta
Valid/small16/AVX-32  1.08GB/s ± 0%  1.17GB/s ± 0%  +8.13%  (p=0.029 n=4+4)
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    0b28eff View commit details
    Browse the repository at this point in the history

  34. Remove some branches

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    7e46a50 View commit details
    Browse the repository at this point in the history

  35. Remove magic numbers

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    cdd5fac View commit details
    Browse the repository at this point in the history

  36. Re-comment out exhaustive tests

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    ffb05ac View commit details
    Browse the repository at this point in the history

  37. Don't set nextD if not necessary

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    79dc2f0 View commit details
    Browse the repository at this point in the history

  38. Remove extra CMP of byte 2 check 

    name                  old time/op    new time/op    delta
Valid/small16/AVX-32    14.2ns ± 7%    13.1ns ± 2%  -7.31%  (p=0.008 n=5+5)

name                  old speed      new speed      delta
Valid/small16/AVX-32  1.13GB/s ± 6%  1.22GB/s ± 2%  +7.77%  (p=0.008 n=5+5)
    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    a85ba22 View commit details
    Browse the repository at this point in the history

  39. Don't xor x register

    @pelletier
    pelletier committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    f43d04d View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2022

  1. Switch to AVX for inputs of 32 bytes or more

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    561943e View commit details
    Browse the repository at this point in the history

  2. Remove scratch space

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    16e6738 View commit details
    Browse the repository at this point in the history

  3. Experiment with copying small inputs to memory

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    5852227 View commit details
    Browse the repository at this point in the history

  4. Revert "Experiment with copying small inputs to memory" 

    This reverts commit 5852227.

Not really better after benchmarking on an intel CPU.
    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    8c4915d View commit details
    Browse the repository at this point in the history

  5. Change the exhaustive test to check only bounds 

    Less exhaustive, but better than disabling it.
    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    76523ef View commit details
    Browse the repository at this point in the history

  6. Consolidated tests 

    Added middle-of-range element for bounds checks. Run bound checks on the three
variants of inputs
    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    c383a6f View commit details
    Browse the repository at this point in the history

  7. Introduce Validate function 

    Also fixes ASCII-fast-path.
    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    bfadc07 View commit details
    Browse the repository at this point in the history

  8. Change import names for fallback code

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    b05fa38 View commit details
    Browse the repository at this point in the history

  9. Check for ASCII validation in fuzzer

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    e960768 View commit details
    Browse the repository at this point in the history

  10. Fuzzer found new bug

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    fc094cd View commit details
    Browse the repository at this point in the history

  11. Fix leading zero bits count

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    7667e0f View commit details
    Browse the repository at this point in the history

  12. Bring back copy for leftovers

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    f02d3bc View commit details
    Browse the repository at this point in the history

  13. Use the actual stdlib

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    4a7bb03 View commit details
    Browse the repository at this point in the history

  14. Don't run ascii.Valid

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    71f4c92 View commit details
    Browse the repository at this point in the history

  15. Rename packages and return values

    @pelletier
    pelletier committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    aaba3a8 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2022

  1. Change signature of Validate

    @pelletier
    pelletier committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    9a2449b View commit details
    Browse the repository at this point in the history

  2. Remove unused imports

    @pelletier
    pelletier committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    ed0e17b View commit details
    Browse the repository at this point in the history

  3. Move MSB mask loading out of hot path

    @pelletier
    pelletier committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    0814fd8 View commit details
    Browse the repository at this point in the history

  4. Reuse intermediate vector for pushN operations

    @pelletier
    pelletier committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    e7c8a9e View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2022

  1. Load last block near page boundary

    @pelletier
    pelletier committed Jan 6, 2022
    Configuration menu
    Copy the full SHA
    aa912a4 View commit details
    Browse the repository at this point in the history

  2. Fix invalid offset for large tail load

    @pelletier
    pelletier committed Jan 6, 2022
    Configuration menu
    Copy the full SHA
    e4da37d View commit details
    Browse the repository at this point in the history

  3. Add tail benchmarks

    @pelletier
    pelletier committed Jan 6, 2022
    Configuration menu
    Copy the full SHA
    0f3e231 View commit details
    Browse the repository at this point in the history

  4. Fix API change in fuzzer

    @pelletier
    pelletier committed Jan 6, 2022
    Configuration menu
    Copy the full SHA
    473f5ac View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2022

  1. Add valid command line

    @pelletier
    pelletier committed Jan 8, 2022
    Configuration menu
    Copy the full SHA
    f802637 View commit details
    Browse the repository at this point in the history

  2. Remove prompt

    @pelletier
    pelletier committed Jan 8, 2022
    Configuration menu
    Copy the full SHA
    e603701 View commit details
    Browse the repository at this point in the history

  3. Add test for page boundary loads

    @pelletier
    pelletier committed Jan 8, 2022
    4 Configuration menu
    Copy the full SHA
    f02a719 View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2022

  1. Move cmd/valid to utf8/cmd/valid

    @pelletier
    pelletier committed Jan 9, 2022
    Configuration menu
    Copy the full SHA
    1835aa3 View commit details
    Browse the repository at this point in the history

  2. Perform non-AVX2 fallback in go

    @pelletier
    pelletier committed Jan 9, 2022
    Configuration menu
    Copy the full SHA
    a9b7485 View commit details
    Browse the repository at this point in the history