Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add google-highway #1611

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Add google-highway #1611

wants to merge 1 commit into from

Conversation

WillAyd
Copy link
Contributor

@WillAyd WillAyd commented Aug 13, 2024

xref conversation with @jan-wassenberg in google/highway#2294

This intentionally does not include any of the contrib packages that are part of the CMake config

@WillAyd WillAyd force-pushed the highway branch 9 times, most recently from 155c01f to fb88876 Compare August 13, 2024 23:04
@WillAyd
Copy link
Contributor Author

WillAyd commented Aug 13, 2024

Not sure what to do with the current CI failures:

  • VisualStudio-clang-cl: maybe an upstream issue with the failing test?
  • MSYS2 (UCRT64) - error: '_LIBCPP_VERSION' was not declared in this scope - issue with the cpp compiler?
  • MSYS2 (CLANG64) - error: undefined symbol: __truncsfbf2 - is this an issue with float16?

@jan-wassenberg
Copy link

__truncsfbf2 was an ABI breakage in llvm. This might indicate the compiler and compiler-rt packages shipped with msys2 were not matching.

The abort_test failure is:
"gtest-port.cc(859): error: Failed
Syntax error at index 16 in simple regular expression "Test Abort from [0-9]+ of abort_test.cc": '[' is unsupported."

We'll change this to use \d.

copybara-service bot pushed a commit to google/highway that referenced this pull request Aug 14, 2024
copybara-service bot pushed a commit to google/highway that referenced this pull request Aug 14, 2024
copybara-service bot pushed a commit to google/highway that referenced this pull request Aug 19, 2024
copybara-service bot pushed a commit to google/highway that referenced this pull request Aug 19, 2024
copybara-service bot pushed a commit to google/highway that referenced this pull request Aug 19, 2024
copybara-service bot pushed a commit to google/highway that referenced this pull request Aug 19, 2024
copybara-service bot pushed a commit to google/highway that referenced this pull request Aug 19, 2024
releases.json Outdated Show resolved Hide resolved
subprojects/packagefiles/google-highway/meson.build Outdated Show resolved Hide resolved
Copy link
Member

@dcbaker dcbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some general review below.

Additionally, is there a chance that upstream would take the meson.build files directly? Our preference would be to have them upstream if possible, since that tends to give a better user experience and is generally less work than us having to maintain it out of tree.

subprojects/packagefiles/google-highway/meson.build Outdated Show resolved Hide resolved
subprojects/packagefiles/google-highway/meson.build Outdated Show resolved Hide resolved
Comment on lines 40 to 43
'hwy/tests/hwy_gtest.h',
'hwy/tests/test_util-inl.h',
'hwy/tests/test_util.cc',
'hwy/tests/test_util.h',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Headers are not needed here, they'll just be dropped anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good - was just trying to keep this in sync with the upstream CMake configuration, but assuming that's not a big deal

subprojects/packagefiles/google-highway/meson.build Outdated Show resolved Hide resolved
subprojects/google-highway.wrap Outdated Show resolved Hide resolved
@dcbaker dcbaker marked this pull request as draft August 27, 2024 16:41
@WillAyd WillAyd force-pushed the highway branch 2 times, most recently from 0b01473 to 0d2c2e5 Compare August 27, 2024 17:20
@WillAyd
Copy link
Contributor Author

WillAyd commented Aug 27, 2024

Additionally, is there a chance that upstream would take the meson.build files directly? Our preference would be to have them upstream if possible, since that tends to give a better user experience and is generally less work than us having to maintain it out of tree.

I have no affiliation with google / google-highway so I'll leave that up to @jan-wassenberg

@jan-wassenberg
Copy link

Additionally, is there a chance that upstream would take the meson.build files directly? Our preference would be to have them upstream if possible, since that tends to give a better user experience and is generally less work than us having to maintain it out of tree.

hm, the interesting question is with what SLO. What we might be able to offer is to include it in our tree, and to make an effort to add new source files (in particular tests) to the meson build. But if it breaks, we won't be able to help :)

@WillAyd
Copy link
Contributor Author

WillAyd commented Aug 27, 2024

@dcbaker I'm not super familiar with MSYS2 - do you know how to resolve this issue?

2024-08-27T17:50:06.4502660Z D:/a/_temp/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/as.exe: subprojects/highway-389fbc6c0e2b91dc1aba92709b14962344915f1d/cast_test.exe.p/hwy_tests_cast_test.cc.obj: too many sections (69839)
2024-08-27T17:50:06.4503089Z D:\a\_temp\msys64\tmp\ccSBPk3T.s: Assembler messages:
2024-08-27T17:50:06.4504829Z D:\a\_temp\msys64\tmp\ccSBPk3T.s: Fatal error: can't write 152 bytes to section .text of subprojects/highway-389fbc6c0e2b91dc1aba92709b14962344915f1d/cast_test.exe.p/hwy_tests_cast_test.cc.obj: 'file too big'
2024-08-27T17:50:06.4506749Z D:/a/_temp/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/as.exe: subprojects/highway-389fbc6c0e2b91dc1aba92709b14962344915f1d/cast_test.exe.p/hwy_tests_cast_test.cc.obj: too many sections (69839)
2024-08-27T17:50:06.4508135Z D:\a\_temp\msys64\tmp\ccSBPk3T.s: Fatal error: subprojects/highway-389fbc6c0e2b91dc1aba92709b14962344915f1d/cast_test.exe.p/hwy_tests_cast_test.cc.obj: file too big
2024-08-27T17:50:06.4508213Z 

@dcbaker
Copy link
Member

dcbaker commented Aug 27, 2024

@WillAyd I'm not sure, but google suggests that -Wa,-mbig-obj might help, so adding something like:

cpp = meson.get_compiler('cpp')
add_project_arguments(cpp.get_supported_arguments(['-Wa,-mbig-obj']), language : 'cpp')

might help. If not I'll try to rope in one of the resident msys2 experts

edit: not link args, doing too many things at once :D

@WillAyd WillAyd force-pushed the highway branch 4 times, most recently from 3380164 to 535b52b Compare August 29, 2024 00:13
@WillAyd
Copy link
Contributor Author

WillAyd commented Aug 29, 2024

OK I think the meson build definition works for those now, but I am seeing this in the MSYS2 UCRT64 build:

2024-08-29T00:15:39.5808278Z Command line: `c++ D:/a/wrapdb/wrapdb/_build/meson-private/tmp6wjk2er3/testfile.cpp -o D:/a/wrapdb/wrapdb/_build/meson-private/tmp6wjk2er3/output.obj -c -D_FILE_OFFSET_BITS=64 -O0 -fpermissive` -> 1
2024-08-29T00:15:39.5809242Z stderr:
2024-08-29T00:15:39.5809752Z D:/a/wrapdb/wrapdb/_build/meson-private/tmp6wjk2er3/testfile.cpp: In function 'int main()':
2024-08-29T00:15:39.5810785Z D:/a/wrapdb/wrapdb/_build/meson-private/tmp6wjk2er3/testfile.cpp:6:17: error: '_LIBCPP_VERSION' was not declared in this scope
2024-08-29T00:15:39.5811482Z     6 |                 _LIBCPP_VERSION;

Which I don't think has anything to do with highway? Seems like an issue with the compiler in the environment?

Similarly, I don't think the CLANG64 build will be green on account of #1611 (comment)

@WillAyd WillAyd force-pushed the highway branch 2 times, most recently from 465f7e0 to 816fd27 Compare September 6, 2024 17:57
@WillAyd
Copy link
Contributor Author

WillAyd commented Sep 6, 2024

Is there a way to skip the tests just for the windows / mysy platforms? I think the test failure in version 1.2.0 for windows is pretty pedantic, and the msys issue seems out of the control of the library

aarongable pushed a commit to chromium/chromium that referenced this pull request Oct 16, 2024
https://chromium.googlesource.com/external/github.com/google/highway.git/+log/8295336dd70f..a97b5d371d69

$ git log 8295336dd..a97b5d371 --date=short --no-merges --format='%ad %ae %s'
2024-10-15 janwas Wrap tests in anon namespace to enable integration in Chromium
2024-10-14 janwas fix topology detection on RVV, where LPs can be under-reported
2024-10-10 janwas fix missing cast
2024-10-10 janwas update MODULE.bazel versions, add missing platforms
2024-10-10 janwas Minor formatting + DASSERT fixes
2024-10-08 janwas add NUMA node to topology
2024-10-07 janwas fix IsNaN codegen for clang < 18.1. Refs numpy/numpy/issues/27313
2024-10-06 siruilu use 1/8 LMUL to process Eq128Upper and Ne128Upper mask
2024-10-06 siruilu use 1/8 LMUL to process Ne128 mask
2024-10-06 siruilu use 1/8 LMUL to process Eq128 mask
2024-10-06 siruilu use 1/8 LMUL to process Lt128Upper mask
2024-09-17 raj.khem Add cmake check for deducing 32bit or 64bit RISCV
2024-10-04 janwas Fix profiler ubsan crash
2024-10-03 no-reply Fix HWY_RCAST_ALIGNED
2024-10-01 john_platts Fixed compilation error in sort_test
2024-09-30 janwas fix Github Action failure: incorrect dependency and cast
2024-09-30 siruilu use HWY_MAX instead of std::max
2024-09-30 janwas fix printf specifier warnings when profiler enabled
2024-09-27 siruilu reduce LMUL for Dup128MaskFromMaskBits
2024-09-19 janwas update Highway intro slides
2024-09-18 janwas disable RVV for older GCC/Clang. Fixes #2328, thanks @johnplatts
2024-09-10 janwas add test for already sorted inputs
2024-09-09 janwas fix NumWorkers: main thread also participates.
2024-09-09 janwas log transform for less overflow in GeometricMean
2024-09-06 john_platts Made unit testing enhancements if HWY_TEST_STANDALONE is 1
2024-09-03 john_platts Fixes for Clang 19 bugs on the Z14/Z15 targets
2024-09-03 janwas consistent stats.cc %e formatting for Mean/SD
2024-08-27 janwas disable runtime dispatch experiment on RVV, clang 19 is not yet ready. Fixes #2311
2024-08-27 janwas add vectorlite user
2024-08-26 janwas remove finished TODO
2024-08-20 paulchang Temporarily disable tests for Load/StoreInterleaved of special floats on arm7
2024-08-20 k0tran Add RVV linker flag in case of building with RVV
2024-08-19 janwas fix abort_test for limited Windows regex. Refs mesonbuild/wrapdb#1611
2024-08-16 paulchang Provide generic emulated Load/StoreInterleaved for special floats
2024-08-15 john_platts Added arithmetic assignment operators from F16/BF16
2024-08-13 john_platts Fixes to hwy/base.h and vqsort-inl.h
2024-08-13 doak Minor QoL improvements.
2024-08-12 john_platts Enabled F64 on WASM_EMU256 target
2024-08-12 janwas update user list from sourcegraph
2024-08-10 john_platts Enable AVX3_SPR F16 support with Clang 19.1 or later
2024-08-08 janwas also enable CompressStore workaround for SPR
2024-08-07 john_platts Made fixes to hwy::IsInteger
2024-08-07 janwas Experimental support for RVV runtime dispatch in clang19, refs llvm/llvm-project#56592
2024-08-02 janwas document SVE operator< issue
2024-08-02 janwas split sort_test
2024-08-02 janwas fix remaining unsafe BlendedStore in vqsort
2024-07-29 janwas add gemma.cpp to Highway users
2024-07-29 janwas add 8-bit test case for vqsort to cover MaybePartitionTwoValue. Refs #2281
2024-07-26 chris Replace BlendedStore at edges with StoreN
2024-07-23 john_platts Fix compilation errors on 32-bit PowerPC
(...)
2024-06-12 janwas add FAQ on HWY_RESTRICT
2024-06-11 janwas split up logical test into if_test and sign_test for GCC. fixes #2240, thanks @malaterre
2024-06-11 john_platts Added Clang 16/17/18 and GCC 13 to GitHub workflow
2024-06-11 john_platts Added x86_32 GCC 12/Clang 15 to GitHub workflow
2024-06-10 john_platts Fixed warning in TestForeach in transform_test.cc
2024-06-10 john_platts Made enhancements to Exp2
2024-06-07 49699333+dependabot[bot] Bump step-security/harden-runner from 2.8.0 to 2.8.1
2024-06-05 paulchang Improve testing of transform edge cases
2024-06-03 janwas opt-in to reduce set of MSVC targets. Fixes #2220
2024-06-03 janwas clarify ZEN4 enabling
2024-06-03 jdorfman Internal
2024-06-03 john_platts Made fixes to generic_ops-inl.h BitShuffle impl on big-endian
2024-06-03 janwas Disable RVV runtime dispatch. Fixes #2227
2024-06-03 janwas update signing process post 1.2 - use git archive result
2024-06-03 janwas fix incorrect DLLEXPORT, fixes #2225
2024-06-03 janwas mention key server for signing, fixes #2224
2024-05-31 john_platts Added DemoteToNearestInt and F16/F64 NearestInt ops
2024-05-31 janwas remove HWY_HAS_INCLUDE - mishandled by clang
2024-05-31 janwas fix RVV TestPartition - cap vector as in the full sort. Fixes #2078
2024-05-30 janwas add HWY_DYNAMIC_POINTER_T for use with HWY_EXPORT_T
2024-05-30 janwas fix RVV missing macro, simplify D args
2024-05-30 janwas fix ppc 32-bit build, which lacks __int128
2024-05-29 mathieu.malaterre Fix compilation of shared libraries
2024-05-29 janwas split arithmetic_test into div/saturated. Fixes #2208
2024-05-29 mathieu.malaterre Allow compilation with -Werror on GNU/Hurd system
2024-05-29 janwas move bit_set to core
2024-05-28 janwas fix non-Apple bf16 detection, fixes #2199, thanks @jgouly
2024-05-28 janwas doc: define lanes, mention new targets and ConvertScalarTo
2024-05-27 janwas temporarily delete codeql - failing due to disk space.
2024-05-24 janwas Clang RVV workarounds
2024-05-23 janwas GCC cast fix
2024-05-23 john_platts Fix ICX detection
2024-05-23 janwas Fix bf16 target and ICC compiler detection
2024-05-22 john_platts Updated Undefined implementation on NEON/PPC/Z14
2024-05-22 janwas disable failing GCC 13 RVV tests
2024-05-22 janwas Re-enable float to int test; UB is fixed in #2189
2024-05-22 49699333+dependabot[bot] --- updated-dependencies: - dependency-name: step-security/harden-runner   dependency-type: direct:production   update-type: version-update:semver-minor ...
2024-05-22 padogra Add frame of reference to Pack32/Pack64
2024-05-21 49699333+dependabot[bot] --- updated-dependencies: - dependency-name: github/codeql-action   dependency-type: direct:production   update-type: version-update:semver-patch ...
2024-05-20 john_platts Workaround for UB in x86 ConvertInRangeTo with GCC
2024-05-21 janwas simplify interleaved_test, fixes GCC AVX2/3
2024-05-17 no-reply re-enable vqsort for sanitizers on arm
2024-05-17 janwas fix GCC warnings (cast after arithmetic, but not for Iota)
2024-05-17 janwas avoid GCC "UB" in truncating cases. Fixes #2183
2024-05-17 janwas internal test infra
2024-05-16 janwas Clang RVV fixes:
2024-05-16 john_platts Updated CMakeLists.txt to compile in C++17 mode by default
2024-05-15 john_platts Fixed compilation errors with HwyInterleavedTest on x86
2024-05-10 john_platts Added support for dynamic dispatch for macOS/iOS/iPadOS on AArch64
2024-04-10 siruilu Add the synthesized, efficient Lt128 implementation.

Created with:
  roll-dep src/third_party/highway/src

Bug: 372239930
Change-Id: Ifc034f7f11b444bacdb4e244e8cdf0b47a6c0440
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5935891
Reviewed-by: Nico Weber <[email protected]>
Commit-Queue: Arthur Eubanks <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1369232}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants