-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[onnx,onnxruntime] new port for v1.19.2 with onnx 1.16.0 #36850
Draft
luncliff
wants to merge
23
commits into
microsoft:master
Choose a base branch
from
luncliff:port/onnxruntime
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 14 commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
41a5b63
[onnx] update to v1.16.0
luncliff e27a0d1
[onnx-optimizer] provide ONNX_TARGET_NAME
luncliff 10d5405
[openvino] support onnx 1.16.0
luncliff 02635ad
[onnxruntime] create port with v1.18.0
luncliff 5c945e4
[onnxruntime] support TensorRT build
luncliff 72177b0
[libtorch] support onnx v1.16.0
luncliff 4e24728
[onnxruntime-gpu] use onnxruntime[cuda], without TensorRT
luncliff 4e4e44e
[onnxruntime] update to v1.19.0
luncliff 7d9da66
[onnxruntime] suppress NVCC warnings from Microsoft.GSL attribute
luncliff 28302ed
[onnxruntime] back to v1.18.0
luncliff 92ea700
Merge branch 'master' into port/onnxruntime
luncliff 5408cb3
[libtorch] fix port version after merge
luncliff 942b310
[onnxruntime] update baseline
luncliff 1fc4cc0
[onnxruntime] remove safeint_interface in PUBLIC
luncliff 47b8289
[onnxruntime] remove nested eigen sources
luncliff e8cc89a
[onnxruntime] bump to 1.19.0 again
luncliff 74d7adf
[onnxruntime] create revert patch, PR 21492
luncliff 6bc3ffc
[onnxruntime] bump to v1.19.2
luncliff ca1dba4
Merge branch 'master' into port/onnxruntime
luncliff aeaa20b
[openvino] support onnx 1.16.0
luncliff be8880a
[onnxruntime-gpu] update port-version, baseline
luncliff b1a1c39
x64-windows skips onnxruntime-gpu
luncliff 4abfe13
Merge branch 'master' into port/onnxruntime
luncliff File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,102 +1,2 @@ | ||
| vcpkg_check_linkage(ONLY_DYNAMIC_LIBRARY) | ||
|
|
||
| vcpkg_download_distfile(ARCHIVE | ||
| URLS "https://github.com/microsoft/onnxruntime/releases/download/v${VERSION}/onnxruntime-win-x64-gpu-${VERSION}.zip" | ||
| FILENAME "onnxruntime-win-x64-gpu-${VERSION}.zip" | ||
| SHA512 7ab350a2ede0fc8c716cf083e16a9303acbcc855982e53900f8843773ec32fd20a7396f0bde82bb29f382012b1d05dea41708797f112d9096c8b5048fc5eb7d8 | ||
| ) | ||
|
|
||
| vcpkg_extract_source_archive( | ||
| SOURCE_PATH | ||
| ARCHIVE "${ARCHIVE}" | ||
| NO_REMOVE_ONE_LEVEL | ||
| ) | ||
|
|
||
| # Download repo for experimental features | ||
| vcpkg_from_github( | ||
| OUT_SOURCE_PATH REPO_PATH | ||
| REPO microsoft/onnxruntime | ||
| REF v${VERSION} | ||
| SHA512 f2fec4ded88da6bf67ae7d0aa3082736cb3b8ba29e723b5a516d7632b68ce02aed461f24d3e82cbab20757729e0ab45d736bd986c9b7395f2879b16a091c12a1 | ||
| ) | ||
|
|
||
| file(COPY | ||
| ${REPO_PATH}/include/onnxruntime/core/session/experimental_onnxruntime_cxx_api.h | ||
| ${REPO_PATH}/include/onnxruntime/core/session/experimental_onnxruntime_cxx_inline.h | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/include | ||
| ) | ||
|
|
||
| file(MAKE_DIRECTORY | ||
| ${CURRENT_PACKAGES_DIR}/include | ||
| ${CURRENT_PACKAGES_DIR}/lib | ||
| ${CURRENT_PACKAGES_DIR}/bin | ||
| ${CURRENT_PACKAGES_DIR}/debug/lib | ||
| ${CURRENT_PACKAGES_DIR}/debug/bin | ||
| ) | ||
|
|
||
| file(COPY | ||
| ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/include | ||
| DESTINATION ${CURRENT_PACKAGES_DIR} | ||
| ) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/lib) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_cuda.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/lib) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_cuda.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_cuda.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_cuda.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_shared.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_shared.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_tensorrt.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_tensorrt.pdb | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_shared.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/lib) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_shared.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_tensorrt.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/lib) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_tensorrt.lib | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_shared.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_shared.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_tensorrt.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_tensorrt.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
|
|
||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_cuda.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/bin) | ||
| file(COPY ${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/lib/onnxruntime_providers_cuda.dll | ||
| DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin) | ||
| # # Handle copyright | ||
| vcpkg_install_copyright(FILE_LIST "${SOURCE_PATH}/onnxruntime-win-x64-gpu-${VERSION}/LICENSE") | ||
| set(VCPKG_POLICY_EMPTY_PACKAGE enabled) | ||
| message(WARNING "${PORT} is deprecated. Please use port onnxruntime instead.") | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,16 @@ | ||
| { | ||
| "name": "onnxruntime-gpu", | ||
| "version": "1.16.3", | ||
| "description": "onnxruntime (GPU)", | ||
| "version": "1.18.0", | ||
| "description": "Build onnxruntime port with all available GPU related features", | ||
| "homepage": "https://github.com/microsoft/onnxruntime", | ||
| "license": "MIT", | ||
| "supports": "windows & !x86 & !uwp & !static & !arm" | ||
| "supports": "windows & !x86 & !uwp & !static & !arm", | ||
| "dependencies": [ | ||
| { | ||
| "name": "onnxruntime", | ||
| "features": [ | ||
| "cuda" | ||
| ] | ||
| } | ||
| ] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| diff --git a/cmake/onnxruntime_mlas.cmake b/cmake/onnxruntime_mlas.cmake | ||
| index 682dcfc..405c65b 100644 | ||
| --- a/cmake/onnxruntime_mlas.cmake | ||
| +++ b/cmake/onnxruntime_mlas.cmake | ||
| @@ -158,15 +158,31 @@ function(setup_mlas_source_for_windows) | ||
| ) | ||
| set_source_files_properties(${mlas_platform_srcs_avx2} PROPERTIES COMPILE_FLAGS "/arch:AVX2") | ||
|
|
||
| + file(GLOB_RECURSE mlas_platform_srcs_avx512 CONFIGURE_DEPENDS | ||
| + "${MLAS_SRC_DIR}/intrinsics/avx512/*.cpp" | ||
| + ) | ||
| + set(mlas_platform_srcs_amx "${MLAS_SRC_DIR}/qgemm_kernel_amx.cpp") | ||
| + | ||
| + # clang-cl requires us to enable the platform feature flags explicitly to compile the intrinsics code | ||
| + # unlike MSVC. See: https://github.com/llvm/llvm-project/issues/53520 | ||
| + if (CMAKE_CXX_COMPILER_ID MATCHES "Clang") | ||
| + set_source_files_properties(${mlas_platform_srcs_avx512} PROPERTIES COMPILE_FLAGS "/arch:AVX512") | ||
| + set_source_files_properties(${mlas_platform_srcs_amx} PROPERTIES COMPILE_FLAGS "/arch:AVX512 -mamx-tile -mamx-int8") | ||
| + # https://clang.llvm.org/docs/UsersManual.html#cpu-architectures-features-and-limitations | ||
| + set_source_files_properties(${MLAS_SRC_DIR}/qgemm_kernel_sse.cpp PROPERTIES COMPILE_FLAGS "-march=x86-64") | ||
| + set_source_files_properties(${MLAS_SRC_DIR}/qgemm_kernel_sse41.cpp PROPERTIES COMPILE_FLAGS "-march=x86-64-v2") | ||
| + endif() | ||
| + | ||
| target_sources(onnxruntime_mlas PRIVATE | ||
| ${MLAS_SRC_DIR}/dgemm.cpp | ||
| ${mlas_platform_srcs_avx} | ||
| ${mlas_platform_srcs_avx2} | ||
| - ${MLAS_SRC_DIR}/qgemm_kernel_amx.cpp | ||
| + ${mlas_platform_srcs_avx512} | ||
| + ${mlas_platform_srcs_amx} | ||
| + # ... | ||
| ${MLAS_SRC_DIR}/qgemm_kernel_avx2.cpp | ||
| ${MLAS_SRC_DIR}/qgemm_kernel_sse.cpp | ||
| ${MLAS_SRC_DIR}/qgemm_kernel_sse41.cpp | ||
| - ${MLAS_SRC_DIR}/intrinsics/avx512/quantize_avx512f.cpp | ||
| ${MLAS_SRC_DIR}/sqnbitgemm_kernel_avx2.cpp | ||
| ${MLAS_SRC_DIR}/sqnbitgemm_kernel_avx512.cpp | ||
| ${MLAS_SRC_DIR}/sqnbitgemm_kernel_avx512vnni.cpp | ||
| @@ -208,9 +224,15 @@ function(setup_mlas_source_for_windows) | ||
| ${MLAS_SRC_DIR}/amd64/ErfKernelFma3.asm | ||
| ) | ||
| if (NOT onnxruntime_ORT_MINIMAL_BUILD) | ||
| - target_sources(onnxruntime_mlas PRIVATE | ||
| + set(onnxruntime_mlas_q4gemm_avx512 | ||
| ${MLAS_SRC_DIR}/q4gemm_avx512.cpp | ||
| ) | ||
| + if (CMAKE_CXX_COMPILER_ID MATCHES "Clang") | ||
| + # clang-cl requires us to enable the platform feature flags explicitly to compile the intrinsics code | ||
| + # unlike MSVC. See: https://github.com/llvm/llvm-project/issues/53520 | ||
| + set_source_files_properties(${onnxruntime_mlas_q4gemm_avx512} PROPERTIES COMPILE_FLAGS "/arch:AVX512 -mavx512vnni") | ||
| + endif() | ||
| + target_sources(onnxruntime_mlas PRIVATE ${onnxruntime_mlas_q4gemm_avx512}) | ||
| endif() | ||
| else() | ||
| target_sources(onnxruntime_mlas PRIVATE | ||
| diff --git a/onnxruntime/core/mlas/lib/qgemm_kernel_sse41.cpp b/onnxruntime/core/mlas/lib/qgemm_kernel_sse41.cpp | ||
| index 68931c5..6c095bd 100644 | ||
| --- a/onnxruntime/core/mlas/lib/qgemm_kernel_sse41.cpp | ||
| +++ b/onnxruntime/core/mlas/lib/qgemm_kernel_sse41.cpp | ||
| @@ -16,6 +16,10 @@ Abstract: | ||
|
|
||
| #include "mlasi.h" | ||
| #include "qgemm.h" | ||
| +#if defined(__clang__) | ||
| +#include <smmintrin.h> | ||
| +#include <tmmintrin.h> | ||
| +#endif | ||
|
|
||
| // N.B. MSVC does not require turning on SSE 4.1 intrinsics and the current use | ||
| // for this code is Windows only, so restrict this kernel to that environment. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we just outright deindex it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any rules for deindexing? I simply followed the case of
qtThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As written is OK; it's more a user experience thing. If you think there are lots of existing users of
onnxruntime-gpuwho would be confused by it suddenly not existing, then keeping this makes sense. That was the case with qt, in particular becausevcpkg install qtis something one is likely to do.If you think there aren't a huge number of users who would be confused by it not existing, making it not exist might be clearer for most users because they won't be shown that a port exists that they shouldn't use.
To make it not exist:
versions/o-/onnxruntime-gpu.jsononnxruntime-gpuentry fromversions/baseline.jsonports/onnxruntime-gpuOtherwise it's fine as written