Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix gemm_float8 build failure on CUDA 11.3-11.7 #18760

Merged
merged 2 commits into from
Dec 8, 2023
Merged

Conversation

pengwa
Copy link
Contributor

@pengwa pengwa commented Dec 8, 2023

Fix gemm_float8 build failure on CUDA 11.3 ~ 11.7

User env: CUDA 11.3, build option include "--disable_types float8"


/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/math/gemm_float8.cu(256): error: identifier "CUBLASLT_MATMUL_DESC_SM_COUNT_TARGET" is undefined

/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/math/gemm_float8.cu(264): error: enum "cublasLtMatmulDescAttributes_t" has no member "CUBLASLT_MATMUL_DESC_FAST_ACCUM"

/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/math/gemm_float8.cu(268): error: identifier "CUBLASLT_MATMUL_DESC_A_SCALE_POINTER" is undefined

/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/math/gemm_float8.cu(271): error: identifier "CUBLASLT_MATMUL_DESC_B_SCALE_POINTER" is undefined

/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/math/gemm_float8.cu(274): error: identifier "CUBLASLT_MATMUL_DESC_D_SCALE_POINTER" is undefined

5 errors detected in the compilation of "/tmp/onnxruntime/onnxruntime/contrib_ops/cu

Here is a versions (major version) diff on the requested attributes:


cuda 11.5.1

no CUBLASLT_MATMUL_DESC_SM_COUNT_TARGET


cuda 11.6

https://docs.nvidia.com/cuda/archive/11.6.0/pdf/CUBLAS_Library.pdf

has CUBLASLT_MATMUL_DESC_SM_COUNT_TARGET



cuda 11.7

no CUBLASLT_MATMUL_DESC_FAST_ACCUM

no CUBLASLT_MATMUL_DESC_A_SCALE_POINTER

no CUBLASLT_MATMUL_DESC_B_SCALE_POINTER

no CUBLASLT_MATMUL_DESC_D_SCALE_POINTER



cuda 11.8

https://docs.nvidia.com/cuda/archive/11.8.0/pdf/CUBLAS_Library.pdf

has CUBLASLT_MATMUL_DESC_FAST_ACCUM

has CUBLASLT_MATMUL_DESC_A_SCALE_POINTER

has CUBLASLT_MATMUL_DESC_A_SCALE_POINTER

has CUBLASLT_MATMUL_DESC_B_SCALE_POINTER

has CUBLASLT_MATMUL_DESC_D_SCALE_POINTER


Motivation and Context

@pengwa pengwa merged commit 44b5843 into main Dec 8, 2023
92 checks passed
@pengwa pengwa deleted the pengwa/fix_float8 branch December 8, 2023 13:01
@pengwa
Copy link
Contributor Author

pengwa commented Dec 8, 2023

Thanks @xadupre !!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants