[Build][Bug] The compiler doesn't support BFLOAT16!!!

[HumamHelfawi]

Describe the issue

I am getting

CMake Error at CMakeLists.txt:677 (message):
  The compiler doesn't support BFLOAT16!!!

Urgency

Urgent

Target platform

Jetson AGX Orin - Ubuntu 20.04 - Cuda 11.8

Build script

sh build.sh --use_tensorrt --use_cuda --cuda_home "/usr/local/cuda-11.8" --config Release --parallel --build_shared_lib --cudnn_home="/usr" --tensorrt_home="/usr"

Error / output

-- ******** Summary ********
--   CMake version                     : 3.27.7
--   CMake command                     : /home/***/Downloads/cmake-3.27.7-linux-aarch64/bin/cmake
--   System                            : Linux
--   C++ compiler                      : /usr/bin/c++
--   C++ compiler version              : 9.4.0
--   CXX flags                         :  -ffunction-sections -fdata-sections -Wno-restrict  -DCPUINFO_SUPPORTED -Wnon-virtual-dtor
--   Build type                        : Release
--   Compile definitions               : ORT_ENABLE_STREAM;EIGEN_MPL2_ONLY;_GNU_SOURCE;__STDC_FORMAT_MACROS
--   CMAKE_PREFIX_PATH                 : /home/****/libs/onnxruntime/build/Linux/Release/installed
--   CMAKE_INSTALL_PREFIX              : /usr/local
--   CMAKE_MODULE_PATH                 : /home/****/libs/onnxruntime/cmake/external
-- 
--   ONNX version                      : 1.16.1
--   ONNX NAMESPACE                    : onnx
--   ONNX_USE_LITE_PROTO               : ON
--   USE_PROTOBUF_SHARED_LIBS          : OFF
--   Protobuf_USE_STATIC_LIBS          : ON
--   ONNX_DISABLE_EXCEPTIONS           : OFF
--   ONNX_DISABLE_STATIC_REGISTRATION  : OFF
--   ONNX_WERROR                       : OFF
--   ONNX_BUILD_TESTS                  : OFF
--   ONNX_BUILD_BENCHMARKS             : OFF
--   ONNX_BUILD_SHARED_LIBS            : 
--   BUILD_SHARED_LIBS                 : OFF
-- 
--   Protobuf compiler                 : 
--   Protobuf includes                 : 
--   Protobuf libraries                : 
--   BUILD_ONNX_PYTHON                 : OFF
Finished fetching external dependencies
-- Found CUDAToolkit: /usr/local/cuda-11.8/include (found version "11.8.89") 
-- Performing Test HAS_AMBIGUOUS_REVERSED_OPERATOR
-- Performing Test HAS_AMBIGUOUS_REVERSED_OPERATOR - Failed
-- Performing Test HAS_BITWISE_INSTEAD_OF_LOGICAL
-- Performing Test HAS_BITWISE_INSTEAD_OF_LOGICAL - Failed
-- Performing Test HAS_CAST_FUNCTION_TYPE
-- Performing Test HAS_CAST_FUNCTION_TYPE - Success
-- Performing Test HAS_CATCH_VALUE
-- Performing Test HAS_CATCH_VALUE - Success
-- Performing Test HAS_CLASS_MEMACCESS
-- Performing Test HAS_CLASS_MEMACCESS - Success
-- Performing Test HAS_DEPRECATED_ANON_ENUM_ENUM_CONVERSION
-- Performing Test HAS_DEPRECATED_ANON_ENUM_ENUM_CONVERSION - Failed
-- Performing Test HAS_DEPRECATED_BUILTINS
-- Performing Test HAS_DEPRECATED_BUILTINS - Failed
-- Performing Test HAS_DEPRECATED_COPY
-- Performing Test HAS_DEPRECATED_COPY - Success
-- Performing Test HAS_DEPRECATED_DECLARATIONS
-- Performing Test HAS_DEPRECATED_DECLARATIONS - Success
-- Performing Test HAS_ENUM_CONSTEXPR_CONVERSION
-- Performing Test HAS_ENUM_CONSTEXPR_CONVERSION - Failed
-- Performing Test HAS_FORMAT_TRUNCATION
-- Performing Test HAS_FORMAT_TRUNCATION - Success
-- Performing Test HAS_IGNORED_ATTRIBUTES
-- Performing Test HAS_IGNORED_ATTRIBUTES - Success
-- Performing Test HAS_MAYBE_UNINITIALIZED
-- Performing Test HAS_MAYBE_UNINITIALIZED - Success
-- Performing Test HAS_MISSING_BRACES
-- Performing Test HAS_MISSING_BRACES - Success
-- Performing Test HAS_NONNULL_COMPARE
-- Performing Test HAS_NONNULL_COMPARE - Success
-- Performing Test HAS_PARENTHESES
-- Performing Test HAS_PARENTHESES - Success
-- Performing Test HAS_SHORTEN_64_TO_32
-- Performing Test HAS_SHORTEN_64_TO_32 - Failed
-- Performing Test HAS_STRICT_ALIASING
-- Performing Test HAS_STRICT_ALIASING - Success
NVCC_ERROR = nvcc fatal   : Unknown option '-Wstrict-aliasing'

NVCC_OUT = 1
-- Performing Test HAS_TAUTOLOGICAL_POINTER_COMPARE
-- Performing Test HAS_TAUTOLOGICAL_POINTER_COMPARE - Failed
-- Performing Test HAS_UNDEFINED_VAR_TEMPLATE
-- Performing Test HAS_UNDEFINED_VAR_TEMPLATE - Failed
-- Performing Test HAS_UNUSED_BUT_SET_PARAMETER
-- Performing Test HAS_UNUSED_BUT_SET_PARAMETER - Success
-- Performing Test HAS_UNUSED_BUT_SET_VARIABLE
-- Performing Test HAS_UNUSED_BUT_SET_VARIABLE - Success
-- Performing Test HAS_UNUSED_VARIABLE
-- Performing Test HAS_UNUSED_VARIABLE - Success
-- Performing Test HAS_USELESS_CAST
-- Performing Test HAS_USELESS_CAST - Success
-- Performing Test HAS_STRINGOP_OVERFLOW
-- Performing Test HAS_STRINGOP_OVERFLOW - Success
-- Looking for reallocarray
-- Looking for reallocarray - found
-- Performing Test HAS_ARM64_BFLOAT16
-- Performing Test HAS_ARM64_BFLOAT16 - Failed
CMake Error at CMakeLists.txt:677 (message):
  The compiler doesn't support BFLOAT16!!!

Visual Studio Version

No response

GCC / Compiler Version

9.4

[tianleiwu]

See Jetson Build Instructions

[snnn]

Please use JetPack 6.0 instead.

[Know-Lee]

I got the same problem, how do you resolve it in the end?

[maaft]

JetPack 6.0 is not an option for Xavier or Nano users!

Otherwise bfloat16 was only introduced after 1.17.0.

So currently compiling 1.16.3. Will update here when I succeed.

But I don't have to mention that it would be very bad for Jetson users if only Orin is supported with newer Onnxruntime versions.

please consider to reopen @snnn

[lida2003]

@maaft I have the same issue on Jetson Orin. We have JP6, but JP5 is widely used.

Have you found a way to solve or get around it?

[HumamHelfawi]

I got the same problem, how do you resolve it in the end?

I went for JetPack 6