Failure of TensorRT 10.5 to install when building on AGX Jetson

[rhelck]

Description

Tried to install tensorrt on AGX Jetson

Environment

Python: 3.8
Torch: 2.0 (for Jetson)
Jetpack 5.1.2
Pip: 24

Steps To Reproduce

pip install tensorrt
-->

Commands or scripts:

pip install tensorrt==10.1
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorrt==10.1
Downloading tensorrt-10.1.0.tar.gz (16 kB)
Preparing metadata (setup.py) ... done
Collecting tensorrt-cu12 (from tensorrt==10.1)
Downloading tensorrt-cu12-10.5.0.tar.gz (18 kB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/tmp/pip-install-vat_hkn8/tensorrt-cu12_71c17ab2858443af8b1c6b5210714b49/setup.py", line 69, in
raise RuntimeError("TensorRT does not currently build wheels for Tegra systems")
RuntimeError: TensorRT does not currently build wheels for Tegra systems
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

NOTE
Basically I cannot install tensorrt on AGX Jetson anymore, and I cannot find a solution for the given error message online.

[rhelck]

I used 10.1 in this example but I also used the default 10.5.*

pip install tensorrt
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorrt
Downloading tensorrt-10.5.0.tar.gz (16 kB)
Preparing metadata (setup.py) ... done
Collecting tensorrt-cu12==10.5.0 (from tensorrt)
Downloading tensorrt-cu12-10.5.0.tar.gz (18 kB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/tmp/pip-install-pe03b9gg/tensorrt-cu12_8f575f2f04004605aa3086b83b1838d1/setup.py", line 69, in
raise RuntimeError("TensorRT does not currently build wheels for Tegra systems")
RuntimeError: TensorRT does not currently build wheels for Tegra systems
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
(perivenv) orbital@orb-uig146:~/yolo$

[lix19937]

raise RuntimeError("TensorRT does not currently build wheels for Tegra systems")

It is said that the right Jetson SDK installation is required.

[Samyak-Verma]

I am encountering the same exact issue running Jetpack 6.1 (CUDA 12.6) on a NVIDIA Jetson Orin Nano Development Kit.

$ pip install tensorrt
Defaulting to user installation because normal site-packages is not writeable
Collecting tensorrt
  Using cached tensorrt-10.5.0.tar.gz (16 kB)
  Preparing metadata (setup.py) ... done
Collecting tensorrt-cu12==10.5.0 (from tensorrt)
  Using cached tensorrt-cu12-10.5.0.tar.gz (18 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-s97on5w_/tensorrt-cu12_a329c527bf13469ea2e5b4cbec08f586/setup.py", line 69, in <module>
          raise RuntimeError("TensorRT does not currently build wheels for Tegra systems")
      RuntimeError: TensorRT does not currently build wheels for Tegra systems
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

This totally precludes the ability to use torch2trt (as well as any other Python scripts that leverage this library) since that is dependent on being able to have a Python-accessible installation of TensorRT.

[Samyak-Verma]

Never mind, I used the TensorRT .deb file for Jetson/Jetpack systems (instead of the TAR I was originally trying to make work) and that cleared the error I was originally having (where the system could not find TensorRT at all in the Python setup.py script that required it to be import'd) that resulted in the above errorcode. Hope this helps others on the similar setup I'm on- unsure if it works for OP's setup.

[kairin]

where did you get the tensorrt .deb file?

my nvidia sdk manager install a tar.gz file right now i get the same error.

[kairin]

k@ubuntu:~/Desktop/ven1$ pyenv virtualenv --system-site-packages ven2py3-11-11
k@ubuntu:~/Desktop/ven1$ pyenv virtualenvs
  3.11.11/envs/ven2py3-11-11 (created from /home/k/.pyenv/versions/3.11.11)
  3.12.8/envs/ven1py3-12-8 (created from /home/k/.pyenv/versions/3.12.8)
  ven1py3-12-8 (created from /home/k/.pyenv/versions/3.12.8)
  ven2py3-11-11 (created from /home/k/.pyenv/versions/3.11.11)
k@ubuntu:~/Desktop/ven1$ pyenv activate ven2py3-11-11
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ pip --version
pip 24.0 from /home/k/.pyenv/versions/3.11.11/envs/ven2py3-11-11/lib/python3.11/site-packages/pip (python 3.11)
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ pip install --upgrade pip
Requirement already satisfied: pip in /home/k/.pyenv/versions/3.11.11/envs/ven2py3-11-11/lib/python3.11/site-packages (24.0)
Collecting pip
  Downloading pip-24.3.1-py3-none-any.whl.metadata (3.7 kB)
Downloading pip-24.3.1-py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 20.6 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 24.0
    Uninstalling pip-24.0:
      Successfully uninstalled pip-24.0
Successfully installed pip-24.3.1
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ pip check
No broken requirements found.
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ tenssorrt --version
bash: tenssorrt: command not found
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ tensorrt --version
bash: tensorrt: command not found
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ python3 -m pip install --upgrade tensorrt
Collecting tensorrt
  Using cached tensorrt-10.7.0.tar.gz (16 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tensorrt_cu12==10.7.0 (from tensorrt)
  Using cached tensorrt_cu12-10.7.0.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/home/k/.pyenv/versions/ven2py3-11-11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/k/.pyenv/versions/ven2py3-11-11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/k/.pyenv/versions/ven2py3-11-11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-bm3zizm1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-bm3zizm1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-bm3zizm1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-bm3zizm1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 71, in <module>
      RuntimeError: TensorRT does not currently build wheels for Tegra systems
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ dpkg -l | grep nvinfer
ii  libnvinfer-bin                             10.3.0.30-1+cuda12.5                        arm64        TensorRT binaries
ii  libnvinfer-dev                             10.3.0.30-1+cuda12.5                        arm64        TensorRT development libraries
ii  libnvinfer-dispatch-dev                    10.3.0.30-1+cuda12.5                        arm64        TensorRT development dispatch runtime libraries
ii  libnvinfer-dispatch10                      10.3.0.30-1+cuda12.5                        arm64        TensorRT dispatch runtime library
ii  libnvinfer-headers-dev                     10.3.0.30-1+cuda12.5                        arm64        TensorRT development headers
ii  libnvinfer-headers-plugin-dev              10.3.0.30-1+cuda12.5                        arm64        TensorRT plugin headers
ii  libnvinfer-lean-dev                        10.3.0.30-1+cuda12.5                        arm64        TensorRT lean runtime libraries
ii  libnvinfer-lean10                          10.3.0.30-1+cuda12.5                        arm64        TensorRT lean runtime library
ii  libnvinfer-plugin-dev                      10.3.0.30-1+cuda12.5                        arm64        TensorRT plugin libraries
ii  libnvinfer-plugin10                        10.3.0.30-1+cuda12.5                        arm64        TensorRT plugin libraries
ii  libnvinfer-samples                         10.3.0.30-1+cuda12.5                        all          TensorRT samples
ii  libnvinfer-vc-plugin-dev                   10.3.0.30-1+cuda12.5                        arm64        TensorRT vc-plugin library
ii  libnvinfer-vc-plugin10                     10.3.0.30-1+cuda12.5                        arm64        TensorRT vc-plugin library
ii  libnvinfer10                               10.3.0.30-1+cuda12.5                        arm64        TensorRT runtime libraries
ii  python3-libnvinfer                         10.3.0.30-1+cuda12.5                        arm64        Python 3 bindings for TensorRT standard runtime
ii  python3-libnvinfer-dev                     10.3.0.30-1+cuda12.5                        arm64        Python 3 development package for TensorRT standard runtime
ii  python3-libnvinfer-dispatch                10.3.0.30-1+cuda12.5                        arm64        Python 3 bindings for TensorRT dispatch runtime
ii  python3-libnvinfer-lean                    10.3.0.30-1+cuda12.5                        arm64        Python 3 bindings for TensorRT lean runtime
(ven2py3-11-11) k@ubuntu:~/Desktop/ven1$ pip install tensorrt
Collecting tensorrt
  Using cached tensorrt-10.7.0.tar.gz (16 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tensorrt_cu12==10.7.0 (from tensorrt)
  Using cached tensorrt_cu12-10.7.0.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/home/k/.pyenv/versions/3.11.11/envs/ven2py3-11-11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/k/.pyenv/versions/3.11.11/envs/ven2py3-11-11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/k/.pyenv/versions/3.11.11/envs/ven2py3-11-11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-bjwyajgi/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-bjwyajgi/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-bjwyajgi/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-bjwyajgi/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 71, in <module>
      RuntimeError: TensorRT does not currently build wheels for Tegra systems
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[lix19937]

@kairin NOTE HERE: TensorRT does not currently build wheels for Tegra systems