From 9809c91bfb7f876c315d7547ebf1cc4d6ceb4172 Mon Sep 17 00:00:00 2001 From: Tianlei Wu Date: Tue, 24 Sep 2024 18:04:00 +0000 Subject: [PATCH] update installation doc for GPU package. --- .../CUDA-ExecutionProvider.md | 8 ++++---- .../TensorRT-ExecutionProvider.md | 6 +++--- docs/get-started/with-python.md | 10 +++++----- docs/install/index.md | 19 ++++++++++++------- 4 files changed, 24 insertions(+), 19 deletions(-) diff --git a/docs/execution-providers/CUDA-ExecutionProvider.md b/docs/execution-providers/CUDA-ExecutionProvider.md index fb4851a822cb2..81c0c4d270de3 100644 --- a/docs/execution-providers/CUDA-ExecutionProvider.md +++ b/docs/execution-providers/CUDA-ExecutionProvider.md @@ -35,14 +35,13 @@ Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/dep ONNX Runtime built with cuDNN 8.x is not compatible with cuDNN 9.x, and vice versa. You can choose the package based on CUDA and cuDNN major versions that match your runtime environment (For example, PyTorch 2.3 uses cuDNN 8.x, while PyTorch 2.4 or later used cuDNN 9.x). -Note: starting ORT 1.19, **CUDA 12.x** becomes default version when distributing ONNX Runtime GPU packages. +Note: starting ORT 1.19, **CUDA 12.x** becomes default version when distributing ONNX Runtime GPU packages in pypi. ### CUDA 12.x -To install CUDA 12 package, please look at [Install ORT](../install). - | ONNX Runtime | CUDA | cuDNN | Notes | |---------------|--------|-------|----------------------------------------------------------------------| +| 1.19.x | 12.x | 9.x | Avaiable in pypi. Compatible with PyTorch >= 2.4.0 for cuda 12.x. | | 1.18.1 | 12.x | 9.x | cuDNN 9 is required. No Java package. | | 1.18.0 | 12.x | 8.x | Java package is added. | | 1.17.x | 12.x | 8.x | Only C++/C# Nuget and Python packages are released. No Java package. | @@ -51,7 +50,8 @@ To install CUDA 12 package, please look at [Install ORT](../install). | ONNX Runtime | CUDA | cuDNN | Notes | |----------------------|--------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------| -| 1.18.x | 11.8 | 8.x | | +| 1.19.x | 11.8 | 8.x | Not available in pypi. See [Install ORT](../install) for detail. Compatible with PyTorch <= 2.3.1 for CUDA 11.8. | +| 1.18.x | 11.8 | 8.x | Available in pypi | | 1.17
1.16
1.15 | 11.8 | 8.2.4 (Linux)
8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2 up to 8.9 | | 1.14
1.13 | 11.6 | 8.2.4 (Linux)
8.5.0.96 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.5.2
libcublas 11.6.5.2
libcudnn 8.2.4 | | 1.12
1.11 | 11.4 | 8.2.4 (Linux)
8.2.2.26 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.5.2
libcublas 11.6.5.2
libcudnn 8.2.4 | diff --git a/docs/execution-providers/TensorRT-ExecutionProvider.md b/docs/execution-providers/TensorRT-ExecutionProvider.md index d6b9d76de656d..ded86899eee6e 100644 --- a/docs/execution-providers/TensorRT-ExecutionProvider.md +++ b/docs/execution-providers/TensorRT-ExecutionProvider.md @@ -31,9 +31,9 @@ Note: starting ORT 1.19, **CUDA 12** becomes default version when distributing O | ONNX Runtime | TensorRT | CUDA | | :----------- | :------- | :------------- | -| 1.19-main | 10.2 | **12.2**, 11.8 | -| 1.18 | 10.0 | 11.8, 12.2 | -| 1.17 | 8.6 | 11.8, 12.2 | +| 1.19-main | 10.2 | **12.x**, 11.8 | +| 1.18 | 10.0 | 11.8, 12.x | +| 1.17 | 8.6 | 11.8, 12.x | | 1.16 | 8.6 | 11.8 | | 1.15 | 8.6 | 11.8 | | 1.14 | 8.5 | 11.6 | diff --git a/docs/get-started/with-python.md b/docs/get-started/with-python.md index 8311eb091753b..ba7ba27baa2d6 100644 --- a/docs/get-started/with-python.md +++ b/docs/get-started/with-python.md @@ -30,7 +30,7 @@ pip install onnxruntime ### Install ONNX Runtime GPU (CUDA 12.x) -The default CUDA version for ORT is 12.2. +The default CUDA version for ORT is 12.x. ```bash pip install onnxruntime-gpu @@ -260,8 +260,8 @@ If using pip, run `pip install --upgrade pip` prior to downloading. |[onnxruntime](https://pypi.org/project/onnxruntime)|CPU (Release)| Windows (x64), Linux (x64, ARM64), Mac (X64), | |[ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly)|CPU (Dev) | Same as above | |[onnxruntime-gpu](https://pypi.org/project/onnxruntime-gpu)|GPU (Release)| Windows (x64), Linux (x64, ARM64) | -|[ort-nightly-gpu for CUDA 11.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) | -|[ort-nightly-gpu for CUDA 12.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-12-nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) | +|[ort-nightly-gpu for CUDA 11.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-11-nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) | +|[ort-nightly-gpu for CUDA 12.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) | Before installing nightly package, you will need install dependencies first. ``` @@ -270,12 +270,12 @@ python -m pip install coloredlogs flatbuffers numpy packaging protobuf sympy Example to install ort-nightly-gpu for CUDA 11.*: ``` -python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ +python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-11-nightly/pypi/simple/ ``` Example to install ort-nightly-gpu for CUDA 12.*: ``` -python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/ +python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ``` For Python compiler version notes, see [this page](https://github.com/microsoft/onnxruntime/tree/main/docs/Python_Dev_Notes.md) diff --git a/docs/install/index.md b/docs/install/index.md index 94cec0d9b34f6..766e6b8489898 100644 --- a/docs/install/index.md +++ b/docs/install/index.md @@ -47,17 +47,22 @@ pip install onnxruntime ``` #### Install ONNX Runtime GPU (CUDA 11.x) -The default CUDA version for ORT is 11.8. +For Cuda 11.x, please use the following instructions to install from [ORT Azure Devops Feed](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-11/PyPI/onnxruntime-gpu/overview) for 1.19.2 or later. ```bash -pip install onnxruntime-gpu +pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/ +``` + +For older version <= 1.18.1, you can download from pypi with explicit version like the following: +```bash +pip install onnxruntime-gpu==1.18.1 ``` #### Install ONNX Runtime GPU (CUDA 12.x) -For Cuda 12.x, please use the following instructions to install from [ORT Azure Devops Feed](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12/PyPI/onnxruntime-gpu/overview) +The default CUDA version for ORT in pypi is 12.x since 1.19.0. ```bash -pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/ +pip install onnxruntime-gpu ``` #### Install ONNX Runtime GPU (ROCm) @@ -96,7 +101,7 @@ dotnet add package Microsoft.ML.OnnxRuntime #### Install ONNX Runtime GPU (CUDA 12.x) -The default CUDA version for ORT is 12.2 +The default CUDA version for ORT is 12.x ```bash # GPU @@ -405,8 +410,8 @@ below: |--------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------| | Python | If using pip, run `pip install --upgrade pip` prior to downloading. | | | | | CPU: [**onnxruntime**](https://pypi.org/project/onnxruntime) | [ort-nightly (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly/overview) | | -| | GPU (CUDA/TensorRT) for CUDA 11.x: [**onnxruntime-gpu**](https://pypi.org/project/onnxruntime-gpu) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) | -| | GPU (CUDA/TensorRT) for CUDA 12.x: [**onnxruntime-gpu**](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12/PyPI/onnxruntime-gpu/overview/) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-12-nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) | +| | GPU (CUDA/TensorRT) for CUDA 12.x: [**onnxruntime-gpu**](https://pypi.org/project/onnxruntime-gpu) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) | +| | GPU (CUDA/TensorRT) for CUDA 11.x: [**onnxruntime-gpu**](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-11/PyPI/onnxruntime-gpu/overview/) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-11-nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) | | | GPU (DirectML): [**onnxruntime-directml**](https://pypi.org/project/onnxruntime-directml/) | [ort-nightly-directml (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-directml/overview/) | [View](../execution-providers/DirectML-ExecutionProvider.md#requirements) | | | OpenVINO: [**intel/onnxruntime**](https://github.com/intel/onnxruntime/releases/latest) - *Intel managed* | | [View](../build/eps.md#openvino) | | | TensorRT (Jetson): [**Jetson Zoo**](https://elinux.org/Jetson_Zoo#ONNX_Runtime) - *NVIDIA managed* | | |