Skip to content

Commit

Permalink
[TensorRT EP] Update trt link (#23532)
Browse files Browse the repository at this point in the history
  • Loading branch information
yf711 authored Jan 29, 2025
1 parent 39c4083 commit a363196
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 8 deletions.
2 changes: 1 addition & 1 deletion docs/build/eps.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ See more information on the TensorRT Execution Provider [here](../execution-prov
{: .no_toc }

* Follow [instructions for CUDA execution provider](#cuda) to install CUDA and cuDNN, and setup environment variables.
* Follow [instructions for installing TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html)
* Follow [instructions for installing TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/latest/installing-tensorrt/installing.html)
* The TensorRT execution provider for ONNX Runtime is built and tested with TensorRT 10.0.
* The path to TensorRT installation must be provided via the `--tensorrt_home` parameter.
* ONNX Runtime uses TensorRT built-in parser from `tensorrt_home` by default.
Expand Down
12 changes: 6 additions & 6 deletions docs/execution-providers/TensorRT-ExecutionProvider.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,22 +417,22 @@ TensorRT configurations can be set by execution provider options. It's useful wh
* Description: control if sparsity can be used by TRT.
* Check `--sparsity` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).
* Check `--sparsity` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/latest/reference/command-line-programs.html#commonly-used-command-line-flags).
##### trt_builder_optimization_level
* Description: set the builder optimization level.
> WARNING: levels below 3 do not guarantee good engine performance, but greatly improve build time. Default 3, valid range [0-5]. Check `--builderOptimizationLevel` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).
> WARNING: levels below 3 do not guarantee good engine performance, but greatly improve build time. Default 3, valid range [0-5]. Check `--builderOptimizationLevel` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/latest/reference/command-line-programs.html#commonly-used-command-line-flags).
##### trt_auxiliary_streams
* Description: set maximum number of auxiliary streams per inference stream.
* Setting this value to 0 will lead to optimal memory usage.
* Default -1 = heuristics.
* Check `--maxAuxStreams` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).
* Check `--maxAuxStreams` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/latest/reference/command-line-programs.html#commonly-used-command-line-flags).
##### trt_tactic_sources
Expand Down Expand Up @@ -519,11 +519,11 @@ Following environment variables can be set for TensorRT execution provider. Clic
* `ORT_TENSORRT_BUILD_HEURISTICS_ENABLE`: Build engine using heuristics to reduce build time. Default 0 = false, nonzero = true.
* `ORT_TENSORRT_SPARSITY_ENABLE`: Control if sparsity can be used by TRT. Default 0 = false, 1 = true. Check `--sparsity` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).
* `ORT_TENSORRT_SPARSITY_ENABLE`: Control if sparsity can be used by TRT. Default 0 = false, 1 = true. Check `--sparsity` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/latest/reference/command-line-programs.html#commonly-used-command-line-flags).
* `ORT_TENSORRT_BUILDER_OPTIMIZATION_LEVEL`: Set the builder optimization level. WARNING: levels below 3 do not guarantee good engine performance, but greatly improve build time. Default 3, valid range [0-5]. Check `--builderOptimizationLevel` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).
* `ORT_TENSORRT_BUILDER_OPTIMIZATION_LEVEL`: Set the builder optimization level. WARNING: levels below 3 do not guarantee good engine performance, but greatly improve build time. Default 3, valid range [0-5]. Check `--builderOptimizationLevel` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/latest/reference/command-line-programs.html#commonly-used-command-line-flags).
* `ORT_TENSORRT_AUXILIARY_STREAMS`: Set maximum number of auxiliary streams per inference stream. Setting this value to 0 will lead to optimal memory usage. Default -1 = heuristics. Check `--maxAuxStreams` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).
* `ORT_TENSORRT_AUXILIARY_STREAMS`: Set maximum number of auxiliary streams per inference stream. Setting this value to 0 will lead to optimal memory usage. Default -1 = heuristics. Check `--maxAuxStreams` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/latest/reference/command-line-programs.html#commonly-used-command-line-flags).
* `ORT_TENSORRT_TACTIC_SOURCES`: Specify the tactics to be used by adding (+) or removing (-) tactics from the default tactic sources (default = all available tactics) e.g. "-CUDNN,+CUBLAS" available keys: "CUBLAS", "CUBLAS_LT", "CUDNN" or "EDGE_MASK_CONVOLUTIONS".
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/csharp/csharp-gpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ See this table for supported versions:
NOTE: Full table can be found [here](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements)


- Follow section [2. Installing cuDNN on Windows](https://docs.nvidia.com/deeplearning/cudnn/installation/windows.html). NOTE: Skip step 5 in section 2.3 on updating Visual Studio settings, this is only for C++ projects.
- Follow section [2. Installing cuDNN on Windows](https://docs.nvidia.com/deeplearning/cudnn/installation/latest/windows.html). NOTE: Skip step 5 in section 2.3 on updating Visual Studio settings, this is only for C++ projects.

- Restart your computer and verify the installation by running the following command or in python with PyTorch:

Expand Down

0 comments on commit a363196

Please sign in to comment.