The trt model is used on different graphics cards

[wahaha]

Hello, is there any way that a trt model running on a 30 series graphics card can be inferred on a 20 series graphics card? For example, modify a parameter? Looking forward to your reply!

[lix19937]

You has better to build and run the engine a the same machine(gpu arch).

[wahaha]

Thank your very much for your answer! I want to ask is there no way to build and run engine at the different machine(gpu arch), ex. build the engine on the 30-series gpu arch and then run the engine on the 20-series gpu arch? Or is build and run engine at the different machine(gpu arch) prone to other problems? Looking forward to your reply!

[lix19937]

If GPU clock speeds differ between engine serialization and runtime systems, the tactics chosen by the serialization system may not be optimal for the runtime system and may incur some performance degradation.

If it is impossible to build a TensorRT engine for each individual type of GPU, you can select several GPUs to build engines with and run the engine on different GPUs with the same architecture. For example, among the NVIDIA RTX 40xx GPUs, you can build an engine with RTX 4090 and an engine with RTX 4070. At runtime, you can use the RTX 4090 engine on an RTX 4080 GPU and the 4070 engine on all smaller GPUs. In most cases, the engine will run without functional issues and with only a small performance drop compared to running the engine built with the same GPU.

you can ref https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#hardware-compat and https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#compatibility-checks