Performance about TensorRT

[zhangchbin]

I tried to accelerate my TensorFlow code by using TensorRT, but it didn't get any improvement?
in frozen_model, num of all_nodes = 893 in TensorRT_model, num of trt_engine_nodes = 0 in TensorRT_model, num of all_nodes = 831
Is there anything wrong? Thanks!

[ardianumam]

I ever encountered such problem in Jetson TX2. My solution was updating Tensorflow version to 1.11.

[zhangchbin]

I ever encountered such problem in Jetson TX2. My solution was updating Tensorflow version to 1.11.

Actually, I used cuda10.0, cudnn7.4.3, Tensorflow 1.13. Is there anything wrong?

[ardianumam]

I'm not sure. Maybe you can try to optimize another model first, e.g., in code 1 in this repo, to make sure all settings work.

[PythonImageDeveloper]

Hi @ardianumam @godman2016
Can you solved this problem?
I optimized my trained model (1 class), the ssdlite_mobilenetv2 and ssd_resnet50, with TensorRT, but the performance did't improve significantly, I reach from 0.12 sec to 0.11 sec on GTX 1080 ti, why?
I installed Tensorflow 1.12.0 , cuda 9 , TensorRT 4.0.1.6 packages on Ubuntu 16.04.

[zhangchbin]

Hi @ardianumam @godman2016
Can you solved this problem?
I optimized my trained model (1 class), the ssdlite_mobilenetv2 and ssd_resnet50, with TensorRT, but the performance did't improve significantly, I reach from 0.12 sec to 0.11 sec on GTX 1080 ti, why?
I installed Tensorflow 1.12.0 , cuda 9 , TensorRT 4.0.1.6 packages on Ubuntu 16.04.

I haven't solved this problem. Maybe it can accelerate some models, not all.

[ardianumam]

Hi @ardianumam @godman2016
Can you solved this problem?
I optimized my trained model (1 class), the ssdlite_mobilenetv2 and ssd_resnet50, with TensorRT, but the performance did't improve significantly, I reach from 0.12 sec to 0.11 sec on GTX 1080 ti, why?
I installed Tensorflow 1.12.0 , cuda 9 , TensorRT 4.0.1.6 packages on Ubuntu 16.04.

I haven't solved this problem. Maybe it can accelerate some models, not all.

I also tried to perform TRT optimization several days ago to SSD MobileNet1 with 1 class. I got 45 FPS in Jetson TX2 for both before & after TRT optimization. My temporary conclusion is: (i) TRT is less optimized for network like MobileNet, maybe due its separable convolution that already performs very small computation so that there is less space for optimization. (ii) When I use more classes (e.g., 80 classes in COCO), there is more difference after TRT optimization (TRT seems optimizing conv. operation for the output prediction which is proportional to the number of classes).