Different versions of TensorRT get different model inference results on GroundingDino model

[demuxin]

Description

I inference the groundingDino model using C++ TensorRT.

For the same model and the same image, TensorRT 8.6 can gets the correct detection boxes.

But when I update TensorRT to 10.4, can't get detection boxes.

Possible model result error caused by TensorRT 10.4, How can I analyze this issue?

By the way, I've tried multiple versions other than 8.6 (eg 9.3, 10.0, 10.1), None of them get detection boxes.

additional information below:

I load the save onnx model via C++ TensorRT and print the information for each layer.

TensorRT 8.6 loaded a model with 21060 layers and TensorRT 10.4 loaded a model with 37921 layers, why is the difference in the number of layers so large?

rt104_layers.txt
rt86_layers.txt

Environment

TensorRT Version: 8.6.1.6 / 10.4.0.26

NVIDIA GPU: GeForce RTX 3090

NVIDIA Driver Version: 535.183.06

CUDA Version: 12.2

Relevant Files

Model link: https://drive.google.com/file/d/1VRHKT7cswtDVXNUUmebbPmBSAOyd-fJN/view?usp=drive_link

[yuanyao-nv]

Can you please try TRT 10.5? There was a known accuracy bug that was fixed in 10.5. Thanks!

[demuxin]

Hi @yuanyao-nv , I tried TRT 10.5, but the model still has no output box.

How can this problem be solved?

[demuxin]

Hi @yuanyao-nv , I tried TRT 10.7, but the model output many wrong boxes.

How can this problem be solved?

[yuanyao-nv]

@demuxin Can you please provide the exact command or script you used to run this ONNX model?
Also, have you checked using onnxruntime to see if it produces the correct output?

[demuxin]

I doesn't check model using onnxruntime, but I tested the onnx model using C++ TensorRT 8.6, the output result is correct.

And I tested the same model using other version of C++ TensorRT, the output result is wrong.

So the onnx model shouldn't be a problem.

[demuxin]

I found that there are some different warning log when build the trt engine:

TensorRT 8.6:

[libprotobuf WARNING google/protobuf/io/coded_stream.cc:604] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:81] The total number of bytes read was 968870665
WARN: [AlgoLib] /nvidia/engine_impl.h : 18 , onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARN: [AlgoLib] /nvidia/engine_impl.h : 18 , onnx2trt_utils.cpp:400: One or more weights outside the range of INT32 was clamped

TensorRT 10.4:

WARN: [AlgoLib] /nvidia/engine_impl.h : 18 , ModelImporter.cpp:420: Make sure input input_ids has Int64 binding.
WARN: [AlgoLib] /nvidia/engine_impl.h : 18 , ModelImporter.cpp:420: Make sure input position_ids has Int64 binding.
WARN: [AlgoLib] /nvidia/engine_impl.h : 18 , ModelImporter.cpp:420: Make sure input token_type_ids has Int64 binding.

Would these warning logs be useful?
.
.
.

Can you please provide the exact command or script you used to run this ONNX model?

Our code has a relatively high degree of encapsulation, Extracting the code snippet out is not very convenient.

Can you run the onnx model using TensorRT 10.4, I can offer you input data files .

input86_0.txt
input86_1.txt
input86_2.txt
input86_3.txt
input86_4.txt
input86_5.txt