Polygraphy: how to compare the precision layer by layer with TensorRT if I have a custom operator in my onnx (and a corresponding plugin in TensorRT)?

[MyraYu2022]

Description

I used to check my engine(tensorRT) layer by layer with onnx using polygraphy tool. But I have a custom operator writen by myself in my onnx, and a corresponding plugin in my tensorRT now and how can I compare them layer by layer using polygraphy?

I write the .sh script like this:

 polygraphy run my_model_with_custom_operator.onnx  \
                --onnxrt --trt  \
                --trt-outputs mark all  \
                --onnx-outputs mark all  \
                --atol 1e-3  --rtol 1e-3  \
                --fail-fast  \
                --val-range [0,1]  \
                --verbose 

And I get error as below:
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: customPlugin(-1) is not a registered function/op

Thank you very much.

Environment

TensorRT Version:
8.4.0.6

NVIDIA Driver Version:
535.183.01

CUDA Version:
11.3

CUDNN Version:
8.3.2.44

Operating System:

Python Version (if applicable):
3.9.7

PyTorch Version (if applicable):
1.13.0

[lix19937]

polygraphy support plugins load.

[lix19937]

Like follow

# Build and run TensorRT engine with plugins

polygraphy run modelB.onnx \
    --trt \
    --plugins ./AddScalarPlugin.so \
    > result.log 2>&1

ref https://docs.nvidia.com/deeplearning/tensorrt/polygraphy/docs/backend/trt/loader.html#polygraphy.backend.trt.loader.LoadPlugins