Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

trtexec failure of TensorRT 8.6.1.6 when running trtexec on GPU RTX4090 #3590

Open
peter5232 opened this issue Jan 11, 2024 · 7 comments
Open

trtexec failure of TensorRT 8.6.1.6 when running trtexec on GPU RTX4090 #3590

peter5232 opened this issue Jan 11, 2024 · 7 comments
Assignees
@zerollzeng
Labels
triaged Issue has been triaged by maintainers

Comments

@peter5232
Copy link

Environment

TensorRT Version:

NVIDIA GPU: RTX 4090

NVIDIA Driver Version: 535.129.03

CUDA Version: 11.8

CUDNN Version: 8.9.6.50

Operating System: ubuntu 22.04

Python Version (if applicable): 3.9

Tensorflow Version (if applicable):

PyTorch Version (if applicable): 1.13

Baremetal or Container (if so, version):

Relevant Files

Model link: https://www.dropbox.com/scl/fi/vzgb4iew1lvj64h6adnjt/model.onnx?rlkey=vqq56hc2t91r7b1m078ks7ycl&dl=0

Steps To Reproduce

I want to transform onnx to engine by using code trtexec --onnx=model.onnx --saveEngine=model.trt --verbose. Then an error was reported. /MaxPool: at least 5 dimensions are required for input.
But I don't know why! The maxpool that reported the error is used here.
image

&&&& RUNNING TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=model.onnx --saveEngine=model.trt --verbose
[01/11/2024-17:39:45] [I] === Model Options ===
[01/11/2024-17:39:45] [I] Format: ONNX
[01/11/2024-17:39:45] [I] Model: model.onnx
[01/11/2024-17:39:45] [I] Output:
[01/11/2024-17:39:45] [I] === Build Options ===
[01/11/2024-17:39:45] [I] Max batch: explicit batch
[01/11/2024-17:39:45] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[01/11/2024-17:39:45] [I] minTiming: 1
[01/11/2024-17:39:45] [I] avgTiming: 8
[01/11/2024-17:39:45] [I] Precision: FP32
[01/11/2024-17:39:45] [I] LayerPrecisions: 
[01/11/2024-17:39:45] [I] Layer Device Types: 
[01/11/2024-17:39:45] [I] Calibration: 
[01/11/2024-17:39:45] [I] Refit: Disabled
[01/11/2024-17:39:45] [I] Version Compatible: Disabled
[01/11/2024-17:39:45] [I] TensorRT runtime: full
[01/11/2024-17:39:45] [I] Lean DLL Path: 
[01/11/2024-17:39:45] [I] Tempfile Controls: { in_memory: allow, temporary: allow }
[01/11/2024-17:39:45] [I] Exclude Lean Runtime: Disabled
[01/11/2024-17:39:45] [I] Sparsity: Disabled
[01/11/2024-17:39:45] [I] Safe mode: Disabled
[01/11/2024-17:39:45] [I] Build DLA standalone loadable: Disabled
[01/11/2024-17:39:45] [I] Allow GPU fallback for DLA: Disabled
[01/11/2024-17:39:45] [I] DirectIO mode: Disabled
[01/11/2024-17:39:45] [I] Restricted mode: Disabled
[01/11/2024-17:39:45] [I] Skip inference: Disabled
[01/11/2024-17:39:45] [I] Save engine: model.trt
[01/11/2024-17:39:45] [I] Load engine: 
[01/11/2024-17:39:45] [I] Profiling verbosity: 0
[01/11/2024-17:39:45] [I] Tactic sources: Using default tactic sources
[01/11/2024-17:39:45] [I] timingCacheMode: local
[01/11/2024-17:39:45] [I] timingCacheFile: 
[01/11/2024-17:39:45] [I] Heuristic: Disabled
[01/11/2024-17:39:45] [I] Preview Features: Use default preview flags.
[01/11/2024-17:39:45] [I] MaxAuxStreams: -1
[01/11/2024-17:39:45] [I] BuilderOptimizationLevel: -1
[01/11/2024-17:39:45] [I] Input(s)s format: fp32:CHW
[01/11/2024-17:39:45] [I] Output(s)s format: fp32:CHW
[01/11/2024-17:39:45] [I] Input build shapes: model
[01/11/2024-17:39:45] [I] Input calibration shapes: model
[01/11/2024-17:39:45] [I] === System Options ===
[01/11/2024-17:39:45] [I] Device: 0
[01/11/2024-17:39:45] [I] DLACore: 
[01/11/2024-17:39:45] [I] Plugins:
[01/11/2024-17:39:45] [I] setPluginsToSerialize:
[01/11/2024-17:39:45] [I] dynamicPlugins:
[01/11/2024-17:39:45] [I] ignoreParsedPluginLibs: 0
[01/11/2024-17:39:45] [I] 
[01/11/2024-17:39:45] [I] === Inference Options ===
[01/11/2024-17:39:45] [I] Batch: Explicit
[01/11/2024-17:39:45] [I] Input inference shapes: model
[01/11/2024-17:39:45] [I] Iterations: 10
[01/11/2024-17:39:45] [I] Duration: 3s (+ 200ms warm up)
[01/11/2024-17:39:45] [I] Sleep time: 0ms
[01/11/2024-17:39:45] [I] Idle time: 0ms
[01/11/2024-17:39:45] [I] Inference Streams: 1
[01/11/2024-17:39:45] [I] ExposeDMA: Disabled
[01/11/2024-17:39:45] [I] Data transfers: Enabled
[01/11/2024-17:39:45] [I] Spin-wait: Disabled
[01/11/2024-17:39:45] [I] Multithreading: Disabled
[01/11/2024-17:39:45] [I] CUDA Graph: Disabled
[01/11/2024-17:39:45] [I] Separate profiling: Disabled
[01/11/2024-17:39:45] [I] Time Deserialize: Disabled
[01/11/2024-17:39:45] [I] Time Refit: Disabled
[01/11/2024-17:39:45] [I] NVTX verbosity: 0
[01/11/2024-17:39:45] [I] Persistent Cache Ratio: 0
[01/11/2024-17:39:45] [I] Inputs:
[01/11/2024-17:39:45] [I] === Reporting Options ===
[01/11/2024-17:39:45] [I] Verbose: Enabled
[01/11/2024-17:39:45] [I] Averages: 10 inferences
[01/11/2024-17:39:45] [I] Percentiles: 90,95,99
[01/11/2024-17:39:45] [I] Dump refittable layers:Disabled
[01/11/2024-17:39:45] [I] Dump output: Disabled
[01/11/2024-17:39:45] [I] Profile: Disabled
[01/11/2024-17:39:45] [I] Export timing to JSON file: 
[01/11/2024-17:39:45] [I] Export output to JSON file: 
[01/11/2024-17:39:45] [I] Export profile to JSON file: 
[01/11/2024-17:39:45] [I] 
[01/11/2024-17:39:45] [I] === Device Information ===
[01/11/2024-17:39:45] [I] Selected Device: NVIDIA GeForce RTX 4090
[01/11/2024-17:39:45] [I] Compute Capability: 8.9
[01/11/2024-17:39:45] [I] SMs: 128
[01/11/2024-17:39:45] [I] Device Global Memory: 24217 MiB
[01/11/2024-17:39:45] [I] Shared Memory per SM: 100 KiB
[01/11/2024-17:39:45] [I] Memory Bus Width: 384 bits (ECC disabled)
[01/11/2024-17:39:45] [I] Application Compute Clock Rate: 2.52 GHz
[01/11/2024-17:39:45] [I] Application Memory Clock Rate: 10.501 GHz
[01/11/2024-17:39:45] [I] 
[01/11/2024-17:39:45] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
[01/11/2024-17:39:45] [I] 
[01/11/2024-17:39:45] [I] TensorRT version: 8.6.1
[01/11/2024-17:39:45] [I] Loading standard plugins
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::ModulatedDeformConv2d version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::Proposal version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::ROIAlign_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::Split version 1
[01/11/2024-17:39:45] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[01/11/2024-17:39:45] [I] [TRT] [MemUsageChange] Init CUDA: CPU +13, GPU +0, now: CPU 19, GPU 5895 (MiB)
[01/11/2024-17:39:45] [V] [TRT] Trying to load shared library libnvinfer_builder_resource.so.8.6.1
[01/11/2024-17:39:45] [V] [TRT] Loaded shared library libnvinfer_builder_resource.so.8.6.1
[01/11/2024-17:39:50] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +1450, GPU +266, now: CPU 1545, GPU 6161 (MiB)
[01/11/2024-17:39:50] [V] [TRT] CUDA lazy loading is enabled.
[01/11/2024-17:39:50] [I] Start parsing network model.
[01/11/2024-17:39:50] [I] [TRT] ----------------------------------------------------------------
[01/11/2024-17:39:50] [I] [TRT] Input filename:   model.onnx
[01/11/2024-17:39:50] [I] [TRT] ONNX IR version:  0.0.8
[01/11/2024-17:39:50] [I] [TRT] Opset version:    16
[01/11/2024-17:39:50] [I] [TRT] Producer name:    pytorch
[01/11/2024-17:39:50] [I] [TRT] Producer version: 1.13.0
[01/11/2024-17:39:50] [I] [TRT] Domain:           
[01/11/2024-17:39:50] [I] [TRT] Model version:    0
[01/11/2024-17:39:50] [I] [TRT] Doc string:       
[01/11/2024-17:39:50] [I] [TRT] ----------------------------------------------------------------
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::ModulatedDeformConv2d version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::ROIAlign_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::Split version 1
[01/11/2024-17:39:50] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1
[01/11/2024-17:39:50] [V] [TRT] Adding network input: input.1 with dtype: float32, dimensions: (1, 1, 682, 1024)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: input.1 for ONNX tensor: input.1
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv1a.weight with dtype: float32, dimensions: (64, 1, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv1a.weight for ONNX tensor: conv1a.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv1a.bias with dtype: float32, dimensions: (64)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv1a.bias for ONNX tensor: conv1a.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv1b.weight with dtype: float32, dimensions: (64, 64, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv1b.weight for ONNX tensor: conv1b.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv1b.bias with dtype: float32, dimensions: (64)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv1b.bias for ONNX tensor: conv1b.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv2a.weight with dtype: float32, dimensions: (64, 64, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv2a.weight for ONNX tensor: conv2a.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv2a.bias with dtype: float32, dimensions: (64)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv2a.bias for ONNX tensor: conv2a.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv2b.weight with dtype: float32, dimensions: (64, 64, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv2b.weight for ONNX tensor: conv2b.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv2b.bias with dtype: float32, dimensions: (64)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv2b.bias for ONNX tensor: conv2b.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv3a.weight with dtype: float32, dimensions: (128, 64, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv3a.weight for ONNX tensor: conv3a.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv3a.bias with dtype: float32, dimensions: (128)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv3a.bias for ONNX tensor: conv3a.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv3b.weight with dtype: float32, dimensions: (128, 128, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv3b.weight for ONNX tensor: conv3b.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv3b.bias with dtype: float32, dimensions: (128)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv3b.bias for ONNX tensor: conv3b.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv4a.weight with dtype: float32, dimensions: (128, 128, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv4a.weight for ONNX tensor: conv4a.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv4a.bias with dtype: float32, dimensions: (128)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv4a.bias for ONNX tensor: conv4a.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv4b.weight with dtype: float32, dimensions: (128, 128, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv4b.weight for ONNX tensor: conv4b.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: conv4b.bias with dtype: float32, dimensions: (128)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: conv4b.bias for ONNX tensor: conv4b.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convPa.weight with dtype: float32, dimensions: (256, 128, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convPa.weight for ONNX tensor: convPa.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convPa.bias with dtype: float32, dimensions: (256)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convPa.bias for ONNX tensor: convPa.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convPb.weight with dtype: float32, dimensions: (65, 256, 1, 1)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convPb.weight for ONNX tensor: convPb.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convPb.bias with dtype: float32, dimensions: (65)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convPb.bias for ONNX tensor: convPb.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convDa.weight with dtype: float32, dimensions: (256, 128, 3, 3)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convDa.weight for ONNX tensor: convDa.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convDa.bias with dtype: float32, dimensions: (256)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convDa.bias for ONNX tensor: convDa.bias
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convDb.weight with dtype: float32, dimensions: (256, 256, 1, 1)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convDb.weight for ONNX tensor: convDb.weight
[01/11/2024-17:39:50] [V] [TRT] Adding network input: convDb.bias with dtype: float32, dimensions: (256)
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: convDb.bias for ONNX tensor: convDb.bias
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv1a/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: input.1
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv1a.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv1a.bias
[01/11/2024-17:39:50] [V] [TRT] /conv1a/Conv [Conv] inputs: [input.1 -> (1, 1, 682, 1024)[FLOAT]], [conv1a.weight -> (64, 1, 3, 3)[FLOAT]], [conv1a.bias -> (64)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv1a/Conv for ONNX node: /conv1a/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv1a/Conv_output_0 for ONNX tensor: /conv1a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv1a/Conv [Conv] outputs: [/conv1a/Conv_output_0 -> (1, 64, 682, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv1a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu/Relu [Relu] inputs: [/conv1a/Conv_output_0 -> (1, 64, 682, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu/Relu for ONNX node: /relu/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu/Relu_output_0 for ONNX tensor: /relu/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu/Relu [Relu] outputs: [/relu/Relu_output_0 -> (1, 64, 682, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv1b/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv1b.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv1b.bias
[01/11/2024-17:39:50] [V] [TRT] /conv1b/Conv [Conv] inputs: [/relu/Relu_output_0 -> (1, 64, 682, 1024)[FLOAT]], [conv1b.weight -> (64, 64, 3, 3)[FLOAT]], [conv1b.bias -> (64)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv1b/Conv for ONNX node: /conv1b/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv1b/Conv_output_0 for ONNX tensor: /conv1b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv1b/Conv [Conv] outputs: [/conv1b/Conv_output_0 -> (1, 64, 682, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_1/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv1b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_1/Relu [Relu] inputs: [/conv1b/Conv_output_0 -> (1, 64, 682, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_1/Relu for ONNX node: /relu_1/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_1/Relu_output_0 for ONNX tensor: /relu_1/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_1/Relu [Relu] outputs: [/relu_1/Relu_output_0 -> (1, 64, 682, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /pool/MaxPool [MaxPool]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_1/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /pool/MaxPool [MaxPool] inputs: [/relu_1/Relu_output_0 -> (1, 64, 682, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /pool/MaxPool for ONNX node: /pool/MaxPool
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /pool/MaxPool_output_0 for ONNX tensor: /pool/MaxPool_output_0
[01/11/2024-17:39:50] [V] [TRT] /pool/MaxPool [MaxPool] outputs: [/pool/MaxPool_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv2a/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /pool/MaxPool_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv2a.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv2a.bias
[01/11/2024-17:39:50] [V] [TRT] /conv2a/Conv [Conv] inputs: [/pool/MaxPool_output_0 -> (1, 64, 341, 512)[FLOAT]], [conv2a.weight -> (64, 64, 3, 3)[FLOAT]], [conv2a.bias -> (64)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv2a/Conv for ONNX node: /conv2a/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv2a/Conv_output_0 for ONNX tensor: /conv2a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv2a/Conv [Conv] outputs: [/conv2a/Conv_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_2/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv2a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_2/Relu [Relu] inputs: [/conv2a/Conv_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_2/Relu for ONNX node: /relu_2/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_2/Relu_output_0 for ONNX tensor: /relu_2/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_2/Relu [Relu] outputs: [/relu_2/Relu_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv2b/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_2/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv2b.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv2b.bias
[01/11/2024-17:39:50] [V] [TRT] /conv2b/Conv [Conv] inputs: [/relu_2/Relu_output_0 -> (1, 64, 341, 512)[FLOAT]], [conv2b.weight -> (64, 64, 3, 3)[FLOAT]], [conv2b.bias -> (64)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv2b/Conv for ONNX node: /conv2b/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv2b/Conv_output_0 for ONNX tensor: /conv2b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv2b/Conv [Conv] outputs: [/conv2b/Conv_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_3/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv2b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_3/Relu [Relu] inputs: [/conv2b/Conv_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_3/Relu for ONNX node: /relu_3/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_3/Relu_output_0 for ONNX tensor: /relu_3/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_3/Relu [Relu] outputs: [/relu_3/Relu_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /pool_1/MaxPool [MaxPool]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_3/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /pool_1/MaxPool [MaxPool] inputs: [/relu_3/Relu_output_0 -> (1, 64, 341, 512)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /pool_1/MaxPool for ONNX node: /pool_1/MaxPool
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /pool_1/MaxPool_output_0 for ONNX tensor: /pool_1/MaxPool_output_0
[01/11/2024-17:39:50] [V] [TRT] /pool_1/MaxPool [MaxPool] outputs: [/pool_1/MaxPool_output_0 -> (1, 64, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv3a/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /pool_1/MaxPool_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv3a.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv3a.bias
[01/11/2024-17:39:50] [V] [TRT] /conv3a/Conv [Conv] inputs: [/pool_1/MaxPool_output_0 -> (1, 64, 170, 256)[FLOAT]], [conv3a.weight -> (128, 64, 3, 3)[FLOAT]], [conv3a.bias -> (128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv3a/Conv for ONNX node: /conv3a/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv3a/Conv_output_0 for ONNX tensor: /conv3a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv3a/Conv [Conv] outputs: [/conv3a/Conv_output_0 -> (1, 128, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_4/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv3a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_4/Relu [Relu] inputs: [/conv3a/Conv_output_0 -> (1, 128, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_4/Relu for ONNX node: /relu_4/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_4/Relu_output_0 for ONNX tensor: /relu_4/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_4/Relu [Relu] outputs: [/relu_4/Relu_output_0 -> (1, 128, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv3b/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_4/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv3b.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv3b.bias
[01/11/2024-17:39:50] [V] [TRT] /conv3b/Conv [Conv] inputs: [/relu_4/Relu_output_0 -> (1, 128, 170, 256)[FLOAT]], [conv3b.weight -> (128, 128, 3, 3)[FLOAT]], [conv3b.bias -> (128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv3b/Conv for ONNX node: /conv3b/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv3b/Conv_output_0 for ONNX tensor: /conv3b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv3b/Conv [Conv] outputs: [/conv3b/Conv_output_0 -> (1, 128, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_5/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv3b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_5/Relu [Relu] inputs: [/conv3b/Conv_output_0 -> (1, 128, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_5/Relu for ONNX node: /relu_5/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_5/Relu_output_0 for ONNX tensor: /relu_5/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_5/Relu [Relu] outputs: [/relu_5/Relu_output_0 -> (1, 128, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /pool_2/MaxPool [MaxPool]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_5/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /pool_2/MaxPool [MaxPool] inputs: [/relu_5/Relu_output_0 -> (1, 128, 170, 256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /pool_2/MaxPool for ONNX node: /pool_2/MaxPool
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /pool_2/MaxPool_output_0 for ONNX tensor: /pool_2/MaxPool_output_0
[01/11/2024-17:39:50] [V] [TRT] /pool_2/MaxPool [MaxPool] outputs: [/pool_2/MaxPool_output_0 -> (1, 128, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv4a/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /pool_2/MaxPool_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv4a.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv4a.bias
[01/11/2024-17:39:50] [V] [TRT] /conv4a/Conv [Conv] inputs: [/pool_2/MaxPool_output_0 -> (1, 128, 85, 128)[FLOAT]], [conv4a.weight -> (128, 128, 3, 3)[FLOAT]], [conv4a.bias -> (128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv4a/Conv for ONNX node: /conv4a/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv4a/Conv_output_0 for ONNX tensor: /conv4a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv4a/Conv [Conv] outputs: [/conv4a/Conv_output_0 -> (1, 128, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_6/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv4a/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_6/Relu [Relu] inputs: [/conv4a/Conv_output_0 -> (1, 128, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_6/Relu for ONNX node: /relu_6/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_6/Relu_output_0 for ONNX tensor: /relu_6/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_6/Relu [Relu] outputs: [/relu_6/Relu_output_0 -> (1, 128, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /conv4b/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_6/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv4b.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: conv4b.bias
[01/11/2024-17:39:50] [V] [TRT] /conv4b/Conv [Conv] inputs: [/relu_6/Relu_output_0 -> (1, 128, 85, 128)[FLOAT]], [conv4b.weight -> (128, 128, 3, 3)[FLOAT]], [conv4b.bias -> (128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /conv4b/Conv for ONNX node: /conv4b/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /conv4b/Conv_output_0 for ONNX tensor: /conv4b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /conv4b/Conv [Conv] outputs: [/conv4b/Conv_output_0 -> (1, 128, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_7/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /conv4b/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_7/Relu [Relu] inputs: [/conv4b/Conv_output_0 -> (1, 128, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_7/Relu for ONNX node: /relu_7/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_7/Relu_output_0 for ONNX tensor: /relu_7/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_7/Relu [Relu] outputs: [/relu_7/Relu_output_0 -> (1, 128, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /convPa/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_7/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: convPa.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: convPa.bias
[01/11/2024-17:39:50] [V] [TRT] /convPa/Conv [Conv] inputs: [/relu_7/Relu_output_0 -> (1, 128, 85, 128)[FLOAT]], [convPa.weight -> (256, 128, 3, 3)[FLOAT]], [convPa.bias -> (256)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /convPa/Conv for ONNX node: /convPa/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /convPa/Conv_output_0 for ONNX tensor: /convPa/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /convPa/Conv [Conv] outputs: [/convPa/Conv_output_0 -> (1, 256, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /relu_8/Relu [Relu]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /convPa/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_8/Relu [Relu] inputs: [/convPa/Conv_output_0 -> (1, 256, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /relu_8/Relu for ONNX node: /relu_8/Relu
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /relu_8/Relu_output_0 for ONNX tensor: /relu_8/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] /relu_8/Relu [Relu] outputs: [/relu_8/Relu_output_0 -> (1, 256, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /convPb/Conv [Conv]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /relu_8/Relu_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: convPb.weight
[01/11/2024-17:39:50] [V] [TRT] Searching for input: convPb.bias
[01/11/2024-17:39:50] [V] [TRT] /convPb/Conv [Conv] inputs: [/relu_8/Relu_output_0 -> (1, 256, 85, 128)[FLOAT]], [convPb.weight -> (65, 256, 1, 1)[FLOAT]], [convPb.bias -> (65)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Kernel weights are not set yet. Kernel weights must be set using setInput(1, kernel_tensor) API call.
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /convPb/Conv for ONNX node: /convPb/Conv
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /convPb/Conv_output_0 for ONNX tensor: /convPb/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /convPb/Conv [Conv] outputs: [/convPb/Conv_output_0 -> (1, 65, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Softmax [Softmax]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /convPb/Conv_output_0
[01/11/2024-17:39:50] [V] [TRT] /Softmax [Softmax] inputs: [/convPb/Conv_output_0 -> (1, 65, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /Softmax for ONNX node: /Softmax
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /Softmax_output_0 for ONNX tensor: /Softmax_output_0
[01/11/2024-17:39:50] [V] [TRT] /Softmax [Softmax] outputs: [/Softmax_output_0 -> (1, 65, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant [Constant] inputs: 
[01/11/2024-17:39:50] [W] [TRT] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[01/11/2024-17:39:50] [V] [TRT] /Constant [Constant] outputs: [/Constant_output_0 -> ()[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant_1 [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant_1 [Constant] inputs: 
[01/11/2024-17:39:50] [V] [TRT] /Constant_1 [Constant] outputs: [/Constant_1_output_0 -> (1)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant_2 [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant_2 [Constant] inputs: 
[01/11/2024-17:39:50] [V] [TRT] /Constant_2 [Constant] outputs: [/Constant_2_output_0 -> (1)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant_3 [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant_3 [Constant] inputs: 
[01/11/2024-17:39:50] [V] [TRT] /Constant_3 [Constant] outputs: [/Constant_3_output_0 -> (1)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant_4 [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant_4 [Constant] inputs: 
[01/11/2024-17:39:50] [V] [TRT] /Constant_4 [Constant] outputs: [/Constant_4_output_0 -> (1)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Slice [Slice]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Softmax_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Constant_2_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Constant_3_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Constant_1_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Constant_4_output_0
[01/11/2024-17:39:50] [V] [TRT] /Slice [Slice] inputs: [/Softmax_output_0 -> (1, 65, 85, 128)[FLOAT]], [/Constant_2_output_0 -> (1)[INT32]], [/Constant_3_output_0 -> (1)[INT32]], [/Constant_1_output_0 -> (1)[INT32]], [/Constant_4_output_0 -> (1)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /Slice for ONNX node: /Slice
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /Slice_output_0 for ONNX tensor: /Slice_output_0
[01/11/2024-17:39:50] [V] [TRT] /Slice [Slice] outputs: [/Slice_output_0 -> (1, 64, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Transpose [Transpose]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Slice_output_0
[01/11/2024-17:39:50] [V] [TRT] /Transpose [Transpose] inputs: [/Slice_output_0 -> (1, 64, 85, 128)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /Transpose for ONNX node: /Transpose
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /Transpose_output_0 for ONNX tensor: /Transpose_output_0
[01/11/2024-17:39:50] [V] [TRT] /Transpose [Transpose] outputs: [/Transpose_output_0 -> (1, 85, 128, 64)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant_5 [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant_5 [Constant] inputs: 
[01/11/2024-17:39:50] [V] [TRT] /Constant_5 [Constant] outputs: [/Constant_5_output_0 -> (5)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Reshape [Reshape]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Transpose_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Constant_5_output_0
[01/11/2024-17:39:50] [V] [TRT] /Reshape [Reshape] inputs: [/Transpose_output_0 -> (1, 85, 128, 64)[FLOAT]], [/Constant_5_output_0 -> (5)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /Reshape for ONNX node: /Reshape
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /Reshape_output_0 for ONNX tensor: /Reshape_output_0
[01/11/2024-17:39:50] [V] [TRT] /Reshape [Reshape] outputs: [/Reshape_output_0 -> (1, 85, 128, 8, 8)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Transpose_1 [Transpose]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Reshape_output_0
[01/11/2024-17:39:50] [V] [TRT] /Transpose_1 [Transpose] inputs: [/Reshape_output_0 -> (1, 85, 128, 8, 8)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /Transpose_1 for ONNX node: /Transpose_1
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /Transpose_1_output_0 for ONNX tensor: /Transpose_1_output_0
[01/11/2024-17:39:50] [V] [TRT] /Transpose_1 [Transpose] outputs: [/Transpose_1_output_0 -> (1, 85, 8, 128, 8)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant_6 [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant_6 [Constant] inputs: 
[01/11/2024-17:39:50] [V] [TRT] /Constant_6 [Constant] outputs: [/Constant_6_output_0 -> (3)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Reshape_1 [Reshape]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Transpose_1_output_0
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Constant_6_output_0
[01/11/2024-17:39:50] [V] [TRT] /Reshape_1 [Reshape] inputs: [/Transpose_1_output_0 -> (1, 85, 8, 128, 8)[FLOAT]], [/Constant_6_output_0 -> (3)[INT32]], 
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /Reshape_1 for ONNX node: /Reshape_1
[01/11/2024-17:39:50] [V] [TRT] Registering tensor: /Reshape_1_output_0 for ONNX tensor: /Reshape_1_output_0
[01/11/2024-17:39:50] [V] [TRT] /Reshape_1 [Reshape] outputs: [/Reshape_1_output_0 -> (1, 680, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /Constant_7 [Constant]
[01/11/2024-17:39:50] [V] [TRT] /Constant_7 [Constant] inputs: 
[01/11/2024-17:39:50] [V] [TRT] /Constant_7 [Constant] outputs: [/Constant_7_output_0 -> (1, 680, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Parsing node: /MaxPool [MaxPool]
[01/11/2024-17:39:50] [V] [TRT] Searching for input: /Reshape_1_output_0
[01/11/2024-17:39:50] [V] [TRT] /MaxPool [MaxPool] inputs: [/Reshape_1_output_0 -> (1, 680, 1024)[FLOAT]], 
[01/11/2024-17:39:50] [V] [TRT] Original shape: (1, 680, 1024), unsqueezing to: (1, 680, 1024, 1)
[01/11/2024-17:39:50] [V] [TRT] Registering layer: /MaxPool for ONNX node: /MaxPool
[01/11/2024-17:39:50] [E] Error[3]: /MaxPool: at least 5 dimensions are required for input.
[01/11/2024-17:39:50] [E] Error[4]: [graphShapeAnalyzer.cpp::needTypeAndDimensions::2212] Error Code 4: Internal Error (/MaxPool: output shape can not be computed)
[01/11/2024-17:39:50] [E] [TRT] ModelImporter.cpp:771: While parsing node number 36 [MaxPool -> "/MaxPool_output_0"]:
[01/11/2024-17:39:50] [E] [TRT] ModelImporter.cpp:772: --- Begin node ---
[01/11/2024-17:39:50] [E] [TRT] ModelImporter.cpp:773: input: "/Reshape_1_output_0"
output: "/MaxPool_output_0"
name: "/MaxPool"
op_type: "MaxPool"
attribute {
  name: "ceil_mode"
  i: 0
  type: INT
}
attribute {
  name: "kernel_shape"
  ints: 9
  ints: 9
  type: INTS
}
attribute {
  name: "pads"
  ints: 4
  ints: 4
  ints: 4
  ints: 4
  type: INTS
}
attribute {
  name: "strides"
  ints: 1
  ints: 1
  type: INTS
}

[01/11/2024-17:39:50] [E] [TRT] ModelImporter.cpp:774: --- End node ---
[01/11/2024-17:39:50] [E] [TRT] ModelImporter.cpp:777: ERROR: ModelImporter.cpp:178 In function parseGraph:
[6] Invalid Node - /MaxPool
std::bad_alloc
[01/11/2024-17:39:50] [E] Failed to parse onnx file
[01/11/2024-17:39:50] [I] Finished parsing network model. Parse time: 0.0403079
[01/11/2024-17:39:50] [E] Parsing model failed
[01/11/2024-17:39:50] [E] Failed to create engine from model or file.
[01/11/2024-17:39:50] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=model.onnx --saveEngine=model.trt --verbose
@zerollzeng
Copy link
Collaborator

Your onnx is invalid, it failed with onnxruntime

$ polygraphy run model.onnx --onnxrt
[I] RUNNING | Command: /home/scratch.zeroz_sw/miniconda3/bin/polygraphy run model.onnx --onnxrt
[I] onnxrt-runner-N0-01/12/24-08:07:21  | Activating and starting inference
[I] Creating ONNX-Runtime Inference Session with providers: ['CPUExecutionProvider']
Traceback (most recent call last):
  File "/home/scratch.zeroz_sw/miniconda3/bin/polygraphy", line 8, in <module>
    sys.exit(main())
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/tools/_main.py", line 70, in main
    status = selected_tool.run(args)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/tools/base/tool.py", line 171, in run
    status = self.run_impl(args)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/tools/run/run.py", line 228, in run_impl
    exec(str(script))
  File "<string>", line 21, in <module>
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/comparator/comparator.py", line 213, in run
    run_results.append((runner.name, execute_runner(runner, loader_cache)))
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/comparator/comparator.py", line 98, in execute_runner
    with runner as active_runner:
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/backend/base/runner.py", line 60, in __enter__
    self.activate()
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/backend/base/runner.py", line 95, in activate
    self.activate_impl()
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/util/util.py", line 694, in wrapped
    return func(*args, **kwargs)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/backend/onnxrt/runner.py", line 44, in activate_impl
    self.sess, _ = util.invoke_if_callable(self._sess)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/util/util.py", line 663, in invoke_if_callable
    ret = func(*args, **kwargs)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/backend/base/loader.py", line 40, in __call__
    return self.call_impl(*args, **kwargs)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/util/util.py", line 694, in wrapped
    return func(*args, **kwargs)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/polygraphy/backend/onnxrt/loader.py", line 68, in call_impl
    return onnxrt.InferenceSession(model_bytes, providers=providers)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 383, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/scratch.zeroz_sw/miniconda3/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 424, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /home/scratch.zeroz_sw/github_bug/3590/model.onnx failed:Node (/MaxPool) Op (MaxPool) [ShapeInferenceError] Attribute strides has incorrect size

@zerollzeng zerollzeng self-assigned this Jan 12, 2024
@zerollzeng zerollzeng added the triaged Issue has been triaged by maintainers label Jan 12, 2024
@peter5232
Copy link
Author

I used the torch.export function to export onnx and use trtexec to generate engine.
trtexec --onnx=model.onnx --saveEngine=model.engine --verbose
onnx
engine

I used the polygraphy run model.onnx --onnxrt command to test the onnx file and the test passed.
image
I also used the polygraphy run model.engine --trt command to test the correctness of the engine file.
image
However, I used tensorRT python api to load the engine file and perform inference, but an error occurred.
image

The code for python inference is as follows.

import numpy as np
import tensorrt as trt
import pycuda.driver as cuda
import pycuda.autoinit
import time

import common

from cuda import cuda,cudart 

class TensorRTInfer:
    """
    Implements inference for the Model TensorRT engine.
    """

    def __init__(self, engine_path):
        """
        :param engine_path: The path to the serialized engine to load from disk.
        """

        # Load TRT engine
        self.logger = trt.Logger(trt.Logger.INFO)
        trt.init_libnvinfer_plugins(self.logger, namespace="")
        with open(engine_path, "rb") as f, trt.Runtime(self.logger) as runtime:
            assert runtime
            self.engine = runtime.deserialize_cuda_engine(f.read())
        assert self.engine
        self.context = self.engine.create_execution_context()
        assert self.context

        # Setup I/O bindings
        self.inputs = []
        self.outputs = []
        self.allocations = []
        for i in range(self.engine.num_bindings):
            is_input = False
            name = self.engine.get_tensor_name(i)
            # print(self.engine.get_tensor_mode(name))
            if self.engine.binding_is_input(i):
                is_input = True
            name = self.engine.get_binding_name(i)
            dtype = self.engine.get_binding_dtype(i)
            shape = self.engine.get_binding_shape(i)
            # if name == '427':
            #     shape = (1, 9382, 2)
            # elif name == '430':
            #     shape = (1, 9382)
            # elif name == "433":
            #     shape = (1, 256, 9382)
            if is_input:
                self.batch_size = shape[0]
            size = np.dtype(trt.nptype(dtype)).itemsize
            for s in shape:
                size *= s
            print(size,shape, self.engine.get_tensor_shape(name),name )
            allocation = common.cuda_call(cudart.cudaMalloc(size))
            binding = {
                'index': i,
                'name': name,
                'dtype': np.dtype(trt.nptype(dtype)),
                'shape': list(shape),
                'allocation': allocation,
                'size': size
            }
            self.allocations.append(allocation)
            if self.engine.binding_is_input(i):
                self.inputs.append(binding)
            else:
                self.outputs.append(binding)

        assert self.batch_size > 0
        assert len(self.inputs) > 0
        assert len(self.outputs) > 0
        assert len(self.allocations) > 0

    def input_spec(self):
        """
        Get the specs for the input tensor of the network. Useful to prepare memory allocations.
        :return: Two items, the shape of the input tensor and its (numpy) datatype.
        """
        return self.inputs[0]['shape'], self.inputs[0]['dtype']

    def output_spec(self):
        """
        Get the specs for the output tensors of the network. Useful to prepare memory allocations.
        :return: A list with two items per element, the shape and (numpy) datatype of each output tensor.
        """
        specs = []
        for o in self.outputs:
            specs.append((o['shape'], o['dtype']))
        return specs

    def infer(self, batch):
        """
        Execute inference on a batch of images. The images should already be batched and preprocessed, as prepared by
        the ImageBatcher class. Memory copying to and from the GPU device will be performed here.
        :param batch: A numpy array holding the image batch.
        :param scales: The image resize scales for each image in this batch. Default: No scale postprocessing applied.
        :return: A nested list for each image in the batch and each detection in the list.
        """

        # Prepare the output data.
        outputs = []
        for shape, dtype in self.output_spec():
            outputs.append(np.zeros(shape, dtype))

        # Process I/O and execute the network.
        common.memcpy_host_to_device(self.inputs[0]['allocation'], np.ascontiguousarray(batch))

        self.context.execute_v2(self.allocations)
        for o in range(len(outputs)):
            common.memcpy_device_to_host(outputs[o], self.outputs[o]['allocation'])
            print(o, outputs[o].shape)
        # Process the results.
        # nums = outputs[0]
        # boxes = outputs[1]
        # scores = outputs[2]
        # pred_classes = outputs[3]
        # masks = outputs[4]

        # detections = []
        # for i in range(self.batch_size):
        #     detections.append([])
        #     for n in range(int(nums[i])):
        #         # Select a mask.
        #         mask = masks[i][n]

        #         # Calculate scaling values for bboxes.
        #         scale = self.inputs[0]['shape'][2]
        #         scale /= scales[i]
        #         scale_y = scale
        #         scale_x = scale

        #         if nms_threshold and scores[i][n] < nms_threshold:
        #             continue
        #         # Append to detections
        #         detections[i].append({
        #             'ymin': boxes[i][n][0] * scale_y,
        #             'xmin': boxes[i][n][1] * scale_x,
        #             'ymax': boxes[i][n][2] * scale_y,
        #             'xmax': boxes[i][n][3] * scale_x,
        #             'score': scores[i][n],
        #             'class': int(pred_classes[i][n]),
        #             'mask': mask,
        #         })
        # return detections



def main():
    trt_infer = TensorRTInfer("model.engine")
    
    input_batch = np.random.randn(1, 1, 682, 1024).astype(np.float64)
    
    trt_infer.infer(input_batch)
    
main()

@zerollzeng
Copy link
Collaborator

Your new model pass with TRT 9.2

[I] Finished engine building in 7.928 seconds
[I] trt-runner-N0-01/19/24-09:42:26    
    ---- Inference Input(s) ----
    {input.1 [dtype=float32, shape=(1, 1, 682, 1024)],
     conv1a.weight [dtype=float32, shape=(64, 1, 3, 3)],
     conv1a.bias [dtype=float32, shape=(64,)],
     conv1b.weight [dtype=float32, shape=(64, 64, 3, 3)],
     conv1b.bias [dtype=float32, shape=(64,)],
     conv2a.weight [dtype=float32, shape=(64, 64, 3, 3)],
     conv2a.bias [dtype=float32, shape=(64,)],
     conv2b.weight [dtype=float32, shape=(64, 64, 3, 3)],
     conv2b.bias [dtype=float32, shape=(64,)],
     conv3a.weight [dtype=float32, shape=(128, 64, 3, 3)],
     conv3a.bias [dtype=float32, shape=(128,)],
     conv3b.weight [dtype=float32, shape=(128, 128, 3, 3)],
     conv3b.bias [dtype=float32, shape=(128,)],
     conv4a.weight [dtype=float32, shape=(128, 128, 3, 3)],
     conv4a.bias [dtype=float32, shape=(128,)],
     conv4b.weight [dtype=float32, shape=(128, 128, 3, 3)],
     conv4b.bias [dtype=float32, shape=(128,)],
     convPa.weight [dtype=float32, shape=(256, 128, 3, 3)],
     convPa.bias [dtype=float32, shape=(256,)],
     convPb.weight [dtype=float32, shape=(65, 256, 1, 1)],
     convPb.bias [dtype=float32, shape=(65,)],
     convDa.weight [dtype=float32, shape=(256, 128, 3, 3)],
     convDa.bias [dtype=float32, shape=(256,)],
     convDb.weight [dtype=float32, shape=(256, 256, 1, 1)],
     convDb.bias [dtype=float32, shape=(256,)]}
[I] trt-runner-N0-01/19/24-09:42:26    
    ---- Inference Output(s) ----
    {427 [dtype=float32, shape=(1, 9382, 2)],
     430 [dtype=float32, shape=(1, 9382)],
     433 [dtype=float32, shape=(1, 256, 9382)]}
[I] trt-runner-N0-01/19/24-09:42:26     | Completed 1 iteration(s) in 16.05 ms | Average inference time: 16.05 ms.
[I] PASSED | Runtime: 10.422s | Command: /home/scratch.zeroz_sw/miniconda3/bin/polygraphy run model.onnx --trt

@zerollzeng
Copy link
Collaborator

You can download it from
https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/9.2.0/tensorrt-9.2.0.5.linux.x86_64-gnu.cuda-11.8.tar.gz
https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/9.2.0/tensorrt-9.2.0.5.linux.x86_64-gnu.cuda-12.2.tar.gz
https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/9.2.0/tensorrt-9.2.0.5.ubuntu-22.04.aarch64-gnu.cuda-12.2.tar.gz

@peter5232
Copy link
Author

I was able to output normally using tensorrt in 8.6, but an error occurred while calling the python api for inference. The details are in the third picture.

@RajUpadhyay
Copy link

Hey, which Object Detection model are you using?

@lix19937
Copy link

I was able to output normally using tensorrt in 8.6, but an error occurred while calling the python api for inference. The details are in the third picture.

You can use trt v10 to try it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zerollzeng zerollzeng

Labels
triaged Issue has been triaged by maintainers
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@zerollzeng @lix19937 @RajUpadhyay @peter5232