Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] fail to export rtmdet and rtmdet-ins to onnx #1586

Closed
3 tasks
zhangzscn opened this issue Dec 29, 2022 · 13 comments
Closed
3 tasks

[Bug] fail to export rtmdet and rtmdet-ins to onnx #1586

zhangzscn opened this issue Dec 29, 2022 · 13 comments

Comments

@zhangzscn
Copy link

zhangzscn commented Dec 29, 2022

Checklist

  • I have searched related issues but cannot get the expected help.
  • 2. I have read the FAQ documentation but cannot get the expected help.
  • 3. The bug has not been fixed in the latest version.

Describe the bug

fail to export rtmdet and rtmdet-ins to onnx

Reproduction

python ./tools/deploy.py configs/mmdet/detection/detection_onnxruntime_static.py ../mmdetection/configs/rtmdet/rtmdet_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth demo/resources/det.jpg --device cuda

python ./tools/deploy.py configs/mmdet/instance-seg/instance-seg_onnxruntime_static.py ../mmdetection/configs/rtmdet/rtmdet-ins_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet-ins_s_8xb32-300e_coco_20221121_212604-fdc5d7ec.pth demo/resources/det.jpg --device cuda

Environment

12/29 15:21:24 - mmengine - INFO - 

12/29 15:21:24 - mmengine - INFO - **********Environmental information**********
12/29 15:21:24 - mmengine - INFO - sys.platform: linux
12/29 15:21:24 - mmengine - INFO - Python: 3.8.15 (default, Nov 24 2022, 15:19:38) [GCC 11.2.0]
12/29 15:21:24 - mmengine - INFO - CUDA available: True
12/29 15:21:24 - mmengine - INFO - numpy_random_seed: 2147483648
12/29 15:21:24 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 4090
12/29 15:21:24 - mmengine - INFO - CUDA_HOME: /usr/local/cuda-11.8
12/29 15:21:24 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.8, V11.8.89
12/29 15:21:24 - mmengine - INFO - GCC: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
12/29 15:21:24 - mmengine - INFO - PyTorch: 1.13.1
12/29 15:21:24 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.7
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.5
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

12/29 15:21:24 - mmengine - INFO - TorchVision: 0.14.1
12/29 15:21:24 - mmengine - INFO - OpenCV: 4.6.0
12/29 15:21:24 - mmengine - INFO - MMEngine: 0.4.0
12/29 15:21:24 - mmengine - INFO - MMCV: 2.0.0rc3
12/29 15:21:24 - mmengine - INFO - MMCV Compiler: GCC 9.3
12/29 15:21:24 - mmengine - INFO - MMCV CUDA Compiler: 11.7
12/29 15:21:24 - mmengine - INFO - MMDeploy: 1.0.0rc0+d6fdb3e
12/29 15:21:24 - mmengine - INFO - 

12/29 15:21:24 - mmengine - INFO - **********Backend information**********
12/29 15:21:24 - mmengine - INFO - tensorrt:    None
12/29 15:21:24 - mmengine - INFO - ONNXRuntime: 1.13.1
12/29 15:21:24 - mmengine - INFO - ONNXRuntime-gpu:     None
12/29 15:21:24 - mmengine - INFO - ONNXRuntime custom ops:      NotAvailable
12/29 15:21:24 - mmengine - INFO - pplnn:       None
12/29 15:21:24 - mmengine - INFO - ncnn:        None
12/29 15:21:24 - mmengine - INFO - snpe:        None
12/29 15:21:24 - mmengine - INFO - openvino:    None
12/29 15:21:24 - mmengine - INFO - torchscript: 1.13.1
12/29 15:21:24 - mmengine - INFO - torchscript custom ops:      NotAvailable
12/29 15:21:24 - mmengine - INFO - rknn-toolkit:        None
12/29 15:21:24 - mmengine - INFO - rknn2-toolkit:       None
12/29 15:21:24 - mmengine - INFO - ascend:      None
12/29 15:21:24 - mmengine - INFO - coreml:      None
12/29 15:21:24 - mmengine - INFO - tvm: None
12/29 15:21:24 - mmengine - INFO - 

12/29 15:21:24 - mmengine - INFO - **********Codebase information**********
12/29 15:21:24 - mmengine - INFO - mmdet:       3.0.0rc4
12/29 15:21:24 - mmengine - INFO - mmseg:       None
12/29 15:21:24 - mmengine - INFO - mmcls:       None
12/29 15:21:24 - mmengine - INFO - mmocr:       None
12/29 15:21:24 - mmengine - INFO - mmedit:      None
12/29 15:21:24 - mmengine - INFO - mmdet3d:     None
12/29 15:21:24 - mmengine - INFO - mmpose:      None
12/29 15:21:24 - mmengine - INFO - mmrotate:    None
12/29 15:21:24 - mmengine - INFO - mmaction:    None

Error traceback

(openmmlab3.0) zhangzs36@midea:~/代码/mmdeploy$ python ./tools/deploy.py configs/mmdet/detection/detection_onnxruntime_static.py ../mmdetection/configs/rtmdet/rtmdet_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth demo/resources/det.jpg --device cuda
12/29 15:14:35 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
12/29 15:14:35 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/29 15:14:35 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/29 15:14:35 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/29 15:14:35 - mmengine - WARNING - The "model" registry in mmdet did not set import location. Fallback to call `mmdet.utils.register_all_modules` instead.
12/29 15:14:35 - mmengine - WARNING - The "task util" registry in mmdet did not set import location. Fallback to call `mmdet.utils.register_all_modules` instead.
Loads checkpoint by local backend from path: ../mmdetection/work_dirs/rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: data_preprocessor.mean, data_preprocessor.std

12/29 15:14:36 - mmengine - WARNING - The "transform" registry in mmdet did not set import location. Fallback to call `mmdet.utils.register_all_modules` instead.
12/29 15:14:36 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
12/29 15:14:36 - mmengine - INFO - Export PyTorch model to ONNX: /home/zhangzs36/代码/mmdeploy/end2end.onnx.
/home/zhangzs36/代码/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:84: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  img_shape = [int(val) for val in img_shape]
/home/zhangzs36/代码/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:84: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  img_shape = [int(val) for val in img_shape]
/home/zhangzs36/代码/mmdeploy/mmdeploy/core/optimizers/function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/aten/src/ATen/native/TensorShape.cpp:3190.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/home/zhangzs36/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:392: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if not is_dynamic_batch(deploy_cfg) and batch_size == 1:
/home/zhangzs36/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:307: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
/home/zhangzs36/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:308: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
/home/zhangzs36/代码/mmdeploy/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
/home/zhangzs36/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:44: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  score_threshold = float(score_threshold)
/home/zhangzs36/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:45: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  iou_threshold = float(iou_threshold)
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmcv/ops/nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(1) == 4
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmcv/ops/nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(0) == scores.size(0)
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/_internal/jit_utils.py:258: UserWarning: Constant folding in symbolic shape inference fails: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:413.)
  _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/zhangzs36/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:86: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please open a GitHub issue if you need this functionality..
  max_output_boxes_per_class = g.op(
/home/zhangzs36/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:100: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please open a GitHub issue if you need this functionality..
  return g.op('NonMaxSuppression', boxes, scores,
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:5408: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  warnings.warn(
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py:687: UserWarning: Constant folding in symbolic shape inference fails: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:413.)
  _C._jit_pass_onnx_graph_shape_type_inference(
Process Process-2:
Traceback (most recent call last):
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 98, in torch2onnx
    export(
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/onnx/export.py", line 131, in export
    torch.onnx.export(
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 504, in export
    _export(
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 1529, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/onnx/optimizer.py", line 11, in model_to_graph__custom_optimizer
    graph, params_dict, torch_out = ctx.origin_func(*args, **kwargs)
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 1172, in _model_to_graph
    params_dict = _C._jit_pass_onnx_constant_fold(
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
12/29 15:14:38 - mmengine - ERROR - /home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.



(openmmlab3.0) zhangzs36@midea:~/代码/mmdeploy$ python ./tools/deploy.py configs/mmdet/instance-seg/instance-seg_onnxruntime_static.py ../mmdetection/configs/rtmdet/rtmdet-ins_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet-ins_s_8xb32-300e_coco_20221121_212604-fdc5d7ec.pth demo/resources/det.jpg --device cuda
12/29 15:41:13 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
12/29 15:41:14 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/29 15:41:14 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/29 15:41:14 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/29 15:41:14 - mmengine - WARNING - The "model" registry in mmdet did not set import location. Fallback to call `mmdet.utils.register_all_modules` instead.
12/29 15:41:14 - mmengine - WARNING - The "task util" registry in mmdet did not set import location. Fallback to call `mmdet.utils.register_all_modules` instead.
Loads checkpoint by local backend from path: ../mmdetection/work_dirs/rtmdet-ins_s_8xb32-300e_coco_20221121_212604-fdc5d7ec.pth
12/29 15:41:14 - mmengine - WARNING - The "transform" registry in mmdet did not set import location. Fallback to call `mmdet.utils.register_all_modules` instead.
12/29 15:41:14 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
12/29 15:41:14 - mmengine - INFO - Export PyTorch model to ONNX: /home/zhangzs36/代码/mmdeploy/end2end.onnx.
/home/zhangzs36/代码/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:84: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  img_shape = [int(val) for val in img_shape]
/home/zhangzs36/代码/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:84: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  img_shape = [int(val) for val in img_shape]
/home/zhangzs36/代码/mmdeploy/mmdeploy/core/optimizers/function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/aten/src/ATen/native/TensorShape.cpp:3190.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/home/zhangzs36/代码/mmdetection/mmdet/models/dense_heads/rtmdet_ins_head.py:353: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/home/zhangzs36/代码/mmdetection/mmdet/models/utils/misc.py:336: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  num_topk = min(topk, valid_idxs.size(0))
/home/zhangzs36/代码/mmdeploy/mmdeploy/codebase/mmdet/models/task_modules/coders/distance_point_bbox_coder.py:34: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert points.size(0) == pred_bboxes.size(0)
/home/zhangzs36/代码/mmdeploy/mmdeploy/codebase/mmdet/models/task_modules/coders/distance_point_bbox_coder.py:35: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert points.size(-1) == 2
/home/zhangzs36/代码/mmdeploy/mmdeploy/codebase/mmdet/models/task_modules/coders/distance_point_bbox_coder.py:36: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(-1) == 4
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmengine/structures/instance_data.py:296: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  return len(self.values()[0])
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmengine/structures/instance_data.py:139: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  assert len(value) == len(self), 'The length of ' \
/home/zhangzs36/代码/mmdetection/mmdet/models/dense_heads/rtmdet_ins_head.py:478: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if not valid_mask.all():
/home/zhangzs36/代码/mmdetection/mmdet/models/dense_heads/rtmdet_ins_head.py:483: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if results.bboxes.numel() > 0:
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmcv/ops/nms.py:276: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if boxes.size(-1) == 5:
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmcv/ops/nms.py:293: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  max_coordinate + torch.tensor(1).to(boxes))
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmcv/ops/nms.py:301: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if boxes_for_nms.shape[0] < split_thr:
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmcv/ops/nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(1) == 4
/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/mmcv/ops/nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(0) == scores.size(0)
/home/zhangzs36/代码/mmdetection/mmdet/models/dense_heads/rtmdet_ins_head.py:559: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if num_inst < 1:
Process Process-2:
Traceback (most recent call last):
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 98, in torch2onnx
    export(
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/onnx/export.py", line 131, in export
    torch.onnx.export(
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 504, in export
    _export(
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 1529, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/zhangzs36/代码/mmdeploy/mmdeploy/apis/onnx/optimizer.py", line 11, in model_to_graph__custom_optimizer
    graph, params_dict, torch_out = ctx.origin_func(*args, **kwargs)
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 1111, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args)
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 987, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/onnx/utils.py", line 891, in _trace_and_get_graph_from_model
    trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/jit/_trace.py", line 1184, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/jit/_trace.py", line 127, in forward
    graph, out = torch._C._create_graph_by_tracing(
  File "/home/zhangzs36/anaconda3/envs/openmmlab3.0/lib/python3.8/site-packages/torch/jit/_trace.py", line 121, in wrapper
    out_vars, _ = _flatten(outs)
RuntimeError: Only tuples, lists and Variables are supported as JIT inputs/outputs. Dictionaries and strings are also accepted, but their usage is not recommended. Here, received an input of unsupported type: InstanceData
12/29 15:41:16 - mmengine - ERROR - /home/zhangzs36/代码/mmdeploy/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.
@lansfair
Copy link

I also encountered this problem in the latest version

@zhangzscn
Copy link
Author

how to show more log ?it difficult to debug。Having trouble finding what's wrong

@hanrui1sensetime
Copy link
Collaborator

Hi, friends: I will take a look at it.

@hanrui1sensetime
Copy link
Collaborator

hanrui1sensetime commented Jan 3, 2023

Hi @zhangzscn , @lansfair:

  1. The bug in
python ./tools/deploy.py configs/mmdet/detection/detection_onnxruntime_static.py ../mmdetection/configs/rtmdet/rtmdet_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth demo/resources/det.jpg --device cuda

has fixed in #1603.
2. The bug in

python ./tools/deploy.py configs/mmdet/instance-seg/instance-seg_onnxruntime_static.py ../mmdetection/configs/rtmdet/rtmdet-ins_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet-ins_s_8xb32-300e_coco_20221121_212604-fdc5d7ec.pth demo/resources/det.jpg --device cuda

is because the RTMDet-Ins is not supported now. @RangiLyu will support this model in a few days, please stay tuned.

@zhangzscn
Copy link
Author

thanks a lot

@zhangzscn
Copy link
Author

@RangiLyu any update? When will export to onnx be supported?

@mattiasbax
Copy link

@zhangzscn May I ask how you installed all the mm libraries to get the deploy script working?
If I try running the above command to convert the RTMDet, I get a return error saying AttributeError: module 'mmcv' has no attribute 'Config'.
If I try to dump the system info using 'python ./tools/deploy.py' I get the return error saying "ImportError: cannot import name 'get_git_hash' from 'mmcv.utils'"

I'm guessing I went wrong with the installation somewhere.
How did you install mmengine, mmcv, mmdet and mmdeploy? All from source or all via mim?

/ M

@hanrui1sensetime
Copy link
Collaborator

hanrui1sensetime commented Jan 16, 2023

@RangiLyu any update? When will export to onnx be supported?

See #1662

@hanrui1sensetime
Copy link
Collaborator

@zhangzscn May I ask how you installed all the mm libraries to get the deploy script working? If I try running the above command to convert the RTMDet, I get a return error saying AttributeError: module 'mmcv' has no attribute 'Config'. If I try to dump the system info using 'python ./tools/deploy.py' I get the return error saying "ImportError: cannot import name 'get_git_hash' from 'mmcv.utils'"

I'm guessing I went wrong with the installation somewhere. How did you install mmengine, mmcv, mmdet and mmdeploy? All from source or all via mim?

/ M

Please run python tools/check_env.py and post your output.
Maybe some of your codebase is still Openmmlab 1.0 version. But RTMDet is an Openmmlab 2.0 feature, should use mmdet==v3.0.0rcx, mmengine, mmcv==v2.0.0rcx and mmdeploy==v1.0.0rcx

@zhangzscn
Copy link
Author

@hanrui1sensetime @RangiLyu

new error

(openmmlab3) zhangzs36@midea:/mnt/data1/代码/mmdeploy$ python ./tools/deploy.py configs/mmdet/instance-seg/instance-seg_rtmdet-ins_onnxruntime_static-640x640.py ../mmdetection/configs/rtmdet/rtmdet-ins_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet-ins_s_8xb32-300e_coco_20221121_212604-fdc5d7ec.pth demo/resources/det.jpg --device cuda:0
02/13 14:51:02 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
02/13 14:51:02 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
02/13 14:51:02 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
02/13 14:51:02 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: ../mmdetection/work_dirs/rtmdet-ins_s_8xb32-300e_coco_20221121_212604-fdc5d7ec.pth
02/13 14:51:03 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
02/13 14:51:03 - mmengine - INFO - Export PyTorch model to ONNX: /mnt/data1/代码/mmdeploy/end2end.onnx.
02/13 14:51:03 - mmengine - WARNING - Can not find mmdet.models.dense_heads.DETRHead.forward_single, function rewrite will not be applied
/mnt/data1/代码/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:84: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
img_shape = [int(val) for val in img_shape]
/mnt/data1/代码/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:84: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
img_shape = [int(val) for val in img_shape]
/mnt/data1/代码/mmdeploy/mmdeploy/core/optimizers/function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/aten/src/ATen/native/TensorShape.cpp:3190.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/mnt/data1/代码/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/rtmdet_ins_head.py:143: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
/mnt/data1/代码/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/rtmdet_ins_head.py:144: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
/mnt/data1/代码/mmdeploy/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
k = torch.tensor(k, device=input.device, dtype=torch.long)
/mnt/data1/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:44: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
score_threshold = float(score_threshold)
/mnt/data1/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:45: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
iou_threshold = float(iou_threshold)
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/mmcv/ops/nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(1) == 4
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/mmcv/ops/nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(0) == scores.size(0)
/mnt/data1/代码/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/rtmdet_ins_head.py:261: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if num_inst < 1:
/mnt/data1/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:86: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please open a GitHub issue if you need this functionality..
max_output_boxes_per_class = g.op(
/mnt/data1/代码/mmdeploy/mmdeploy/mmcv/ops/nms.py:100: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please open a GitHub issue if you need this functionality..
return g.op('NonMaxSuppression', boxes, scores,
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:5408: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
warnings.warn(
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/torch/onnx/_internal/jit_utils.py:258: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
_C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/torch/onnx/utils.py:687: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
_C._jit_pass_onnx_graph_shape_type_inference(
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/torch/onnx/utils.py:1178: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525541702/work/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
_C._jit_pass_onnx_graph_shape_type_inference(
02/13 14:51:05 - mmengine - INFO - Execute onnx optimize passes.
02/13 14:51:05 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
02/13 14:51:05 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
02/13 14:51:05 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
02/13 14:51:05 - mmengine - INFO - visualize onnxruntime model start.
02/13 14:51:06 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
02/13 14:51:06 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
02/13 14:51:06 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
02/13 14:51:06 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
02/13 14:51:06 - mmengine - WARNING - The library of onnxruntime custom ops doesnot exist:
/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:54: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
warnings.warn(
2023-02-13:14:51:06 - root - ERROR - Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]
Traceback (most recent call last):
File "/mnt/data1/代码/mmdeploy/mmdeploy/utils/utils.py", line 41, in target_wrapper
result = target(*args, **kwargs)
File "/mnt/data1/代码/mmdeploy/mmdeploy/apis/visualize.py", line 72, in visualize_model
result = model.test_step(model_inputs)[0]
File "/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
return self._run_forward(data, mode='predict') # type: ignore
File "/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 326, in _run_forward
results = self(**data, mode=mode)
File "/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/data1/代码/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 190, in forward
outputs = self.predict(inputs)
File "/mnt/data1/代码/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 286, in predict
outputs = self.wrapper({self.input_name: imgs})
File "/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/data1/代码/mmdeploy/mmdeploy/backend/onnxruntime/wrapper.py", line 84, in forward
self.io_binding.bind_input(
File "/home/zhangzs36/anaconda3/envs/openmmlab3/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 469, in bind_input
self._iobinding.bind_input(
RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]
02/13 14:51:07 - mmengine - ERROR - ./tools/deploy.py - create_process - 82 - visualize onnxruntime model failed.

Reproduction

python ./tools/deploy.py configs/mmdet/instance-seg/instance-seg_rtmdet-ins_onnxruntime_static-640x640.py ../mmdetection/configs/rtmdet/rtmdet-ins_s_8xb32-300e_coco.py ../mmdetection/work_dirs/rtmdet-ins_s_8xb32-300e_coco_20221121_212604-fdc5d7ec.pth demo/resources/det.jpg --device cuda:0

Environment

02/13 14:54:45 - mmengine - INFO -

02/13 14:54:45 - mmengine - INFO - Environmental information
02/13 14:54:45 - mmengine - INFO - sys.platform: linux
02/13 14:54:45 - mmengine - INFO - Python: 3.8.16 (default, Jan 17 2023, 23:13:24) [GCC 11.2.0]
02/13 14:54:45 - mmengine - INFO - CUDA available: True
02/13 14:54:45 - mmengine - INFO - numpy_random_seed: 2147483648
02/13 14:54:45 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 4090
02/13 14:54:45 - mmengine - INFO - CUDA_HOME: /usr/local/cuda
02/13 14:54:45 - mmengine - INFO - NVCC: Cuda compilation tools, release 12.0, V12.0.140
02/13 14:54:45 - mmengine - INFO - GCC: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
02/13 14:54:45 - mmengine - INFO - PyTorch: 1.13.1
02/13 14:54:45 - mmengine - INFO - PyTorch compiling details: PyTorch built with:

  • GCC 9.3
  • C++ Version: 201402
  • Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  • Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  • OpenMP 201511 (a.k.a. OpenMP 4.5)
  • LAPACK is enabled (usually provided by MKL)
  • NNPACK is enabled
  • CPU capability usage: AVX2
  • CUDA Runtime 11.7
  • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  • CuDNN 8.5
  • Magma 2.6.1
  • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

02/13 14:54:45 - mmengine - INFO - TorchVision: 0.14.1
02/13 14:54:45 - mmengine - INFO - OpenCV: 4.7.0
02/13 14:54:45 - mmengine - INFO - MMEngine: 0.5.0
02/13 14:54:45 - mmengine - INFO - MMCV: 2.0.0rc4
02/13 14:54:45 - mmengine - INFO - MMCV Compiler: GCC 9.3
02/13 14:54:45 - mmengine - INFO - MMCV CUDA Compiler: 11.7
02/13 14:54:45 - mmengine - INFO - MMDeploy: 1.0.0rc1+2efb42d
02/13 14:54:45 - mmengine - INFO -

02/13 14:54:45 - mmengine - INFO - Backend information
02/13 14:54:45 - mmengine - INFO - tensorrt: None
02/13 14:54:45 - mmengine - INFO - ONNXRuntime: 1.14.0
02/13 14:54:45 - mmengine - INFO - ONNXRuntime-gpu: None
02/13 14:54:45 - mmengine - INFO - ONNXRuntime custom ops: NotAvailable
02/13 14:54:45 - mmengine - INFO - pplnn: None
02/13 14:54:45 - mmengine - INFO - ncnn: None
02/13 14:54:45 - mmengine - INFO - snpe: None
02/13 14:54:45 - mmengine - INFO - openvino: None
02/13 14:54:45 - mmengine - INFO - torchscript: 1.13.1
02/13 14:54:45 - mmengine - INFO - torchscript custom ops: NotAvailable
02/13 14:54:45 - mmengine - INFO - rknn-toolkit: None
02/13 14:54:45 - mmengine - INFO - rknn-toolkit2: None
02/13 14:54:45 - mmengine - INFO - ascend: None
02/13 14:54:45 - mmengine - INFO - coreml: None
02/13 14:54:45 - mmengine - INFO - tvm: None
02/13 14:54:45 - mmengine - INFO -

02/13 14:54:45 - mmengine - INFO - Codebase information
02/13 14:54:45 - mmengine - INFO - mmdet: 3.0.0rc5
02/13 14:54:45 - mmengine - INFO - mmseg: None
02/13 14:54:45 - mmengine - INFO - mmcls: None
02/13 14:54:45 - mmengine - INFO - mmocr: None
02/13 14:54:45 - mmengine - INFO - mmedit: None
02/13 14:54:45 - mmengine - INFO - mmdet3d: None
02/13 14:54:45 - mmengine - INFO - mmpose: None
02/13 14:54:45 - mmengine - INFO - mmrotate: None
02/13 14:54:45 - mmengine - INFO - mmaction: None

@zhangzscn
Copy link
Author

the output onnxfile hase so many inputs?
2023-02-13 15-04-52屏幕截图

@github-actions
Copy link

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

@github-actions github-actions bot added the Stale label Mar 23, 2023
@github-actions
Copy link

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants