Converting ava action detection from mmaction2 to Tensorrt

[nyanmn]

I am using this ava model for action detection.
The following config file is used.
https://github.com/open-mmlab/mmaction2/blob/master/configs/detection/ava/slowfast_kinetics_pretrained_r50_8x8x1_cosine_10e_ava22_rgb.py

Follwed in the instruction in Developer Guide for new model. https://github.com/open-mmlab/mmdeploy/blob/master/docs/en/07-developer-guide/support_new_model.md

But no idea for where to start?
Can somebody show me how to start if I want to convert ava activity detection model?

[irexyc]

Currently, mmdeploy only support recognition task in mmaction2.

[nyanmn]

Thanks for the reply.
I need to use the model in Tensorrt.
So I start convert to onnx first then I will convert to Tensorrt.
This is my code.
https://www.dropbox.com/s/aqspqttz94yahne/conversion.py?dl=0

I have error as follows. What could be wrong?

(mmdeploy) root@32d3d12ffe2a:/home/docker_data/mmdeploy_model/ONNX_TRT_conversion/ava# python ava_to_onnx.py
/home/mmcv/mmcv/__init__.py:21: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
  'On January 1, 2023, MMCV will release v2.0.0, in which it will remove '
load checkpoint from local path: /home/docker_data/mmdeploy_model/worksdir/slowfast_kinetics_pretrained_r50_8x8x1_cosine_10e_ava22_rgb/latest.pth
/home/anaconda/envs/mmdeploy/lib/python3.7/site-packages/torch/nn/functional.py:3500: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
  "The default behavior for interpolate/upsample with float scale_factor changed "
Traceback (most recent call last):
  File "ava_to_onnx.py", line 65, in <module>
    main(parse_args())
  File "ava_to_onnx.py", line 49, in main
    torch_out = model(**in_)
  File "/home/anaconda/envs/mmdeploy/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/mmcv/mmcv/runner/fp16_utils.py", line 119, in new_func
    return old_func(*args, **kwargs)
  File "/home/mmdetection/mmdet/models/detectors/base.py", line 174, in forward
    return self.forward_test(img, img_metas, **kwargs)
  File "/home/mmdetection/mmdet/models/detectors/fast_rcnn.py", line 52, in forward_test
    **kwargs)
  File "/home/mmdetection/mmdet/models/detectors/two_stage.py", line 184, in simple_test
    x, proposal_list, img_metas, rescale=rescale)
  File "/home/mmaction2/mmaction/models/heads/roi_head.py", line 85, in simple_test
    thr=self.test_cfg.action_thr)
  File "/home/mmaction2/mmaction/core/bbox/transforms.py", line 32, in bbox2result
    scores = labels.cpu().numpy()  # rename for clarification
RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.
(mmdeploy) root@32d3d12ffe2a:/home/docker_data/mmdeploy_model/ONNX_TRT_conversion/ava# python ava_to_onnx.py
/home/mmcv/mmcv/__init__.py:21: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
  'On January 1, 2023, MMCV will release v2.0.0, in which it will remove '
load checkpoint from local path: /home/docker_data/mmdeploy_model/worksdir/slowfast_kinetics_pretrained_r50_8x8x1_cosine_10e_ava22_rgb/latest.pth
/home/anaconda/envs/mmdeploy/lib/python3.7/site-packages/torch/nn/functional.py:3500: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
  "The default behavior for interpolate/upsample with float scale_factor changed "
Traceback (most recent call last):
  File "ava_to_onnx.py", line 65, in <module>
    main(parse_args())
  File "ava_to_onnx.py", line 49, in main
    torch_out = model(**in_)
  File "/home/anaconda/envs/mmdeploy/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/mmcv/mmcv/runner/fp16_utils.py", line 119, in new_func
    return old_func(*args, **kwargs)
  File "/home/mmdetection/mmdet/models/detectors/base.py", line 174, in forward
    return self.forward_test(img, img_metas, **kwargs)
  File "/home/mmdetection/mmdet/models/detectors/fast_rcnn.py", line 52, in forward_test
    **kwargs)
  File "/home/mmdetection/mmdet/models/detectors/two_stage.py", line 184, in simple_test
    x, proposal_list, img_metas, rescale=rescale)
  File "/home/mmaction2/mmaction/models/heads/roi_head.py", line 85, in simple_test
    thr=self.test_cfg.action_thr)
  File "/home/mmaction2/mmaction/core/bbox/transforms.py", line 32, in bbox2result
    scores = labels.cpu().numpy()  # rename for clarification
RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.