[Feature]Support centerpoint (#252)

* bump version to v0.4.0 * [Enhancement] Make rewriter more powerful (#150) * Finish function tests * lint * resolve comments * Fix tests * docstring & fix * Complement informations * lint * Add example * Fix version * Remove todo Co-authored-by: RunningLeon <[email protected]> * Torchscript support (#159) * support torchscript * add nms * add torchscript configs and update deploy process and dump-info * typescript -> torchscript * add torchscript custom extension support * add ts custom ops again * support mmseg unet * [WIP] add optimizer for torchscript (#119) * add passes * add python api * Torchscript optimizer python api (#121) * add passes * add python api * use python api instead of executable * Merge Master, update optimizer (#151) * [Feature] add yolox ncnn (#29) * add yolox ncnn * add ncnn android performance of yolox * add ut * fix lint * fix None bugs for ncnn * test codecov * test codecov * add device * fix yapf * remove if-else for img shape * use channelshuffle optimize * change benchmark after channelshuffle * fix yapf * fix yapf * fuse continuous reshape * fix static shape deploy * fix code * drop pad * only static shape * fix static * fix docstring * Added mask overlay to output image, changed fprintf info messages to … (#55) * Added mask overlay to output image, changed fprintf info messages to stdout * Improved box filtering (filter area/score), make sure roi coordinates stay within bounds * clang-format * Support UNet in mmseg (#77) * Repeatdataset in train has no CLASSES & PALETTE * update result for unet * update docstring for mmdet * remove ppl for unet in docs * fix ort wrap about input type (#81) * Fix memleak (#86) * delete [] * fix build error when enble MMDEPLOY_ACTIVE_LEVEL * fix lint * [Doc] Nano benchmark and tutorial (#71) * add cls benchmark * add nano zh-cn benchmark and en tutorial * add device row * add doc path to index.rst * fix typo * [Fix] fix missing deploy_core (#80) * fix missing deploy_core * mv flag to demo * target link * [Docs] Fix links in Chinese doc (#84) * Fix docs in Chinese link * Fix links * Delete symbolic link and add links to html * delete files * Fix link * [Feature] Add docker files (#67) * add gpu and cpu dockerfile * fix lint * fix cpu docker and remove redundant * use pip instead * add build arg and readme * fix grammar * update readme * add chinese doc for dockerfile and add docker build to build.md * grammar * refine dockerfiles * add FAQs * update Dpplcv_DIR for SDK building * remove mmcls * add sdk demos * fix typo and lint * update FAQs * [Fix]fix check_env (#101) * fix check_env * update * Replace convert_syncbatchnorm in mmseg (#93) * replace convert_syncbatchnorm with revert_sync_batchnorm from mmcv * change logger * [Doc] Update FAQ for TensorRT (#96) * update FAQ * comment * [Docs]: Update doc for openvino installation (#102) * fix docs * fix docs * fix docs * fix mmcv version * fix docs * rm blank line * simplify non batch nms (#99) * [Enhacement] Allow test.py to save evaluation results (#108) * Add log file * Delete debug code * Rename logger * resolve comments * [Enhancement] Support mmocr v0.4+ (#115) * support mmocr v0.4+ * 0.4.0 -> 0.4.1 * fix onnxruntime wrapper for gpu inference (#123) * fix ncnn wrapper for ort-gpu * resolve comment * fix lint * Fix typo (#132) * lock mmcls version (#131) * [Enhancement] upgrade isort in pre-commit config (#141) * [Enhancement] upgrade isort in pre-commit config by refering to mmflow pr #87 * fix lint * remove .isort.cfg and put its known_third_party to setup.cfg * Fix ci for mmocr (#144) * fix mmocr unittests * remove useless * lock mmdet maximum version to 2.20 * pip install -U numpy * Fix capture_output (#125) Co-authored-by: hanrui1sensetime <[email protected]> Co-authored-by: Johannes L <[email protected]> Co-authored-by: RunningLeon <[email protected]> Co-authored-by: VVsssssk <[email protected]> Co-authored-by: lvhan028 <[email protected]> Co-authored-by: AllentDan <[email protected]> Co-authored-by: Yifan Zhou <[email protected]> Co-authored-by: 杨培文 (Yang Peiwen) <[email protected]> Co-authored-by: Semyon Bevzyuk <[email protected]> * configs for all tasks * use torchvision roi align * remote unnecessary code * fix ut * fix ut * export * det dynamic * det dynamic * add ut * fix ut * add ut and docs * fix ut * skip torchscript ut if no ops available * add torchscript option to build.md * update benchmark and resolve comments * resolve conflicts * rename configs * fix mrcnn cuda test * remove useless * add version requirements to docs and comments to codes * enable empty image exporting for torchscript and accelerate ORT inference for MRCNN * rebase * update example for torchscript.md * update FAQs for torchscript.md * resolve comments * only use torchvision roi_align for torchscript * fix ut * use torchvision roi align when pool model is avg * resolve comments Co-authored-by: grimoire <[email protected]> Co-authored-by: grimoire <[email protected]> Co-authored-by: hanrui1sensetime <[email protected]> Co-authored-by: Johannes L <[email protected]> Co-authored-by: RunningLeon <[email protected]> Co-authored-by: VVsssssk <[email protected]> Co-authored-by: lvhan028 <[email protected]> Co-authored-by: Yifan Zhou <[email protected]> Co-authored-by: 杨培文 (Yang Peiwen) <[email protected]> Co-authored-by: Semyon Bevzyuk <[email protected]> * Update supported mmseg models (#181) * fix ocrnet cascade decoder * update mmseg support models * update mmseg configs * support emanet and icnet * set max K of TopK for tensorrt * update supported models for mmseg in docs * add test for emamodule * add configs and update docs * Update docs * update benchmark * [Features]Support mmdet3d (#103) * add mmdet3d code * add code * update code * [log]This commit finish pointpillar export and evaluate on onnxruntime.The model is sample with nvidia repo model * add tensorrt config * fix config * update * support for tensorrt * add config * fix config` * fix apis about torch2onnx * update * mmdet3d deploy version1.0 * map is ok * fix code * version1.0 * fix code * fix visual * fix bug * tensorrt support success * add docstring * add docs * fix docs * fix comments * fix comment * fix comment * fix openvino wrapper * add unit test * fix device about cpu * fix comment * fix show_result * fix lint * fix requirments * remove ci about det3d * fix ut * add ut data * support for new version pointpillars * fix comment * fix support_list * fix comments * fix config name * [Enhancement] Update pad logic in detection heads (#168) * pad with register * fix lint Co-authored-by: AllentDan <[email protected]> * [Enhancement] Additional arguments support for OpenVINO Model Optimizer (#178) * Add mo args. * [Docs]: update docs and argument descriptions (#196) * bump version to v0.4.0 * update docs and argument descriptions * revert version change * fix unnecessary change of config for dynamic exportation (#199) * fix mmcls get classes (#215) * fix mmcls get classes * resolve comment * resolve comment * Add ModelOptimizerOptions. * Fix merge bugs. * Update mmpose.md (#224) * [Dostring]add example in apis docstring (#214) * add example in apis docstring * add backend example in docstring * rm blank line * Fixed get_mo_options_from_cfg args * fix l2norm test Co-authored-by: RunningLeon <[email protected]> Co-authored-by: Haofan Wang <[email protected]> Co-authored-by: VVsssssk <[email protected]> Co-authored-by: grimoire <[email protected]> * [Enhancement] Switch to statically typed Value::Any (#209) * replace std::any with StaticAny * fix __compare_typeid * remove fallback id support * constraint on traits::TypeId<T>::value * fix includes * support for centerpoint * [Enhancement] TensorRT DCN support (#205) * add tensorrt dcn support * fix lint * add docstring and dcn model support * add centerpoint ut and docs * add config and fix input rank * fix merge error * fix a bug * fix comment * [Doc] update benchmark add supported-model-list (#286) * update benchmark add supported-model-list * fix lint * fix lint * loc mmocr maximum version * fix ut Co-authored-by: maningsheng <[email protected]> Co-authored-by: Yifan Zhou <[email protected]> Co-authored-by: AllentDan <[email protected]> Co-authored-by: grimoire <[email protected]> Co-authored-by: grimoire <[email protected]> Co-authored-by: hanrui1sensetime <[email protected]> Co-authored-by: Johannes L <[email protected]> Co-authored-by: lvhan028 <[email protected]> Co-authored-by: 杨培文 (Yang Peiwen) <[email protected]> Co-authored-by: Semyon Bevzyuk <[email protected]> Co-authored-by: AllentDan <[email protected]> Co-authored-by: Haofan Wang <[email protected]> Co-authored-by: lzhangzz <[email protected]>
open-mmlab · Apr 1, 2022 · be86bc7 · be86bc7
1 parent 43e6bea
commit be86bc7
Show file tree

Hide file tree

Showing 16 changed files with 454 additions and 23 deletions.
diff --git a/...ction/voxel-detection_openvino_dynamic.py → ...voxel-detection_openvino_dynamic_kitti.py b/...ction/voxel-detection_openvino_dynamic.py → ...voxel-detection_openvino_dynamic_kitti.py
diff --git a/configs/mmdet3d/voxel-detection/voxel-detection_openvino_dynamic_nus.py b/configs/mmdet3d/voxel-detection/voxel-detection_openvino_dynamic_nus.py
@@ -0,0 +1,9 @@
+_base_ = ['./voxel-detection_dynamic.py', '../../_base_/backends/openvino.py']
+
+onnx_config = dict(input_shape=None)
+
+backend_config = dict(model_inputs=[
+    dict(
+        opt_shapes=dict(
+            voxels=[20000, 20, 5], num_points=[20000], coors=[20000, 4]))
+])
diff --git a/configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-nus.py b/configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-nus.py
@@ -0,0 +1,18 @@
+_base_ = ['./voxel-detection_dynamic.py', '../../_base_/backends/tensorrt.py']
+backend_config = dict(
+    common_config=dict(max_workspace_size=1 << 30),
+    model_inputs=[
+        dict(
+            input_shapes=dict(
+                voxels=dict(
+                    min_shape=[5000, 20, 5],
+                    opt_shape=[20000, 20, 5],
+                    max_shape=[30000, 20, 5]),
+                num_points=dict(
+                    min_shape=[5000], opt_shape=[20000], max_shape=[30000]),
+                coors=dict(
+                    min_shape=[5000, 4],
+                    opt_shape=[20000, 4],
+                    max_shape=[30000, 4]),
+            ))
+    ])
diff --git a/docs/en/benchmark.md b/docs/en/benchmark.md
@@ -326,10 +326,10 @@ Users can directly test the speed through [how_to_measure_performance_of_models.
     <td align="center">Mask-RCNN</td>
     <td align="center">COCO</td>
     <td align="center">1x3x800x1344</td>
-    <td align="center">320.86 </td>
-    <td align="center">3.12</td>
-    <td align="center">241.32</td>
-    <td align="center">4.14</td>
+    <td align="center">104.83</td>
+    <td align="center">9.54</td>
+    <td align="center">58.27</td>
+    <td align="center">17.16</td>
     <td align="center">-</td>
     <td align="center">-</td>
     <td align="center">-</td>

diff --git a/docs/en/supported_models.md b/docs/en/supported_models.md
@@ -65,6 +65,8 @@ The table below lists the models that are guaranteed to be exportable to other b
 | MSPN                      | MMPose           |      ?      |      Y      |    Y     |  Y   |   N   |    Y     |    [config](https://mmpose.readthedocs.io/en/latest/papers/backbones.html#mspn-arxiv-2019)     |
 | LiteHRNet                 | MMPose           |      ?      |      Y      |    Y     |  N   |   N   |    Y     |  [config](https://mmpose.readthedocs.io/en/latest/papers/backbones.html#litehrnet-cvpr-2021)   |
 | PointPillars              | MMDetection3d    |      ?      |      Y      |    Y     |  N   |   N   |    Y     | [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars)         |
+| CenterPoint (pillar)      | MMDetection3d    |      ?      |      Y      |    Y     |   N   |   N   |    Y     | [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/centerpoint)          |
+
 
 ### Note
 

diff --git a/docs/zh_cn/benchmark.md b/docs/zh_cn/benchmark.md
@@ -327,10 +327,10 @@ GPU: ncnn, TensorRT, PPLNN
     <td align="center">Mask-RCNN</td>
     <td align="center">COCO</td>
     <td align="center">1x3x800x1344</td>
-    <td align="center">320.86 </td>
-    <td align="center">3.12</td>
-    <td align="center">241.32</td>
-    <td align="center">4.14</td>
+    <td align="center">104.83</td>
+    <td align="center">9.54</td>
+    <td align="center">58.27</td>
+    <td align="center">17.16</td>
     <td align="center">-</td>
     <td align="center">-</td>
     <td align="center">-</td>

diff --git a/mmdeploy/codebase/mmdet3d/deploy/voxel_detection_model.py b/mmdeploy/codebase/mmdet3d/deploy/voxel_detection_model.py
@@ -7,6 +7,7 @@
 from torch.nn import functional as F
 
 from mmdeploy.codebase.base import BaseBackendModel
+from mmdeploy.core import RewriterContext
 from mmdeploy.utils import (Backend, get_backend, get_codebase_config,
                             get_root_logger, load_config)
 
@@ -92,7 +93,8 @@ def forward(self,
                 'coors': coors
             }
             outputs = self.wrapper(input_dict)
-            result = VoxelDetectionModel.post_process(self.model_cfg, outputs,
+            result = VoxelDetectionModel.post_process(self.model_cfg,
+                                                      self.deploy_cfg, outputs,
                                                       img_metas[i],
                                                       self.device)[0]
             result_list.append(result)
@@ -171,6 +173,7 @@ def voxelize(model_cfg: Union[str, mmcv.Config], points: torch.Tensor):
 
     @staticmethod
     def post_process(model_cfg: Union[str, mmcv.Config],
+                     deploy_cfg: Union[str, mmcv.Config],
                      outs: torch.Tensor,
                      img_metas: Dict,
                      device: str,
@@ -179,6 +182,8 @@ def post_process(model_cfg: Union[str, mmcv.Config],
 
         Args:
             model_cfg (str | mmcv.Config): The model config.
+            deploy_cfg (str|mmcv.Config): Deployment config file or loaded
+            Config object.
             outs (torch.Tensor): Output of model's head.
             img_metas(Dict): Meta info for pcd.
             device (str): A string specifying device type.
@@ -189,7 +194,13 @@ def post_process(model_cfg: Union[str, mmcv.Config],
         from mmdet3d.core import bbox3d2result
         from mmdet3d.models.builder import build_head
         model_cfg = load_config(model_cfg)[0]
-        head_cfg = dict(**model_cfg.model['bbox_head'])
+        deploy_cfg = load_config(deploy_cfg)[0]
+        if 'bbox_head' in model_cfg.model.keys():
+            head_cfg = dict(**model_cfg.model['bbox_head'])
+        elif 'pts_bbox_head' in model_cfg.model.keys():
+            head_cfg = dict(**model_cfg.model['pts_bbox_head'])
+        else:
+            raise NotImplementedError('Not supported model.')
         head_cfg['train_cfg'] = None
         head_cfg['test_cfg'] = model_cfg.model['test_cfg']
         head = build_head(head_cfg)
@@ -206,12 +217,16 @@ def post_process(model_cfg: Union[str, mmcv.Config],
         cls_scores = [outs['scores'].to(device)]
         bbox_preds = [outs['bbox_preds'].to(device)]
         dir_scores = [outs['dir_scores'].to(device)]
-        bbox_list = head.get_bboxes(
-            cls_scores, bbox_preds, dir_scores, img_metas, rescale=False)
-        bbox_results = [
-            bbox3d2result(bboxes, scores, labels)
-            for bboxes, scores, labels in bbox_list
-        ]
+        with RewriterContext(
+                cfg=deploy_cfg,
+                backend=deploy_cfg.backend_config.type,
+                opset=deploy_cfg.onnx_config.opset_version):
+            bbox_list = head.get_bboxes(
+                cls_scores, bbox_preds, dir_scores, img_metas, rescale=False)
+            bbox_results = [
+                bbox3d2result(bboxes, scores, labels)
+                for bboxes, scores, labels in bbox_list
+            ]
         return bbox_results
 
 

diff --git a/mmdeploy/codebase/mmdet3d/models/__init__.py b/mmdeploy/codebase/mmdet3d/models/__init__.py
@@ -1,5 +1,7 @@
 # Copyright (c) OpenMMLab. All rights reserved.
 from .base import *  # noqa: F401,F403
+from .centerpoint import *  # noqa: F401,F403
+from .mvx_two_stage import *  # noqa: F401,F403
 from .pillar_encode import *  # noqa: F401,F403
 from .pillar_scatter import *  # noqa: F401,F403
 from .voxelnet import *  # noqa: F401,F403
diff --git a/mmdeploy/codebase/mmdet3d/models/base.py b/mmdeploy/codebase/mmdet3d/models/base.py
@@ -11,7 +11,7 @@ def base3ddetector__forward_test(ctx,
                                  coors,
                                  img_metas=None,
                                  img=None,
-                                 rescale=True):
+                                 rescale=False):
     """Rewrite this function to run simple_test directly."""
     return self.simple_test(voxels, num_points, coors, img_metas, img)
 

diff --git a/mmdeploy/codebase/mmdet3d/models/centerpoint.py b/mmdeploy/codebase/mmdet3d/models/centerpoint.py
@@ -0,0 +1,189 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch
+from mmdet3d.core import circle_nms
+
+from mmdeploy.core import FUNCTION_REWRITER
+
+
+@FUNCTION_REWRITER.register_rewriter(
+    'mmdet3d.models.detectors.centerpoint.CenterPoint.extract_pts_feat')
+def centerpoint__extract_pts_feat(ctx, self, voxels, num_points, coors,
+                                  img_feats, img_metas):
+    """Extract features from points. Rewrite this func to remove voxelize op.
+
+    Args:
+        voxels (torch.Tensor): Point features or raw points in shape (N, M, C).
+        num_points (torch.Tensor): Number of points in each voxel.
+        coors (torch.Tensor): Coordinates of each voxel.
+        img_feats (list[torch.Tensor], optional): Image features used for
+            multi-modality fusion. Defaults to None.
+        img_metas (list[dict]): Meta information of samples.
+
+    Returns:
+        torch.Tensor: Points feature.
+    """
+    if not self.with_pts_bbox:
+        return None
+
+    voxel_features = self.pts_voxel_encoder(voxels, num_points, coors)
+    batch_size = coors[-1, 0] + 1
+    x = self.pts_middle_encoder(voxel_features, coors, batch_size)
+    x = self.pts_backbone(x)
+    if self.with_pts_neck:
+        x = self.pts_neck(x)
+    return x
+
+
+@FUNCTION_REWRITER.register_rewriter(
+    'mmdet3d.models.detectors.centerpoint.CenterPoint.simple_test_pts')
+def centerpoint__simple_test_pts(ctx, self, x, img_metas, rescale=False):
+    """Rewrite this func to format model outputs.
+
+    Args:
+        x (torch.Tensor): Input points feature.
+        img_metas (list[dict]): Meta information of samples.
+        rescale (bool): Whether need rescale.
+
+    Returns:
+        List: Result of model.
+    """
+    outs = self.pts_bbox_head(x)
+    bbox_preds, scores, dir_scores = [], [], []
+    for task_res in outs:
+        bbox_preds.append(task_res[0]['reg'])
+        bbox_preds.append(task_res[0]['height'])
+        bbox_preds.append(task_res[0]['dim'])
+        if 'vel' in task_res[0].keys():
+            bbox_preds.append(task_res[0]['vel'])
+        scores.append(task_res[0]['heatmap'])
+        dir_scores.append(task_res[0]['rot'])
+    bbox_preds = torch.cat(bbox_preds, dim=1)
+    scores = torch.cat(scores, dim=1)
+    dir_scores = torch.cat(dir_scores, dim=1)
+    return scores, bbox_preds, dir_scores
+
+
+@FUNCTION_REWRITER.register_rewriter(
+    'mmdet3d.models.dense_heads.centerpoint_head.CenterHead.get_bboxes')
+def centerpoint__get_bbox(ctx,
+                          self,
+                          cls_scores,
+                          bbox_preds,
+                          dir_scores,
+                          img_metas,
+                          img=None,
+                          rescale=False):
+    """Rewrite this func to format func inputs.
+
+    Args
+        cls_scores (list[torch.Tensor]): Classification predicts results.
+        bbox_preds (list[torch.Tensor]): Bbox predicts results.
+        dir_scores (list[torch.Tensor]): Dir predicts results.
+        img_metas (list[dict]): Point cloud and image's meta info.
+        img (torch.Tensor): Input image.
+        rescale (Bool): Whether need rescale.
+
+    Returns:
+        list[dict]: Decoded bbox, scores and labels after nms.
+    """
+    rets = []
+    scores_range = [0]
+    bbox_range = [0]
+    dir_range = [0]
+    self.test_cfg = self.test_cfg['pts']
+    for i, task_head in enumerate(self.task_heads):
+        scores_range.append(scores_range[i] + self.num_classes[i])
+        bbox_range.append(bbox_range[i] + 8)
+        dir_range.append(dir_range[i] + 2)
+    for task_id in range(len(self.num_classes)):
+        num_class_with_bg = self.num_classes[task_id]
+
+        batch_heatmap = cls_scores[
+            0][:, scores_range[task_id]:scores_range[task_id + 1],
+               ...].sigmoid()
+
+        batch_reg = bbox_preds[0][:,
+                                  bbox_range[task_id]:bbox_range[task_id] + 2,
+                                  ...]
+        batch_hei = bbox_preds[0][:, bbox_range[task_id] +
+                                  2:bbox_range[task_id] + 3, ...]
+
+        if self.norm_bbox:
+            batch_dim = torch.exp(bbox_preds[0][:, bbox_range[task_id] +
+                                                3:bbox_range[task_id] + 6,
+                                                ...])
+        else:
+            batch_dim = bbox_preds[0][:, bbox_range[task_id] +
+                                      3:bbox_range[task_id] + 6, ...]
+
+        batch_vel = bbox_preds[0][:, bbox_range[task_id] +
+                                  6:bbox_range[task_id + 1], ...]
+
+        batch_rots = dir_scores[0][:,
+                                   dir_range[task_id]:dir_range[task_id + 1],
+                                   ...][:, 0].unsqueeze(1)
+        batch_rotc = dir_scores[0][:,
+                                   dir_range[task_id]:dir_range[task_id + 1],
+                                   ...][:, 1].unsqueeze(1)
+
+        temp = self.bbox_coder.decode(
+            batch_heatmap,
+            batch_rots,
+            batch_rotc,
+            batch_hei,
+            batch_dim,
+            batch_vel,
+            reg=batch_reg,
+            task_id=task_id)
+        assert self.test_cfg['nms_type'] in ['circle', 'rotate']
+        batch_reg_preds = [box['bboxes'] for box in temp]
+        batch_cls_preds = [box['scores'] for box in temp]
+        batch_cls_labels = [box['labels'] for box in temp]
+        if self.test_cfg['nms_type'] == 'circle':
+
+            boxes3d = temp[0]['bboxes']
+            scores = temp[0]['scores']
+            labels = temp[0]['labels']
+            centers = boxes3d[:, [0, 1]]
+            boxes = torch.cat([centers, scores.view(-1, 1)], dim=1)
+            keep = torch.tensor(
+                circle_nms(
+                    boxes.detach().cpu().numpy(),
+                    self.test_cfg['min_radius'][task_id],
+                    post_max_size=self.test_cfg['post_max_size']),
+                dtype=torch.long,
+                device=boxes.device)
+
+            boxes3d = boxes3d[keep]
+            scores = scores[keep]
+            labels = labels[keep]
+            ret = dict(bboxes=boxes3d, scores=scores, labels=labels)
+            ret_task = [ret]
+            rets.append(ret_task)
+        else:
+            rets.append(
+                self.get_task_detections(num_class_with_bg, batch_cls_preds,
+                                         batch_reg_preds, batch_cls_labels,
+                                         img_metas))
+
+    # Merge branches results
+    num_samples = len(rets[0])
+
+    ret_list = []
+    for i in range(num_samples):
+        for k in rets[0][i].keys():
+            if k == 'bboxes':
+                bboxes = torch.cat([ret[i][k] for ret in rets])
+                bboxes[:, 2] = bboxes[:, 2] - bboxes[:, 5] * 0.5
+                bboxes = img_metas[i]['box_type_3d'](bboxes,
+                                                     self.bbox_coder.code_size)
+            elif k == 'scores':
+                scores = torch.cat([ret[i][k] for ret in rets])
+            elif k == 'labels':
+                flag = 0
+                for j, num_class in enumerate(self.num_classes):
+                    rets[j][i][k] += flag
+                    flag += num_class
+                labels = torch.cat([ret[i][k].int() for ret in rets])
+        ret_list.append([bboxes, scores, labels])
+    return ret_list