Group Pose

This repository is an official paddlepaddle implementation of the ICCV 2023 paper "Group Pose: A Simple Baseline for End-to-End Multi-person Pose Estimation".

☀️ If you find this work useful for your research, please kindly star our repo and cite our paper! ☀️

TODO

We are working hard on following items.

• Release arXiv paper
• Release PyTorch scripts

Introduction

In this paper, we study the end-to-end multi-person pose estimation and present a simple yet effective transformer approach, named Group Pose. We simply regard $K$-keypoint pose estimation as predicting a set of $N\times K$ keypoint positions, each from a keypoint query, as well as representing each pose with an instance query for scoring $N$ pose predictions.

Motivated by the intuition that the interaction, among across-instance queries of different types, is not directly helpful, we make a simple modification to decoder self-attention. We replace single self-attention over all the $N\times(K+1)$ queries with two subsequent group self-attentions: (i) $N$ within-instance self-attention, with each over $K$ keypoint queries and one instance query, and (ii) $(K+1)$ same-type across-instance self-attention, each over $N$ queries of the same type. The resulting decoder removes the interaction among across-instance type-different queries, easing the optimization and thus improving the performance. Experimental results on MS COCO and CrowdPose show that our approach without human box supervision is superior to previous methods with complex decoders, and even is slightly better than ED-Pose that uses human box supervision.

Inference

The checkpoints can be found here(baidu & onedrive & Google Drive) with the .pdparams suffix.

Requirements

The code is developed and validated with python=3.8.17,paddlepadlle=2.5.1,cuda=11.2. Higher versions might be as well. Please follows the Installation instruction to prepare the environment.

Resnet-50 on COCO

Please first download the pretrained model and save it into the output folder (or you can change the weight param in the config file).

python tools/eval.py -c configs/keypoint/group_pose/group_pose_r50_4scale_coco.yml

Main Results

Results on MS COCO val2017 set

Method	Backbone	Loss Type	AP	AP50	AP75	APM	APL
PETR	ResNet-50	HM+KR	68.8	87.5	76.3	62.7	77.7
PETR	Swin-L	HM+KR	73.1	90.7	80.9	67.2	81.7
QueryPose	ResNet-50	BR+RLE	68.7	88.6	74.4	63.8	76.5
QueryPose	Swin-L	BR+RLE	73.3	91.3	79.5	68.5	81.2
ED-Pose	ResNet-50	BR+KR	71.6	89.6	78.1	65.9	79.8
ED-Pose	Swin-L	BR+KR	74.3	91.5	81.6	68.6	82.6
GroupPose	ResNet-50	KR	72.0	89.4	79.1	66.8	79.7
GroupPose	Swin-T	KR	73.6	90.4	80.5	68.7	81.2
GroupPose	Swin-L	KR	74.8	91.6	82.1	69.4	83.0

HM, BR and KR denote heatmap, human box regression and keypoint regression.

Results on MS COCO test2017 set

Method	Backbone	Loss Type	AP	AP50	AP75	APM	APL
PETR	ResNet-50	HM+KR	67.6	89.8	75.3	61.6	76.0
PETR	Swin-L	HM+KR	70.5	91.5	78.7	65.2	78.0
QueryPose	Swin-L	BR+RLE	72.2	92.0	78.8	67.3	79.4
ED-Pose	ResNet-50	BR+KR	69.8	90.2	77.2	64.3	77.4
ED-Pose	Swin-L	BR+KR	72.7	92.3	80.9	67.6	80.0
GroupPose	ResNet-50	KR	70.2	90.5	77.8	64.7	78.0
GroupPose	Swin-T	KR	72.1	91.4	79.9	66.7	79.5
GroupPose	Swin-L	KR	72.8	92.5	81.0	67.7	80.3

Results on CrowdPose test set

Method	Loss	AP	AP50	AP75	APE	APM	APH
PETR	HM+KR	71.6	90.4	78.3	77.3	72.0	65.8
QueryPose	BR+RLE	72.7	91.7	78.1	79.5	73.4	65.4
ED-Pose	BR+KR	73.1	90.5	79.8	80.5	73.8	63.8
GroupPose	KR	74.1	91.3	80.4	80.8	74.7	66.4

All methods are with the Swin-L backbone.

License

Group Pose is released under the Apache 2.0 license. Please see the LICENSE file for more information.

Acknowledgement

This project is built on the open source repositories PaddleDetection. Thanks them for their well-organized codes!

Citation

@inproceedings{liu2023GroupPose,
  title       = {Group Pose: A Simple Baseline for End-to-End Multi-person Pose Estimation},
  author      = {Liu, Huan and Chen, Qiang and Tan, Zichang and Liu, Jiangjiang and Wang, Jian and Su, Xiangbo and Li, Xiaolong and Yao, Kun and Han, Junyu and Ding, Errui and Zhao, Yao and Wang, Jingdong},
  booktitle   = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
  year        = {2023}
}