STGCN is one of the first algorithms that adopt Graph Convolution Neural Networks for skeleton processing. We provide STGCN trained on NTURGB+D with 2D skeletons (HRNet) and 3D skeletons in both the original training setting and the PYSKL training setting. We provide checkpoints for four modalities: Joint, Bone, Joint Motion, and Bone Motion. The accuracy of each modality links to the weight file.
@inproceedings{yan2018spatial,
title={Spatial temporal graph convolutional networks for skeleton-based action recognition},
author={Yan, Sijie and Xiong, Yuanjun and Lin, Dahua},
booktitle={Thirty-second AAAI conference on artificial intelligence},
year={2018}
}
# If you use the STGCN with PYSKL practices in your work
@misc{duan2022pyskl,
title={PYSKL: a toolbox for skeleton-based video understanding},
author={PYSKL Contributors},
howpublished = {\url{https://github.com/kennymckormick/pyskl}},
year={2022}
}
We release numerous checkpoints trained with various modalities, annotations on NTURGB+D and NTURGB+D 120. The accuracy of each modality links to the weight file.
Dataset | Practice | Annotation | GPUs | Training Epochs | Joint Top1 Config Link: Weight Link |
Bone Top1 Config Link: Weight Link |
Joint Motion Top1 Config Link: Weight Link |
Bone-Motion Top1 Config Link: Weight Link |
Two-Stream Top1 | Four Stream Top1 |
---|---|---|---|---|---|---|---|---|---|---|
NTURGB+D XSub | Vanilla | Official 3D Skeleton | 8 | 80 | joint_config: 81.5 | bone_config: 81.0 | joint_motion_config: 79.9 | bone_motion_config: 81.2 | 84.3 | 86.6 |
NTURGB+D XSub | Vanilla | HRNet 2D Skeleton | 8 | 80 | joint_config: 85.7 | bone_config: 85.8 | joint_motion_config: 81.6 | bone_motion_config: 83.9 | 88.8 | 90.1 |
NTURGB+D XSub | PYSKL | Official 3D Skeleton | 8 | 80 | joint_config: 87.8 | bone_config: 88.6 | joint_motion_config: 85.8 | bone_motion_config: 86.2 | 90.0 | 90.7 |
NTURGB+D XSub | PYSKL | HRNet 2D Skeleton | 8 | 80 | joint_config: 89.0 | bone_config: 91.2 | joint_motion_config: 86.7 | bone_motion_config: 87.8 | 92.0 | 92.4 |
NTURGB+D XView | Vanilla | Official 3D Skeleton | 8 | 80 | joint_config: 90.1 | bone_config: 87.7 | joint_motion_config: 88.8 | bone_motion_config: 88.3 | 91.4 | 93.2 |
NTURGB+D XView | Vanilla | HRNet 2D Skeleton | 8 | 80 | joint_config: 92.4 | bone_config: 90.0 | joint_motion_config: 92.0 | bone_motion_config: 86.5 | 93.8 | 95.1 |
NTURGB+D XView | PYSKL | Official 3D Skeleton | 8 | 80 | joint_config: 95.5 | bone_config: 95.0 | joint_motion_config: 93.7 | bone_motion_config: 92.8 | 96.2 | 96.5 |
NTURGB+D XView | PYSKL | HRNet 2D Skeleton | 8 | 80 | joint_config: 98.0 | bone_config: 96.5 | joint_motion_config: 95.6 | bone_motion_config: 95.4 | 98.2 | 98.3 |
NTURGB+D 120 XSub | PYSKL | Official 3D Skeleton | 8 | 80 | joint_config: 82.1 | bone_config: 83.7 | joint_motion_config: 80.3 | bone_motion_config: 80.6 | 85.6 | 86.2 |
NTURGB+D 120 XSub | PYSKL | HRNet 2D Skeleton | 8 | 80 | joint_config: 80.1 | bone_config: 83.4 | joint_motion_config: 78.6 | bone_motion_config: 79.8 | 84.0 | 84.7 |
NTURGB+D 120 XSet | PYSKL | Official 3D Skeleton | 8 | 80 | joint_config: 84.5 | bone_config: 85.8 | joint_motion_config: 82.7 | bone_motion_config: 83.0 | 87.5 | 88.4 |
NTURGB+D 120 XSet | PYSKL | HRNet 2D Skeleton | 8 | 80 | joint_config: 84.2 | bone_config: 87.7 | joint_motion_config: 82.5 | bone_motion_config: 83.5 | 88.3 | 89.0 |
Note
- We use the linear-scaling learning rate (Initial LR ∝ Batch Size). If you change the training batch size, remember to change the initial LR proportionally.
- For Two-Stream results, we adopt the 1 (Joint):1 (Bone) fusion. For Four-Stream results, we adopt the 2 (Joint):2 (Bone):1 (Joint Motion):1 (Bone Motion) fusion.
You can use the following command to train a model.
bash tools/dist_train.sh ${CONFIG_FILE} ${NUM_GPUS} [optional arguments]
# For example: train STGCN on NTURGB+D XSub (3D skeleton, Joint Modality) with 8 GPUs, with validation, with PYSKL practice, and test the last and the best (with best validation metric) checkpoint.
bash tools/dist_train.sh configs/stgcn/stgcn_pyskl_ntu60_xsub_3dkp/j.py 8 --validate --test-last --test-best
You can use the following command to test a model.
bash tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${NUM_GPUS} [optional arguments]
# For example: test STGCN on NTURGB+D XSub (3D skeleton, Joint Modality) with metrics `top_k_accuracy`, and dump the result to `result.pkl`.
bash tools/dist_test.sh configs/stgcn/stgcn_pyskl_ntu60_xsub_3dkp/j.py checkpoints/SOME_CHECKPOINT.pth 8 --eval top_k_accuracy --out result.pkl