Skip to content

Latest commit

 

History

History
181 lines (151 loc) · 8.39 KB

README.md

File metadata and controls

181 lines (151 loc) · 8.39 KB
Raw

High Quality Entity Segmentation (ICCV 2023 Oral)

Lu Qi, Jason Kuen, Tiancheng Shen, Jiuxiang Gu, Weidong Guo, Jiaya Jia, Zhe Lin, Ming-Hsuan Yang

This project offers an implementation of the paper, "High-Quality Entity Segmentation". This repository serves as an unofficial extension to the Adobe EntitySeg Github, where you can directly download the EntitySeg Dataset and the source code of our proposed CropFormer. For a more comprehensive view of our results and visualizations, we invite you to explore our project website.


News

2023-09-07 The dataset, code and pretrained models are released.

2023-08-01 Our paper is accepted as ICCV2023 oral.

Data

Please refer to the official repo EntitySeg-Dataset for annotation files and image URLs. For convenience, we provide the images in several links including Google Drive and Hugging Face, but we do not own the copyright of the images. It is solely your responsibility to check the original licenses of the images before using them. Any use of the images are at your own discretion and risk. Furthermore, please refer to the dataset description on how to set up the dataset before running our code.

Code

We offer the instructions on installation, train, evaluation and visualization for the proposed CropFormer in the code description.

Model Zoo

Overall, we also provide the pretrained segmentation models in two links including Google Drive or Hugging Face. We illustrate them in the following. For the class-aware segmentation tasks, we directly use the COCO-pretrained Mask2Former Model as our pretrain weights.

Entity Segmentation

We provide several entity segmentation models. For all the training, we use the COCO-Entity pretrained models as our initialization that are provided in Google Drive or Hugging Face. We evaluate the model on both the overlapped-free $AP$ and no-overlapped $AP^e$ in low-resolution (L) and high-resolution images (H).

Method Backbone Sched AP_L AP_L^e AP_H AP_H^e download
Mask2Former Swin-T 3x - 38.8 - 40.7 model
CropFormer Swin-T 3x - 40.6 - 43.0 model
Mask2Former Swin-L 3x - 44.4 - 46.2 model
CropFormer Swin-L 3x - 45.8 - 48.2 model
Mask2Former Hornet-L 3x 51.0 47.1 53.6 49.2 model
CropFormer Hornet-L 3x - 49.1 - 51.5 model

Instance Segmentation

Method Backbone Sched AP_H download
Mask2Former Swin-T 3x 22.7 model
Mask2Former Swin-L 3x 30.3 model

Semantic Segmentation

Method Backbone Sched mIoU_H download
Mask2Former Swin-T 3x 45.2 model
Mask2Former Swin-L 3x 51.1 model

Panoptic Segmentation

Method Backbone Sched PQ_H download
Mask2Former Swin-T 3x 9.8 model
Mask2Former Swin-L 3x 13.5 model

Citing Ours

Consider to cite High Quality Entity Segmentation if it helps your research.

@inproceedings{qi2022high,
  title={High Quality Entity Segmentation},
  author={Qi, Lu and Kuen, Jason and Shen, Tiancheng and Gu, Jiuxiang and Guo, Weidong and Jia, Jiaya and Lin, Zhe and Yang, Ming-Hsuan},
  booktitle={ICCV},
  year={2023}
}

License

The code and models are released under the CC BY-NC 4.0 license.