Skip to content

Commit

Permalink
Annotation formats documentation (#719)
Browse files Browse the repository at this point in the history 
* added handling of truncated and difficult attributes for pascal voc
loader/dumper
added descriptions of supported annotation formats
* added YOLO example
* made match_frame as Annotations method
changed 'image/source_id' field TF feature from int64 to string
(according to TF OD API dataset utlis)
* updated README
improved match_frame function
* added unit tests for dump/load
  • Loading branch information
@azhavoro @nmanovic
azhavoro authored and nmanovic committed Oct 31, 2019
1 parent a0f083d commit 05c5230
Show file tree
Hide file tree
Showing 7 changed files with 689 additions and 115 deletions.
352 changes: 351 additions & 1 deletion cvat/apps/annotation/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ It allows to download and upload annotations in different formats and easily add

annotations.add_shape(shape)
```
Full examples can be found in [builtin](builtin) folder.
Full examples can be found in corrseponding *.py files (cvat.py, coco.py, yolo.py, etc.).
1. Add path to a new python script to the annotation app settings:

```python
Expand All @@ -150,3 +150,353 @@ It allows to download and upload annotations in different formats and easily add
Possible solutions: install additional modules via pip call to a separate directory for each Annotation Format
to reduce version conflicts, etc. Thus, custom code can be run in an extended environment, and core CVAT modules
should not be affected. As well, this functionality can be useful for Auto Annotation module.

## Format specifications

### CVAT
This is native CVAT annotation format.
[Detailed format description](cvat/apps/documentation/xml_format.md)

#### CVAT XML for images dumper
- downloaded file: Single unpacked XML
- supported shapes - Rectangles, Polygons, Polylines, Points

#### CVAT XML for videos dumper
- downloaded file: Single unpacked XML
- supported shapes - Rectangles, Polygons, Polylines, Points

#### CVAT XML Loader
- uploaded file: Single unpacked XML
- supported shapes - Rectangles, Polygons, Polylines, Points

### [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/)

#### Pascal dumper description
- downloaded file: a zip archive with following structure:
```bash
taskname.zip
├── frame_000001.xml
├── frame_000002.xml
├── frame_000003.xml
└── ...
```
Each annotation `*.xml` file has a name that corresponds to the name of the image file
(e.g. `frame_000001.xml` is the annotation for the `frame_000001.jpg` image).
Detailed structure specification of the `*.xml` file can be found
[here](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/devkit_doc.pdf).
- supported shapes - Rectangles
- additional comments: If you plan to use 'truncated' and 'difficult' attributes please add the corresponding
items to the CVAT label attributes:
`~checkbox=difficult:false ~checkbox=truncated:false`

#### Pascal loader description
- uploaded file: a zip archive with following structure:
```bash
taskname.zip
├── frame_000001.xml
├── frame_000002.xml
├── frame_000003.xml
└── ...
```
It should be possible to match the CVAT frame(imagename) and image filename from the annotation \*.xml
file (the tag filename, e.g. `<filename>2008_004457.jpg</filename>`). There are 2 options:
1. full match between image name and filename from annotation *.xml
file (in case of a task was created from images or archive of images).
1. match by frame number (if CVAT cannot match by name). File name should be in the following format `frame_%6d.jpg`.
It will be used when task was created from a video.

- supported shapes: Rectangles
- limitations: Support of Pascal VOC object detection format
- additional comments: the CVAT task should be created with the full label set that may be in the annotation files

#### How to create a task from Pascal VOC dataset
1. Download the Pascal Voc dataset (Can be downloaded from the
[PASCAL VOC website](http://host.robots.ox.ac.uk/pascal/VOC/))
1. Create a CVAT task with the following labels:
```bash
aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
```
You can add `~checkbox=difficult:false ~checkbox=truncated:false` attributes for each label if you want to use them.

Select interesting image files
(See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
guide for details)
1. zip the corresponding annotation files
1. click `Upload annotation` button, choose `Pascal VOC ZIP 1.0`
and select the *.zip file with annotations from previous step.
It may take some time.

### [YOLO](https://pjreddie.com/darknet/yolo/)
#### Yolo dumper description
- downloaded file: a zip archive with following structure:
```bash
taskname.zip
├── frame_000001.txt
├── frame_000002.txt
├── ...
└── obj.names
```
Each annotation `*.txt` file has a name that corresponds to the name of the image file
(e.g. `frame_000001.txt` is the annotation for the `frame_000001.jpg` image).
Short description of `*.txt` file structure: each line describes label and bounding box
in the following format `label_id cx cy w h`.
`obj.names` contains the ordered list of label names.
- supported shapes - Rectangles

#### Yolo loader description
- uploaded file: a zip archive with following structure:
```bash
taskname.zip
├── frame_000001.txt
├── frame_000002.txt
├── frame_000003.txt
├── ...
└──obj.names
```
It should be possible to match the CVAT frame(imagename) and annotation filename
There are 2 options:
1. full match between image name and name of annotation `*.txt` file
(in case of a task was created from images or archive of images).
1. match by frame number (if CVAT cannot match by name). File name should be in the following format `frame_%6d.jpg`.
It will be used when task was created from a video.

- supported shapes: Rectangles
- additional comments: the CVAT task should be created with the full label set that may be in the annotation files

#### How to create a task from YOLO formatted dataset (from VOC for example)
1. Follow the official [guide](https://pjreddie.com/darknet/yolo/)(see Training YOLO on VOC section)
and prepare the YOLO formatted annotation files.
1. Zip train images
```bash
zip images.zip -j -@ < train.txt
```
1. Create a CVAT task with the following labels:
```bash
aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
```
Select images.zip as data. Most likely you should use `share`
functionality because size of images.zip is more than 500Mb.
See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
guide for details.
1. Create `obj.names` with the following content:
```bash
aeroplane
bicycle
bird
boat
bottle
bus
car
cat
chair
cow
diningtable
dog
horse
motorbike
person
pottedplant
sheep
sofa
train
tvmonitor
```
1. Zip all label files together (we need to add only label files that correspond to the train subset)
```bash
cat train.txt | while read p; do echo ${p%/*/*}/labels/${${p##*/}%%.*}.txt; done | zip labels.zip -j -@ obj.names
```
1. Click `Upload annotation` button, choose `YOLO ZIP 1.0` and select the *.zip file with labels from previous step.
It may take some time.

### [MS COCO Object Detection](http://cocodataset.org/#format-data)
#### COCO dumper description
- downloaded file: single unpacked `json`. Detailed description of the MS COCO format can be found [here](http://cocodataset.org/#format-data)
- supported shapes - Polygons, Rectangles (interpreted as polygons)

#### COCO loader description
- uploaded file: single unpacked `*.json`.
- supported shapes: Polygons (the `segmentation` must not be empty)
- additional comments: the CVAT task should be created with the full label set that may be in the annotation files

#### How to create a task from MS COCO dataset
1. Download the [MS COCO dataset](http://cocodataset.org/#download).
For example [2017 Val images](http://images.cocodataset.org/zips/val2017.zip)
and [2017 Train/Val annotations](http://images.cocodataset.org/annotations/annotations_trainval2017.zip).
1. Create a CVAT task with the following labels:
```bash
person bicycle car motorcycle airplane bus train truck boat "traffic light" "fire hydrant" "stop sign" "parking meter" bench bird cat dog horse sheep cow elephant bear zebra giraffe backpack umbrella handbag tie suitcase frisbee skis snowboard "sports ball" kite "baseball bat" "baseball glove" skateboard surfboard "tennis racket" bottle "wine glass" cup fork knife spoon bowl banana apple sandwich orange broccoli carrot "hot dog" pizza donut cake chair couch "potted plant" bed "dining table" toilet tv laptop mouse remote keyboard "cell phone" microwave oven toaster sink refrigerator book clock vase scissors "teddy bear" "hair drier" toothbrush
```

Select val2017.zip as data
(See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
guide for details)
1. unpack annotations_trainval2017.zip
1. click `Upload annotation` button,
choose `COCO JSON 1.0` and select `instances_val2017.json.json` annotation file. It may take some time.

### [TFRecord](https://www.tensorflow.org/tutorials/load_data/tf_records)
TFRecord is a very flexible format, but we try to correspond the format that used in
[TF object detection](https://github.com/tensorflow/models/tree/master/research/object_detection)
with minimal modifications.
Used feature description:
```python
image_feature_description = {
'image/filename': tf.io.FixedLenFeature([], tf.string),
'image/source_id': tf.io.FixedLenFeature([], tf.string),
'image/height': tf.io.FixedLenFeature([], tf.int64),
'image/width': tf.io.FixedLenFeature([], tf.int64),
# Object boxes and classes.
'image/object/bbox/xmin': tf.io.VarLenFeature(tf.float32),
'image/object/bbox/xmax': tf.io.VarLenFeature(tf.float32),
'image/object/bbox/ymin': tf.io.VarLenFeature(tf.float32),
'image/object/bbox/ymax': tf.io.VarLenFeature(tf.float32),
'image/object/class/label': tf.io.VarLenFeature(tf.int64),
'image/object/class/text': tf.io.VarLenFeature(tf.string),
}
```
#### TFRecord dumper description
- downloaded file: a zip archive with following structure:
```bash
taskname.zip
├── task2.tfrecord
└── label_map.pbtxt
```
- supported shapes - Rectangles

#### TFRecord loader description
- uploaded file: a zip archive with following structure:
```bash
taskname.zip
└── task2.tfrecord
```
- supported shapes: Rectangles
- additional comments: the CVAT task should be created with the full label set that may be in the annotation files

#### How to create a task from TFRecord dataset (from VOC2007 for example)
1. Create label_map.pbtxt file with the following content:
```js
item {
id: 1
name: 'aeroplane'
}
item {
id: 2
name: 'bicycle'
}
item {
id: 3
name: 'bird'
}
item {
id: 4
name: 'boat'
}
item {
id: 5
name: 'bottle'
}
item {
id: 6
name: 'bus'
}
item {
id: 7
name: 'car'
}
item {
id: 8
name: 'cat'
}
item {
id: 9
name: 'chair'
}
item {
id: 10
name: 'cow'
}
item {
id: 11
name: 'diningtable'
}
item {
id: 12
name: 'dog'
}
item {
id: 13
name: 'horse'
}
item {
id: 14
name: 'motorbike'
}
item {
id: 15
name: 'person'
}
item {
id: 16
name: 'pottedplant'
}
item {
id: 17
name: 'sheep'
}
item {
id: 18
name: 'sofa'
}
item {
id: 19
name: 'train'
}
item {
id: 20
name: 'tvmonitor'
}
```
1. Use [create_pascal_tf_record.py](https://github.com/tensorflow/models/blob/master/research/object_detection/dataset_tools/create_pascal_tf_record.py)
to convert VOC2007 dataset to TFRecord format.
As example:
```bash
python create_pascal_tf_record.py --data_dir <path to VOCdevkit> --set train --year VOC2007 --output_path pascal.tfrecord --label_map_path label_map.pbtxt
```
1. Zip train images
```bash
cat <path to VOCdevkit>/VOC2007/ImageSets/Main/train.txt | while read p; do echo <path to VOCdevkit>/VOC2007/JPEGImages/${p}.jpg ; done | zip images.zip -j -@
```
1. Create a CVAT task with the following labels:
```bash
aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
```
Select images.zip as data.
See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
guide for details.
1. Zip pascal.tfrecord and label_map.pbtxt files together
```bash
zip anno.zip -j <path to pascal.tfrecord> <path to label_map.pbtxt>
```
1. Click `Upload annotation` button, choose `TFRecord ZIP 1.0` and select the *.zip file
with labels from previous step. It may take some time.

### PNG mask
#### Mask dumper description
- downloaded file: a zip archive with the following structure:
```bash
taskname.zip
├── frame_000001.png
├── frame_000002.png
├── frame_000003.png
├── ...
└── colormap.txt
```
Mask is a png image with several (RGB) channels where each pixel has own color which corresponds to a label.
Color generation correspond to the Pascal VOC color generation
[algorithm](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/htmldoc/devkit_doc.html#sec:voclabelcolormap).
(0, 0, 0) is used for background.
`colormap.txt` file contains the values of the used colors in RGB format.
- supported shapes - Rectangles, Polygons

#### Mask loader description
Not supported
18 changes: 18 additions & 0 deletions cvat/apps/annotation/annotation.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ def __init__(self, annotation_ir, db_task, scheme='', host='', create_callback=N
self._host = host
self._create_callback=create_callback
self._MAX_ANNO_SIZE=30000
self._frame_info = {}
self._frame_mapping = {}

db_labels = self._db_task.label_set.all().prefetch_related('attributespec_set').order_by('pk')

Expand Down Expand Up @@ -189,6 +191,10 @@ def _init_frame_info(self):
"height": db_image.height,
} for db_image in self._db_task.image_set.all()}

self._frame_mapping = {
self._get_filename(info["path"]): frame for frame, info in self._frame_info.items()
}

def _init_meta(self):
db_segments = self._db_task.segment_set.all().prefetch_related('job_set')
self._meta = OrderedDict([
Expand Down Expand Up @@ -424,3 +430,15 @@ def _len(self):
@property
def frame_info(self):
return self._frame_info

@staticmethod
def _get_filename(path):
return os.path.splitext(os.path.basename(path))[0]

def match_frame(self, filename):
# try to match by filename
_filename = self._get_filename(filename)
if _filename in self._frame_mapping:
return self._frame_mapping[_filename]

raise Exception("Cannot match filename or determinate framenumber for {} filename".format(filename))
Loading

0 comments on commit 05c5230

Please sign in to comment.