[Future Request] 5D Transpose support in the edgetpu compiler

[PINTO0309]

1. Overview

There are many good quality models nowadays, including the 5D Transpose. For example, YOLOv4, YOLOv5, NanoDet, and so on. Currently, EdgeTPU Compiler does not support 5D transpose, so being able to do the transformations will benefit many engineers as it will allow them to generate very high performance edgetpu tflite models.

2. Details

2-1. Environment

• Ubuntu 18.04
• Edge TPU Compiler version 15.0.340273435
• TensorFlow version tf-nightly==2.5.0-dev20201204
• NanoDet (Super fast and lightweight anchor-free object detection model. Real-time on mobile devices)

2-2. NanoDet repository of citations

https://github.com/RangiLyu/nanodet.git

2-3. NanoDet Full Integer Quant tflite:

tflite: https://drive.google.com/file/d/1CXWOabh2Kml-0hAZrDg2fhqaUeOJ4JaL/view?usp=sharing

Model structure diagram of NanoDet Full Integer Quant tflite

This includes the 5D Transpose OP, which is not supported by the EdgeTPU Compiler.

2-4. NanoDet EdgetTPU tflite

tflite: https://drive.google.com/file/d/1qqtj-DndmFTIJBl4tDkuZPeBwA-eHV7T/view?usp=sharing

5D Transpose and later are ignored for conversion to custom op.

Model structure diagram of NanoDet EdgetTPU tflite

3. Benchmarks

Performance is measured on Kirin 980(4xA76+4xA55) ARM CPU based on ncnn. There is no doubt that if we were able to convert this model perfectly in EdgeTPU, the performance would be much higher than the benchmark using NCNN.

Model	Resolution	COCO mAP	Latency(ARM 4xCore)	FLOPS	Params	Model Size(ncnn bin)
NanoDet-m	320*320	20.6	10.23ms	0.72B	0.95M	1.8mb
NanoDet-m	416*416	21.7	16.44ms	1.2B	0.95M	1.8mb
YoloV3-Tiny	416*416	16.6	37.6ms	5.62B	8.86M	33.7mb
YoloV4-Tiny	416*416	21.7	32.81ms	6.96B	6.06M	23.0mb

4. Appendix

I have already been able to convert all OPs except 5D Transpose into OPs supported by the edgetpu compiler.

4-1. PyTorch -> ONNX -> OpenVINO -> TFLite, Conversion Tools

openvino2tensorflow: https://github.com/PINTO0309/openvino2tensorflow.git

4-2. The various models generated by the conversion work are committed to

https://github.com/PINTO0309/PINTO_model_zoo/tree/master/072_NanoDet

[PINTO0309]

EdgeTPU Compiler v16
A lot more layers have been converted than when I posted the issue, but 5D Reshape and Transpose are still not converted.

[HeywardLiu]

@PINTO0309
Hi, Is 5D-transpose still unsupport in the edgetpu compiler?
I'm trying to convert a model from pytroch -> onnx -> tensorflow -> tflite,
but I found there's additional 5D transpose ops have been added during conversion.

So, If edgetpu compiler still not support 5D-transpose operation,
I guess it is impossible to run my model on coral tpu?

Thank you for your attention and patience!

[hjonnala]

@HeywardLiu @PINTO0309
Tensors have to be either 1-, 2-, or 3-dimensional. If a tensor has more than 3 dimensions, then only the 3 innermost dimensions may have a size greater than 1.

For example:
(1,320,320,3) and (1,3,320,320,1) --> supported
(320,1,320, 3) and (3,320,1,320) --> unsupported.

We don't have any plans to support tensors with more than 3 innermost dimensions.

Thanks!

[HeywardLiu]

@hjonnala
I appreciate your message.
Thank you for your quick reply!

[google-coral-bot]

Are you satisfied with the resolution of your issue?
Yes
No

[PINTO0309]

Thanks.