Super Slow TFLite YOLOX-Nano (m) inference time speed compared to yolov5

[Mkarami3]

Hello,
Many thanks for sharing your code,
I was wondering if a comparison between YOLOV5 (ultralytics) and the present YOLOX about inference time speed has been done?

For yoloV5 (Small) TFLite with 1 thread, I get around 62 FPS.
For YOLOX (nano), the FPS value is around 1 FPS and for YOLOX (Tiny), FPS value is around 6.5 FPS. However, I was expecting to obtain a higher FPS value using YOLOX.

Am I missing something here?
Thanks

[tucachmo2202]

How did you get tflite model? And can you tell more detail about your device you use?

[Mkarami3]

thank you for your reply.
My CPU information is: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz 3.60 GHz
and I used the following code for conversion to TFLite

#First create the frozen graph using the following command:

onnx-tf convert -i "yolox_s.onnx" -o "yolox_s.pb"
Then run the following python file
converter = tf.lite.TFLiteConverter.from_saved_model('./yolox_s.pb')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS]
converter.allow_custom_ops = False
converter.experimental_new_converter = True
converter.target_spec.supported_types = [tf.float16]
tf_lite_model = converter.convert()
open('yolox_s.tflite', 'wb').write(tf_lite_model)

[tucachmo2202]

There are something I think it may be helpful:

• Your cpu may not support float 16 quantization so if not it will fall back to float 32 when inference.
• When you convert in this way I see many transpose operation added to graph which cause slow down.

[Mkarami3]

Thanks for the notes
One thing that surprised me is that YOLOX_Nano was slower than YOLOX_Tiny.
Is there another recommended way for converting to TFLite?

[tucachmo2202]

Hi,
In my case, yolox-nano is little faster than yolox-tiny.
About convert to tflite you can see this repo and also their script for each model. But with yolox-nano model, I follow their step it still raise this error. If you can convert succesfully, please tell me. Thanks!

[Mkarami3]

Thanks for your helps
I will try the new conversion and post my observation here

[ghost]

@tucachmo2202 In PINTO_model_zoo/132_YOLOX, there is a script called "download_nano.sh". Please launch "download_nano.sh" to download all files then you can find missing "yolox_nano_320x320_tf.xml".

[tucachmo2202]

@Neo1109-Chang-RTK, Thanks. I fixed my problem!