Compiler fails on model, which runs only with TFLite 2.14.0 and above

[adamp87]

Description

I have a SPPBottleneck module, which is commonly used in object detectors. This module includes 3 separate MaxPool2D layers then it is concatenated. Compiling it, results in ERROR: :80 t->dims->data[d] != t0->dims->data[d] (9 != 13). I have the latest compiler, version 16.0.384591198.

I have been testing around and I got the same message with TFLite below version 2.13, inference done on CPU. Above TFLite 2.14.0, the model inference is done properly.

Question: Can the TFLite Interpreter be updated in the edgetpu_compiler?

I have attached a tflite model which consist of the SPPBottleneck and the visualization of it. Furthermore added the two Dockerfile and the testcode i was using.

TFLite file

FROM python:3.9
RUN pip install numpy==1.26.4 tflite-runtime==2.14.0
# Test code output: Done

FROM python:3.9
RUN pip install numpy==1.26.4 tflite-runtime==2.13.0
# Test code output: RuntimeError: /tensorflow/tensorflow/lite/kernels/concatenation.cc:159 t->dims->data[d] != t0->dims->data[d] (9 != 13)Node number 12 (CONCATENATION) failed to prepare.

# Test Code
#!/usr/bin/env python3
import numpy as np
import tflite_runtime.interpreter as tflite

def tflite_inference(export_name, sample_input):
    # test converted model with TFLite Interpreter
    interpreter = tflite.Interpreter(model_path=export_name)
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    interpreter.allocate_tensors()

    scale_i8, shift_i8 = input_details[0]['quantization']
    input_data = np.array(sample_input / scale_i8 + shift_i8, dtype=np.int8)
    interpreter.set_tensor(input_details[0]['index'], input_data)
    interpreter.invoke()

    scale_i8, shift_i8 = output_details[0]['quantization']
    output_data = interpreter.get_tensor(output_details[0]['index'])
    output_np = (output_data.astype(np.float32) - shift_i8) * scale_i8
    return output_np


if __name__ == '__main__':
    export_name = "./test.tflite"
    sample_input = np.random.random((1, 320, 13, 13))
    tf_output = tflite_inference(export_name, sample_input)
    print("Done")

Click to expand!

Issue Type

Feature Request, Bug

Operating System

Ubuntu

Coral Device

No response

Other Devices

No response

Programming Language

No response

Relevant Log Output

Edge TPU Compiler version 16.0.384591198
Started a compilation timeout timer of 180 seconds.
ERROR: :80 t->dims->data[d] != t0->dims->data[d] (9 != 13)
ERROR: Node number 12 (CONCATENATION) failed to prepare.

Compilation failed: Model failed in Tflite interpreter. Please ensure model can be loaded/run in Tflite interpreter.
Compilation child process completed within timeout period.
Compilation failed!

```</details>

[adamp87]

Tested also with the new ai-edge-litert and the inference runs as expected. An update on the Interpreter for the edgetpu compiler probably would solve this issue.

@dmitriykovalev I saw you uploaded the compiler binaries 5 years ago. What is your opinion?