Converter for pytorch models that have multiple inputs

[bayleef1]

Hi, thanks for your excellent work. I've converted my pytorch model that has one input to tflite model successfully.
However, the converter seems not support pytorch models that have multiple inputs yet. Any plan for it?

[peterjc123]

@bayleef1 We do support models with multiple inputs. You need to pass the inputs as a tuple. e.g.

input_1 = torch.randn(1, 3, 224, 224)
input_2 = torch.randn(1, 3, 224, 224)
converter = TFLiteConverter(model, (input_1, input_2), ...)
converter.convert()

[bayleef1]

OK, I successfully converted pytorch model with multiple inputs and outputs that exclude lstm state parameters.
When I tried to update lstm state by adding parameters in function 'forward(self, lstm_h, lstm_c, ...)' and 'output, (lstm_h, lstm_c) = self.lstm(input, (lstm_h, lstm_c))', I got the error message:
assert len(missing_outputs) == 0, f'Some output nodes are missing: {missing_outputs}'
AssertionError: Some output nodes are missing: ['hx1.1', 'hx2.1']
I wonder if the converter could solve this properly when both model inputs and outputs include lstm state.

[peterjc123]

@bayleef1 Yes, it's due to a limitation that you cannot read or write variables in the TFLite graph. So you need to export the model without adding them to inputs or outputs. But you can still use them through the variable mechanism.
For example, consider the following subgraph.

You are free to access them through set_tensor and get_tensor, given you've found out the locations of the state tensors.

import tensorflow as tf
import numpy as np

interpreter = tf.lite.Interpreter(model_path='xxx.tflite')
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
tensor_details = interpreter.get_tensor_details()
tensor_shapes = {d['index']: d['shape'] for d in tensor_details}

# set inputs
for i in range(len(input_details)):
    interpreter.set_tensor(input_details[i]['index'], np.random.random(input_details[i]['shape']))

# set states
state_tensors = [3, 21]
for i in state_tensors:
    interpreter.set_tensor(i, np.zeros(tensor_shapes[i]))

# invoke
interpreter.invoke()

# get outputs
outputs = []
for i in range(len(output_details)):
    outputs.append(interpreter.get_tensor(output_details[i]['index']))

# get states
states = []
for i in state_tensors:
    states.append(interpreter.get_tensor(i))

[peterjc123]

https://github.com/alibaba/TinyNeuralNetwork/blob/main/docs/FAQ.md#how-to-convert-a-model-with-lstm
https://github.com/alibaba/TinyNeuralNetwork/blob/main/docs/FAQ_zh-CN.md#%E7%AE%97%E5%AD%90%E4%B8%8D%E6%94%AF%E6%8C%81%E5%A6%82%E4%BD%95%E5%A4%84%E7%90%86