Convert tensorflow tensor to Keras tensor

[sanjeevmk]

I have a trained tensorflow model that I've loaded using the checkpoint and meta files. The model loads properly. But now I'd like to work with the same model using Keras, instead of direct tensorflow.

I got the input and output tensor of the loaded model:

images_placeholder = tf.get_default_graph().get_tensor_by_name("input:0")
embeddings = tf.get_default_graph().get_tensor_by_name("embeddings:0")

Now I want to create a Keras model with these, so I tried this:
model = Model(images_placeholder,embeddings)

This however, gives me this error:
TypeError: Input tensors to a Model must be Keras tensors. Found: Tensor("input:0", shape=(?, 160, 160, 3), dtype=float32) (missing Keras metadata).
Any ideas on how to fix this? I also don't know the architecture of this model, so I wanted to do model.summary() after the keras model is loaded. I thought Keras worked seamlessly with Tensorflow and Theano graphs..

[ahundt]

perhaps this tutorial can help? https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html

[ghost]

Have you been able to solve this issue? I am trying something similar (see here). The link from @ahundt did not help me unfortunately. It appears that Dense#set_input() is no more in the current version of Keras.

[sanjeevmk]

No I haven't been able to solve my issue. I face the problem when I apply keras based operations directly on tensorflow tensors, I don't face it when I use pure tensorflow without keras. Also for your case, I think just -1*img should work, instead of tf.mul or merging with mul etc? Scalar multiplications and additions directly work in both TF and Keras.

[ahundt]

I'm dealing with this as detailed in these releated issues:

• ImageDataGenerator.flow() with tf tensor of images/labels, ex: TFRecord #5356
• sess.run() with Keras model and TFRecord tensor based images #5368
• http://stackoverflow.com/questions/42184863/how-do-you-make-tensorflow-keras-fast-with-a-tfrecord-dataset

So far it seems you have to either convert the tf tensors to numpy arrays or write/use TF only training code as supplied in tf itself, or potentially making use of https://github.com/ppwwyyxx/tensorpack or https://github.com/zsdonghao/tensorlayer.

My code where I'm trying out some of these things is here:
https://github.com/ahundt/tf-image-segmentation/blob/ahundt-keras/tf_image_segmentation/recipes/pascal_voc/FCNs/densenet_fcn.py

I'd be happy to collaborate and/or interested in your solutions to these holes in the API as well.

[ahundt]

Oh this stackoverflow question also looks like it might be very helpful:
http://stackoverflow.com/questions/36026892/how-can-i-convert-tfrecords-into-numpy-arrays

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs, but feel free to re-open it if needed.

[mddrill]

I'm having this issue also. Has anyone found a solution?
https://stackoverflow.com/questions/44466066/how-can-i-convert-a-trained-tensorflow-model-to-keras

[ahundt]

ah whoops! I thought this was about tfrecords.

You can extract numpy arrays from a tensorflow model, and you can set keras weights from a numpy array. You can use the layer name strings to access specific layers. Therefore:

1. create a keras model that matches the tf model
2. figure out the exact name string of each tf model layer and its corresponding keras layer
3. extract the numpy array from each tf layer, and load it into each keras layer
4. save the keras model

That's the way to do the conversion.

extracting weights:
https://www.quora.com/How-do-I-save-weights-of-training-data-from-MNIST-testing-on-tensorflow-for-future-use

Inspecting tfrecord model weight checkpoints (get the names):

python ~/src/tensorflow/tensorflow/python/tools/inspect_checkpoint.py --file_name=/path/to/tf_image_segmentation_checkpoints/resnet_v1_101.ckpt > ~/tf_image_segmentation_checkpoints/resnet_v1_101_chkpt_tensors.txt

[mddrill]

@ahundt So there's no way to do this without rewriting the whole model in keras? (It's kind of a complex model, so this wouldn't be trivial)

I was hoping there would be some way to either convert the tf.Session to a keras Model object after initializing the model with the weights vector, or convert the protobuf file to a JSON file containing a keras model.

[ahundt]

Maybe someone wrote something that can convert models from tf to keras, try googling around? To my knowledge it isn't built into in the Keras API.

[ahundt]

@mddrill This has an example of how to convert models from TF to keras, you could implement something similar:
https://github.com/titu1994/MobileNetworks

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[bottydim]

Has anybody found an answer to this one?
I am training to do the following:

model = VGG16(weights='imagenet', include_top=True)
input_1 = model.input
output_1 = model.output
new_model = Model(inputs=input_1,outputs=output_1)

TypeError: Input tensors to a Model must be Keras tensors. Found: Tensor("input_1:0", shape=(?, 224, 224, 3), dtype=float32) (missing Keras metadata).

[mddrill]

@bottydim I stopped working on this problem a year ago. But you may want to check out https://stackoverflow.com/questions/44466066/how-can-i-convert-a-trained-tensorflow-model-to-keras

[fornaciari]

Make your own layer: https://keras.io/layers/writing-your-own-keras-layers/

[HarborZeng]

Make your own layer: https://keras.io/layers/writing-your-own-keras-layers/

please explain more specificly, how and way to use a custom layer can solve this problem. I am stuck in this for a long time. Thanks

[fornaciari]

The core is the build function, where you create the trainable weights.
If you only want to wrap your weights into a Keras layer, pass them to the class and skip the definition of the new tensor.
For example, from:

class MyLayer(Layer):

def __init__(self, output_dim, **kwargs):
    self.output_dim = output_dim
    super(MyLayer, self).__init__(**kwargs)

def build(self, input_shape):
    assert isinstance(input_shape, list)
    # Create a trainable weight variable for this layer.
    self.kernel = self.add_weight(name='kernel',
                                  shape=(input_shape[0][1], self.output_dim),
                                  initializer='uniform',
                                  trainable=True)
    super(MyLayer, self).build(input_shape)  # Be sure to call this at the end

to:

class MyLayer(Layer):
def init(self, your_matrix, **kwargs):
self.your_matrix = your_matrix
super(MyLayer, self).init(**kwargs)

def build(self, class_param): 
    super(MyLayer, self).build(class_param)  # Be sure to call this at the end

I hope this helps.

[HarborZeng]

@fornaciari It helped me a lot! I wonder how to call this MyLayer and what is this your_matrix means?

[HarborZeng]

@fornaciari I raised a question at https://stackoverflow.com/questions/55354612/how-to-turn-a-tf-tensor-to-some-form-that-keras-can-fit, and someone metioned your custom layer way. You can see my code shown below, I tried as you explained to me just now, and still got some error:

_AttributeError: 'NoneType' object has no attribute 'inbound_nodes'

def pointnet2(nb_classes):
    input_points = tf.placeholder(tf.float32, shape=(16, 1024, 3))
    model_input = Input(tensor=input_points)

    sa1_xyz, sa1_points = set_abstraction_msg(model_input,
                                              None,
                                              512,
                                              [0.1, 0.2, 0.4],
                                              [16, 32, 128],
                                              [[32, 32, 64], [64, 64, 128], [64, 96, 128]])

    sa2_xyz, sa2_points = set_abstraction_msg(sa1_xyz,
                                              sa1_points,
                                              128,
                                              [0.2, 0.4, 0.8],
                                              [32, 64, 128],
                                              [[64, 64, 128], [128, 128, 256], [128, 128, 256]])

    sa3_xyz, sa3_points = set_abstraction(sa2_xyz,
                                          sa2_points,
                                          [256, 512, 1024])

    # point_net_cls
    c = Dense(512, activation='relu')(sa3_points)
    c = BatchNormalization()(c)
    c = Dropout(0.5)(c)
    c = Dense(256, activation='relu')(c)
    c = BatchNormalization()(c)
    c = Dropout(0.5)(c)
    c = Dense(nb_classes, activation='softmax')(c)
    prediction = Flatten()(c)
    output_keras = TfToKerasLayer()(prediction)
    model = Model(inputs=model_input, outputs=output_keras)

    # turn tf tensor to keras
    return model


class TfToKerasLayer(Layer):
    def __init__(self, **kwargs):
        super(TfToKerasLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        # 为该层创建一个可训练的权重
        super(TfToKerasLayer, self).build(input_shape)  # 一定要在最后调用它