Slower Inference time than normal network

[AntoineRichard]

Hi,

I tried your implementation and made some fixes so that it could be used with tensorflow 1.5 (removed the functions inside the functions) and its estimators. After a couple runs I noticed that the inference time of the binary network on mnist is slower than the normal mnist network while being less accurate .

Binary network: Binarized_Conv_32,HardTanh,Binarized_Conv_64,HardTanh, Binarized_Affine 1024, HardTanh, Binarized Affine 10
Normal network: CV32, Relu, CV64, Relu, Dense 1024, Relu, Dense 10

The normal network runs in about 330 ms for 100 iterations
The binary network runs in about 402 ms for 100 iterations

I am running the networks on a GTX 1060 and a intel 7700hq.

Have you ever encountered this kind of issues or should I move to Torch to try binary networks ?

The reason I didn't used your framework is because I can't get the concat function to work and I'd like to make an inception like network using binary layers.

This is the kind of model I tried with your implementation:
model = Sequential([
BinarizedWeightOnlySpatialConvolution(64,3,3,1,1, padding='VALID', bias=False),
BatchNormalization(),
HardTanh(),
BinarizedSpatialConvolution(64,3,3, padding='SAME', bias=False),
BatchNormalization(),
HardTanh(),
BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False),
BatchNormalization(),
HardTanh(),
Concat([
Sequential([
BinarizedSpatialConvolution(96,3,3, padding='SAME', bias=False),
SpatialMaxPooling(2,2,2,2),
BatchNormalization(),
HardTanh()
]),
Sequential([
BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False),
SpatialMaxPooling(2,2,2,2),
BatchNormalization(),
HardTanh()
])
]),
Concat([
Sequential([
BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False),
SpatialMaxPooling(2,2,2,2),
BatchNormalization(),
HardTanh(),
BinarizedSpatialConvolution(196,3,3, padding='SAME', bias=False),
BatchNormalization(),
HardTanh()
]),
Sequential([
BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False),
BatchNormalization(),
HardTanh(),
BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False),
BatchNormalization(),
HardTanh(),
BinarizedSpatialConvolution(196,3,3, padding='SAME', bias=False),
BatchNormalization(),
HardTanh(),
BinarizedSpatialConvolution(196,3,3, padding='SAME', bias=False),
SpatialMaxPooling(2,2,2,2),
BatchNormalization(),
HardTanh()
])
]),
Concat([
Sequential([
BinarizedSpatialConvolution(396,3,3, padding='SAME', bias=False),
SpatialMaxPooling(2,2,2,2),
BatchNormalization(),
HardTanh()
])
]),
Concat([
Sequential([
BinarizedSpatialConvolution(396,3,3, padding='SAME', bias=False),
SpatialMaxPooling(2,2,2,2),
BatchNormalization(),
HardTanh()
]),
Sequential([
BinarizedSpatialConvolution(396,3,3, padding='SAME', bias=False),
SpatialMaxPooling(2,2,2,2),
BatchNormalization(),
HardTanh()
])
]),
BinarizedAffine(1024, bias=False),
BatchNormalization(),
HardTanh(),
BinarizedAffine(1024, bias=False),
BatchNormalization(),
HardTanh(),
BinarizedAffine(10),
BatchNormalization()
])

[suke27]

Hello, do you resolve this problem, I face same problem