a capsule cnn on cifar-10 #9193

bojone · 2018-01-26T04:10:24Z

this is a fast capsule implement. and got the better performance than https://github.com/keras-team/keras/blob/master/examples/cifar10_cnn.py.

fchollet · 2018-01-26T19:56:16Z

Thanks for the PR. There seems to be various style issues. Please work on fixing the code style.

bojone · 2018-01-27T03:29:20Z

@fchollet please check it again. I am sorry that I am not very clear what style issues means.

fchollet · 2018-01-27T03:31:20Z

You can start by getting your code to pass the checks run by a PEP8 linter: https://pypi.python.org/pypi/pep8

bojone · 2018-01-27T04:26:18Z

@fchollet thank you for introducting me such a useful tool. I have change the code with the guide of pep8

tboquet

Nice addition! Please address or discuss the few modifications suggested.

tboquet · 2018-02-08T14:06:45Z

examples/cifar10_cnn_capsule.py

+    return ex / K.sum(ex, axis=axis, keepdims=True)
+
+
+'''


Please put this comment on top of your file with the description of the example.

tboquet · 2018-02-08T14:07:44Z

examples/cifar10_cnn_capsule.py

+
+class Capsule(Layer):
+    def __init__(self,
+                 num_capsule,  # the number of output capsules


Please put the documentation in the docstring and follow Keras' documentation style. You can find the style in the docstring of any layer.

tboquet · 2018-02-08T14:09:36Z

examples/cifar10_cnn_capsule.py

+model = Model(inputs=input_image, outputs=output)
+
+# we use a margin loss
+model.compile(loss=lambda y_true, y_pred: y_true * K.relu(0.9 - y_pred)**2 +


It is a good idea to implement your loss outside of compile so that your code is readable.

tboquet · 2018-02-08T14:10:00Z

examples/cifar10_cnn_capsule.py

+                                            self.num_capsule,
+                                            self.dim_capsule))
+        u_hat_vecs = K.permute_dimensions(u_hat_vecs, (0, 2, 1, 3))
+        # final u_hat_vecs.shape = [None, num_capsule,


Are those comments needed?

tboquet · 2018-02-08T14:12:52Z

examples/cifar10_cnn_capsule.py

+        # final u_hat_vecs.shape = [None, num_capsule,
+        #                           input_num_capsule, dim_capsule]
+
+        b = K.zeros_like(u_hat_vecs[:, :, :, 0])  # shape = [None, num_capsule,


Put your comment on top of the operation please.

tboquet · 2018-02-08T14:13:13Z

examples/cifar10_cnn_capsule.py

+        return (None, self.num_capsule, self.dim_capsule)
+
+
+# some parameters


Clarify or remove.

tboquet · 2018-02-08T14:13:44Z

examples/cifar10_cnn_capsule.py

+              metrics=['accuracy'])
+model.summary()
+
+# we can compare the perfermace with or without data augmentation


Typo: perfermace -> performance

tboquet · 2018-02-08T14:15:13Z

examples/cifar10_cnn_capsule.py

+    print('Using real-time data augmentation.')
+    # This will do preprocessing and realtime data augmentation:
+    datagen = ImageDataGenerator(
+        featurewise_center=False,  # set input mean to 0 over the dataset


Everything in the ImageDataGenerator is already documented, I'm not 100% sure but I would remove the inline comments.

This is copied verbatim from another example. If comments are removed, it'll be inconsistent with other examples, where this is pervasive throughout.

tboquet · 2018-02-09T15:03:46Z

@bojone any news? Your example is a nice addition and you only need to change a few things 👍 .

bojone · 2018-02-10T01:23:04Z

@tboquet thanks your useful suggestions! I will fix it as soon as possible.

fchollet · 2018-02-10T21:34:35Z

examples/cifar10_cnn_capsule.py

+        super(Capsule, self).build(input_shape)
+        input_dim_capsule = input_shape[-1]
+        if self.share_weights:
+            self.W = self.add_weight(


Call this kernel for consistency with other Keras layers

fchollet · 2018-02-10T21:35:10Z

examples/cifar10_cnn_capsule.py

+        self.dim_capsule = dim_capsule
+        self.routings = routings
+        self.share_weights = share_weights
+        if activation == 'default':


Why default? What does that mean?

default means we use squashing function as activation. I rename it now.

fchollet · 2018-02-10T21:36:01Z

examples/cifar10_cnn_capsule.py

+        if activation == 'default':
+            self.activation = squash
+        else:
+            self.activation = Activation(activation)


This should be activations.get(activation) where activations is from keras import activations

fchollet · 2018-02-10T21:36:41Z

examples/cifar10_cnn_capsule.py

+            self.activation = Activation(activation)
+
+    def build(self, input_shape):
+        super(Capsule, self).build(input_shape)


No point in calling super here

fchollet · 2018-02-10T21:37:35Z

examples/cifar10_cnn_capsule.py

+
+# A common Conv2D model
+input_image = Input(shape=(None, None, 3))
+cnn = Conv2D(64, (3, 3), activation='relu')(input_image)


cnn sounds like a model instance, but it's a tensor. Call it x

fchollet · 2018-02-10T21:37:54Z

examples/cifar10_cnn_capsule.py

+                trainable=True)
+
+    def call(self, u_vecs):
+        """


Format your docstrings like other docstrings in the codebase

fchollet · 2018-02-10T21:38:08Z

examples/cifar10_cnn_capsule.py

+
+class Capsule(Layer):
+    """
+    A Capsule Implement with Pure Keras


Format your docstrings like other docstrings in the codebase

bojone · 2018-02-11T04:26:04Z

@fchollet my latest submit has a "SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)" error. what is the reason?

fchollet · 2018-02-11T08:45:29Z

examples/cifar10_cnn_capsule.py

+
+Without Data Augmentation:
+It gets to 75% validation accuracy in 10 epochs,
+and 79% after 15 epochs, and overfitting after 20 epcohs


epochs (typo)

fchollet · 2018-02-11T08:45:55Z

examples/cifar10_cnn_capsule.py

+
+
+# a squashing function. but it has litte difference from the Hinton's paper.
+# it seems that this form of squashing performs better.


Explain what the difference is and what motivates it

fchollet · 2018-02-11T08:46:14Z

examples/cifar10_cnn_capsule.py

+    return scale * x
+
+
+# define our own softmax function instead of K.softmax


Why though?

fchollet · 2018-02-12T19:31:32Z

examples/cifar10_cnn_capsule.py

+
+
+# define our own softmax function instead of K.softmax
+# because K.softmax can not specify axis.


That's something we should fix in the Keras backend.

Wasn't it implemented in #8841?

fchollet

LGTM, thanks

* 'master' of github.com:fchollet/keras: (57 commits) Minor README edit Speed up Travis tests (keras-team#9386) fix typo (keras-team#9391) Fix style issue in docstring Prepare 2.1.4 release. Fix activity regularizer + model composition test Corrected copyright years (keras-team#9375) Change default interpolation from nearest to bilinear. (keras-team#8849) a capsule cnn on cifar-10 (keras-team#9193) Enable us to use sklearn to do cv for functional api (keras-team#9320) Add support for stateful metrics. (keras-team#9253) The type of list keys was float (keras-team#9324) Fix mnist sklearn wrapper example (keras-team#9317) keras-team#9287 Fix most of the file-handle resource leaks. (keras-team#9309) Pass current learning rate to schedule() in LearningRateScheduler (keras-team#8865) Simplify with from six.moves import input (keras-team#9216) fixed RemoteMonitor: Json to handle np.float32 and np.int32 types (keras-team#9261) Update tweet length from 140 to 280 in docs Add `depthconv_conv2d` tests (keras-team#9225) Remove `force` option in progbar ...

* a capsule cnn on cifar-10 * Update cifar10_cnn_capsule.py * update the style * Update cifar10_cnn_capsule.py * Update cifar10_cnn_capsule.py * Update cifar10_cnn_capsule.py * Update cifar10_cnn_capsule.py * pass pep8 verify * Update cifar10_cnn_capsule.py * Update cifar10_cnn_capsule.py

a capsule cnn on cifar-10

81531af

fchollet added the Reviewers wanted label Jan 26, 2018

Update cifar10_cnn_capsule.py

ab0b308

update the style

6c53e54

Update cifar10_cnn_capsule.py

9ae1789

tboquet suggested changes Feb 8, 2018

View reviewed changes

bojone added 4 commits February 10, 2018 10:07

Update cifar10_cnn_capsule.py

8ef2397

Update cifar10_cnn_capsule.py

f9cc297

Update cifar10_cnn_capsule.py

6594763

pass pep8 verify

4f1cbb9

fchollet reviewed Feb 10, 2018

View reviewed changes

Update cifar10_cnn_capsule.py

307de84

fchollet reviewed Feb 11, 2018

View reviewed changes

Update cifar10_cnn_capsule.py

25ff097

fchollet reviewed Feb 12, 2018

View reviewed changes

fchollet approved these changes Feb 12, 2018

View reviewed changes

fchollet merged commit f6958ab into keras-team:master Feb 12, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

a capsule cnn on cifar-10 #9193

a capsule cnn on cifar-10 #9193

bojone commented Jan 26, 2018

fchollet commented Jan 26, 2018

bojone commented Jan 27, 2018

fchollet commented Jan 27, 2018

bojone commented Jan 27, 2018

tboquet left a comment

tboquet Feb 8, 2018

tboquet Feb 8, 2018

tboquet Feb 8, 2018

tboquet Feb 8, 2018

tboquet Feb 8, 2018

tboquet Feb 8, 2018

tboquet Feb 8, 2018

tboquet Feb 8, 2018

ozabluda Feb 10, 2018

tboquet commented Feb 9, 2018

bojone commented Feb 10, 2018

fchollet Feb 10, 2018

fchollet Feb 10, 2018

bojone Feb 11, 2018

fchollet Feb 10, 2018

fchollet Feb 10, 2018

fchollet Feb 10, 2018

fchollet Feb 10, 2018

fchollet Feb 10, 2018

bojone commented Feb 11, 2018

fchollet Feb 11, 2018

fchollet Feb 11, 2018

fchollet Feb 11, 2018

fchollet Feb 12, 2018

ozabluda Feb 12, 2018

fchollet left a comment

		return (None, self.num_capsule, self.dim_capsule)


		# some parameters



		# a squashing function. but it has litte difference from the Hinton's paper.
		# it seems that this form of squashing performs better.

		return scale * x


		# define our own softmax function instead of K.softmax



		# define our own softmax function instead of K.softmax
		# because K.softmax can not specify axis.

a capsule cnn on cifar-10 #9193

a capsule cnn on cifar-10 #9193

Conversation

bojone commented Jan 26, 2018

fchollet commented Jan 26, 2018

bojone commented Jan 27, 2018

fchollet commented Jan 27, 2018

bojone commented Jan 27, 2018

tboquet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tboquet commented Feb 9, 2018

bojone commented Feb 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bojone commented Feb 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fchollet left a comment

Choose a reason for hiding this comment