SampleTest.NearestNeighbor is loading data incorrectly

[migueldeicaza]

I have isolated the code to do the operations and they look fine with small data sets, I suspect my data loader is wrong.

For reference, this is a port of this sample:

https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/2_BasicModels/nearest_neighbor.py

[cesarsouza]

Hi Miguel,

If I understood correctly, do you believe the problem is in the MNIST parser code at https://github.com/migueldeicaza/TensorFlowSharp/blob/master/Learn/Datasets/MNIST.cs ?

If this is the case, I have also implemented a MNIST parser that you can find it here: https://github.com/accord-net/framework/blob/development/Sources/Accord.IO/IdxReader.cs

I had originally written it under the LGPL but I can relicense it to TensorFlowSharp under MIT or any other license you might need.

I will try to use to load the data in the kNN sample application and check if it works better.

[JimSEOW]

@cesarsouza Welcome! I hope you see that we are gathering help from different communitiess to make .NET deep learning a common phenomenon.

[cesarsouza]

@JimSEOW Hey! Please count me in. Now that I finally got TensorFlowSharp working for me, I am in the process of writing a tool similar to Keras in C#. I hope it can be useful to finally leverage .NET/C# as a useful/serious language for deep learning.

[migueldeicaza]

@cesarsouza Thanks for reaching out - I do not recall the details, but I spend a couple of evenings comparing the different outputs (with small data sets) and could not found the culprit, everything looks ok. It might be the MNIST loader, would love if you could try your loader and see if that is the source of the issue, I suspected that.

[migueldeicaza]

@cesarsouza I shared with you my KerasSharp port, it is not complete by any means, but it shows the work in progress. One problem is that I had not quite grasped that Keras is not intended to be a framework, it optimizes for ease-of-use, and when I started my effort, I was going for "framework". So some code can be simplified

[mdabros]

@migueldeicaza First off, thanks for your big efforts in starting a .net wrapping of tensorflow.

I have been working on my own humble machine learning library, SharpLearning, for the last few years, trying to make machine learning more accessible to .net developers. This library also contains a project for neural nets and deep learning with a Keras-like interface.

Like many others, I also see a great opportunity in using TensorFlowSharp as a provider of deep learning and other machine learning algorithms for .net. So I would very much like to contribute in porting some of the higher level concepts from the Python parts of tensorflow. Especailly the optimizers from python/tf/train and the python/layers, so we can start creating some examples that shows how to train neural nets using TensorFlowSharp.

Is there any coordination on this part of the porting yet?

[JimSEOW]

@mdabros I already put SharpLearning on this discussion

[cesarsouza]

Hi @migueldeicaza,

(Well, first of all sorry for the long delay before I could give a proper reply to your post)

Thanks a bunch for sharing your KerasSharp port with me - I had taken a look on it a few days ago and I have to say, I completely understood what you meant when you said you were going for a "framework" type of project.

In this way, I have to say that the idea I had for a KerasSharp project was a bit different. My intention was to do an actual, complete, line-by-line port of the Keras library to C# - meaning achieving an almost line-by-line translation of the current Keras python code including the abstraction flayers or different tensor processing frameworks which for me would be one of the most critical feature to have so far.

I have at least three reasons for wanting a true, line-by-line port rather than a new similar project that happens to share a similar name, which I hope I will be able to explain below:

1. I understand that the Keras Python community is going to be much more active than any number of developers we might get the attention of in the first days of this project. Therefore, my main goal would be to make it easy to port any new fixes, functionality and features from mainstream Keras into KerasSharp from the very start - by the time we finish the first working version of our .NET library, I am sure there would have been hundreds of improvements and pull-requests to the original Keras project we would be interested in porting right after.

2. I have a strong suspicion (but feel free to disagree) that most developers looking for a KerasSharp project on NuGet would have been already familiar with the Keras, but are looking for something that could be usable from C# or .NET projects (Well, otherwise they could have looked for other libraries/frameworks directly instead of something called "Keras"). As such, by keeping the transition path as smooth as possible, we could increase the chances that more people can get interested in our projects and not get drifted away once they realize their existing Keras/Python knowledge would not apply.

3. The third reason for trying to follow the Keras design as close as possible would be to leave the possibility for, one day, implementing a CNTK backend in additional to TensorFlowSharp. You may surely know better since you are now in Microsoft, but CNTK is also becoming a very interesting alternative to TensorFlow and Theano since they started providing bindings for Python not so long ago, and plus since @fchollet started adding support for CNTK in the latest releases of Keras.

All this said, I want to disclose that I have also gone through most of Keras python codebase and have manually converted about 90% of the Python code into C# - although I recognize that there is still a long, long way until all this code could be actually usable. My initial plan was to be able to commit a compilable version of this initial repository to GitHub before the CVPR conference starts next week, but given my lack of free time, I might have to settle with an initial version filled with NotImplementedException placeholders.

PS: By the way, if any C#/.NET fans also attending CVPR in end of next week would like to discuss the current state of deep learning in C#/.NET, please let me know, as I am not sure we are many these days.

Regards,
Cesar

[mdabros]

@cesarsouza I apologize if I am intruding in your discussion, but a direct port of Keras to C#/.Net would be great, and I think you are spot on with your observations.

• Keras is optimized for ease of use, which is important, when trying to engage more .net developers not previously familiar with deep learning.
• A direct port would make it easy to learn from and translate the huge amount of Keras examples already available in python.
• One interface with support for different tensor processing frameworks, will make the transition between them trivial, and developers can pick the framework most suitable for their platform.

If you have already ported 90% of the Keras codebase, then please put it on github, and if you accept contributions, I will be happy to contribute.

I am not attending CVPR, but a discussion on the current state of deep learning in C#/.Net would have been great. I will be at NIPS this december, in case anybody is attending, and want to continue the discussion.

[cesarsouza]

Hi everyone,

Thanks for the all feedback. I just wanted to say that I have finally put my version of a "Keras-Sharp" project on GitHub. It is also not complete just like @migueldeicaza's version, but most of the classes related to the engine of Keras are already there (but not necessarily tested).

Most of my work was actually in trying to get the types right in all method calls, given that in Python things can change types at the blink of an eye. For example, here is the current status of the implementation for Sequential, Model, and Container classes which by far are one of the most complex in Keras:

• https://github.com/cesarsouza/keras-sharp/blob/master/Sources/Models/Sequential.cs
• https://github.com/cesarsouza/keras-sharp/blob/master/Sources/Engine/Training/Model.cs
• https://github.com/cesarsouza/keras-sharp/blob/master/Sources/Engine/Topology/Container.cs

Most of the activation functions, constraints, weight initializers, regularizers, losses have been ported (some still have NotImplementedException placeholders):

• https://github.com/cesarsouza/keras-sharp/tree/master/Sources/Activations
• https://github.com/cesarsouza/keras-sharp/tree/master/Sources/Constraints
• https://github.com/cesarsouza/keras-sharp/tree/master/Sources/Regularizers
• https://github.com/cesarsouza/keras-sharp/tree/master/Sources/Initializers
• https://github.com/cesarsouza/keras-sharp/tree/master/Sources/Losses

For now, the only optimizer with a partial implementation is SGD:

• https://github.com/cesarsouza/keras-sharp/blob/master/Sources/Optimizers/StochasticGradientDescent.cs

And besides the interface being already defined, most of the implementation of the TensorFlowBackend is still empty:

• https://github.com/cesarsouza/keras-sharp/blob/master/Sources/Backends/TensorFlowBackend.cs

Of course, contributions are more than welcome. My plan of action was to develop it in a structural risk minimization-style:

• Start with the simplest example in Keras so far currently available that we still don't support (i.e. first example of Dense layer),
• Write a unit test for it ,
• Make it pass (the aforementioned test still doesn't)

I've marked the Keras commit I used as a base when converting the code above in a README.txt in the project's source folder. In order to make a test pass, it should be possible to go back to that commit and locate the Python code that should had been implemented by the code sections being covered by the test and check where they don't match.

Additionally, another way to contribute would be to go gradually filling the TensorFlowBackend class with calls to TensorFlowSharp.

Hope this can be useful.

Regards,
Cesar

[cesarsouza]

Ops, actually I apologize to @migueldeicaza because we completely diverged from the original topic of this issue, which is about SampleTest.NearestNeighbor and had nothing to do with Keras Sharp. Let's please not diverge further and continue the discussion somewhere else, if desired.

[F0123X]

Any updates here?

If there is an issue with the MNIST loader, I haven't found it. Not sure if its anything, but I have noticed that nn_index has been converging to 1/4 of trainCount.

[migueldeicaza]

Andrew Violette emailed me a pointer to the fix!

It is embarrassing.