Low accuracy of LeNet

[llCurious]

Hey. I try to run the experiments for LeNet model. However i noticed that the convergence of LeNet is weird. The test accuracy of LeNet remains low (far from 98%). I only changed the localhost_config.json to run locally with the following configurations:

1. 3PC computation based on Falcon
2. network: files/models/lenet.json
3. batch size: 128
4. epoch: 5
5. lr: 2^-4
6. precision: 16

The output is as follows:

[llCurious]

Hey, @jlwatson. Sry to bother you. But do you have any ideas?

[jlwatson]

Hey! I can't immediately run some tests, but as a starting point, could you try running it with models/lenet-norelu.json?

That's the actual LeNet architecture we evaluated with, it just doesn't have the last relu layer, which doesn't interact well with our loss function.

[llCurious]

Thanks to you quick reply!!
I just run the model of lenet-norelu. The accuracy results seems to work fine. BTW, do you mean for other networks like secureml, minionn and so on, I should remove the last relu layer?

[jlwatson]

Hey just to follow up, you should use the versions without the last ReLU layer -- it plays nicer with our loss function. And LeNet is pretty easy to train, so I'm not very surprised to see high accuracy after only 5 rounds. Looks like things are working properly.

…

On Fri, Apr 7, 2023 at 2:36 AM llCurious ***@***.***> wrote: I just check the figures on the paper. The presented figure is only about 97% for training 10 epochs. The above figures that I obtain is for 5 epochs. Is there something wrong with this part? [image: image] <https://user-images.githubusercontent.com/21785695/230585340-9c2b1514-730d-4432-a4a6-ca75932d5a23.png> — Reply to this email directly, view it on GitHub <#11 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABMIC4622H26EBOFTXHCA7DW77N3TANCNFSM6AAAAAAWVKBAQY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[allenlin316]

I also tried the same training using local_running.sh with MNIST dataset with files/models/lenet-norelu.json model. But somehow I only got the result as below image.
I wonder what did I do wrong? Thanks in advance!

[allenlin316]

I also tried the same training using local_running.sh with MNIST dataset with files/models/lenet-norelu.json model. But somehow I only got the result as below image. I wonder what did I do wrong? Thanks in advance!

I just solved it by remaking the Makefile with 23 bits because of paper mentioned LeNet secure training of neural network
command remake is as: make -j8 PIRANHA_FLAGS="-DFLOAT_PRECISION=23 -DTWOPC"