How to disable the spiking behaviour of LIF neurons associated with slayer.block.cuba.Conv and slayer.block.cuba.Dense so that the membrane voltage can be accumulated until reset is done without generating spikes(because if spike happen the voltage reset)

[gwgkarunasena57]

Dear Experts,

How can i disable the spiking behaviour of LIF neurons associated with slayer.block.cuba.Dense and slayer.block.cuba.Conv blocks?
Because i want to accumulate the membrane voltage of output layer of the neural network until the neural network is reset for net input sample. During this period, the neurons of the output layers should not generate any spikes because if a spike is generated the membrane voltage is reset which is not desired for my case.

Thank you.
Thanks and Rgds,
Udayanga

[rileywhite-noblis]

To do this set the threshold of the neurons to zero, and then set graded_spike to true. Then apply a sum to the output of your network over the time dimension. So each time step your output neurons will always fire, but you will sum up their output over each timestep after the model has finished running.

[gwgknudayanga]

Thank you for the reply. So, is it cumsum or sum over the time dimension ?
And is there any specific current_decay and voltage_decay values that need to be set for this case.
I feel like current_decay should be 1.0 since then there is not accumulation of the input current, isn't it?

[rileywhite-noblis]

Yes sum, idk why I wrote cumsum.

Current decay can be anything you want, it's just another hyperparamter and will behave in the sameway as all your other neurons. voltage_decay is what becomes irrelevant since your voltage is reset every timestep.

[rileywhite-noblis]

I will note that you will have some awkwardness with the gradient because it will be a * exp(-f(x)/beta) rather than just passed through. You could get around this by just using a PyTorch linear layer, and then write some custom saving/loading code to put this layer into a graded spike layer in Lava. You could also just run the last linear layer on the CPU instead of Loihi.

[gwgknudayanga]

Thank you. I thought like if i set the current_decay to 1.0 then the input current/spike of the current time step will be almost zero at the next time step even though the current decay is governed as exp(-f(x)/beta. If we set current_decay to 1.0, at the next time step if the current can be decayed from 1.0 to 0.01 at least we can set the threshold to 0.01 instead zero. Again we loose something which are less than 0.01.
Thanks for the suggesting the CPU thing also.

[gwgknudayanga]

Hi,

When i set the graded_spike to true, the program is hanging. Do you have any idea?
neuron_params_2 = {
'threshold' : 0.0,
'graded_spike' : True,
'current_decay' : 0.25,
'voltage_decay' : 0.03,
'tau_grad' : 0.03,
'scale_grad' : 3,
'requires_grad' : False,
}
self.head = torch.nn.ModuleList([
slayer.synapse.Conv(16,10,kernel_size=1, stride=1, padding=0,pre_hook_fx = quantizer),
slayer.neuron.cuba.Neuron(**neuron_params_2),
])

Thanks .