lava-Slayer SNN Sampling larger number of time bins.

[tangores]

In the case of employing NMNIST, I've noticed that the time_bins tend to be quite limited, which, of course, is contingent on the specific dataset needs. However, when dealing with a larger number of event files, such as one containing nearly 10-20K timebins distributed across 2-3 million events, I'm uncertain how Lava Slayer can handle this. I would greatly appreciate your guidance on this matter. Thank you. (Just to provide some context, I'm attempting to execute a custom dataset. Unfortunately, when I attempted to do so, it caused the Vs-code window to close abruptly.)

    def __getitem__(self, i):
        filename = self.samples[i]
        label = int(filename.split('/')[-2])
        event = slayer.io.read_2d_spikes(filename)
        if self.transform is not None:
            event = self.transform(event)
        spike = event.fill_tensor(
                torch.zeros(2, 34, 34, self.**num_time_bins**),
                sampling_time=self.sampling_time,
            )
        return spike.reshape(-1, self.**num_time_bins**), label

[bamsumit]

@tangores for your use case of 10-20K timesteps, it is incredibly memory inefficient to use a single tensor at once to represent the input spike. Do you need to process all of them at once? It is advisable to break it down into smaller chunks, say 1000 time bins at a time and process the data in a loop.

You can also take a look at persistent_state parameter in the neuron parameters. Enabling it will allow you to have continuity between subsequence calls in the loop.

[tangores]

@bamsumit Thank you for your prompt response. I'm currently working on a custom dataset, and I've encountered two significant challenges:

1. Spatial Resolution vs. Time Bins: One challenge I'm facing is related to the dataset's spatial resolution, which is quite large at 640x480. This high spatial resolution seems to limit the number of time bins (typically around 50-100) that the network can effectively process. However, this constraint also depends on the available memory resources. Could you provide any insights or strategies for optimizing network performance in scenarios with high spatial resolutions like this?
2. Microsecond Timestamps: The second issue I'm grappling with involves the custom dataset's microsecond-level timestamps. To work effectively with Spiking Neural Network (SNN) simulations, the network requires timestamps in milliseconds (integers). Real-time conversion of events from microseconds to milliseconds doesn't seem ideal (loss of fine-grained temporal resolution). Do you have any recommendations or best practices for efficiently handling this timestamp conversion while maintaining the integrity of the data?

I genuinely appreciate your expertise and any guidance you can offer on these challenges. Your insights will be invaluable in helping me make progress on this project. Thank you in advance for your assistance.

[bamsumit]

@tangores You can have more sophisticated binning method. For e.g. you are using 1 ms time bins and you have an event with time-stamp of 10.75 ms. Then you can add 0.25 to t=10ms bin and 0.75 to t=11ms bin. The amount you add depends on the closeness of the event with the time bin. We do not have such binning mechanism in lava-dl. So you will have to implement one.

For spatial resolution vs time bins, you will have to sacrifice in time bins if you want bigger spatial resolution. You can always process a smaller number of time bins at a time.