sT_pre,prev_sT_pre and sT_post, prev_sT_post in GeNN seem strange!

[gimunBae]

Hello!
I'm trying to understand your pavlovian conditioning code and I have a question in sT_pre,prev_sT_pre,sT_post,and prev_sT_post, the values used in the GeNN model.
According to my understanding, sT_pre and sT_post mean latest spiking time in presynaptic neuron and postsynaptic neuron respectively, and prev_sT_pre and prev_sT_post means previous spiking time.
But, when I tried to see the value of them using "std::cout<<$(sT_pre)<<std::endl;" in sim_code and learn_post_code in "create_custom_weight_update_class", I found the values seemed stranged.(I used pygenn)
To be specific, when I checked the value of "sT_pre" and "prev_sT_pre" in "learn_post_code", the two values were same. Also, when I checked the value of "sT_post" and "prev_sT_post" in "sim_code", the two values were same!(I mean sT_pre and prev_sT_pre had same value and similar for post)
I thought using std::cout can influence the value saved in sT_pre or other.
So I checked the values of them directly by using "pull_spike_times_from_device()" and "pull_prev_spike_times_from_device()".
And by seeing them, I could know error can happen when we called the values of sT_pre and prev_sT_pre in learn_post_code and vice versa(sT_post and prev_sT_post in sim_code).
Besides, another problem happened. When I gave nonzero axonal time delay in "add_synapse_population" and tried to check the value of sT_pre and prev_sT_pre in sim_code, where no error happened in no time delay situation, the two values were same!.
What is the problem and can't I use this code for time delay situation?
(At first, I thought my simulation setting can be the reason, but the same error happened when I did same for other neural network system)

[neworderofjamie]

I'm afraid they are indeed a bit strange. Key things are:

• If a neuron spikes, it immediately sets it's spike time meaning that if you read sT_pre in the sim_code or sT_post in the learn_post_code which is triggered by that spike you will find that sT_pre/sT_post are equal to t which isn't helpful when calculating the type of event-driven updates used in the pavlovian conditioning model.
• We added prev_sT_pre and prev_sT_post to solve this but the names are a little confusing. Essentially they are updated after the spike-triggered weight update model code is called so are essentially the spike times from the previous timestep.
• You might notice different bits of dopamine-modulated STDP weight update model code use the prev_sT_XXX values in the same context others use sT_XXX. This is also a bit ugly because it relies on knowing that presynaptic updates happen before postsynaptic updates and ensures that, if both updates happen in the same timestep, you won't get a double weight update.
• Delays further complicate things - the values stored in the variables you are pulling with pull_spike_times_from_device etc are spike times at the neuron so won't reflect delays. The delays are only added on at the point you reference them in the weight update model code strings.

Hope that makes sense - I'm afraid this bit of GeNN is complex and a little confusing!

[gimunBae]

Thank you for answering!
According to my understanding of your answer, prev_sT_pre and prev_sT_post are updated after sim_code and learn_post_code are called respectively.
But, the dopamine-modulated STDP weight update code which I used, needed sT_pre and prev_sT_pre in sim_code and prev_sT_post in learn_post_code.
And I also saw that value of prev_sT_pre in sim_code was a little bit strange.(it was same with sT_pre as I said initial question, and now I think this was because it was not updated yet).
Then, is it okay for me to judge that problem can happen when we used prev_sT_pre in sim_code and prev_sT_post in learn_post_code?? Thank you

[neworderofjamie]

According to my understanding of your answer, prev_sT_pre and prev_sT_post are updated after sim_code and learn_post_code are called respectively.

They are actually both updated after all synapse code has run but, yes, this is basically correct.

But, the dopamine-modulated STDP weight update code which I used, needed sT_pre and prev_sT_pre in sim_code and prev_sT_post in learn_post_code.

I'm not quite sure what you mean. The key thing to remember is that each of the code strings in the weight update model is intended to update the synapse variables from the time of the last event of any kind to the current time. Specifically (and assuming you're talking about the code in https://github.com/BrainsOnBoard/pygenn_paper ):

• In sim_code (https://github.com/BrainsOnBoard/pygenn_paper/blob/master/models/izhikevich_pavlovian/common.py#L63), $(prev_sT_pre) is used because, as a presynaptic spike is currently being processed, $(sT_pre) will equal $(t). $(prev_sT_post) and $(prev_seT_pre) are used because this update is called before event_code and learn_post_code so if a postsynaptic spike or spike-like event occurred in this timestep, it hasn't yet been processed.
• In event_code (https://github.com/BrainsOnBoard/pygenn_paper/blob/master/models/izhikevich_pavlovian/common.py#L80), $(prev_seT_pre) is used because, as a spike-like event is currently being processed, $(seT_pre) will equal $(t). $(sT_pre) is used because, if a presynaptic spike occurred in the same timestep, it will already have been processed in the sim_code so the synaptic state will have already been updated. $(prev_sT_post) is used because this update is called before learn_post_code so if a postsynaptic spike occurred in this timestep, it hasn't yet been processed.
• In learn_post_code (https://github.com/BrainsOnBoard/pygenn_paper/blob/master/models/izhikevich_pavlovian/common.py#L90), $(prev_sT_post) is used because, as a postsynaptic spike is currently being processed, $(sT_post) will equal $(t). $(sT_pre) and $(seT_pre) are used because, if a presynaptic spike or spike-like event occurred in the same timestep, it will already have been processed in the sim_code or event_code so the synaptic state will have already been updated.

And I also saw that value of prev_sT_pre in sim_code was a little bit strange.(it was same with sT_pre as I said initial question, and now I think this was because it was not updated yet).

This will be the case if a presynaptic spike hasn't occured in the current timestep

Then, is it okay for me to judge that problem can happen when we used prev_sT_pre in sim_code and prev_sT_post in learn_post_code?? Thank you

What exactly is "that problem"?

[gimunBae]

Sincerely thank you for your specific explanation!
"That problem" in last part means when I checked sT_pre and prev_sT_pre in sim_code with time delay, two values were same, which had not appeared in the zero delay situation. Based on your explanation, I guess it is because presynaptic spike hasn't occured in the current timestep.

But it is still strange that when I saw sT_pre and prev_sT_pre in sim_code or learn_post_code, two were not same but strange. (Actually, prev_sT_pre was bigger than sT_pre whenever I check it at sim_code or learn_post_code)

[neworderofjamie]

Hmm, I don't think prev_sT_pre should ever be bigger than sT_pre - can you confirm exactly what you're running (i.e. what code and what version of GeNN) so I can try and reproduce?

[gimunBae]

Sorry for confusing.
The error I told you, happened only when I took too long time delay so that the delay was bigger than inter_stimuli_interval. And now, I think that this can be problematic in the real system as well.
Just so you know, I send you the code I used. But now, I guess the error I said was not because of the code error, It is unrealistic situation. Thank you!
(This is situation for system I made with 3 presynaptic neurons and 2 post synaptic neurons with 50 time step. When I checked the value of prev_sT_pre and sT_pre using std::cout<<$(sT_pre)<<std::endl; and std::cout<<$(prev_sT_pre)<<std::endl; in sim_code, or learn_post_code, prev_sT_pre is bigger than sT_pre.)

import numpy as np
from copy import copy
from matplotlib import pyplot as plt
from pygenn import genn_model
from scipy.signal import find_peaks
I=10.0
delay=50
# Simulation time step
DT = 0.1
# Number of Kenyon Cells in model (defines memory capacity)
NUM_KC = 3
NUM_MBON = 2
# How long to present each image to model
PRESENT_TIME_MS = 20.0
# Standard LIF neurons parameters
LIF_PARAMS = {
    "C": 0.2,
    "TauM": 20.0,
    "Vrest": -60.0,
    "Vreset": -60.0,
    "Vthresh": -50.0,
    "Ioffset": 0.0,
    "TauRefrac": 2.0}
# We only want PNs to spike once
KC_PARAMS = copy(LIF_PARAMS)
KC_PARAMS["TauRefrac"] = 100.0
KC_MBON_TAU_SYN = 3.0
KC_MBON_PARAMS = {"tau": 15.0,
                  "rho": 0.01,
                  "eta": 0.00002,
                  "wMin": 0.0,
                  "wMax": 100.0}
# Current source model, allowing current to be injected into neuron from variable
cs_model = genn_model.create_custom_current_source_class(
    "cs_model",
    var_name_types=[("magnitude", "scalar")],
    injection_code="$(injectCurrent, $(magnitude));")
# Minimal integrate and fire neuron model
if_model = genn_model.create_custom_neuron_class(
    "IF",
    param_names=["Vthresh"],
    var_name_types=[("V", "scalar")],
    sim_code=
    """
    $(V)+= $(Isyn);
    """,
    threshold_condition_code=
    """
    $(V) >= $(Vthresh)
    """,
    reset_code=
    """
    $(V)= 0.0;
    """)
symmetric_stdp = genn_model.create_custom_weight_update_class(
    "symmetric_stdp",
    param_names=["tau", "rho", "eta", "wMin", "wMax"],
    var_name_types=[("g", "scalar")],
    sim_code=
    """    
    std::cout<<$(prev_sT_pre)<<std::endl;
    const scalar dt = $(t) - $(sT_post);
    const scalar timing = exp(-dt / $(tau)) - $(rho);
    const scalar newWeight = $(g) + 10;
    $(g) = fmin($(wMax), fmax($(wMin), newWeight));
    """,
    learn_post_code=
    """
    const scalar dt = $(t) ;
    const scalar timing = fmax(exp(-dt / $(tau)) - $(rho), -0.1*$(rho));
    const scalar newWeight = $(g) -10;
    $(g) = fmin($(wMax), fmax($(wMin), newWeight));
    """,
    is_pre_spike_time_required=True,
    is_post_spike_time_required=True,
    is_prev_pre_spike_time_required=True,
    is_prev_post_spike_time_required=True)
# Create model
model = genn_model.GeNNModel("float", "mnist_mb_third_layer")
model.dT = DT
# Create neuron populations
lif_init = {"V": LIF_PARAMS["Vreset"], "RefracTime": 0.0}
if_init = {"V": 0.0}
kc = model.add_neuron_population("kc", NUM_KC, "LIF", LIF_PARAMS, lif_init)
mbon = model.add_neuron_population("mbon", NUM_MBON, "LIF", LIF_PARAMS, lif_init)
model.add_current_source("CurrentSource", "DC", kc, {"amp": 10.0}, {});
kc.spike_recording_enabled = True
mbon.spike_recording_enabled = True
mbon_input = model.add_current_source("mbon_input", cs_model, mbon , {}, {"magnitude": 15.0})
# Create current sources to deliver input and supervision to network
kc_mbon = model.add_synapse_population("kc_mbon", "DENSE_INDIVIDUALG", delay,
                                       kc, mbon,
                                       symmetric_stdp, KC_MBON_PARAMS, {"g": 0.0}, {}, {},
                                       "ExpCurr", {"tau": KC_MBON_TAU_SYN}, {})
# Convert present time into timesteps
present_timesteps = int(round(PRESENT_TIME_MS / DT))
model.build()
model.load(num_recording_timesteps=present_timesteps)
voltage_kc=[]
voltage_mbon=[]
weights=[]
sT_pre=[]
prev_sT_pre=[]
model.t = 0.0
model.timestep = 0
voltage_view = kc.vars["V"].view
voltage_view2=mbon.vars["V"].view

while model.t < 20:
    model.step_time()    
    model.pull_recording_buffers_from_device()
    kc.pull_var_from_device("V")
    mbon.pull_var_from_device("V")
    voltage_kc.append(np.copy(voltage_view))
    voltage_mbon.append(np.copy(voltage_view2))
    kc_mbon.pull_var_from_device("g")
    ee = kc_mbon.get_var_values("g")
    weights.append(np.copy(ee))

    kc.pull_spike_times_from_device()
    kc.pull_prev_spike_times_from_device()
    sT_pre.append(np.copy(kc.spike_times))
    prev_sT_pre.append(np.copy(kc.prev_spike_times))
    
voltage_kc=np.vstack(voltage_kc)
voltage_mbon=np.vstack(voltage_mbon)
sT_pre=np.vstack(sT_pre)
prev_sT_pre=np.vstack(prev_sT_pre)
weights=np.vstack(weights)
sT_pre=np.vstack(sT_pre)
prev_sT_pre=np.vstack(prev_sT_pre)
fig, axes = plt.subplots(NUM_KC+NUM_MBON+(NUM_KC*NUM_MBON), sharex=True, figsize=(15, 8))

for j in range(NUM_KC):
    axes[j].set_ylabel("V")
    axes[j].plot(np.arange(0.0, 20.0, 0.1), voltage_kc[:,j])
for k in range(NUM_MBON):
    axes[NUM_KC+k].set_ylabel("V")
    axes[NUM_KC+k].plot(np.arange(0.0, 20.0, 0.1), voltage_mbon[:,k])
for i in range(NUM_KC*NUM_MBON):
    axes[NUM_KC+NUM_MBON+i].set_ylabel("G")
    axes[NUM_KC+NUM_MBON+i].plot(np.arange(0.0, 20.0, 0.1), weights[:,i])
axes[-1].set_xlabel("Time [ms]");