Problem running pynam model on SpiNNaker via HBP platform

[apdavison]

When running the "net16_vary_sigma" benchmark through the HBP Neuromorphic Platform, I get the following error:

Traceback (most recent call last):
  File "/localhome/nmpidemo/RemoteSpiNNaker/temp/job1179751006082691553.tmp/run.py", line 548, in <module>
    mode == "analyse-exec"):
  File "/localhome/nmpidemo/RemoteSpiNNaker/temp/job1179751006082691553.tmp/run.py", line 329, in execute_networks
    output = sim.run(input_network)
  File "/localhome/nmpidemo/RemoteSpiNNaker/temp/job1179751006082691553.tmp/lib/pynnless/pynnless/pynnless_isolated.py", line 129, in run
    raise Exception(exception)
Exception: Traceback (most recent call last):
  File "/localhome/nmpidemo/RemoteSpiNNaker/temp/job1179751006082691553.tmp/lib/pynnless/pynnless/pynnless_isolated.py", line 48, in _PyNNLessIsolatedMain
    res = inst.run(network, duration)
  File "/localhome/nmpidemo/RemoteSpiNNaker/temp/job1179751006082691553.tmp/lib/pynnless/pynnless/pynnless.py", line 1238, in run
    network["populations"][i], timestep)
  File "/localhome/nmpidemo/RemoteSpiNNaker/temp/job1179751006082691553.tmp/lib/pynnless/pynnless/pynnless.py", line 682, in _build_population
    res.tset(k, tvals[k])
  File "/localhome/nmpidemo/sPyNNaker/sPyNNaker/spynnaker/pyNN/models/pynn_population.py", line 746, in tset
    self.set(parametername, value_array)
  File "/localhome/nmpidemo/sPyNNaker/sPyNNaker/spynnaker/pyNN/models/pynn_population.py", line 671, in set
    self._parameters[parameter] = value
  File "/localhome/nmpidemo/sPyNNaker/sPyNNaker/spynnaker/pyNN/utilities/parameters_surrogate.py", line 41, in __setitem__
    self.vertex.n_atoms)
  File "/localhome/nmpidemo/sPyNNaker/sPyNNaker/spynnaker/pyNN/utilities/utility_calls.py", line 49, in convert_param_to_numpy
    return numpy.array(param, dtype=float)
ValueError: setting an array element with a sequence.

Any ideas, or should we ask @alan-stokes and @rowleya ?

[astoeckel]

This is a known problem with the current release version of sPyNNaker.

Let me quickly rephrase what the underlying issue is: originally my code generated a single Population object for each neuron and spike source in the network, as this was the most convenient way of setting individual neuron parameters. However, this is not well supported by SpiNNaker. Each processor can only simulate neurons from a single population, which severely limits the size of the network in the above use case.

I've since tried to split my network into two large population objects and set the parameters for each neuron in the population individually. However, this approach was long blocked by several bugs in sPyNNaker (see the epic discussion in https://groups.google.com/d/msg/spinnakerusers/lY13bfXQodQ/BadpFItHDwAJ ). However, those problems are fixed in the current development version of sPyNNaker and I've verified that my code runs with a local 4-chip SpiNNaker board at the Code Jam in Manchester. Correspondingly I've updated my code to use large populations. However, the neuromorphic compute platform still has the old sPyNNaker version installed. See: https://groups.google.com/d/msg/spinnakerusers/gMSPzKCsvlY/rkhMn3RQFQAJ

Probably you ( @apdavison ) could try to persuade @rowleya to finally update the software available on the neuromorphic compute platform.

[alan-stokes]

Just to clarify, the software version to which @astoeckel is referring to is the master branch on github, not the current release (Arbitrary) which the portal is using. There are plans to do a release before the end of SGA0, at which point the portal will be updated with that, which will include the fixes he is referring to (known as "Another Fine Product From The Nonsense Factory").

There is a discussion needed with @alan-stokes, @rowleya and @apdavison about supporting the different releases directly on the portal (Little-Rascal, Arbitrary, Just-Testing, etc and a development version which is the master branch on github). But as this has not been discussed nor implemented. It must be worth noting that we shouldnt update the portal with a alpha version of the software for all users for at least the sake of provenance until such a capability is available on the portal.

Alan

[apdavison]

It would be really helpful for the benchmarking activity in SGA0 if AFPFTNF were released by the end of February. Do you think this is possible/likely?

[rowleya]

We were hoping to already have released this, but other things are taking priority at the moment. I am fairly sure that this will be released by the end of February; we could update the portal version to the git version for the time being as well as the platform is only internally released. As Alan stated, we would like to allow users to select a version of the software to run things on, which would include the git version (latest unstable) if this was desired. We aren’t quite ready to do this immediately, but hopefully soon. I am guessing that this would be specified in the “Hardware Config” section.

I will have a look at updating the portal shortly.