Multi-threading and/or multi-processing approach

[sridharraman]

I need to run the PySD model on a lot of input files. Running it sequentially means it takes a lot of time: almost 30-40 seconds per file (i.e. model run). I would like to parallelise this execution. I have tried both multi-threading and multi-processing.

Multi-processing:
This works fine. The running time (per file) reduces to 12 seconds on average.

Multi-threading:
I get an error when I try to run multiple threads. The error is this:

KeyError: 'OUTPUTS'
ValueError: All arrays must be of the same length

When I looked online, it suggested that PySD might not be thread-safe.

Any idea why this is? Is anyone else facing similar problems?