Add external data functions

[JamesPHoughton]

Read from an external source, commands that call excel:
GET DIRECT DATA(...) etc.

[JamesPHoughton]

Also an issue when you are interpolating from a timeseries:
variable:INTERPOLATE::=...

[JamesPHoughton]

The data loading is actually a really interesting question, and one I haven’t given much thought to. The use case I have addressed is when you want to load the dataseries separately, and then replace an existing (constant) value with that data. In those cases, a new function is created (see here ) which is then substituted for the original.

I suppose GET XLS DATA should probably use pandas to read from excel when the model is loaded. (After translation, when the translated model file is imported into the .components namespace, here ). There would probably want to be a line in the translated model file that behaves similar to the way stocks are instantiated at load time, as similar to here, something like:
my_dataset = functions.get_xls_data(‘filepath/filename.xls’, sheet=1)
or whatever arguments translate between vensim’s syntax and pandas’.

You can see that the stock instantiation line is created in the builder.add_stock function here.

My guess is that we want to add to the builder module a new function (maybe add_data_loader or something) which creates this (above) line of code to add to the translated model file. This line would read the data into an object, in the same way that the stock is instantiated as an object within the .components namespace. The add_data_loader builder function would also return code to create a function that would be called every timestep, and would access the data object in the correct way. This would be analogous to how these lines of code provide functional access to the stock.

We then need to add hooks to the vensim2py file that called the builder function you’ve just created at the appropriate time, similar to the way we deal with stocks and other stateful model components here

Before doing any of that, we'll probably want to add a test file to the test suite that uses the functions we’re hoping to add, and make sure it is called in the tests/integration_test_pysd module.

[JamesPHoughton]

Having had another 24 hrs to think about this, I think it would make sense to create a class for doing data loading and interpolation. It would probably extend the Stateful class, in the same way that the Integ class or the Delay class does. Then it could overload the initialize method, to read the data from the excel file not at load time, but when the model is initialized. The initialization routines are called after load, but can also be manually called by the user, giving it more flexibility.

The __call__ method would then want to be overloaded to return the correct value given the current time.

This would probably fit better within the existing paradigm that any model element which is not purely an equation is instantiated as a Stateful element.