Does SingleEnvMultiProcess support simulate function?

[hlhsu]

Is your feature request related to a problem? Please describe.

I am trying to run the code under multi-process and I glad to know that we can handle it via SingleEnvMultiProcess. However, if I would like to call obs.simulate(action) like what we do in one process, it shows raise NoForecastAvailable("No forecasts are available for this instance of BaseObservation (no action_space "
grid2op.Exceptions.ObservationExceptions.NoForecastAvailable: Grid2OpException NoForecastAvailable "No forecasts are available for this instance of BaseObservation (no action_space and no simulated environment are set).

Describe the solution you'd like

Could we also have the same implementation like multi_obs[process_id].simulate[action[process_id]?

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context about the feature request here.

[BDonnot]

Hello,

Unfortunately no it's not possible at the moment as stated in the documentation (admitedly a bit hidden):
https://grid2op.readthedocs.io/en/latest/environment.html?highlight=BaseMultiProcessEnvironment#grid2op.Environment.BaseMultiProcessEnvironment

A broader support of regular grid2op environment capabilities as well as support for grid2op.Observation.BaseObservation.simulate() call might be added in the future.

It's not done yet because it's unclear how it would work.
Using multiEnv you receive a list of observations, what would it mean to do a "simulate" on this list ?
Do that mean you simulate the effect of the same action on all the observations of the list ? Or that you pass a list of actions ?

It's not very clear but if you have a use case that you can illustrate with a few lines of code, we will study how we could implement it :-)

[hlhsu]

Thank you for your response. For example, we have 8 cores in a single machine. Then we can use SingleEnvMultiProcess to run 8
different chronics at the same time (different observations). Then I expect that we can use the simulate function to simulate the effect of 8 actions corresponding to these 8 observations. Therefore, it just one action for one observation, and then we can train it with 8 observations at the same time. It's just for saving time.

[bwitherspoon]

I would like to explore this enhancement also. I can currently work around it with an Actor/Learner like framework designed for multiple machines where I create N processes with N actors each with its own single process environment so I can simulate. This is a bit overkill for a single machine though because each actor is perfectly capable if generating batches of actions (and batches of actions it wants to simulate). I need to be able to generate as much experience as possible in parallel with the ability to simulate with forecasts.

I would expect to be able to pass a list of actions for all (or maybe just some) of the previously received observations.

So usage like:

obs_n, reward_n, done_n, info_n = multi_env.step(action_n)
...
sim_action_n = [env.action_space() for _ in range(nb_env)]
sim_obs_n, sim_reward_n, sim_done_n, sim_info_n = multi_env.simulate(sim_action_n)

To help describe, I went ahead and implemented something like this at bwitherspoon/Grid2Op@81ba03c33907829ae158c0bedf03683f2ee317b9.

It appears to work with my simple if __name__ == "__main__": test, but if we can agree on how it should work I would be willing to implement and submit a pull request.

[BDonnot]

Hello 😊

If that is the best usage then using "simulate" just like you did is the way to go indeed.

The observations themselves leave on the main process, so it's not really possible to do "obs.simulate" (it's possible but it would not be "multi process").
So I guess the solution you propose is pretty elegant and it should work. Maybe it would be worth it to implement a "env.simulate" for regular too.

I'm currently on holiday right now.
For sure if there is an implementation for such feature, I'll review it and I'm not against integrating it, quite the opposite actually 😊