Would a ML model for edge deployment and long-term predictions be fitting for OCF?

[RupeshMangalam21]

Hey OCF community! ☀️
I’m thinking of proposing a new model for solar forecasting—optimized for edge devices (like NVIDIA Jetson) and focused on 24–48h predictions. It’d use cross-modal attention to blend satellite/NWP/PV data and active learning for global scalability. Could this complement PVNet’s short-term forecasts? What do you all think?

[Sukh-P]

Hey @RupeshMangalam21, thanks for sharing this idea it is an interesting one, on cross-modal attention for the different data sources/modalities that is definitely something we are interested in experimenting more and would welcome any experimentation on that!

On models being deployed on the edge I could see this potentially having some use cases if it was part of a device which collected on the ground data that would be useful for PV prediction, e.g. a solar farm device which also has a sky camera attached so it takes photos of the sky and then passes this into the model at inference (we currently haven't trained a model with sky camera data), this is generally thought to improve forecasting skill in the short term horizon (< a couple of hours or even < a hour). Or perhaps if a device was connected to someones home PV inverter so it gets live PV readings and then uses that as an input to the model.

Given this I would say that it perhaps has potential but we aren't quite setup currently for use cases where it could help, but that is not to say it would not be interesting to try! I'll let others chime in too as they may have other thoughts

[RupeshMangalam21]

Hi, @Sukh-P Thanks for the thoughtful feedback! Regarding edge deployment, a lightweight model (<10 MB) deployable on microgrid controllers is what i am leaning towards, yes a sky camera setup could be a potential implementation along with this. I am actually prototyping a cross-modal attention model now and would love to test it on OCF’s datasets (e.g., satellite/NWP/PV) to benchmark against PVNet. A few quick questions:

1. Are there specific datasets or preprocessing steps you’d recommend for training/evaluation? It would be a huge help if you could guide me in building a data pipeline for the prototype/model.

2. Could we simulate edge deployment using existing PV inverter telemetry streams (like OCF’s PVlive)?

3. What metrics matter most for long-term forecasts in OCF’s context (e.g., MAE, CRPS)?

[Sukh-P]

Hi @RupeshMangalam21,

1. I would point you to this repo/project https://github.com/openclimatefix/open-data-pvnet, the tooling to train PVNet is open source, i.e the PVNet repo and the ocf-data-sampler repo can be used to train the model provided you have the data inputs available and they are in the correct formats (e.g. zarr files), however you will need to use open source data sets if you want to train PVNet, the ones we used are closed source/purchased from NWP/satellite imagery providers so we can't reshare them, the best open source ones I know of are e.g. on our HF we have the ICON datasets available, on an S3 bucket we have put some GFS NWP data, check out the repo linked for the exact source and there are other open data sources mentioned there
2. Yes I imagine you could do that
3. MAE matter most to us overall, for long term forecasts at the moment we care about the 36-48 hour forecast horizons the most