WeatherMesh

[jacobbieker]

Arxiv/Blog/Paper Link

https://windbornesystems.com/blog/how-we-built-our-record-breaking-ai-model-weathermesh

Detailed Description

Unfortunately very annoyingly quite limited details on the model, much less than any other weather forecasting model so far that I've seen, but supposedly good results. As there is no implementation to test, not entirely sure if it does do as well as it says, but some of the changes seem like they should work.

Primarily new things that they mention:

1. Do autoregressive steps inside the weather latent space, so the processor step is repeated multiple times to get longer forecasts.
2. SWIN Transformer for the model
3. Input adapters (U-Net) to go from HRES, GFS, etc. to ERA5-type data, to match the training data being ERA5. Allows for multiple compound ensembles of input analysis -> forecast.
4. Trained on RTX 4090 cluster of 33 of them. Supposedly some special enhancement on vision transformers which let them scale much larger without using too much VRAM.

Context

Overall, could be some interesting ideas for ML weather models. Follows the encode-process-decode setup from the original graph weather paper, and a lot of more recent ones. Wish they released a paper or more details, quite disappointing on that front.

[morestart]

Hello, in my experiments, I have found that SwinTransformer exhibits significant non-smoothness after multiple autoregressive iterations. Are there any methods to alleviate or solve this issue?

[jacobbieker]

Hi, I am not sure, I haven't tried the SwinTransformer for these, and we haven't reimplemented the WeatherMesh model yet either. Good to know about its potential downsides though!

[morestart]

I have found that using other initial fields in models using Swintransformer can result in more severe non smoothness issues