GraphCast: Learning skillful medium-range global weather forecasting

Everything in AI seems to be focused on LLMs lately. But in my opinion, powerful problem-specific models like this are even more interesting. The SOTA AI models used in weather forecasting are already outperforming the classic NWP models while being orders of magnitude more efficient (inference). Most are based on multi scale (hierarchical) Graph Neural Networks, an architecture which is not often talked about. The original Graphcast paper is worth a read if you think this is interesting: https://arxiv.org/abs/2212.12794

I used GAN to mean graph attention network in my comment, which is how the GraphCast paper defines transformers. https://arxiv.org/pdf/2212.12794 I was looking at this part in particular: > And while Transformers [48] can also compute arbitrarily long-range computations, they do not scale well with very large inputs (e.g., the 1 million-plus grid points in GraphCast’s global inputs) because of the quadratic memory complexity induced by computing all-to-all interactions. Contemporary extensions of Transformers often sparsify possible interactions to reduce the complexity, which in effect makes them analogous to GNNs (e.g., graph attention networks [49]). Which kind of makes a soup of the whole thing and suggests that LLMs/Graph Attention Networks are "extensions to transformers" and not exactly transformers themselves.

> a weather continuation model could take in all the available weather data up to the present moment and provide a continuation of that And it turns out to work really well: https://arxiv.org/abs/2212.12794 predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute