Getting Started with AI Weather Forecasting: Predict Next Week's Weather with Hugging Face

For decades, weather forecasting has been the domain of supercomputers and physics equations. In recent years, neural network-based AI weather models are transforming the field — they run faster, consume far less energy, and can even run on a regular laptop. In a public livestream hosted by the Hugging Face team, researcher Emma walked through her blog post Making AI Weather Forecasting Models Easy to Run, giving a systematic introduction to the principles, toolchain, and practical methods behind AI weather forecasting. This article summarizes the key takeaways.

What Is Weather Forecasting, and How Is AI Changing the Game?

Weather forecasting itself is nothing new. Emma points out that humans have been doing it for at least 50 years, with the core goal being to predict the state of the atmosphere. The discussion focuses on "medium-range forecasting" — predicting atmospheric conditions up to two weeks out. Whether to bring an umbrella or what to wear in summer, these are the kinds of questions this category of models helps answer.

The real turning point came around 2018. Before then, all forecasting models were physics-based: take a "snapshot" of the atmospheric state at a given moment, feed it into a massive physical model full of equations, and simulate forward. This approach works well, but it's costly — requiring enormous amounts of computing power, energy, and time. Every small experiment or new idea meant a heavy engineering effort.

Emma emphasizes that AI models haven't replaced physics-based models — they serve as a complement. Their biggest advantage is efficiency: typically 10x faster and 1,000x more energy-efficient than physics-based models. That means researchers can run their models more frequently, iterate quickly, and explore new directions. That, in Emma's view, is the most important shift.

The core method in traditional weather forecasting is known as Numerical Weather Prediction (NWP). The basic idea is to discretize the atmosphere into a 3D grid and solve the Navier-Stokes equations for fluid motion, along with thermodynamic and humidity equations, at each grid point — stepping forward in time to simulate atmospheric evolution. The most authoritative physics-based systems today include ECMWF's IFS model and NOAA's GFS model. These systems typically require parallel computation across thousands of CPU cores running for hours to complete a single global forecast, at significant electricity cost. AI models, by contrast, learn the rules of atmospheric evolution from massive historical weather datasets, compressing the simulation process into neural network weights. Inference requires only a fraction of the compute — essentially trading upfront training cost for real-time efficiency.

How AI Weather Models Resemble Large Language Models

Interestingly, AI weather models and large language models (LLMs) emerged around the same time and share quite a few underlying ideas. Emma explains that while the specific architectures vary — some use Transformers, others use graph methods or diffusion models — the core logic is strikingly similar.

You can think of the initial atmospheric state as a token or word, pass it into the model as input, and the model predicts the "next state" — say, the atmosphere 6 hours later. To forecast further ahead, you feed the prediction back into the model and continue stepping forward. This is essentially the same autoregressive token-by-token generation process as an LLM.

There are key differences at the input/output level, though. A weather model's input is multiple variables at every point on a global grid: temperature, wind, precipitation, snow cover, and more. Emma uses a diagram from a Microsoft paper to illustrate the pipeline — the state is encoded into a representation, the model reasons over that representation, and then decodes it back into the next Earth state. She also acknowledges that processing weather data is itself a fascinating challenge that sets this field apart from LLMs.

The structural challenges of weather model input differ significantly from text tasks. The global atmosphere is typically divided into a lat-lon grid (e.g., 0.25° resolution yields ~1.4 million grid points), with multiple pressure levels in the vertical dimension and dozens of physical variables per level — a single timestep's input tensor can reach several gigabytes. How to efficiently encode spherical geometry while preserving spatial structure, and how to handle grid distortion near the poles, is where architectural choices diverge most sharply. For example, Pangu-Weather uses a 3D Transformer with Earth-specific positional encoding; GraphCast converts the Earth's grid into a graph structure to model long-range atmospheric teleconnections; and diffusion-based models (like GenCast) naturally support ensemble forecasting through probabilistic generation. These design choices directly affect a model's ability to capture extreme events and the trade-off with computational efficiency.

Where Does the Data Come From: EarthMover Marketplace

The first step in running a weather model is loading initial conditions. Emma uses the EarthMover Marketplace — a platform that adds a management layer on top of data storage specifically designed to make working with scientific data easier. For anyone unfamiliar with climate data, it significantly reduces the overhead of setup and configuration.

EarthMover is a data marketplace — you can register for a free account and access the same datasets as everyone else. It lists a wide variety of datasets, some of them free. One major advantage is on-demand data access: you don't need to download all variables at once. You only fetch a specific variable (like temperature or wind) at a specific moment when you need it. As a result, even on a personal laptop doing observation and visualization, you only need a few GB of memory — completely feasible. Data is in Zarr format, well-suited for scientific data, and you can log in and stream data with just three lines of Python.

Running a Model in Practice: Aurora as a Case Study

The tutorial demonstrates Aurora — a model released by Microsoft a few years ago and one of the earliest AI weather models to show strong performance. Microsoft has since released more modern, specialized versions (for forecasting air pollution, cyclones, etc.), but Aurora as a foundation model is surprisingly approachable.

The workflow is fairly standard: fetch the required variables from EarthMover (such as 2-meter temperature, two wind components, etc. — specifics are listed in the model documentation and Hugging Face model card), download the model weights with a few lines of Hub library code (takes a few minutes on first run, then cached), organize the data into the batch format the model expects, and then specify the number of steps to get predictions.

On compute, Emma gives practical numbers: on a GPU, predicting 6 hours ahead (one step) takes under 1 second — very fast. On a CPU, it's about 100 seconds (roughly 1.5 minutes) per step — slower, but feasible. Forecasting a week out means dozens of steps, so CPU inference can be time-consuming. Note that Aurora currently does not support Apple silicon.

Ensemble Forecasting and Evaluation: Still Open Questions

A single deterministic model produces just one forecast, but the atmosphere is full of "butterfly effects" — tiny factors can lead to vastly different outcomes. This is why researchers use ensemble models: run the same scenario multiple times with small perturbations (say, 10 runs), then average the results to capture the range of possibilities. Emma uses her own city of Zurich as an example, showing how 10 individual forecast curves each vary, but their average closely matches the actual observed values.

Evaluation is particularly tricky in climate science. Emma candidly notes that what we call "ground truth" is not entirely true — we can't directly measure temperature or precipitation in the middle of the ocean or in uninhabited regions. The field constructs approximate ground truth through "analysis" or "reanalysis" — using physics or AI models to extrapolate from sparse observation points to a complete global grid. Common references include ERA5 and IFS.

Evaluation metrics have their limitations too. Metrics like RMSE and mean absolute error work reasonably well for smooth variables like temperature, but poorly for highly skewed distributions like precipitation — they tend to smooth out the extremes. The result: models score well on these metrics yet perform noticeably worse when it comes to predicting rare events like heat waves, cyclones, or extreme precipitation. How to better evaluate these models remains an open research question.

ERA5 is ECMWF's global atmospheric reanalysis dataset, covering 1940 to the present at roughly 31 km spatial resolution, with hundreds of variables including temperature, wind speed, and humidity. "Reanalysis" means that unevenly distributed historical observations (weather stations, radiosondes, satellites, etc.) are assimilated into a physical model to reconstruct past atmospheric states on a uniform global grid. ERA5 is widely used both as training data and as an evaluation benchmark for AI weather models — but since it is itself a model-derived product rather than direct observation, this is exactly why "ground truth is not entirely true," as noted in the article. Using ERA5 simultaneously as a training target and evaluation reference introduces a risk of circular reasoning, and building more independent evaluation frameworks is an important open research direction in the field.

The Tool Ecosystem and Hugging Face's Direction

Emma sees the biggest pain point in today's AI weather models as the lack of standardized tooling. These high-performance models have only existed for four or five years, and each one has its own configuration, dependencies, and initialization data requirements — setting up the environment for one model might take a month, only to start over for the next. What makes this worse is that climate researchers, while expert in physics, are not necessarily AI specialists, meaning these models remain largely siloed within the AI teams that built them.

On Hugging Face's end, the team is pushing on two fronts. On the infrastructure side, they're providing compute (HF Jobs), storage (buckets compatible with EarthMover's Zarr format), and other capabilities so users can run forecasts on personal laptops, clusters, or in the cloud. On the ecosystem side, Emma has outlined three directions: standardizing inference workflows; building a benchmark that lets the community add custom evaluation metrics (such as for heat waves or extreme precipitation); and making model fine-tuning straightforward.

One notable development comes from her colleague Kashif, who is integrating Google's Weather Next 2 (Weather Next 3 was announced during the livestream) into the Transformers library. This means the familiar fine-tuning, inference, and tooling workflows that LLM practitioners know could soon apply to weather models as well — dramatically simplifying testing and evaluation.

Final Thoughts

What made this discussion compelling is how it broke down a once highly specialized, high-barrier field into steps that any developer can understand and experiment with. Emma closed with an invitation to readers: share feedback on the blog and project — whether questions or collaboration proposals — to help push this young field forward. For anyone curious about getting into AI weather forecasting, now might just be the right time to start.