Image: © Andrey VP/Stock.adobe.com
Huawei’s You Peng spoke to SiliconRepublic.com about the company’s new AI weather prediction model, Pangu-Weather.
As the dust settles on the summer, many discussions are being had regarding the eye-opening weather events that occurred, and for all the wrong reasons.
Numerous weather-related catastrophes dotted the summer, from destructive Canadian wildfires to record-breaking temperatures signalling the onset of serious global implications of the climate crisis. Extreme heatwaves and equally disastrous rainfall levels have led to growing concerns of climate experts, as the next five years are predicted to be the warmest on record.
As a result of these record-breaking worries, climate change discussions are increasingly changing from mitigation to adaption, as concerns about global preparedness begin to heat up.
One key area significant to adaption discussions is that of improved weather-forecasting systems, as the more notice we have of impending weather events, the better prepared we will be.
To improve forecasting abilities, some companies are experimenting with the ever-evolving technology of generative AI.
Huawei’s Pangu-Weather AI model recently became available on the European Centre for Medium-Range Weather Forecasts (ECMWF) website, where users can now view the model’s 10-day global weather forecasts.
To find out more about the Pangu-Weather system, we spoke to You Peng who is the director of Huawei’s cloud EI service product department.
Using the past to predict the future
Firstly, what makes the Pangu-Weather AI prediction model special?
According to Peng, Pangu-Weather demonstrates a 10,000-times faster prediction speed than traditional numerical weather forecast methods, and can accurately predict meteorological features such as humidity, wind speed, temperature and sea-level pressure “in seconds”.
To build the model, researchers at Huawei built a deep neural network based on about 40 years of global reanalysis data. Reanalysis data combines historical weather observations with modern weather forecasting models. According to the ECMWF, reanalysis data provides “the most complete picture currently possible” of past weather and climate.
“The methodology involves training deep neural networks to take reanalysis weather data at a given point in time as input, and then produce reanalysis weather data at a future point in time as output,” says Peng.
The model uses a 3D Earth-Specific Transformer (3DEST) architecture to process “complex non-uniform 3D meteorological data”.
“We integrated height information into a new dimension so that the input and output of our deep neural networks can be conceptualised in three dimensions,” says Peng, adding that they then used the 3DEST architecture to “inject Earth-specific priors” into the deep networks.
The team then applied a “hierarchical temporal aggregation algorithm that involves training a series of models with increasing forecast lead times”.
For a complex model like Pangu-Weather to work properly, Peng says it has to be trained on a “tremendous amount” of publicly-available data, which then requires massive amounts of computing power.
The impact of AI weather prediction
Peng believes that one of the ways Pangu-Weather will benefit society is by giving the general public first-hand access to AI weather prediction.
“The increasing frequency of extreme weather has piqued significant interest in the topic beyond meteorological circles,” he says. “More importantly, however, it serves to demystify how AI prediction works in weather for the general public, who generally isn’t exposed to this.
“As far as future societal benefits, from a scientific community perspective, it provides further indication that AI is a true contender when it comes to weather prediction.”
In a report published by MIT Technology Review, the ECMWF’s head of Earth system modelling, Peter Dueben, said that models like Pangu-Weather are making meteorologists “reconsider how we use machine learning and weather forecasts”.
Peng believes, however, that there is room for improvement for both AI-based prediction methods and traditional numerical weather prediction (NWP) methods.
“On the AI side, further gains can be found by incorporating more vertical levels and/or atmospheric variables, integrating the time dimension and training four-dimensional deep networks, using deeper and/or wider networks, or simply increasing the number of training epochs.”
As for NWP methods, Peng believes that post-processing methods can be developed to “alleviate the predictable biases of NWP models”.
“We expect that AI-based and NWP methods will be combined in the future to bring about even stronger performance.”
10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.
