With supercomputers and large models, can any meteorological disaster be predicted?

The Elusive Butterfly

The world-wide method of weather forecasting is called numerical calculation. After collecting enough and rich enough meteorological data, they are substituted into complex equations, and the result is the so-called forecast. Repeating this process, the forecast is continuous.

Behind these equations are some solid theories, such as fluid motion equations, thermodynamic equations and different interfaces, air, water, land ice material energy exchange equations, etc. The physical and chemical laws they represent have been rigorously calculated and verified.

But the weather forecast is still not absolutely accurate. This kind of numerical prediction is to predict the future trend through the changes of meteorological elements that have occurred, and its accuracy is seriously affected by the accuracy of observation data.

What’s more, with the change of time and space scales, the complexity of the meteorological system will increase dramatically. On the one hand, when analyzing the causes of the weather, there will be a lot of missing things, and on the other hand, there will be more errors in the data, which in turn affects the prediction. the result of.

And when the scale is large enough, the weather will enter a chaotic state. This is not a metaphor. The meteorological system is a typical chaotic system. Chaos theory was first proposed by meteorologists in 1963.

A chaotic system means that very small perturbations to the system can have hugely different consequences—another name for it is the butterfly effect, where a butterfly flaps its wings in California and a storm kicks up in Texas and the Gulf of Mexico.

This requires that the initial calculation of the value must be very accurate, but we all know that absolute accuracy does not exist, not to mention that in the face of such complex meteorological movements, there are too many types and quantities of data to be collected.

Human prediction of the weather is like trying to learn the relationship between the wings of a butterfly and the trajectory of the eye of a storm. It is magnificent, sometimes hopeless, and even esoteric.

On the one hand, more and more complex function equations are constructed in terms of accuracy, and as much data as possible are collected at shorter intervals, on the other hand, an almost “intuitive” experience is relied on.

To this day, all weather forecast results are still determined by two parts-the result of numerical calculation, plus the judgment of the forecaster. Weather forecasting is still highly dependent on human experience.

To some extent, human beings are using their own feelings to fight against the chaos of nature.

supercomputer

Therefore, meteorological forecasting presents a weird characteristic—because as long as one relies on human experience and feeling, there will always be failures, especially in some extreme climates, because the data on extreme climates is too scarce, causing human There is no coherent experience of it.

However, as the accuracy of numerical prediction has improved over the past few decades, the current weather forecast can reach a fairly accurate level in many cases. The medium-term usability forecast is close to 10 days, and the short-term forecast—for example, when the cold air will arrive. How much the temperature will drop, humans have been able to judge very accurately. This is because although the overall global climate environment changes are chaotic, at many microscopic levels, such as the formation of air masses, changes in air pressure, and the trajectory of cloud movements, are being described vividly and accurately by more and more functions and formulas .

For example, the recent torrential rains in Beijing, Tianjin and Hebei.

From July 29th to August 1st, Typhoon Dusurui moved northward with abundant water vapor and was intercepted by high pressure in North China. Extreme rainstorms rarely seen in history occurred in the Beijing-Tianjin-Hebei region.

The extreme is reflected in several aspects, the first is the large quantity. The rainfall in Lincheng County, Xingtai, Hebei Province exceeds 1,000 millimeters, or 1 meter, which is equal to the total amount of precipitation in the original two years here. Secondly, it lasted for nearly four days starting from July 29, and it rained continuously for 83 hours even in Beijing. Hebei, Shaanxi, Henan and other places also experienced heavy rain for two consecutive days. Finally, it has a wide range of influence, and the entire North China is shrouded in rain.

Forecasters predicted this extreme rain with almost complete accuracy. Behind this is the great progress of weather forecasting over the past few decades-the numerical calculation has developed from a simple set of equations to one of the most complex algorithms in the world. After all, the more data and algorithms are used to capture unpredictable weather Well, and in order to deal with this massive amount of data and complex algorithms, supercomputers were introduced.

Many people may have no concept of the huge amount of weather data, but here is a figure: Every day, China’s weather data grows up to 40TB. my country has established a complete set of three-dimensional weather observation network, with more than 70,000 meteorological observation stations covering 99.6% of the country’s towns and villages. The data transmission time has been shortened from 1 hour in the past to 1 minute in recent years.

It is quite difficult to deal with such a huge amount of data itself, but it is almost impossible to combine them with numerical calculation equations—complex partial differential equations and floating-point calculations require a lot of calculations. Power, especially weather forecast, also requires extremely high timeliness. Under various constraints, only supercomputers can meet the demand.

Fortunately, China is at the forefront of the world in this regard.

Supercomputing has always been a business card of China. For example, China has 162 of the world’s fastest 500 supercomputers. Another example is that I have won the highest award for international supercomputing applications - the Gordon Bell Award three times. Two of them are The two award-winning contents “Atmospheric Dynamics Framework” and “Earthquake Simulation” are both related to meteorology.

The supercomputer currently used in China is called “Pai-Dawn”. Its peak computing speed reaches 8189.5 trillion times per second, and its storage capacity reaches 23088TB. The scale has jumped to the third place in the world in the field of meteorology.

This is not over. With the rapid growth of data volume and wear and tear of hardware, the service life of supercomputers is mostly 6 to 8 years. Therefore, after “Pai-Dawn”, new supercomputers are also being launched one after another. This year’s first A batch of new state-level high-performance computing (HPC) subsystem 1 has been installed and constructed. Its performance has been further improved compared with “Pai-Dawning”, reaching 13PFlops, and its storage capacity has also increased to 76PB.

More supercomputers are on the way.

The concept map of the domestic high-performance computer system “Pai-Shuguang”. The picture comes from the National Meteorological Information Center.

** “Stagnation” and the New Way**

In general, the supercomputing composed of hundreds or thousands of CPUs constitutes a solid foundation for weather forecasting, but there are still things that cannot be done or are difficult to do.

Such as typhoon. Our prediction of Typhoon Dusurui is far from the accuracy of the prediction of heavy rain. The mainstream typhoon prediction methods in the world all use dynamic models, which simulate atmospheric motion based on the laws of atmospheric physics, including fluid dynamics and thermodynamics, and then predict changes in typhoons.

There are many inaccuracies in the dynamical model, which made everyone predict Du Surui’s path in a variety of ways at the beginning. The European Medium-Range Weather Forecast Center, which is recognized as more authoritative and accurate in this field, once believed that Du Surui would go straight to the Pearl River Delta.

In fact, typhoon forecasting has always been a worldwide problem, because a typhoon can undergo drastic changes in intensity in a short period of time, and its path is unpredictable due to the influence of many factors on a very large time-space scale.

It is a typical chaotic system.

Therefore, the weather forecast based on numerical prediction has become a complex of contradictions, which makes it have a huge room for improvement, but it is very limited-the only way for people to approach accuracy is to accumulate experience Collecting data and constructing more and more complex functions, but its marginal benefits are becoming stronger and stronger. More data cannot greatly and comprehensively improve the accuracy of weather forecasts on the macro and micro scales, while general The cost of computing continues to grow.

At this point, the situation seems to have reached an impasse. Massive data, complex algorithms, and high computing power requirements make numerical calculations become a bottleneck to some extent, but do these three conditions look familiar?

Seems a bit like the three elements of a big model?

In fact, in the material of an internal seminar on meteorological technology seen by Pinwan, some experts directly declared that artificial intelligence and meteorological work are similar in methodology.

Since numerical calculation formulas can’t actually exhaust all the causes and effects in the entire chaotic system, why not use the neural network method to import massive data into it, and let the computer find and learn the laws by itself?

This seems to be an indescribable, yet extremely mysterious and reasonable explanation.

ENIAC, the first electronic computer of mankind, and EDSAC, the first computer built according to the von Neumann structure, were actually used to carry out scientific calculations related to meteorology. The development of meteorology has always been in line with the development of computer science. Progress is closely connected-high-performance computing has promoted the solution of complex problems in meteorological science, and after jumping out of the general computing thinking, neural networks and large models seem to be a very interesting and reasonable direction for meteorology and weather forecasting.

In fact, the Central Meteorological Observatory referred to the results of the artificial intelligence large-scale model for the prediction of the trajectory of Typhoon Dusurui.

Another way

Large meteorological models did not appear with the popularity of generative AI at the end of last year. There have been many large models that use powerful computing power, huge amounts of data, and various deep learning architectures to predict the weather. The more famous ones abroad are Nvidia FourCastNet, DeepMind, Google’s GraphCast, and Microsoft’s ClimaX, the latest domestic models are the Huawei Pangu model and the Shanghai Fengwu model, the latter from the Shanghai Artificial Intelligence Laboratory.

Both Pangu and Fengwu used the European Meteorological Center’s meteorological reanalysis data set named ERA5, which provides a total of 2000TB of various meteorological data covering 37 isobaric surfaces on the earth’s surface over the past 60 years.

These data provide a space for large models to play - instead of using precise and corresponding numerical calculations, AI can more flexibly mine various complex relationships between data.

Pangu built a 3D transformer structure to integrate the complex interactive physical processes of the ocean, atmosphere, and land. In the past, large climate models, such as Nvidia’s FourCastNet, used a 2D structure—each isobaric surface is a layer , obviously, there is no way for the 2D plane to better reflect the process of meteorological changes.

According to the materials seen by Pinwan, Huawei used absolute position encoding in this 3D structure, which significantly accelerated the convergence of the model. In addition, Huawei trained four basic models corresponding to 1 hour, 3 hours, 6 hours and 24 hours respectively. Hourly prediction, the time-domain fusion method significantly reduces the error caused by small step size and multiple iterations in the previous model.

For the prediction of the old and difficult typhoon, Pangu used the mean sea level pressure as the prediction benchmark, and the result was better than that of the European Meteorological Agency.

The idea of Fengwu large model is to use 6 independent encoders to encode and decode different meteorological parameters (humidity, wind, temperature, etc.), and use independent transformer networks to learn between different variables.

Different from Huawei’s method of time-domain fusion to reduce errors, Fengwu designed a cache space to store and replay the results of the training process, so that the neural network can adapt to its errors, thereby achieving better prediction results.

Although the results of these new meteorological large-scale model predictions are still preliminary, and the published results are better than traditional numerical prediction methods in some specific indicators, comprehensive testing and optimization are still needed, but there is no doubt that they represent a kind of A new paradigm radically different from traditional weather forecasting.

At present, Huawei’s Pangu large-scale model has achieved preliminary business operation in the European medium-range weather forecast center, while Fengwu claims to be the most accurate global high-resolution AI weather forecast model. 38.7 kilometers, better than the 54.11 kilometers of the European Center for Medium-Range Weather Forecasts.

Long Road Ahead

Change is coming. Big weather models have the potential to upend weather forecasting paradigms in many ways, such as speed. Fengwu uses a GPU to generate 37 levels of high-precision and high-resolution weather forecast results for all regions of the world in the next 14 days within 1 minute, while Pangu’s 7-day weather forecast takes only 9.8 seconds to generate.

The real-time weather data transmission system and general computing power infrastructure established for numerical calculations will therefore change. According to the insiders of the Meteorological Bureau learned by Pinwan, although it is only used as a forecast reference in the short term, in the medium and long term, the potential of large meteorological models is undoubtedly huge.

Now, there may be two main problems faced by these large meteorological models. The first is tuning. How to correctly label data such as extreme climate in the large model is a problem. The AI model has a strong fitting ability, so errors may occur in the long-term forecast. This may be called the “illusion” of the large meteorological model.

In addition, the current large-scale models do not have a deeper industry know how reserve, and the construction of the models is done by computer scientists rather than meteorological experts, which means that these vertical large-scale models still lack “human feedback” tuning.

Another problem is the commonplace computing power. Although my country ranks among the top in the world in terms of supercomputer development, the large model does not use the general computing power of traditional supercomputers. The operation of deep networks requires large-scale parallel computing. It is the high-performance GPU, and we are still stuck in this regard.

Fortunately, the cost of computing power directly involved in the weather large model is not high. According to the situation learned by Pinwan, dozens of Nvidia graphics cards (not even the most advanced models) can be completed in a few weeks, at most 2 months Model training. This has also become an advantage over traditional numerical prediction models.

With the blessing of large weather models, the phrase “unpredictable” may soon become a false proposition, and one day, perhaps we can really predict where the fluttering butterflies and violent hurricanes come from at the same time. Where are you going.