OpenLB
Open Library of Bioscience
Advanced Search
Nature
Jul, 2023;619 (7970): 533-538.

Accurate medium-range global weather forecasting with 3D neural networks.

Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, Qi Tian
Author Information
  1. Kaifeng Bi: Huawei Cloud, Shenzhen, China. ORCID
  2. Lingxi Xie: Huawei Cloud, Shenzhen, China. ORCID
  3. Hengheng Zhang: Huawei Cloud, Shenzhen, China. ORCID
  4. Xin Chen: Huawei Cloud, Shenzhen, China. ORCID
  5. Xiaotao Gu: Huawei Cloud, Shenzhen, China. ORCID
  6. Qi Tian: Huawei Cloud, Shenzhen, China. tian.qi1@huawei.com. ORCID
PMID: 37407823 DOI: 10.1038/s41586-023-06185-3

Abstract

Weather forecasting is important for science and society. At present, the most accurate forecast system is the numerical weather prediction (NWP) method, which represents atmospheric states as discretized grids and numerically solves partial differential equations that describe the transition between those states. However, this procedure is computationally expensive. Recently, artificial-intelligence-based methods have shown potential in accelerating weather forecasting by orders of magnitude, but the forecast accuracy is still significantly lower than that of NWP methods. Here we introduce an artificial-intelligence-based method for accurate, medium-range global weather forecasting. We show that three-dimensional deep networks equipped with Earth-specific priors are effective at dealing with complex patterns in weather data, and that a hierarchical temporal aggregation strategy reduces accumulation errors in medium-range forecasting. Trained on 39 years of global data, our program, Pangu-Weather, obtains stronger deterministic forecast results on reanalysis data in all tested variables when compared with the world's best NWP system, the operational integrated forecasting system of the European Centre for Medium-Range Weather Forecasts (ECMWF). Our method also works well with extreme weather forecasts and ensemble forecasts. When initialized with reanalysis data, the accuracy of tracking tropical cyclones is also higher than that of ECMWF-HRES.

References

  1. Bauer, P., Thorpe, A. & Brunet, G. The quiet revolution of numerical weather prediction. Nature 525, 47–55 (2015). [DOI: 10.1038/nature14956]
  2. Pathak, J. et al. FourCastNet: a global data-driven high-resolution weather model using adaptive Fourier neural operators. Preprint at https://arxiv.org/abs/2202.11214 (2022).
  3. Bougeault, P. et al. The THORPEX interactive grand global ensemble. Bull. Am. Meteorol. Soc. 91, 1059–1072 (2010). [DOI: 10.1175/2010BAMS2853.1]
  4. Skamarock, W. C. et al. A Description of the Advanced Research WRF Version 2 (National Center For Atmospheric Research Mesoscale and Microscale Meteorology Division, 2005).
  5. Molteni, F., Buizza, R., Palmer, T. N. & Petroliagis, T. The ECMWF ensemble prediction system: methodology and validation. Q. J. R. Meteorol. Soc. 122, 73–119 (1996). [DOI: 10.1002/qj.49712252905]
  6. Ritchie, H. et al. Implementation of the semi-Lagrangian method in a high-resolution version of the ECMWF forecast model. Mon. Weather Rev. 123, 489–514 (1995). [DOI: 10.1175/1520-0493(1995)123<0489]
  7. Bauer, P. et al. The ECMWF Scalability Programme: Progress and Plans (European Centre for Medium Range Weather Forecasts, 2020).
  8. Allen, M. R., Kettleborough, J. A. & Stainforth, D. A. Model error in weather and climate forecasting. In ECMWF Predictability of Weather and Climate Seminar 279–304 (European Centre for Medium Range Weather Forecasts, 2022); http://www.ecmwf.int/publications/library/do/references/list/209 .
  9. Palmer, T. N. et al. Representing model uncertainty in weather and climate prediction. Annu. Rev. Earth Planet. Sci. 33, 163–193 (2005). [DOI: 10.1146/annurev.earth.33.092203.122552]
  10. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015). [DOI: 10.1038/nature14539]
  11. Weyn, J. A., Durran, D. R. & Caruana, R. Can machines learn to predict weather? Using deep learning to predict gridded 500‐hPa geopotential height from historical weather data. J. Adv. Model. Earth Syst. 11, 2680–2693 (2019). [DOI: 10.1029/2019MS001705]
  12. Scher, S. & Messori, G. Weather and climate forecasting with neural networks: using general circulation models (GCMs) with different complexity as a study ground. Geosci. Model Dev. 12, 2797–2809 (2019). [DOI: 10.5194/gmd-12-2797-2019]
  13. Rasp, S. et al. WeatherBench: a benchmark data set for data‐driven weather forecasting. J. Adv. Model. Earth Syst. 12, e2020MS002203 (2020). [DOI: 10.1029/2020MS002203]
  14. Weyn, J. A., Durran, D. R., Caruana, R. & Cresswell‐Clay, N. Sub‐seasonal forecasting with a large ensemble of deep‐learning weather prediction models. J. Adv. Model. Earth Syst. 13, e2021MS002502 (2021). [DOI: 10.1029/2021MS002502]
  15. Keisler, R. Forecasting global weather with graph neural networks. Preprint at https://arxiv.org/abs/2202.07575 (2022).
  16. Hu, Y., Chen, L., Wang, Z. & Li, H. SwinVRNN: a data-driven ensemble forecasting model via learned distribution perturbation. J. Adv. Model. Earth Syst. 15, e2022MS003211(2023).
  17. Schultz, M. G. et al. Can deep learning beat numerical weather prediction? Phil. Trans. R. Soc. A 379, 20200097 (2021). [DOI: 10.1098/rsta.2020.0097]
  18. Hersbach, H. et al. The ERA5 global reanalysis. Q. J. R. Meteorol. Soc. 146, 1999–2049 (2020). [DOI: 10.1002/qj.3803]
  19. Liu, Z. et al. Swin transformer: hierarchical vision transformer using shifted windows. In Proc. International Conference on Computer Vision 10012–10022 (IEEE, 2021).
  20. Dosovitskiy, A. et al. An image is worth 16x16 words: transformers for image recognition at scale. Preprint at https://arxiv.org/abs/2010.11929 (2020).
  21. Betts, A. K., Chan, D. Z. & Desjardins, R. L. Near-surface biases in ERA5 over the Canadian Prairies. Front. Environ. Sci. 7, 129 (2019). [DOI: 10.3389/fenvs.2019.00129]
  22. Jiang, Q. et al. Evaluation of the ERA5 reanalysis precipitation dataset over Chinese mainland. J. Hydrol. 595, 125660 (2021). [DOI: 10.1016/j.jhydrol.2020.125660]
  23. Magnusson, L. et al. Tropical Cyclone Activities at ECMWF (European Centre for Medium Range Weather Forecasts, 2021).
  24. Knapp, K. R., Kruk, M. C., Levinson, D. H., Diamond, H. J. & Neumann, C. J. The international best track archive for climate stewardship (IBTrACS) unifying tropical cyclone data. Bull. Am. Meteorol. Soc. 91, 363–376 (2010). [DOI: 10.1175/2009BAMS2755.1]
  25. Knapp, K. R., Diamond, H. J., Kossin, J. P., Kruk, M. C. & Schreck, C. J. International Best Track Archive for Climate Stewardship (IBTrACS) Project, Version 4 (NOAA National Centers for Environmental Information, 2018).
  26. He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 9729–9738 (IEEE, 2020).
  27. Bao, H., Dong, L. & Wei, F. Beit: BERT pre-training of image transformers. Preprint at https://arxiv.org/abs/2106.08254 (2021).
  28. Devlin, J., Chang, M. W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. Conference North American Chapter of the Association of Computational Linguistics Vol. 1, 4171–4186 (NAACL, 2019).
  29. Brown, T. et al. Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020).
  30. Radford, A. et al. Learning transferable visual models from natural language supervision. In Proc. International Conference on Machine Learning 8748–8763 (PMLR, 2021).
  31. Chasteen, M. B. & Koch, S. E. Multiscale aspects of the 26–27 April 2011 tornado outbreak. Part I: outbreak chronology and environmental evolution. Mon. Weather Rev. 150, 309–335 (2022). [DOI: 10.1175/MWR-D-21-0013.1]
  32. Chasteen, M. B. & Koch, S. E. Multiscale aspects of the 26–27 April 2011 tornado outbreak. Part II: environmental modifications and upscale feedbacks arising from latent processes. Mon. Weather Rev. 150, 337–368 (2022). [DOI: 10.1175/MWR-D-21-0014.1]
  33. Choy, C., Gwak, J. Y. & Savarese, S. 4D spatio-temporal convnets: Minkowski convolutional neural networks. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 3075–3084 (IEEE, 2019).
  34. Zhang S., Guo, S., Huang, W., Scott M. R. & Wang, L. V4D: 4D convolutional neural networks for video-level representation learning. Preprint at https://arxiv.org/abs/2002.07442 (2020).
  35. Garg, S., Rasp, S. & Thuerey, N. WeatherBench probability: a benchmark dataset for probabilistic medium-range weather forecasting along with deep learning baseline models. Preprint at https://arxiv.org/abs/2205.00865 (2022).
  36. Zoph, B., Vasudevan, V., Shlens, J. & Le, Q. V. Learning transferable architectures for scalable image recognition. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 8697–8710 (IEEE, 2017).
  37. Fildier, B., Collins, W. D. & Muller, C. Distortions of the rain distribution with warming, with and without self‐aggregation. J. Adv. Model. Earth Syst. 13, e2020MS002256 (2021). [DOI: 10.1029/2020MS002256]
  38. White, P. Newsletter No. 102—Winter 2004/05 (European Centre for Medium Range Weather Forecasts, 2005); https://www.ecmwf.int/node/14623 (2005).
  39. Kalnay, E. Atmospheric Modeling, Data Assimilation and Predictability (Cambridge Univ. Press, 2003).
  40. Lynch, P. The origins of computer weather prediction and climate modeling. J. Comput. Phys. 227, 3431–3444 (2009). [DOI: 10.1016/j.jcp.2007.02.034]
  41. Stensrud, D. J. Parameterization Schemes: Keys to Understanding Numerical Weather Prediction Models (Cambridge Univ. Press 2009).
  42. Bauer, P. et al. The ECMWF Scalability Programme: Progress and Plans (European Centre for Medium Range Weather Forecasts, 2020).
  43. Nakaegawa, T. High-performance computing in meteorology under a context of an era of graphical processing units. Computers 11, 114 (2022). [DOI: 10.3390/computers11070114]
  44. Shi, X. et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. Adv. Neural. Inf. Process. Syst. 28, 802–810 (2015).
  45. Shi, X. et al. Deep learning for precipitation nowcasting: a benchmark and a new model. Adv. Neural. Inf. Process. Syst. 30, 5617–5627 (2017).
  46. Agrawal, S. et al. Machine learning for precipitation nowcasting from radar images. Preprint at https://arxiv.org/abs/1912.12132 (2019).
  47. Ravuri, S. et al. Skilful precipitation nowcasting using deep generative models of radar. Nature 597, 672–677 (2021). [DOI: 10.1038/s41586-021-03854-z]
  48. Lebedev, V. et al. Precipitation nowcasting with satellite imagery. In Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2680–2688 (ACM, 2019).
  49. Sønderby, C. K. et al. Metnet: a neural weather model for precipitation forecasting. Preprint at https://arxiv.org/abs/2003.12140 (2020).
Journal Article
Article has an altmetric score of 1735

Word Cloud

Created with Highcharts 10.0.0forecastingweatherdataforecastsystemNWPmethodmedium-rangeglobalWeatheraccuratestatesartificial-intelligence-basedmethodsaccuracynetworksreanalysisalsoforecastsimportantsciencesocietypresentnumericalpredictionrepresentsatmosphericdiscretizedgridsnumericallysolvespartialdifferentialequationsdescribetransitionHoweverprocedurecomputationallyexpensiveRecentlyshownpotentialacceleratingordersmagnitudestillsignificantlylowerintroduceshowthree-dimensionaldeepequippedEarth-specificpriorseffectivedealingcomplexpatternshierarchicaltemporalaggregationstrategyreducesaccumulationerrorsTrained39 yearsprogramPangu-Weatherobtainsstrongerdeterministicresultstestedvariablescomparedworld'sbestoperationalintegratedEuropeanCentreMedium-RangeForecastsECMWFworkswellextremeensembleinitializedtrackingtropicalcycloneshigherECMWF-HRESAccurate3Dneural

Similar Articles

Cited By