OpenLB
Open Library of Bioscience
Advanced Search
Scientific reports
Feb 09, 2024;14 (1): 3396.

Quantizing reconstruction losses for improving weather data synthesis.

Daniela Szwarcman, Jorge Guevara, Maysa M G Macedo, Bianca Zadrozny, Campbell Watson, Laura Rosa, Dario A B Oliveira
Author Information
  1. Daniela Szwarcman: IBM Research, Rio de Janeiro, Brazil.
  2. Jorge Guevara: IBM Research, São Paulo, Brazil.
  3. Maysa M G Macedo: IBM Research, São Paulo, Brazil.
  4. Bianca Zadrozny: IBM Research, Rio de Janeiro, Brazil.
  5. Campbell Watson: IBM Research, Yorktown Heights, NY, USA.
  6. Laura Rosa: Laboratory of Geo-information Science and Remote Sensing, Wageningen University and Research, Wageningen, The Netherlands.
  7. Dario A B Oliveira: IBM Research, Rio de Janeiro, Brazil. dario.oliveira@ibm.com.
PMID: 38336873 DOI: 10.1038/s41598-024-52773-2

Abstract

The stochastic synthesis of extreme, rare climate scenarios is vital for risk and resilience models aware of climate change, directly impacting society in different sectors. However, creating high-quality variations of under-represented samples remains a challenge for several generative models. This paper investigates quantizing reconstruction losses for helping variational autoencoders (VAE) better synthesize extreme weather fields from conventional historical training sets. Building on the classical VAE formulation using reconstruction and latent space regularization losses, we propose various histogram-based penalties to the reconstruction loss that explicitly reinforces the model to synthesize under-represented values better. We evaluate our work using precipitation weather fields, where models usually strive to synthesize well extreme precipitation samples. We demonstrate that bringing histogram awareness to the reconstruction loss improves standard VAE performance substantially, especially for extreme weather events.

References

  1. National Academies of Sciences, Engineering and Medicine. Attribution of Extreme Weather Events in the Context of Climate Change (The National Academies Press, 2016).
  2. Seneviratne, S. et al. Changes in Climate Extremes and their Impacts on the Natural Physical Environment 109–230 (Cambridge University Press, 2012).
  3. Verdin, A., Rajagopalan, B., Kleiber, W., Podestá, G. & Bert, F. A conditional stochastic weather generator for seasonal to multi-decadal simulations. J. Hydrol. 556, 835–846 (2018). [DOI: 10.1016/j.jhydrol.2015.12.036]
  4. Allard, D., Ailliot, P., Monbet, V. & Naveau, P. Stochastic weather generators: An overview of weather type models. J. Soc. Fr. Stat. 156, 101–113 (2015).
  5. Apipattanavis, S., Podestá, G., Rajagopalan, B. & Katz, R. W. A semiparametric multivariate and multisite weather generator. Water Resour. Res. https://doi.org/10.1029/2006WR005714 (2007). [DOI: 10.1029/2006WR005714]
  6. Peleg, N., Fatichi, S., Paschalis, A., Molnar, P. & Burlando, P. An advanced stochastic weather generator for simulating 2-d high-resolution climate variables. J. Adv. Model. Earth Syst. 9, 1595–1627. https://doi.org/10.1002/2016MS000854 (2017). [DOI: 10.1002/2016MS000854]
  7. van den Oord, A., Vinyals, O. & Kavukcuoglu, K. Neural discrete representation learning. In Advances in Neural Information Processing Systems Vol. 30 (eds Guyon, I. et al.) (Curran Associates, Inc., 2017).
  8. Karras, T., Aila, T., Laine, S. & Lehtinen, J. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations (2018).
  9. Brock, A., Donahue, J. & Simonyan, K. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations (2019).
  10. Bhatia, S., Jain, A. & Hooi, B. Exgan: Adversarial generation of extreme samples. Preprint at http://arxiv.org/abs/2009.08454 (2020).
  11. Wang, C., Tang, G. & Gentine, P. Precipgan: Merging microwave and infrared data for satellite precipitation estimation using generative adversarial network. Geophys. Res. Lett. https://doi.org/10.1029/2020GL092032 (2021). [DOI: 10.1029/2020GL092032]
  12. Oliveira, D. A. B., Guevara, J., Zadrozny, B. & Watson, C. D. Controlling weather field synthesis using variational autoencoders. Preprint at http://arxiv.org/abs/2108.00048 (2021).
  13. Klemmer, K., Saha, S., Kahl, M., Xu, T. & Zhu, X. X. Generative modeling of spatio-temporal weather patterns with extreme event conditioning. CoRR. http://arxiv.org/abs/2104.12469 (2021).
  14. Goodfellow, I. et al. Generative adversarial nets. In Advances in Neural Information Processing Systems Vol. 27 (eds Ghahramani, Z. et al.) (Curran Associates, Inc., 2014).
  15. Bau, D. et al. Seeing what a gan cannot generate. Proc. IEEE/CVF Int. Conf. Comput. Vis. https://doi.org/10.1109/ICCV.2019.00460 (2019). [DOI: 10.1109/ICCV.2019.00460]
  16. Liu, K., Tang, W., Zhou, F. & Qiu, G. Spectral regularization for combating mode collapse in gans. In Proc. of the IEEE/CVF International Conference on Computer Vision, 6382–6390 (2019).
  17. Kingma, D. P. & Welling, M. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings (2014). http://arxiv.org/1312.6114v10 .
  18. Zhao, S., Song, J. & Ermon, S. Infovae: Information maximizing variational autoencoders. CoRR (2017). http://arxiv.org/abs/1706.02262 .
  19. Tolstikhin, I., Bousquet, O., Gelly, S. & Schoelkopf, B. Wasserstein auto-encoders. In International Conference on Learning Representations (2018).
  20. Arbel, M., Korba, A., Salim, A. & Gretton, A. Maximum mean discrepancy gradient flow. In Advances in Neural Information Processing Systems Vol. 32 (eds Wallach, H. et al.) (Curran Associates, Inc., 2019).
  21. Yang, Y., Zha, K., Chen, Y.-C., Wang, H. & Katabi, D. Delving into deep imbalanced regression. Preprint at http://arxiv.org/abs/2102.09554 (2021).
  22. Rezende, D. J., Mohamed, S. & Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. In Xing, E. P. & Jebara, T. (eds) Proc. of the 31st International Conference on Machine Learning, no. 2. In Proc. of Machine Learning Research, 1278–1286 (PMLR, 2014).
  23. Ghosh, P., Sajjadi, M. S. M., Vergari, A., Black, M. & Scholkopf, B. From variational to deterministic autoencoders. In International Conference on Learning Representations (2020).
  24. Razavi, A., van den Oord, A. & Vinyals, O. Generating diverse high-fidelity images with vq-vae-2. In Advances in Neural Information Processing Systems Vol. 32 (eds Wallach, H. et al.) (Curran Associates, Inc., 2019).
  25. Torgo, L. & Ribeiro, R. Utility-based regression. In European Conference on Principles of Data Mining and Knowledge Discovery (eds Torgo, L. & Ribeiro, R.) 597–604 (Springer, 2007).
  26. Ribeiro, R. P. & Moniz, N. Imbalanced regression and extreme value prediction. Mach. Learn. 109, 1803–1835 (2020). [DOI: 10.1007/s10994-020-05900-9]
  27. Torgo, L., Ribeiro, R. P., Pfahringer, B. & Branco, P. Smote for regression. In Portuguese Conference on Artificial Intelligence (eds Torgo, L. et al.) 378–389 (Springer, 2013).
  28. Branco, P., Torgo, L. & Ribeiro, R. P. Smogn: A pre-processing approach for imbalanced regression. In First International Workshop on Learning with Imbalanced Domains: Theory and Applications, 36–50 (PMLR, 2017).
  29. Imani, E. & White, M. Improving regression performance with distributional losses. In International Conference on Machine Learning (eds Imani, E. & White, M.) 2157–2166 (PMLR, 2018).
  30. Lin, T., Goyal, P., Girshick, R., He, K. & Dollár, P. Focal loss for dense object detection. In 2017 IEEE International Conference on Computer Vision (ICCV), 2999–3007, https://doi.org/10.1109/ICCV.2017.324 (2017).
  31. Oksuz, K., Cam, B. C., Kalkan, S. & Akbas, E. Imbalance problems in object detection: A review. IEEE Trans. Pattern Anal. Mach. Intell. https://doi.org/10.1109/TPAMI.2020.2981890 (2020). [DOI: 10.1109/TPAMI.2020.2981890]
  32. Lu, X. et al. Deep regression tracking with shrinkage loss. In Proc. of the European Conference on Computer Vision (ECCV) (2018).
  33. Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B. & Smola, A. A kernel two-sample test. J. Mach. Learn. Res. 13, 723–773 (2012).
  34. Funk, C. et al. The climate hazards infrared precipitation with stations-a new environmental record for monitoring extremes. Sci. Data 2, 1–21 (2015). [DOI: 10.1038/sdata.2015.66]
  35. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016).
Journal Article

Word Cloud

Similar Articles

Cited By