OpenLB
Open Library of Bioscience
Advanced Search
Psychometrika
06, 2024;89 (2): 658-686.

Learning Bayesian Networks: A Copula Approach for Mixed-Type Data.

Federico Castelletti
Author Information
  1. Federico Castelletti: Department of Statistical Sciences, Universitá Cattolica del Sacro Cuore, Milan, Italy. federico.castelletti@unicatt.it. ORCID
PMID: 38609693 DOI: 10.1007/s11336-024-09969-2

Abstract

Estimating dependence relationships between variables is a crucial issue in many applied domains and in particular psychology. When several variables are entertained, these can be organized into a network which encodes their set of conditional dependence relations. Typically however, the underlying network structure is completely unknown or can be partially drawn only; accordingly it should be learned from the available data, a process known as structure learning. In addition, data arising from social and psychological studies are often of different types, as they can include categorical, discrete and continuous measurements. In this paper, we develop a novel Bayesian methodology for structure learning of directed networks which applies to mixed data, i.e., possibly containing continuous, discrete, ordinal and binary variables simultaneously. Whenever available, our method can easily incorporate known dependence structures among variables represented by paths or edge directions that can be postulated in advance based on the specific problem under consideration. We evaluate the proposed method through extensive simulation studies, with appreciable performances in comparison with current state-of-the-art alternative methods. Finally, we apply our methodology to well-being data from a social survey promoted by the United Nations, and mental health data collected from a cohort of medical students. R code implementing the proposed methodology is available at https://github.com/FedeCastelletti/bayes_networks_mixed_data .

Keywords

Bayesian inference Markov chain Monte Carlo directed acyclic graph network psychometrics structural equation model

References

  1. Andersson, S. A., Madigan, D., & Perlman, M. D. (1997). On the Markov equivalence of chain graphs, undirected graphs, and acyclic digraphs. Scandinavian Journal of Statistics, 24, 81–102.
  2. Andrews, B., Ramsey, J., & Cooper, G. F. (2018). Scoring Bayesian networks of mixed variables. International Journal of Data Science and Analytics, 14, 2–18.
  3. Andrews, B., Ramsey, J., & Cooper, G. F. (2019). Learning high-dimensional directed acyclic graphs with mixed data-types. In Proceedings of machine learning research, vol. 104 of Proceedings of Machine Learning Research. PMLR, pp. 4–21.
  4. Barbieri, M. M., & Berger, J. O. (2004). Optimal predictive model selection. The Annals of Statistics, 32, 870–897.
  5. Bhadra, A., Rao, A., & Baladandayuthapani, V. (2018). Inferring network structure in non-normal and mixed discrete-continuous genomic data. Biometrics, 74, 185–195. [PMID: 28437848]
  6. Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64, 1089–1108. [PMID: 18683856]
  7. Borsboom, D., Deserno, M. K., Rhemtulla, M., Epskamp, S., Fried, E. I., McNally, R. J., Robinaugh, D. J., Perugini, M., Dalege, G., Costantini, Jonasand, Isvoranu, A.-M., Wysocki, A. C., van Borkulo, C. D., van Bork, R., & Waldorp, L. J. (2021). Network analysis of multivariate data in psychological science. Nature Reviews Methods Primers, 1, 58.
  8. Briganti, G., Scutari, M., & McNally, R. (2022). A tutorial on Bayesian networks for psychopathology researchers. Psychological Methods, Advance Online Publication NA–NA.
  9. Cao, X., Khare, K., & Ghosh, M. (2019). Posterior graph selection and estimation consistency for high-dimensional Bayesian DAG models. The Annals of Statistics, 47, 319–348.
  10. Carrard, V., Bourquin, C., Berney, S., Schlegel, K., Gaume, J., Bart, P.-A., Preisig, M., Mast, M. S., & Berney, A. (2022). The relationship between medical students’ empathy, mental health, and burnout: A cross-sectional study. Medical Teacher, 44, 1392–1399. [PMID: 35830537]
  11. Castelletti, F., & Consonni, G. (2021). Bayesian causal inference in probit graphical models. Bayesian Analysis, 16, 1113–1137.
  12. Castelletti, F., & Consonni, G. (2023). Bayesian graphical modeling for heterogeneous causal effects. Statistics in Medicine, 42, 15–32. [PMID: 36317356]
  13. Castelletti, F., Consonni, G., Della Vedova, M., & Peluso, S. (2018). Learning Markov equivalence classes of directed acyclic graphs: An objective Bayes approach. Bayesian Analysis, 13, 1231–1256.
  14. Castelletti, F., & Mascaro, A. (2022). BCDAG: An R package for Bayesian structure and causal learning of Gaussian DAGs. arXiv pre-print arXiv:2201.12003 .
  15. Castelletti, F., & Peluso, S. (2021). Equivalence class selection of categorical graphical models. Computational Statistics & Data Analysis, 164, 107304.
  16. Cheng, J., Li, T., Levina, E., & Zhu, J. (2017). High-dimensional mixed graphical models. Journal of Computational and Graphical Statistics, 26, 367–378.
  17. Chickering, D. M. (2002). Learning equivalence classes of Bayesian-network structures. Journal of Machine Learning Research, 2, 445–498.
  18. Cowell, R. G., Dawid, P. A., Lauritzen, S. L., & Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. Springer.
  19. Cramer, A. O. J., Waldorp, L. J., Van der Maas, H. L. J., & Borsboom, D. (2010). Comorbidity: A network perspective. Behavioral and Brain Sciences, 33, 137–150. [PMID: 20584369]
  20. Cui, R., Groot, P., & Heskes, T. (2018). Learning causal structure from mixed data with missing values using Gaussian copula models. Statistics and Computing, 29, 311–333.
  21. Cui, R., Groot, P., & Heskes, T. (2016). Copula PC algorithm for causal discovery from mixed data. In P. Frasconi, N. Landwehr, G. Manco, & J. Vreeken (Eds.), Machine learning and knowledge discovery in databases (pp. 377–392). Springer International Publishing.
  22. Dobra, A., & Lenkoski, A. (2011). Copula Gaussian graphical models and their application to modeling functional disability data. The Annals of Applied Statistics, 2A, 969–993.
  23. Edwards, D. (2000). Introduction to graphical modelling. Springer.
  24. Epskamp, S., Kruis, J., & Marsman, M. (2017). Estimating psychopathological networks: Be careful what you wish for. PLOS ONE, 12, 1–13.
  25. Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9, 432–441. [PMID: 18079126]
  26. Godsill, S. J. (2012). On the relationship between Markov chain monte Carlo methods for model uncertainty. Journal of Computational and Graphical Statistics, 10, 230–248.
  27. Hambleton, R. K., & Cook, L. L. (1977). Latent trait models and their use in the analysis of educational test data. Journal of Educational Measurement, 14, 75–96.
  28. Harris, N., & Drton, M. (2013). PC algorithm for nonparanormal graphical models. Journal of Machine Learning Research, 14, 3365–3383.
  29. Haslbeck, J. M. B., & Waldorp, L. J. (2018). How well do network models predict observations? On the importance of predictability in network models. Behavior Research Methods, 50, 853–861. [PMID: 28718088]
  30. He, Y., Zhang, X., Wang, P., & Zhang, L. (2017). High dimensional Gaussian copula graphical model with FDR control. Computational Statistics & Data Analysis, 113, 457–474.
  31. Heckerman, D., Geiger, D., & Chickering, D. M. (1995). Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20, 197–243.
  32. Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999). Bayesian model averaging: A tutorial. Statistical Science, 14, 382–401.
  33. Hoff, P. (2007). Extending the rank likelihood for semiparametric copula estimation. The Annals of Applied Statistics, 1, 265–283.
  34. Ickstadt, K., Bornkamp, B., Grzegorczyk, M., Wieczorek, J., Sheriff, M. R., Grecco, H. E., & Zamir, E. (2011). Nonparametric Bayesian networks. In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, & M. West (Eds.), Bayesian Statistics 9 (pp. 283–316). Oxford University Press.
  35. Isvoranu, A., Epskamp, S., Waldorp, L. J., & Borsboom, D. E. (2022). Network psychometrics with R: A guide for behavioral and social scientists. Routledge.
  36. Kalisch, M., & Bühlmann, P. (2007). Estimating high-dimensional directed acyclic graphs with the PC-algorithm. Journal of Machine Learning Research, 8, 613–636.
  37. Lauritzen, S. L. (1996). Graphical models. Oxford University Press.
  38. Lauritzen, S. L., & Wermuth, N. (1989). Graphical models for associations between variables, some of which are qualitative and some quantitative. The Annals of Statistics, 17, 31–57.
  39. Lee, J., & Hastie, T. (2013). Structure learning of mixed graphical models. In Carvalho, C. M. & Ravikumar, P. (eds.), Proceedings of the sixteenth international conference on artificial intelligence and statistics, vol. 31 of Proceedings of Machine Learning Research. Scottsdale, Arizona, USA: PMLR, pp. 388–396.
  40. Lee, K. H., Chen, Q., DeSarbo, W. S., & Xue, L. (2022). Estimating finite mixtures of ordinal graphical models. Psychometrika, 87, 83–106. [PMID: 34191228]
  41. Liu, H., Lafferty, J., & Wasserman, L. (2009). The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research, 10, 2295–2328.
  42. Maathuis, M., & Nandy, P. (2016). A review of some recent advances in causal inference. In P. Bühlmann, P. Drineas, M. Kane, & M. van der Laan (Eds.), Handbook of big data (pp. 387–408). Chapman and Hall/CRC.
  43. Marsman, M., Huth, K., Waldorp, L. J., & Ntzoufras, I. (2022). Objective Bayesian edge screening and structure selection for Ising networks. Psychometrika, 87, 47–82. [PMID: 35192102]
  44. Marsman, M., & Rhemtulla, M. (2022). Guest Editors’ introduction to the special issue “Network Psychometrics in action’’: methodological innovations inspired by empirical problems. Psychometrika, 87, 1–11. [PMID: 35397084]
  45. Meinshausen, N., & Bühlmann, P. (2006). High-dimensional graphs and variable selection with the Lasso. The Annals of Statistics, 34, 1436–1462.
  46. Mohammadi, A., Abegaz, F., van den Heuvel, E., & Wit, E. C. (2017). Bayesian modelling of Dupuytren disease by using Gaussian copula graphical models. Journal of the Royal Statistical Society: Series C (Applied Statistics), 66, 629–645.
  47. Moustaki, I., & Knott, M. (2000). Generalized latent trait models. Psychometrika, 65, 391–411.
  48. Müller, D., & Czado, C. (2019). Dependence modelling in ultra high dimensions with vine copulas and the graphical lasso. Computational Statistics & Data Analysis, 137, 211–232.
  49. Ni, Y., Baladandayuthapani, V., Vannucci, M., & Stingo, F. C. (2022). Bayesian graphical models for modern biological applications. Statistical Methods & Applications, 31, 197–225.
  50. Pearl, J. (2000). Causality: Models, reasoning, and inference. Cambridge University Press.
  51. Peluso, S., & Consonni, G. (2020). Compatible priors for model selection of high-dimensional Gaussian DAGs. Electronic Journal of Statistics, 14, 4110–4132.
  52. Peterson, C., Stingo, F. C., & Vannucci, M. (2015). Bayesian inference of multiple Gaussian graphical models. Journal of the American Statistical Association, 110, 159–174. [PMID: 26078481]
  53. Rodríguez, A., Lenkoski, A., & Dobra, A. (2011). Sparse covariance estimation in heterogeneous samples. Electronic Journal of Statistics, 5, 981–1014. [PMID: 26925189]
  54. Scott, J. G., & Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. The Annals of Statistics, 38, 2587–2619.
  55. Spearman, C. (1904). General intelligence, objectively determined and measured. The American Journal of Psychology, 15, 201–292.
  56. Spirtes, P., Glymour, C., & Scheines, R. (2000). Causation, prediction and search (2nd edition) (pp. 1–16). The MIT Press.
  57. Wirth, R., & Edwards, M. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79. [PMID: 17402812]

MeSH Term

Bayes Theorem
Humans
Psychometrics
Computer Simulation
Models, Statistical
Algorithms
Mental Health
Journal Article Research Support, Non-U.S. Gov't

Word Cloud

Created with Highcharts 10.0.0candatavariablesdependencenetworkstructureavailableBayesianmethodologyknownlearningsocialstudiesdiscretecontinuousdirectedmethodproposedEstimatingrelationshipscrucialissuemanyapplieddomainsparticularpsychologyseveralentertainedorganizedencodessetconditionalrelationsTypicallyhoweverunderlyingcompletelyunknownpartiallydrawnaccordinglylearnedprocessadditionarisingpsychologicaloftendifferenttypesincludecategoricalmeasurementspaperdevelopnovelnetworksappliesmixedie possiblycontainingordinalbinarysimultaneouslyWhenevereasilyincorporatestructuresamongrepresentedpathsedgedirectionspostulatedadvancebasedspecificproblemconsiderationevaluateextensivesimulationappreciableperformancescomparisoncurrentstate-of-the-artalternativemethodsFinallyapplywell-beingsurveypromotedUnitedNationsmentalhealthcollectedcohortmedicalstudentsRcodeimplementinghttps://githubcom/FedeCastelletti/bayes_networks_mixed_dataLearningNetworks:CopulaApproachMixed-TypeDatainferenceMarkovchainMonteCarloacyclicgraphpsychometricsstructuralequationmodel

Similar Articles

Cited By

No available data.