Divisive normalization is an efficient code for multivariate Pareto-distributed environments.

Stefan F Bucher, Adam M Brandenburger
Author Information
  1. Stefan F Bucher: Department of Economics, New York University, New York, NY 10012. ORCID
  2. Adam M Brandenburger: Stern School of Business, New York University, New York, NY 10012. ORCID

Abstract

Divisive normalization is a canonical computation in the brain, observed across neural systems, that is often considered to be an implementation of the efficient coding principle. We provide a theoretical result that makes the conditions under which divisive normalization is an efficient code analytically precise: We show that, in a low-noise regime, encoding an -dimensional stimulus via divisive normalization is efficient if and only if its prevalence in the environment is described by a multivariate Pareto distribution. We generalize this multivariate analog of histogram equalization to allow for arbitrary metabolic costs of the representation, and show how different assumptions on costs are associated with different shapes of the distributions that divisive normalization efficiently encodes. Our result suggests that divisive normalization may have evolved to efficiently represent stimuli with Pareto distributions. We demonstrate that this efficiently encoded distribution is consistent with stylized features of naturalistic stimulus distributions such as their characteristic conditional variance dependence, and we provide empirical evidence suggesting that it may capture the statistics of filter responses to naturalistic images. Our theoretical finding also yields empirically testable predictions across sensory domains on how the divisive normalization parameters should be tuned to features of the input distribution.

Keywords

References

  1. Nat Neurosci. 1999 Aug;2(8):740-5 [PMID: 10412064]
  2. Psychol Rev. 1954 May;61(3):183-93 [PMID: 13167245]
  3. Ann N Y Acad Sci. 2012 Mar;1251:13-32 [PMID: 22694213]
  4. Nat Commun. 2019 Dec 11;10(1):5660 [PMID: 31827078]
  5. Nature. 1996 Jun 13;381(6583):560-1 [PMID: 8637587]
  6. Nat Neurosci. 2001 Aug;4(8):819-25 [PMID: 11477428]
  7. J Neurosci. 2011 Oct 26;31(43):15310-9 [PMID: 22031877]
  8. Nat Neurosci. 2016 Apr;19(4):568-70 [PMID: 26878671]
  9. Neuron. 2009 Jan 29;61(2):168-85 [PMID: 19186161]
  10. Annu Rev Psychol. 2008;59:167-92 [PMID: 17705683]
  11. Neuron. 2017 Jul 19;95(2):399-411.e8 [PMID: 28728025]
  12. Perception. 2000;29(9):1041-55 [PMID: 11144818]
  13. Proc Natl Acad Sci U S A. 2010 Jul 6;107(27):12329-34 [PMID: 20616090]
  14. Nat Neurosci. 2021 Jul;24(7):998-1009 [PMID: 34017131]
  15. Proc Natl Acad Sci U S A. 2018 Jan 2;115(1):186-191 [PMID: 29259111]
  16. Proc Biol Sci. 1997 Dec 22;264(1389):1775-83 [PMID: 9447735]
  17. Nat Commun. 2020 May 19;11(1):2513 [PMID: 32427825]
  18. Neural Comput. 2016 Feb;28(2):305-26 [PMID: 26654209]
  19. Nat Neurosci. 2019 Jan;22(1):134-142 [PMID: 30559477]
  20. J Neurosci. 2011 Jul 20;31(29):10627-39 [PMID: 21775606]
  21. J Neurophysiol. 2017 Dec 1;118(6):3051-3091 [PMID: 28835531]
  22. IEEE Trans Image Process. 2008 Oct;17(10):1795-805 [PMID: 18784028]
  23. Neuron. 2009 Dec 24;64(6):931-42 [PMID: 20064398]
  24. Neural Comput. 1995 Nov;7(6):1129-59 [PMID: 7584893]
  25. IEEE Trans Image Process. 1999;8(12):1688-701 [PMID: 18267447]
  26. Nat Hum Behav. 2020 Nov;4(11):1118-1120 [PMID: 32929203]
  27. Z Naturforsch C Biosci. 1981 Sep-Oct;36(9-10):910-2 [PMID: 7303823]
  28. Cell Rep. 2013 Sep 12;4(5):1010-21 [PMID: 23994479]
  29. Neural Comput. 2013 Nov;25(11):2809-14 [PMID: 23895047]
  30. Neural Comput. 2010 Dec;22(12):3179-206 [PMID: 20858127]
  31. Proc Natl Acad Sci U S A. 2022 Oct 4;119(40):e2120581119 [PMID: 36161961]
  32. PLoS Comput Biol. 2021 Jun 7;17(6):e1009028 [PMID: 34097695]
  33. Network. 2003 Aug;14(3):579-93 [PMID: 12938772]
  34. Nature. 2003 May 15;423(6937):267-70 [PMID: 12748636]
  35. J Econ Behav Organ. 2019 Aug;164:148-165 [PMID: 32076358]
  36. Proc Natl Acad Sci U S A. 2013 Apr 9;110(15):6139-44 [PMID: 23530203]
  37. Curr Opin Neurobiol. 2003 Apr;13(2):144-9 [PMID: 12744966]
  38. Annu Rev Neurosci. 2001;24:1193-216 [PMID: 11520932]
  39. Network. 2001 Aug;12(3):241-53 [PMID: 11563528]
  40. Nat Hum Behav. 2020 Jun;4(6):634-645 [PMID: 32015490]
  41. Curr Biol. 2003 Mar 18;13(6):493-7 [PMID: 12646132]
  42. J Neurosci. 2015 Nov 04;35(44):14829-41 [PMID: 26538653]
  43. Nat Commun. 2018 Jan 11;9(1):162 [PMID: 29323110]
  44. Nat Hum Behav. 2020 Nov;4(11):1121-1123 [PMID: 32929204]
  45. Vision Res. 2010 Oct 28;50(22):2223-32 [PMID: 20034510]
  46. Vis Neurosci. 1997 Mar-Apr;14(2):293-309 [PMID: 9147482]
  47. Proc Natl Acad Sci U S A. 2021 Nov 16;118(46): [PMID: 34772812]
  48. J Neurosci. 1999 Jun 15;19(12):5074-84 [PMID: 10366640]
  49. Psychol Rev. 2018 Nov;125(6):985-1001 [PMID: 30431303]
  50. PLoS Comput Biol. 2012;8(3):e1002405 [PMID: 22396635]
  51. Nat Neurosci. 2015 Oct;18(10):1509-17 [PMID: 26343249]
  52. Elife. 2019 Apr 25;8: [PMID: 31021319]
  53. Vision Res. 1992 Aug;32(8):1409-10 [PMID: 1455713]
  54. Nat Rev Neurosci. 2011 Nov 23;13(1):51-62 [PMID: 22108672]
  55. Nat Commun. 2019 Apr 5;10(1):1583 [PMID: 30952855]
  56. Proc Natl Acad Sci U S A. 2021 Nov 23;118(47): [PMID: 34789580]
  57. Vision Res. 1998 Mar;38(5):743-61 [PMID: 9604103]
  58. J Neurosci. 2005 Sep 7;25(36):8150-64 [PMID: 16148223]
  59. Sci Rep. 2019 May 6;9(1):6940 [PMID: 31061521]
  60. Neural Comput. 2011 Nov;23(11):2942-73 [PMID: 21851283]
  61. J Neurosci. 2019 Sep 11;39(37):7344-7356 [PMID: 31387914]
  62. Elife. 2020 Jul 06;9: [PMID: 32628109]
  63. Proc Biol Sci. 1998 Mar 7;265(1394):359-66 [PMID: 9523437]
  64. Neuron. 2010 Apr 29;66(2):287-99 [PMID: 20435004]
  65. Nat Rev Neurosci. 2014 Apr;15(4):264-78 [PMID: 24569488]
  66. Neural Comput. 2009 Jun;21(6):1485-519 [PMID: 19191599]
  67. Network. 2003 Nov;14(4):733-45 [PMID: 14653500]
  68. Vis Neurosci. 1992 Aug;9(2):181-97 [PMID: 1504027]
  69. Neural Comput. 1996 Apr 1;8(3):531-43 [PMID: 8868566]
  70. PLoS One. 2011;6(5):e19779 [PMID: 21720544]
  71. J Neurosci. 1997 Nov 1;17(21):8621-44 [PMID: 9334433]

MeSH Term

Brain
Models, Neurological
Neurons

Word Cloud

Created with Highcharts 10.0.0normalizationdivisiveefficientdistributionstimulusmultivariateParetodistributionsefficientlyDivisiveacrosscodingprovidetheoreticalresultcodeshowhistogramequalizationcostsdifferentmayfeaturesnaturalisticstatisticscanonicalcomputationbrainobservedneuralsystemsoftenconsideredimplementationprinciplemakesconditionsanalyticallyprecise:low-noiseregimeencoding-dimensionalviaprevalenceenvironmentdescribedgeneralizeanalogallowarbitrarymetabolicrepresentationassumptionsassociatedshapesencodessuggestsevolvedrepresentstimulidemonstrateencodedconsistentstylizedcharacteristicconditionalvariancedependenceempiricalevidencesuggestingcapturefilterresponsesimagesfindingalsoyieldsempiricallytestablepredictionssensorydomainsparameterstunedinputPareto-distributedenvironmentsnatural

Similar Articles

Cited By