Replicating outdoor environments using VR and ambisonics: a methodology for accurate audio-visual recording, processing and reproduction.

Fotis Georgiou, Claudia Kawai, Beat Sch��ffer, Reto Pieren
Author Information
  1. Fotis Georgiou: Empa, Swiss Federal Laboratories for Materials Science and Technology, Laboratory for Acoustics/Noise Control, ��berlandstrasse 129, D��bendorf, 8600 Switzerland.
  2. Claudia Kawai: Empa, Swiss Federal Laboratories for Materials Science and Technology, Laboratory for Acoustics/Noise Control, ��berlandstrasse 129, D��bendorf, 8600 Switzerland.
  3. Beat Sch��ffer: Empa, Swiss Federal Laboratories for Materials Science and Technology, Laboratory for Acoustics/Noise Control, ��berlandstrasse 129, D��bendorf, 8600 Switzerland.
  4. Reto Pieren: Empa, Swiss Federal Laboratories for Materials Science and Technology, Laboratory for Acoustics/Noise Control, ��berlandstrasse 129, D��bendorf, 8600 Switzerland.

Abstract

This paper introduces a methodology tailored to capture, post-process, and replicate audio-visual data of outdoor environments (urban or natural) for VR experiments carried out within a controlled laboratory environment. The methodology consists of 360 video and higher order ambisonic (HOA) field recordings and subsequent calibrated spatial sound reproduction with a spherical loudspeaker array and video played back via a head-mounted display using a game engine and a graphical user interface for a perceptual experimental questionnaire. Attention was given to the equalisation and calibration of the ambisonic microphone and to the design of different ambisonic decoders. A listening experiment was conducted to evaluate four different decoders (one 2D first-order ambisonic decoder and three 3D third-order decoders) by asking participants to rate the relative (perceived) realism of recorded outdoor soundscapes reproduced with these decoders. The results showed that the third-order decoders were ranked as more realistic.

Keywords

References

  1. J Acoust Soc Am. 2023 Sep 1;154(3):1882-1895 [PMID: 37756576]
  2. PLoS One. 2013 Dec 02;8(12):e83068 [PMID: 24312677]
  3. Front Robot AI. 2020 Feb 21;7:20 [PMID: 33501189]
  4. Sci Rep. 2021 Feb 17;11(1):3994 [PMID: 33597577]
  5. PLoS One. 2022 Jun 27;17(6):e0270401 [PMID: 35759477]
  6. J Environ Psychol. 2020 Dec;72:101500 [PMID: 33390641]

Word Cloud

Created with Highcharts 10.0.0decodersambisonicmethodologyoutdooraudio-visualenvironmentsVRvideoreproductionusingequalisationdifferentthird-orderpaperintroducestailoredcapturepost-processreplicatedataurbannaturalexperimentscarriedwithincontrolledlaboratoryenvironmentconsists360higherorderHOAfieldrecordingssubsequentcalibratedspatialsoundsphericalloudspeakerarrayplayedbackviahead-mounteddisplaygameenginegraphicaluserinterfaceperceptualexperimentalquestionnaireAttentiongivencalibrationmicrophonedesignlisteningexperimentconductedevaluatefourone2Dfirst-orderdecoderthree3DaskingparticipantsraterelativeperceivedrealismrecordedsoundscapesreproducedresultsshowedrankedrealisticReplicatingambisonics:accuraterecordingprocessingAmbisonicsDecoderPerceptualevaluationVirtualreality

Similar Articles

Cited By

No available data.