publications

2026

1. SciRep

   Semantic processing and individual suggestibility modulate motor preparation and perceived distance for looming sounds entering the peripersonal space

   Roberto Barumerli, Michele Geronazzo, and Paola Cesari

   Scientific Reports, Apr 2026

   DOI
2. JASA

   Perceptual evaluation of an auditory model-based similarity metric for head-related transfer functions

   Rapolas Daugintis, Michele Geronazzo, Katarina C. Poole, and 1 more author

   The Journal of the Acoustical Society of America, Mar 2026

   DOI
3. IJCCI

   Pitch training for children with cochlear implants: Negotiating clinical validity through mobile game-based design

   Andrea Gulli, Michele Geronazzo, Federico Fontana, and 4 more authors

   International Journal of Child-Computer Interaction, Jun 2026

   DOI
4. EmpathyPod: An Interactive XR System for Exploring Hearing Diversity

   Andrea Gulli, Emanuele Zanoni, Giovanni Nicoli, and 2 more authors

   In Extended Reality, 2026
5. DIMA: Toward a Multimedia Inspection Workbench for Musical Cultural Heritage Records

   Andrea Gulli, Michele Geronazzo, and Massimiliano Guido

   In Proceedings of the 2026 IEEE International Conference on Content-Based Multimedia Indexing, 2026
6. Spatialized Looming Sounds in Virtual Reality: Reaction Times and Localization Accuracy

   Roberto Barumerli, Alessandro Giuseppe Privitera, Andrea Fasolato, and 4 more authors

   In Extended Reality, 2026

   Series Title: Lecture Notes in Computer Science

   Abs DOI

   The recent availability of consumer head-mounted displays (HMDs) for virtual reality (VR) offers an immersive and affordable platform to study sound perception and action in plausible settings. Traditional perception-action studies often rely on complex systems such as motion capture, while HMDs integrate dynamic sound delivery and realtime movement tracking, making them more accessible for research outside controlled laboratory environments. In this study, we build upon the experimental protocol from Geronazzo et al. (2023) to evaluate the feasibility of measuring user behavior with a consumer-grade HMD, rather than relying on surface electromyography (EMG) to capture muscular activity and precise motion caption equipment. We focused on reaction times, direction and distance localization performances, utilizing the body tracking capabilities from the HMD’s onboard sensors and controllers to showcase the potential for consumer-grade VR to enable accessible experimental research.

7. Listener Acoustic Personalization Challenge LAP24: Perceptual Evaluation of the HRTF Upsampling

   Fabian Brinkman, Anton Hoyer, Aidan O. T. Hogg, and 3 more authors

   In In Proc. of the 12th Convention of the European Acoustics Association, 2026

2025

1. PPS

   From Digital Twins to Entangled Twins: A Phenomenological Model for Industry 5.0

   Floriana Ferro, Alessandro Giuseppe Privitera, Andrea Gulli, and 1 more author

   Philosophical Problems in Science (Zagadnienia Filozoficzne w Nauce), Dec 2025

   Abs DOI

   This article introduces the concept of Entangled Twins (ETs) as an interdisciplinary framework emerging from a sustained dialogue between Human–Computer Interaction (HCI) research and phenomenological philosophy. Building on the Digital Twin (DT) paradigm while critically reworking its representational logic, ETs are conceived as relational configurations in which human users and AI-driven systems become dynamically entangled, with significant implications for agency, responsibility, and human-centered design. The theoretical framework draws on phenomenological analyses of intersubjectivity and embodiment (Husserl, Merleau-Ponty, and Sartre) together with design-oriented concerns in HCI. Within this perspective, human–machine interaction is conceptualized in terms of a We-subject: a plural and co-constituted configuration of agency. Through this interdisciplinary approach, a prototypical audio-augmented experience is examined as a phenomenologically informed case, elucidating the conditions under which human–machine entanglement can emerge. On this basis, the paper argues that ETs raise distinctive questions concerning responsibility and agency, articulating an ethics of entanglement capable of informing the human-centered aspirations of Industry 5.0.

2. JASA EL

   Strong and weak head-related transfer functions: The eHRTF analytical framework

   Michele Geronazzo

   JASA Express Letters, Aug 2025

   Abs DOI

   This article introduces an analytical framework for modeling head-related transfer functions (HRTFs) from a listener-centered perspective. The distinction between strong (or general) HRTFs, aiming for idealized physical acoustic fidelity, and weak (or narrow) HRTFs, prioritizing perceptual adequacy in task-specific contexts, frames the contrast in multiple distinctive definitions and scientific methodologies by drawing inspiration from the debate in artificial intelligence. The proposed formalism adopts a Bayesian structure that models HRTFs through a state-space formulation capturing anatomical, contextual, experiential, and task-related factors: the eHRTF. The “e” emphasizes the egocentric perspective, transforming HRTFs from static measurements into mutable auditory representations continuously updated through the listener’s feedback. Satisfaction regions are defined in probabilistic terms and characterize how different classes of HRTFs, i.e., individual, generic, super, and personalized, meet perceptual requirements under varying tasks and their complexity.

3. Listener Acoustic Personalization Challenge—LAP24: Head-Related Transfer Function Dataset Harmonization

   Rapolas Daugintis, Roberto Barumerli, Michele Geronazzo, and 3 more authors

   IEEE Open Journal of Signal Processing, 2025

   Abs DOI

   Big data analysis and collation for data-driven head-related transfer function (HRTF) personalization methods are often hindered by systematic differences between HRTF datasets. To address this issue, we designed Task 1 of the inaugural listener acoustic personalisation (LAP) challenge. Researchers were invited to propose strategies for harmonizing HRTFs from a collection of eight different datasets so that dataset-specific artifacts were mitigated while preserving the perceptually relevant attributes of the original HRTFs. Defining the two-sided task required a deeper assessment of the acoustic and perceptual HRTF descriptions to find an evaluation framework that encompassed the two domains. Consequently, a two-stage evaluation was devised to assess the submissions. First, an auditory sound localization model was used to test the perceptual validity of the harmonized HRTFs by estimating the difference in sound localization performance between the original and the harmonized versions. Then, a machine learning classifier was employed to differentiate harmonized HRTF datasets, and its accuracy was used to rank submissions. Three submissions were received, and one was declared a winner according to the evaluation criteria. Further analysis of the submissions revealed some limitations of the evaluation system, prompting a comprehensive review of the task’s inherent complexities. This paper serves as a systematic account of the challenge and relevant considerations, intended to guide future advancements in the field of HRTF personalization research.

4. Listener Acoustic Personalization Challenge - LAP24: Head-Related Transfer Function Upsampling

   Aidan O. T. Hogg, Roberto Barumerli, Rapolas Daugintis, and 4 more authors

   IEEE Open Journal of Signal Processing, 2025

   Abs DOI

   Head-related transfer functions (HRTFs) often play a crucial role in spatial hearing, immersive audio applications for virtual reality (VR) and augmented reality (AR), and help in improving hearing assistive devices. The Listener Acoustic Personalisation (LAP) challenge 2024 aimed at advancing research in spatial audio personalisation, with a focus on head-related transfer functions (HRTFs). The challenge was split into two tasks: Task 1 was on HRTF harmonisation, and Task 2 dealt with spatial HRTF upsampling. This paper presents the results and reports the findings related to Task 2 of the LAP challenge. The submissions to Task 2 employed both algorithmic and machine learning-based approaches, which were evaluated on three key spatial audio objective metrics, including the log-spectral distortion (LSD), interaural time difference (ITD), and the interaural level difference (ILD). The results highlighted the strengths and limitations of various upsampling techniques, with learning-based methods demonstrating superior performance at lower sparsity levels. In terms of the LSD, seven of the submissions achieved an impressive performance of less than 5 dB when upsampling from only three measurement points. The results also highlighted that most submissions were often not able to outperform a generic HRTF created by averaging the HRTFs in the training dataset. One of the main contributions of this paper is that it showcases the limitations of objective metrics when it comes to evaluating HRTF upsampling. Therefore, this paper argues that a more holistic approach is needed going forward, which should include the integration of multiple perceptually relevant measures, as this is the only way to ensure a well-rounded assessment of HRTF upsampling quality.

5. OJSP

   A Survey on Machine Learning Techniques for Head-Related Transfer Function Individualization

   Davide Fantini, Michele Geronazzo, Federico Avanzini, and 1 more author

   IEEE Open Journal of Signal Processing, 2025

   Abs DOI

   Machine learning (ML) has become pervasive in various research fields, including binaural synthesis personalization, which is crucial for sound in immersive virtual environments. Researchers have mainly addressed this topic by estimating the individual head-related transfer function (HRTF). HRTFs are utilized to render audio signals at specific spatial positions, thereby simulating real-world sound wave interactions with the human body. As such, an HRTF that is compliant with individual characteristics enhances the realism of the binaural simulation. This survey systematically examines the HRTF individualization works based on ML proposed in the literature. The analyzed works are organized according to the processing steps involved in the ML workflow, including the employed dataset, input and output types, data preprocessing operations, ML models, and model evaluation. In addition to categorizing the works of the existing literature, this survey discusses their achievements, identifies their limitations, and outlines aspects that require further investigation at the crossroads of research communities in acoustics, audio signal processing, and machine learning.

6. On comparing auditory models and perceptual assessment when rating head-related transfer functions

   Rapolas Daugintis, Michele Geronazzo, and Lorenzo Picinali

   In Forum Acusticum 2025, 2025

   DOI
7. MTA

   The Role of Audio in Immersive Storytelling: a Systematic Review in Cultural Heritage

   Alessandro Giuseppe Privitera, Federico Fontana, and Michele Geronazzo

   Multimedia Tools and Applications, Jun 2025

   Abs DOI

   Nowadays, Virtual and Augmented Reality technologies play a supportive role in many research fields. In cultural heritage, various examples are available, including storytelling and narratives, where they can provide immersive and enhanced experiences to visitors and tourists, especially for entertainment and educational purposes. This review aims to investigate the opportunities that soundscape design and advanced sonic interactions in virtual and augmented environments can bring to cultural heritage sites and museums in terms of presence, emotional content, and cultural dissemination. Nineteen-two papers have been identified through the PRISMA methodology, and a promising positive effect of sonic interaction on user experience in a virtual environment can be observed in various studies, notwithstanding a general lack of specific contributions on the use of sound rendering and audio spatialisation for improving such experiences. Moreover, this work identifies the main involved research areas and discusses the state-of-the-art best practices and case studies where sonic interactions may assume a central role. The final part suggests possible future directions and applications for more engaging and immersive storytelling in the cultural heritage domain.

8. IEEE 8th VR Workshop on Sonic Interactions for Virtual Environments

   Stefania Serafin, Ali Adjorlu, Rolf Nordahl, and 7 more authors

   In 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Mar 2025

   Abs DOI

   Sonic interaction design is defined as the study and exploitation of sound as one of the principal channels conveying information, meaning, and aesthetic/emotional qualities in interactive contexts. This field lies at the intersection of interaction design, and sound and music computing.In the virtual reality community, the focus of research on topics related to auditory feedback has been rather limited when compared, for example, to the focus placed on visual feedback or even on haptic feedback. However, in such communities as film production or product sound design it is well known that sound is a powerful way to communicate meaning and emotion to a scene or a product.SIVE 2025 is the 8th of a series of workshops, whose main goal is to increase awareness among the virtual reality community of the importance of sonic elements when designing virtual/augmented/mixed reality (XR hereafter) environments. Participants to the workshop will also discuss how research in other related fields such as film sound theory, product sound design, sound and music computing, game sound design, and accessibility can inform designers of XR environments. Moreover, the workshop will feature state-of-the-art research in the field of sound for XR environments.

9. Measuring Motor Planning Using an Affordable Sound-Based Virtual Reality Setup for Accessible Perception-Action Studies

   Roberto Barumerli, Alessandro Giuseppe Privitera, Alessandro Carollo, and 5 more authors

   In 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Mar 2025

   Abs DOI

   The recent availability of consumer head-mounted displays (HMDs) for virtual reality (VR) provides an immersive and affordable platform to study sound perception and action outside laboratory settings. While traditional perception-action studies rely on complex systems like motion capture, HMDs integrate dynamic sound delivery and real-time movement tracking. In this study, we adapted the experimental protocol from Geronazzo et al. (2023) to test the feasibility of measuring user behaviour with a consumer-grade HMD. Instead of muscular activity measured from surface electromyography, we analysed reaction times using kinematic data from the HMD’s onboard sensors and controllers. Data from ten participants showed that spatial properties of looming sounds influence auditory perception-action loops, aligning with findings from the original study. Our results demonstrate that reaction time is a scalable and meaningful measure for studying space perception through sound, providing a possible alternative to muscular activity measurements and showcasing the potential of consumer-grade VR for accessible and robust experimental research despite current limitations.

10. Head-related transfer function forte e debole

    Michele Geronazzo

    In Proceedings of the 51st Convegno Nazionale Associazione Italiana di Acustica, 2025
11. Motor Planning Sensitivity to Affective Looming Sounds Within The Peri-personal Space: An Interplay of Exogenous and Endogenous Influences

    Roberto Barumerli, Michele Geronazzo, and Paola Cesari

    Jul 2025

    DOI

2024

1. Effects of binaural rendering personalisation and reverberation on speech-on-speech masking

   Rapolas Daugintis, Benoit Alary, Michele Geronazzo, and 1 more author

   In , Aug 2024

   Abs

   This study investigates the effect of head-related transfer function (HRTF) personalisation on understanding binaurally rendered target speech masked by interfering speakers in reverberant conditions. During a listening test, participants had to identify a correct colour-number combination from a virtual talker rendered in front of them while ignoring two interfering talkers positioned either in front or at the back. The sound was rendered with either an individual HRTF or one of two non-individual ones. These were selected for each participant as the best or the worstmatching from the same HRTF dataset, based on predictions of a computational auditory model for sound localisation. Two types of reverb from measured spatial room impulse responses (SRIRs) were applied to the speech: realistic dichotic reverberation decoded from 4th-order Ambisonic SRIRs or diotic reverb based on the omnidirectional Ambisonic channel IR as a baseline. Preliminary results show that realistic dichotic reverb improves speech perception when interfering speech is co-located with the target. No significant differences were observed across HRTF conditions on a group level, but individual HRTF-related performance differences exist, requiring further intra-subject analyses and data collection to characterise the individual results. Link to paper: https://aes2.org/publications/elibrary-page/?id=22671

2. Un Caso di Studio Relativo al Sovracampionamento Spaziale di Head-Related Transfer Function Attraverso Reti Neurali Informate dalla Fisica

   Fei Zanoni, Andrea Gulli, and Michele Geronazzo

   In In Atti 50° Convegno Nazionale Asoociazione Italiana di Acustica, 2024

   Abs

   La Realtà Virtuale (VR) e Aumentata (AR) stanno cambiando il nostro modo di interagire con il mondo digitale. In questo contesto, l’audio spazializzato gioca un ruolo cruciale per la completa immersione in un ambiente virtuale. Le reti neurali informate dalla fisica, Physics-Informed Neural Networks (PINN), possono essere impiegate per l’upsampling spaziale di Head-Related Transfer Function (HRTF), semplificandone drasticamente i metodi di acquisizione e garantendo l’accessibilità all’audio spazializzato.

3. A mobile game app for adaptive assessment of pitch discrimination in children with different hearing ability

   Andrea Gulli, Federico Fontana, Hanna Jarvelainen, and 1 more author

   In In Proc, of the XXIV Colloquio di Informatica Musicale, 2024

   Abs

   Recent advancements in audiological testing and rehabilitation prioritize utmost personalization and minimal stress in patients, especially the young ones. Hearing health assessments integrate innovative approaches, emphasizing ecological listening scenarios and patient engagement. Driven by these principles, a mobile application tailored for pitch discrimination in children has been designed. The app implements intuitive game mechanics on a captivating graphical interface and harnesses machine learning algorithms to adapt the sound pressure levels to individual comfort levels. It utilizes simple yet effective acoustic stimuli obtained from second-order digital resonators, ensuring a more ecological approach. The pitch discrimination threshold is obtained with adaptive psychometric techniques to guarantee reliable and faster measurements. Preliminary qualitative evaluations involving normal hearing and single-sided deaf with cochlear implant children yield promising outcomes. The resulting perceptual thresholds align with established literature, envisioning the app’s efficacy in delivering accurate assessments. The presented tool paves the way for the use of gameplay in young hearing-impaired individuals’ rehabilitation after treatment with cochlear implants.

4. Designing sonic interactions in intelligent reality with egocentric audio technologies

   Alessandro Giuseppe Privitera and Michele Geronazzo

   In The Routledge Handbook of Sound Design, 2024
5. Technical Report: SONICOM / IEEE Listener Acoustic Personalisation (LAP) Challenge - 2024

   Michele Geronazzo, Lorenzo Picinali, Aidan Hogg, and 7 more authors

   2024

2023

1. Co-immersion in Audio Augmented Virtuality: the Case Study of a Static and Approximated Late Reverberation Algorithm

   Davide Fantini, Giorgio Presti, Michele Geronazzo, and 3 more authors

   IEEE Transactions on Visualization and Computer Graphics, 2023

   Conference Name: IEEE Transactions on Visualization and Computer Graphics

   Abs DOI

   In immersive Audio Augmented Reality, a virtual sound source should be indistinguishable from the existing real ones. This property can be evaluated with the co-immersion criterion, which encompasses scenes constituted by arbitrary configurations of real and virtual objects. Thus, we introduce the term Audio Augmented Virtuality (AAV) to describe a fully virtual environment consisting of auditory content captured from the real world, augmented by synthetic sound generation. We propose an experimental design in AAV investigating how simplified late reverberation (LR) affects the co-immersion of a sound source. Participants listened to simultaneous virtual speakers dynamically rendered through spatial Room Impulse Responses, and were asked to detect the presence of an impostor, i.e., a speaker rendered with one of two simplified LR conditions. Detection rates were found to be close to chance level, especially for one condition, suggesting a limited influence on co-immersion of the simplified LR in the evaluated AAV scenes. This methodology can be straightforwardly extended and applied to different acoustics scenes, complexities, i.e., the number of simultaneous speakers, and rendering parameters in order to further investigate the requirements for immersive audio technologies in AAR and AAV applications.

2. VR

   Shaping the auditory peripersonal space with motor planning in immersive virtual reality

   Michele Geronazzo, Roberto Barumerli, and Paola Cesari

   Virtual Reality, Oct 2023

   Abs DOI

   Immersive audio technologies require personalized binaural synthesis through headphones to provide perceptually plausible virtual and augmented reality (VR/AR) simulations. We introduce and apply for the first time in VR contexts the quantitative measure called premotor reaction time (pmRT) for characterizing sonic interactions between humans and the technology through motor planning. In the proposed basic virtual acoustic scenario, listeners are asked to react to a virtual sound approaching from different directions and stopping at different distances within their peripersonal space (PPS). PPS is highly sensitive to embodied and environmentally situated interactions, anticipating the motor system activation for a prompt preparation for action. Since immersive VR applications benefit from spatial interactions, modeling the PPS around the listeners is crucial to reveal individual behaviors and performances. Our methodology centered around the pmRT is able to provide a compact description and approximation of the spatiotemporal PPS processing and boundaries around the head by replicating several well-known neurophysiological phenomena related to PPS, such as auditory asymmetry, front/back calibration and confusion, and ellipsoidal action fields.

3. A Bayesian model for human directional localization of broadband static sound sources

   Roberto Barumerli, Piotr Majdak, Michele Geronazzo, and 3 more authors

   Acta Acustica, 2023

   Publisher: EDP Sciences

   Abs DOI

   Humans estimate sound-source directions by combining prior beliefs with sensory evidence. Prior beliefs represent statistical knowledge about the environment, and the sensory evidence consists of auditory features such as interaural disparities and monaural spectral shapes. Models of directional sound localization often impose constraints on the contribution of these features to either the horizontal or vertical dimension. Instead, we propose a Bayesian model that flexibly incorporates each feature according to its spatial precision and integrates prior beliefs in the inference process. The model estimates the direction of a single, broadband, stationary sound source presented to a static human listener in an anechoic environment. We simplified interaural features to be broadband and compared two model variants, each considering a different type of monaural spectral features: magnitude profiles and gradient profiles. Both model variants were fitted to the baseline performance of five listeners and evaluated on the effects of localizing with non-individual head-related transfer functions (HRTFs) and sounds with rippled spectrum. We found that the variant equipped with spectral gradient profiles outperformed other localization models. The proposed model appears particularly useful for the evaluation of HRTFs and may serve as a basis for future extensions towards modeling dynamic listening conditions.

4. Sonic Interactions in Virtual Environments

   2023

   DOI
5. Sonic Interactions in Virtual Environments: the Egocentric Audio Perspective of the Digital Twin

   Michele Geronazzo and Stefania Serafin

   In Sonic Interactions in Virtual Environments, 2023
6. Classifying Non-Individual Head-Related Transfer Functions with A Computational Auditory Model: Calibration And Metrics

   Rapolas Daugintis, Roberto Barumerli, Lorenzo Picinali, and 1 more author

   In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Jun 2023

   Abs DOI

   This study explores the use of a multi-feature Bayesian auditory sound localisation model to classify non-individual head-related transfer functions (HRTFs). Based on predicted sound localisation performance, these are grouped into ‘good’ and ‘bad’, and the ‘best’/‘worst’ is selected from each category. Firstly, we present a greedy algorithm for automated individual calibration of the model based on the individual sound localisation data. We then discuss data analysis of predicted directional localisation errors and present an algorithm for categorising the HRTFs based on the localisation error distributions within a limited range of directions in front of the listener. Finally, we discuss the validity of the classification algorithm when using averaged instead of individual model parameters. This analysis of auditory modelling results aims to provide a perceptual foundation for automated HRTF personalisation techniques for an improved experience of binaural spatial audio technologies.

7. An active learning procedure for the interaural time difference discrimination threshold

   Andrea Gulli, Federico Fontana, Stefania Serafin, and 1 more author

   In Proc. of the 26th Int. Conference on Digital Audio Effects (DAFx-23), Sep 2023
8. On the Effect of User Tracking on Perceived Source Positions in Mobile Audio Augmented Reality

   Alessandro Giuseppe Privitera, Federico Fontana, and Michele Geronazzo

   In Proceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter, Sep 2023

   Abs DOI

   Mobile Audio Augmented Reality (AAR) allows users to live sonic experience where virtual sound objects are integrated seamlessly with the real-world acoustic space. This paper focuses on user perception of the virtual source position and the impact of different out-of-the-box user tracking and head orientation methods made available by iPad’s Augmented Reality (AR) camera tracking system and AirPods Pro. We propose an experimental procedure to rank tracking approaches according to perceived accuracy in positioning eye-level and ground-level virtual audio sources compared to real source references. To correctly provide a plausible AAR scenario, the proposed consumer electronic setup simulates a virtual sound source employing the scattering delay network (SDN) algorithm for calibrated dynamic auralisation and customized head-related transfer functions (HRTFs) for personalized user acoustics. The natural listening experience of real sound sources leverages AirPods’s active signal processing algorithms for headphone hear-through capabilities. The main result of this study lies in observing an accommodation effect by users interacting with different tracking approaches and sound source positions. In summary, participants tend to prefer more stable tracking solutions despite accuracy and wide head movement range.

9. Initial Evaluation of an Auditory-Model-Aided Selection Procedure for Non- Individual HRTFs

   R. Daugintis, R. Barumerli, M. Geronazzo, and 1 more author

   In Proceedings of the 10th Convention of the European Acoustics Association Forum Acusticum 2023, Sep 2023

   Abs DOI

   Binaural spatial audio reproduction systems use measured or simulated head-related transfer functions (HRTFs), which encode the effects of the outer ear and body on the incoming sound to recreate a realistic spatial auditory field around the listener. The sound localisation cues embedded in the HRTF are highly personal. Establishing perceptual similarity between different HRTFs in a reliable manner is challenging due to a combination of acoustic and non-acoustic aspects affecting our spatial auditory perception. To account for these factors, we propose an automated procedure to select the ‘best’ non-individual HRTF dataset from a pool of measured ones. For a group of human participants with their own acoustically measured HRTFs, a multi-feature Bayesian auditory sound localisation model is used to predict individual localisation performance with the other HRTFs from within the group. Then, the model selection of the ‘best’ and the ‘worst’ non-individual HRTFs is evaluated via an actual localisation test and a subjective audio quality assessment in comparison with individual HRTFs. A successful model-aided objective selection of the ‘best’ non-individual HRTF may provide relevant insights for effective and handy binaural spatial audio solutions in virtual/augmented reality (VR/AR) applications.

10. Personalization in Audio Storytelling within Virtual and Augmented Reality: State of the Art and Insights

    A.G. Privitera, F. Fontana, and M. Geronazzo

    In Proceedings of the 10th Convention of the European Acoustics Association Forum Acusticum 2023, Sep 2023

    Abs DOI

    Storytelling and narrative are essential components in diverse contexts ranging from entertainment to culture and knowledge. With the progressive evolution of communication, stories also started to be transmitted through digital media. In particular, the emerging and promising field of sonic interactions in virtual environments (SIVE) fosters the creation of immersive experiences in which personalization of user interaction and narrative content can help create engaging and interactive storytelling. The insights presented here aim to define a state of the art of available interactive audio technologies for storytelling and to illustrate the role of audio in augmented and virtual reality (AR/VR) personalized experiences by reviewing a selection of published works. Moreover, we will try to disclose the most important elements enabling an immersive experience and to explain how interactivity, emotional and personalized content help convey messages in the context of serious storytelling. Finally, we will investigate the limitations of the available methods by also outlining some promising research directions.

11. Preliminary Evaluation of the Auralization of a Real Indoor Environment for Augmented Reality Research

    A.G. Privitera, M. Noro, and M. Geronazzo

    In Proceedings of the 10th Convention of the European Acoustics Association Forum Acusticum 2023, Sep 2023

    Abs DOI

    This paper describes the calibration procedure and the technical setup for a realistic real-time acoustic reconstruction of a real indoor environment, a corridor, in the context of Audio Augmented Reality (AAR). The acoustic phenomena inside such space are simulated using the scattering delay network (SDN) algorithm. Wall reflection coefficients have been estimated using room dimensions, wall materials, and RT60 decay measurements. Auralisation has been dynamically conveyed by using a personalized head-related transfer function (HRTF), modeled by combining (i) a spherical head model with ear displacement with (ii) the high-frequency magnitude of an HRTF selected from the CIPIC database by using two 2D images of the user’s head. Moreover, the iPad’s AR camera tracking system and AirPods pro accelerometers have tracked the listener’s head and body position in real space. The proposed preliminary evaluation focuses on the impact of the different rendering factors in a simple AAR environment, suggesting that personalization, room calibration, and volume gain help render a more plausible AAR scene.

12. Effect of Auditory Stimulation to Task and Presence in Selective Blurred Immersive Environment for VR Sickness Reduction

    Satoshi Okuno, Sota Shimizu, Alessandro Giuseppe Privitera, and 2 more authors

    In Proc. of IEEJ International Workshop on Sensing, Actuation, and Motion Control, Mar 2023

2022

1. SPM

   The SONICOM Project: Artificial Intelligence-Driven Immersive Audio, From Personalization to Modeling [Applications Corner]

   Lorenzo Picinali, Brian FG Katz, Michele Geronazzo, and 3 more authors

   IEEE Signal Processing Magazine, Nov 2022

   Conference Name: IEEE Signal Processing Magazine

   Abs DOI

   Every individual perceives spatial audio differently, due in large part to the unique and complex shape of ears and head. Therefore, high-quality, headphone-based spatial audio should be uniquely tailored to each listener in an effective and efficient manner. Artificial intelligence (AI) is a powerful tool that can be used to drive forward research in spatial audio personalization. The SONICOM project aims to employ a data-driven approach that links physiological characteristics of the ear to the individual acoustic filters, which allows us to localize sound sources and perceive them as being located around us. A small amount of data acquired from users could allow personalized audio experiences, and AI could facilitate this by offering a new perspective on the matter. A Bayesian approach to computational neuroscience and binaural sound reproduction will be linked to create a metric for AI-based algorithms that will predict realistic spatial audio quality. Being able to consistently and repeatedly evaluate and quantify the improvements brought by technological advancements, as well as the impact these have on complex interactions in virtual environments, will be key for the development of new techniques and for unlocking new approaches to understanding the mechanisms of human spatial hearing and communication.

2. The Egocentric Audio Perspective in Virtual Environments

   Michele Geronazzo

   In Proc. of the 2nd Symposium: The Acoustics of Ancient Theatres, Jul 2022
3. Egocentric Audio in the Digital Twin of Virtual Environments

   Michele Geronazzo

   In 2022 IEEE 2nd International Conference on Intelligent Reality (ICIR), Dec 2022

   Abs DOI

   In Virtual Environments (VE), audio technologies play a significant role in immersive and interactive experiences. Virtual Reality (VR) simulations must be ecologically enacted by a participatory exploration of sense-making in a network of human and non-human agents, called actors. The guardian of such locus of agency is the digital twin (DT) that fosters intra-actions between humans and technology, dynamically and fluidly redefining all those configurations that are crucial for meaningful sonic experiences. The idea of human-machine entanglement is here mainly declined in an egocentric-spatial perspective related to emerging knowledge of the listener’s subjectivity. Such a systemic view can be interpreted as a working definition of intelligent reality: a perceptual and cognitive co-constitution of physical and virtual worlds through adaptive and reflective behaviors of VR technologies. The main theoretical results reported in this paper reside in the definition of sonic experiences as a multilayer interconnected network of actors lying in two main layers, i.e., immersion and coherence, which are entangled by a DT able to perform transformative actions for the listener.

4. L’Audio Egocentrico negli Ambienti Virtuali

   Michele Geronazzo

   In Proc. XXIII Colloquio di Informatica Musicale (XXII CIM), Oct 2022
5. Visibility control based on approximate expression gaze rate by GMM

   Miwa Takase, Sota Shimizu, Takumi Morimoto, and 3 more authors

   In IEEJ Advanced sensor information processing technology and its application, Jan 2022
6. Effects of Auditory Stimulation on Tasks and Presence in Selective Blur Immersive Virtual Environments to Reduce VR Sickness

   Satoshi Okuno, Sota Shimizu, Kohei Iida, and 3 more authors

   In IEEJ Advanced sensor information processing technology and its application, Jan 2022
7. Development and evaluation of auditory-model-aided non-individual HRTF selection procedure

   Rapolas Daugintis, Michele Geronazzo, Roberto Barumerli, and 1 more author

   In UK Hearing Audiology and Sciences Meeting, Sep 2022
8. Evaluation of spatial tasks in virtual acoustic environments by means of modeling individual localization performances

   R. Barumerli, P. Majdak, M. Geronazzo, and 3 more authors

   In Proceedings: A21, Virtual Acoustics, ICA 2022, 2022
9. Sonic Interactions in Virtual Environments: the Egocentric Audio Perspective of the Digital Twin

   Michele Geronazzo and Stefania Serafin

   Apr 2022

   DOI

2021

1. Action planning and affective states within the auditory peripersonal space in normal hearing and cochlear-implanted listeners

   Mehrdad Bahadori, Roberto Barumerli, Michele Geronazzo, and 1 more author

   Neuropsychologia, May 2021

   Abs DOI

   Fast reaction to approaching stimuli is vital for survival. When sounds enter the auditory peripersonal space (PPS), sounds perceived as being nearer elicit higher motor cortex activation. There is a close relationship between motor preparation and the perceptual components of sounds, particularly of highly arousing sounds. Here we compared the ability to recognize, evaluate, and react to affective stimuli entering the PPS between 20 normal-hearing (NH, 7 women) and 10 cochlear-implanted (CI, 3 women) subjects. The subjects were asked to quickly flex their arm in reaction to positive (P), negative (N), and neutral (Nu) affective sounds ending virtually at five distances from their body. Pre-motor reaction time (pm-RT) was detected via electromyography from the postural muscles to measure action anticipation at the sound-stopping distance; the sounds were also evaluated for their perceived level of valence and arousal. While both groups were able to localize sound distance, only the NH group modulated their pm-RT based on the perceived sound distance. Furthermore, when the sound carried no affective components, the pm-RT to the Nu sounds was shorter compared to the P and the N sounds for both groups. Only the NH group perceived the closer sounds as more arousing than the distant sounds, whereas both groups perceived sound valence similarly. Our findings underline the role of emotional states in action preparation and describe the perceptual components essential for prompt reaction to sounds approaching the peripersonal space.

2. SofaMyRoom: a fast and multiplatform "shoebox" room simulator for binaural room impulse response dataset generation

   Roberto Barumerli, Daniele Bianchi, Michele Geronazzo, and 1 more author

   Jun 2021

   arXiv: 2106.12992

   Abs

   This paper introduces a shoebox room simulator able to systematically generate synthetic datasets of binaural room impulse responses (BRIRs) given an arbitrary set of head-related transfer functions (HRTFs). The evaluation of machine hearing algorithms frequently requires BRIR datasets in order to simulate the acoustics of any environment. However, currently available solutions typically consider only HRTFs measured on dummy heads, which poorly characterize the high variability in spatial sound perception. Our solution allows to integrate a room impulse response (RIR) simulator with different HRTF sets represented in Spatially Oriented Format for Acoustics (SOFA). The source code and the compiled binaries for different operating systems allow to both advanced and non-expert users to benefit from our toolbox, see https://github.com/spatialaudiotools/sofamyroom/ .

3. Comparison of Distortion Products in Headphone Equalization Algorithms for Binaural Synthesis

   Braxton Boren and Michele Geronazzo

   In In Proc. of the Audio Engineering Society Convention 150, May 2021

   Abs

   Headphone design has traditionally focused on creating a frequency response to make commercial stereo audio sound more natural. However, because of the sensitivity of spatial hearing to frequency-dependent cues, binaural reproduction requires headphones’ target spectrum to be as flat as possible. Initial attempts to equalize headphones used a naive inversion of the headphone spectrum, which degraded binaural content because the headphone transfer function (HpTF) changes each time headphones...

4. The SONICOM project: AI-driven immersive audio, from personalisation to modelling

   Lorenzo Picinali, Michele Geronazzo, Dan F. M. Goodman, and 7 more authors

   Oct 2021
5. Modeling Sound Localization within the Framework of Bayesian Inference

   R. Barumerli, P. Majdak, R. Baumgartner, and 2 more authors

   In In Proc. of the DAGA 2021, 47th Annual Conference on Acoustics, Aug 2021

   Abs

   In this work, we propose an auditory model based on Bayesian inference to reproduce the individual human ability to localize a sound source in the acoustic free field. The model combines physiologically motivated front-ends with a probabilistic decision stage in order to estimate both the lateral- and polar-angle components of the incoming sound direction. In our systematical evaluation, the model was able to reproduce the summary statistics from five listeners when localizing broad-band sound sources. In particular, the results indicate that the model required to account for both the acoustic interaction of the sound source with the subject anatomy and for non-acoustic factors as neural uncertainties and sensorimotor mapping. On the other hand, we found little agreement between simulations and experimental data when considering distortions in the stimuli’s spectrum. This mismatch is probably related on how the auditory front-ends combined various spatial cues. We will discuss these results and further extensions of the framework to match actual performances in diverse acoustic scenarios.

2020

1. Pinna-related transfer functions and lossless wave equation using finite-difference methods: Validation with measurements

   Sebastian T. Prepeliță, Javier Gómez Bolaños, Michele Geronazzo, and 2 more authors

   The Journal of the Acoustical Society of America, May 2020

   Publisher: Acoustical Society of America

   Abs DOI

   Nowadays, wave-based simulations of head-related transfer functions (HRTFs) lack strong justifications to replace HRTF measurements. The main cause is the complex interactions between uncertainties and biases in both simulated and measured HRTFs. This paper deals with the validation of pinna-related high-frequency information in the ipsilateral directions-of-arrival, computed by lossless wave-based simulations with finite-difference models. A simpler yet related problem is given by the pinna-related transfer function (PRTF), which encodes the acoustical effects of only the external ear. Results stress that PRTF measurements are generally highly repeatable but not necessarily easily reproducible, leading to critical issues in terms of reliability for any ground truth condition. On the other hand, PRTF simulations exhibit an increasing uncertainty with frequency and grid-dependent frequency changes, which are here quantified analyzing the benefits in the use of a unique asymptotic solution. In this validation study, the employed finite-difference model accurately and reliably predict the PRTF magnitude mostly within ±1 dB up to ≈\textlessmath display="inline" overflow="scroll" altimg="eq-00001.gif"\textgreater \textlessmo\textgreater≈\textless/mo\textgreater\textless/math\textgreater8 kHz and a space- and frequency-averaged spectral distortion within about 2 dB up to ≈\textlessmath display="inline" overflow="scroll" altimg="eq-00002.gif"\textgreater \textlessmo\textgreater≈\textless/mo\textgreater\textless/math\textgreater 18 kHz.

2. TVCG

   Superhuman Hearing - Virtual Prototyping of Artificial Hearing: a Case Study on Interactions and Acoustic Beamforming

   Michele Geronazzo, Luis S. Vieira, Niels Christian Nilsson, and 2 more authors

   IEEE Transactions on Visualization and Computer Graphics, May 2020

   Abs DOI

   Directivity and gain in microphone array systems for hearing aids or hearable devices allow users to acoustically enhance the information of a source of interest. This source is usually positioned directly in front. This feature is called acoustic beamforming. The current study aimed to improve users’ interactions with beamforming via a virtual prototyping approach in immersive virtual environments (VEs). Eighteen participants took part in experimental sessions composed of a calibration procedure and a selective auditory attention voice-pairing task. Eight concurrent speakers were placed in an anechoic environment in two virtual reality (VR) scenarios. The scenarios were a purely virtual scenario and a realistic 360° audio-visual recording. Participants were asked to find an individual optimal parameterization for three different virtual beamformers: (i) head-guided, (ii) eye gaze-guided, and (iii) a novel interaction technique called dual beamformer, where head-guided is combined with an additional hand-guided beamformer. None of the participants were able to complete the task without a virtual beamformer (i.e., in normal hearing condition) due to the high complexity introduced by the experimental design. However, participants were able to correctly pair all speakers using all three proposed interaction metaphors. Providing superhuman hearing abilities in the form of a dual acoustic beamformer guided by head and hand movements resulted in statistically significant improvements in terms of pairing time, suggesting the task-relevance of interacting with multiple points of interests.

3. Reflections from five years of Sonic Interactions in Virtual Environments workshops

   Stefania Serafin, Federico Avanzini, Amalia De Goetzen, and 5 more authors

   Journal of New Music Research, Jan 2020

   Abs DOI

   For the past five years, the authors have been running at the IEEE Virtual Reality Conference a Workshop called Sonic Interactions in Virtual Environments (SIVE). The main goal of the workshop series has been to increase among the virtual reality community awareness of the importance of sonic elements when designing multimodal and immersive virtual environments. Starting from this experience, this paper presents a survey of the main active research topics related to sound in virtual and augmented reality (VR/AR), ranging from basic research in spatial audio rendering and sonic interaction design to applications in interactive environments for training, health, rehabilitation, entertainment, and art. Looking at the different research topics emerging from laboratories worldwide, the paper discusses how different research communities can collaborate and benefit from each other in order to increase sound awareness in VR and AR.

4. Proceedings of the 2020 IEEE 5th VR Workshop on Sonic Interactions for Virtual Environments (SIVE)

   Michele Geronazzo, Rolf Nordahl, Amalia De Götzen, and 5 more authors

   Mar 2020
5. A Minimal Personalization of Dynamic Binaural Synthesis with Mixed Structural Modeling and Scattering Delay Networks

   Michele Geronazzo, Jason Yves Tissieres, and Stefania Serafin

   In Proc. IEEE Int. Conf. on Acoust. Speech Signal Process. (ICASSP 2020), May 2020

   Abs DOI

   This paper provides a small set of essential parameters for a per-sonalized and effective real-time auralization with headphones. An image-guided procedure with two 2D images of the user’s head guides the mixed structural modeling of head-related transfer function (HRTF), combining a spherical head model with ear displacement with the HRTF high-frequency magnitude selected from a database according to ear anthropometry. Room acoustics phenomena are simplified following the scattering delay network (SDN) approach which allows an accurate spatialization of first order reflections. Finally, statically significant improvements in localization performances within a virtual reality (VR) test allow to identify some benefits of the proposed customized auralization model compared to the widely used higher-order ambisonics (HOA) rendering with generic HRTFs.

6. Predicting Directional Sound-Localization of Human Listeners in both Horizontal and Vertical Dimensions

   Roberto Barumerli, Piotr Majdak, Jonas Reijniers, and 3 more authors

   In , May 2020

   Abs

   Measuring and understanding spatial hearing is a fundamental step to create effective virtual auditory displays (VADs). The evaluation of such auralization systems often requires psychoacoustic experiments. This process can be time consuming and error prone, resulting in a bottleneck for the evaluation complexity. In this work we evaluated a probabilistic auditory model for sound localization intended as a tool to assess VAD’s abilities to provide static sound-localization cues to listeners....

7. Reacting to emotional sounds entering peripersonal space

   M. Bahadori, R. Barumerli, M. Geronazzo, and 5 more authors

   In In Proc. of the 4th HBP Student Conf. on Interdisciplinary Brain Research (accepted), 2020
8. Evaluation of a human sound localization model based on Bayesian inference

   R. Barumerli, P. Majdak, R. Baumgartner, and 2 more authors

   In Proc. Forum Acusicum 2020, Apr 2020
9. MSM binsdn – binaural synthesis and scattering delay networks, https://github.com/msmhrtf/binsdn - a Unity plugin

   Michele Geronazzo and Jason Yves Tissieres

   2020

2019

1. Cervical Spine Motion During Vehicle Extrication of Healthy Volunteers

   Alberto Gabrieli, Francesca Nardello, Michele Geronazzo, and 6 more authors

   Prehospital Emergency Care, Dec 2019

   Abs DOI

   (2019). Cervical Spine Motion During Vehicle Extrication of Healthy Volunteers. Prehospital Emergency Care. Ahead of Print.

2. Pinna-related transfer functions and lossless wave equation using finite-difference methods: Verification and asymptotic solution

   Sebastian T. Prepeliță, Javier Gómez Bolaños, Michele Geronazzo, and 2 more authors

   The Journal of the Acoustical Society of America, Nov 2019

   Abs DOI

   A common approach when employing discrete mathematical models is to assess the reliability and credibility of the computation of interest through a process known as solution verification. Present-day computed head-related transfer functions (HRTFs) seem to lack robust and reliable assessments of the numerical errors embedded in the results which makes validation of wave-based models difficult. This process requires a good understanding of the involved sources of error which are systematically reviewed here. The current work aims to quantify the pinna-related high-frequency computational errors in the context of HRTFs and wave-based simulations with finite-difference models. As a prerequisite for solution verification, code verification assesses the reliability of the proposed implementation. In this paper, known and manufactured formal solutions are used and tailored for the wave equation and frequency-independent boundary conditions inside a rectangular room of uniform acoustic wall-impedance. Asymptotic estimates for pinna acoustics are predicted in the frequency domain based on regression models and a convergence study on sub-millimeter grids. Results show an increasing uncertainty with frequency and a significant frequency-dependent change among computations on different grids.

3. Creating an Audio Story with Interactive Binaural Rendering in Virtual Reality

   Michele Geronazzo, Amalie Rosenkvist, David Sebastian Eriksen, and 5 more authors

   Wireless Communications and Mobile Computing, 2019

   Abs DOI

   The process of listening to an audiobook is usually a rather passive act that does not require an active interaction. If spatial interaction is incorporated into a storytelling scenario, can open. Possibilities of a novel experience which allows an active participation might affect the user-experience. The aim of this paper is to create a portable prototype system based on an embedded hardware platform, allowing listeners to get immersed in an interactive audio storytelling experience enhanced by dynamic binaural audio rendering. For the evaluation of the experience, a short story based on the horror narrative of Stephen King’s Strawberry Springs is adapted and designed in virtual environments. A comparison among three different listening experiences, namely, (i) monophonic (traditional audio story), (ii) static binaural rendering (state-of-the-art audio story), and (iii) our prototype, is conducted. We discuss the quality of the experience based on usability testing, physiological data, emotional assessments, and questionnaires for immersion and spatial presence. Results identify a clear trend for an increase in immersion with our prototype compared to traditional audiobooks, showing also an emphasis on story-specific emotions, i.e., terror and fear.

4. C&E

   Mobile virtual reality for musical genre learning in primary education

   Edoardo Degli Innocenti, Michele Geronazzo, Diego Vescovi, and 4 more authors

   Computers & Education, Oct 2019

   Abs DOI

   Mobile virtual reality (VR) is increasingly becoming popular and accessible to everyone that holds a smartphone. In particular, digital didactics can take advantage of natural interaction and immersion in virtual environments, starting from primary education. This paper investigates the problem of enhancing music learning in primary education through the use of mobile VR. To this end, technical and methodological frameworks were developed, and were tested with two classes in the last year of a primary school (10 years old children). The classes were involved in an evaluation study on music genre identification and learning with a multi-platform mobile application called VR4EDU. Students were immersed in music performances of different genres (e.g., classical, country, jazz, and swing), navigating inside several musical rooms. The evaluation of the didactic protocol shows a statistically significant improvement in learning genre characterization (i.e., typical instruments and their spatial arrangements on stage) compared to traditional lessons with printed materials and passive listening. These results show that the use of mobile VR technologies in synergy with traditional teaching methodologies can improve the music learning experience in primary education, in terms of active listening, attention, and time. The inclusion of pupils with certified special needs strengthened our results.

5. Applying a Single-Notch Metric to Image-Guided Head-Related Transfer Function Selection for Improved Vertical Localization

   Michele Geronazzo, Enrico Peruch, Fabio Prandoni, and 1 more author

   J_AES, 2019

   Publisher: Audio Engineering Society
6. Auditory feedback for navigation with echoes in virtual environments: training procedure and orientation strategies

   A. Andreasen, M. Geronazzo, N. C. Nilsson, and 3 more authors

   IEEE Transactions on Visualization and Computer Graphics, May 2019

   Abs DOI

   Being able to hear objects in an environment, for example using echolocation, is a challenging task. The main goal of the current work is to use virtual environments (VEs) to train novice users to navigate using echolocation. Previous studies have shown that musicians are able to differentiate sound pulses from reflections. This paper presents design patterns for VE simulators for both training and testing procedures, while classifying users’ navigation strategies in the VE. Moreover, the paper presents features that increase users’ performance in VEs. We report the findings of two user studies: a pilot test that helped improve the sonic interaction design, and a primary study exposing participants to a spatial orientation task during four conditions which were early reflections (RF), late reverberation (RV), early reflections-reverberation (RR) and visual stimuli (V). The latter study allowed us to identify navigation strategies among the users. Some users (10/26) reported an ability to create spatial cognitive maps during the test with auditory echoes, which may explain why this group performed better than the remaining participants in the RR condition.

7. Action anticipation and sounds’ semantics

   M. Bahadori, R. Barumerli, M. Geronazzo, and 5 more authors

   In In Proc. of XXVII SIPF National Congress, Nov 2019
8. Interactions in Mobile Sound and Music Computing

   Michele Geronazzo, Federico Avanzini, Federico Fontana, and 1 more author

   Wireless Communications and Mobile Computing, Dec 2019

   DOI
9. What Is It Like to Be a Virtual Bat?

   Anastassia Andreasen, Niels Christian Nilsson, Jelizaveta Zovnercuka, and 2 more authors

   In Interactivity, Game Creation, Design, Learning, and Innovation, 2019

   Abs

   Virtual Reality (VR) might give us a glimpse of what it feels to have a different from human shaped body and how to orientate ourselves in virtual environment (VE) with it. Bats’ wings structure has anatomical similarities to a human hand; yet would it be possible to achieve a compelling illusion of virtual body ownership (VBO) over bat’s avatar is questionable. Hence our main aim of research is to imitate bat’s sonar system and achieve embodiment of anatomically similar but morphologically different body – a body of a bat. Test results showed a possibility to achieve VBO illusion using bat’s avatar. VBO was significantly higher when steering through VE, as opposed to steering without a virtual body and exposing to involuntary movement through VE. With our research prototype, users will be able to navigate with echolocation system and fly through a virtual cave.

10. Action anticipation for different sounds’ semantics

    M. Bahadori, R. Barumerli, M. Geronazzo, and 4 more authors

    Sport Sciences for Health, Sep 2019

    DOI
11. On the evaluation of head-related transfer functions with probabilistic auditory models of human sound localization

    Michele Geronazzo, Roberto Barumerli, and Federico Avanzini

    In In Proc. 23rd International Congress on Acoustics, Sep 2019

    Abs

    Understanding spatial hearing leads to implement efficient and effective auralization rendering algorithms with headphones. Two important aspects contribute to sound localization: (i) acoustic filtering of listener body, and (ii) non-acoustic factors introduced by auditory periphery. Accordingly, head-related transfer functions (HRTFs) describe users acoustics in terms of their spatial filtering. Binaural synthesis through generic HRTFs (commonly a dummy head) is the most simple solution for an auralization framework. In this scenario, a high variability in localization tasks between subjects yields to an unreliable rendering. Listener’s acoustic and perceptual characterization require HRTF modeling and auditory models predictions in order to provide an effective auralization on individual basis. Systemic comparisons of HRTF approximations and different user profiles can help to predict listener’s performances. We consider a case study on both vertical and horizontal localization with different HRTFs and two probabilistic auditory models. In our analysis, spatial audio rendering with non-individual HRTFs has a special attention for its commercial relevance compared to unpractical and questionable use of individual HRTFs.

12. Auditory models comparison for horizontal localization of concurrent speakers in adverse acoustic scenarios

    Roberto Barumerli, Andrea Almenari, Michele Geronazzo, and 2 more authors

    In In Proc. 23rd International Congress on Acoustics, Sep 2019

    Abs

    This paper aims at comparing and reproducing the predictions of two public available computational auditory models for speaker localization in different simulated environments. The direction-of-arrival (DOA) of sound sources in the horizontal plane can be extracted by using binaural spatial cues from room and user acoustics. Since our predictions consider the specificity of both models at the level of peripheral processing, the proposed solution for DOA extraction also provides a common multi-conditional training for the Gaussian Mixture Model (GMM) approach. A set of acoustic simulations of adverse conditions (i.e. multi speakers or high reverberant scenarios) supports the evaluation phase on robustness of the synthetic auditory process. Our analysis reproduces two case studies from the scientific literature in order to investigate the reliability of localization predictions in the frontal horizontal plane. Finally, a newly defined acoustic scenario allows to identify differences between auditory models outcome in the entire horizontal plane. The results show a good agreement with previous literature and our machine learning approach emphasizes peculiarities of each approach for auditory peripheral processing.

13. MSM sel – HRTF selection tool, https://github.com/msmhrtf/sel - a Matlab framework for HRTF personalization

    Michele Geronazzo, Alberto Bedin, Enrico Peruch, and 1 more author

    2019

2018

1. TASLP

   Do we need individual head-related transfer functions for vertical localization? The case study of a spectral notch distance metric

   Michele Geronazzo, Simone Spagnol, and Federico Avanzini

   IEEE/ACM Transactions on Audio, Speech, and Language Processing, Jul 2018

   Abs DOI

   This paper deals with the issue of individualizing the head-related transfer function (HRTF) rendering process for auditory elevation perception. Is it possible to find a nonindividual, personalized HRTF set that allows a listener to have an equally accurate localization performance than with his/her individual HRTFs? We propose a psychoacoustically motivated, anthropometry based mismatch function between HRTF pairs that exploits the close relation between the listener’s pinna geometry and localization cues. This is evaluated using an auditory model that computes a mapping between HRTF spectra and perceived spatial locations. Results on a large number of subjects in the center for image processing and integrated computing (CIPIC) and acoustics research institute (ARI) HRTF databases suggest that there exists a nonindividual HRTF set, which allows a listener to have an equally accurate vertical localization than with individual HRTFs. Furthermore, we find the optimal parameterization of the proposed mismatch function, i.e., the one that best reflects the information given by the auditory model. Our findings show that the selection procedure yields statistically significant improvements with respect to dummy-head HRTFs or random HRTF selection, with potentially high impact from an applicative point of view.

2. Sonic interactions in virtual reality: state of the art, current challenges and future directions

   Stefania Serafin, Michele Geronazzo, Niels Christian Nilsson, and 2 more authors

   IEEE Computer Graphics and Applications, 2018

   Abs DOI

   A high fidelity but efficient sound simulation is an essential element of any virtual reality experience. In recent years, several advances in hardware and software technologies are facilitating the development of immersive interactive sound rendering experiences. In this paper we present a review of the state of the art of such simulations, with a focus on the different elements such as physics based simulation of sound effects and their propagation in space to binaural rendering. We present how the different elements of the sound design pipeline have been addressed in the literature, trying to find the trade-off between accuracy and plausibility. Recent applications and current challenges are also presented.

3. Proceedings of the 2018 IEEE 4th VR Workshop on Sonic Interactions for Virtual Environments (SIVE)

   Michele Geronazzo, Amalia De Götzen, Cumhur Erkut, and 4 more authors

   Mar 2018
4. Immersive auralization using headphones

   Michele Geronazzo

   In Encyclopedia of Computer Graphics and Games, 2018

   Abs DOI

   Earphones; Headphone; Headphone impulse response; Headphone transfer function; Headphones; Headset Headphones are electro-acoustic transducers able to convert two electric output channels into two...

5. User acoustics with head-related transfer functions

   Michele Geronazzo

   In Encyclopedia of Computer Graphics and Games, 2018

   Abs DOI

   Binaural hearing; Binaural sound; Head-related impulse response; Head-related transfer function; HRIR; HRTF; ILD; ITD; Spatial hearing; User acoustics Head-related impulse responses (HRIRs) or...

6. Sound spatialization

   Michele Geronazzo

   In Encyclopedia of Computer Graphics and Games, 2018

   Abs DOI

   Auralization; Binuaral headphone reproduction; Loud-speaker reproduction; Room acoustics; Room response equalization; Sound spatialization; Spatial room acoustic; Spatial room impulse response; Virtual...

7. ISMAR

   The impact of an accurate vertical localization with HRTFs on short explorations of immersive virtual reality scenarios

   Michele Geronazzo, Erik Sikström, Jari Kleimola, and 3 more authors

   In Proc. 17th IEEE/ACM Int. Symposium on Mixed and Augmented Reality (ISMAR), Oct 2018

   DOI
8. Localization in Elevation with Non-Individual Head-Related Transfer Functions: Comparing Predictions of Two Auditory Models

   Roberto Barumerli, Michele Geronazzo, and Federico Avanzini

   In 2018 26th European Signal Processing Conference (EUSIPCO), Sep 2018

   ISSN: 2076-1465

   Abs DOI

   This paper explores the limits of human localization of sound sources when listening with non-individual Head-Related Transfer Functions (HRTFs), by simulating performances of a localization task in the mid-sagittal plane. Computational simulations are performed with the CIPIC HRTF database using two different auditory models which mimic human hearing processing from a functional point of view. Our methodology investigates the opportunity of using virtual experiments instead of time- and resource- demanding psychoacoustic tests, which could also lead to potentially unreliable results. Four different perceptual metrics were implemented in order to identify relevant differences between auditory models in a selection problem of best-available non-individual HRTFs. Results report a high correlation between the two models denoting an overall similar trend, however, we discuss discrepancies in the predictions which should be carefully considered for the applicability of our methodology to the HRTF selection problem.

9. Virtual reality exploration with different head-related transfer functions

   Erik Sikström, Michele Geronazzo, Jari Kleimola, and 3 more authors

   In Proc. 15th Int. Conf. Sound and Music Computing (SMC 2018), Jul 2018
10. Navigate as a bat. real-time echolocation system in virtual reality

    Anastassia Andreasen, Jelizaveta Zovnercuka, Kristian Konovalovs, and 3 more authors

    In Proc. 15th Int. Conf. Sound and Music Computing (SMC 2018), Jul 2018
11. Round robin comparison of inter-laboratory HRTF measurements – assessment with an auditory model for elevation

    Roberto Barumerli, Michele Geronazzo, and Federico Avanzini

    In Proc. of IEEE 4th VR Workshop on Sonic Interactions for Virtual Environments (SIVE18), Mar 2018

    DOI
12. An educational experience with motor planning and sound semantics in virtual audio reality

    Michele Geronazzo, Francesca Nardello, and Paola Cesari

    In Proc. of IEEE 4th VR Workshop on Sonic Interactions for Virtual Environments (SIVE18), Mar 2018

    DOI
13. Tecnologie per l’interazione sonora in contesti di realtà virtuale e aumentata immersiva

    Michele Geronazzo

    In Proc. XXII Colloquio di Informatica Musicale (XXII CIM), Nov 2018
14. Tecnologie per la didattica musicale: un’esperienza con la realtà virtuale

    Michele Geronazzo, Edoardo Degli Innocenti, Rolf Nordahl, and 2 more authors

    In Proc. XXII Colloquio di Informatica Musicale (XXII CIM), Nov 2018
15. Hoba-vr: hrtf on demand for binaural audio in immersive virtual reality environments

    Michele Geronazzo, Jari Kleimola, Erik Sikström, and 3 more authors

    In , May 2018

    Abs

    One of the main challenges of spatial audio rendering in headphones is the personalization of the so-called head-related transfer functions (HRTFs). HRTFs capture the listener’s acoustic effects supporting immersive and realistic virtual reality (VR) contexts. This e-brief presents the HOBA-VR framework that provides a full-body VR experience with personalized HRTFs that were individually selected on demand based on anthropometric data (pinnae shapes). The proposed WAVH transfer format allows...

16. When sounds convey emotions: sound localization and action pre-planning

    Paola Cesari and Michele Geronazzo

    In MeeTo – From moving bodies to interactive minds, May 2018

    Abs

    Understanding how people plan their action in relation to a detection of an external sound source is the main aim of this experiment. Several evidences corroborate the strict link between sound and action supporting the idea of the motor system involvement during sound perception. In this research we focus our attention on how individuals react to sounds that reach the space close to their body, the so-called peri-personal space (PPS). Thirty-six individuals were tested and by means of kinematics (MX Ultranet Vicon) and EMG (zero wire System) systems we extracted the individual premotor reaction time to quantify movement planning and action preparation. Subjects were equipped with Hefio headphone individually calibrated for listening to sounds selected from the International Affective Digitized Sounds (IADS) inducing positive negative and neutral emotions. Participants were instructed to keep a standing posture while listening to the sounds having looming characteristics and to raise their arms as fast as possible once the sound stopped. The analysis considered 5 sounds (2 pleasant 2 unpleasant and 1 neutral) stopping at 5 different distances. After listening to each sound they were instructed to indicate with their right index finger on their left upper arm extended forward where the sound stopped (distance estimation). Premotor reaction time modulated significantly with the distances at which the sounds stopped with an accuracy of few centimeters. Individuals were systematically faster at each distance when reacting to a neutral sound when compared to sounds carrying semantics. For distance estimation individuals were highly precise in locating the distances while the neutral sound was systematically perceived as the closest at each distance when compared with the semantic sounds. The results evidenced the role of sound semantic decoding in action preparation and localization.

2017

1. Proceedings of the 2017 IEEE 3rd VR Workshop on Sonic Interactions for Virtual Environments (SIVE)

   Stefania Serafin, Rolf Nordahl, Amalia De Götzen, and 5 more authors

   Mar 2017
2. Improving elevation perception with a tool for image-guided head-related transfer function selection

   Michele Geronazzo, Enrico Peruch, Fabio Prandoni, and 1 more author

   In Proc. of the 20th Int. Conference on Digital Audio Effects (DAFx-17), Sep 2017
3. La cinematica del rachide durante l’estricazione: protocollo di ricerca e studio di fattibilità

   A. Gabrieli, F. Nardello, M. Liberto, and 8 more authors

   In , Oct 2017

2016

1. Auditory navigation with a tubular acoustic model for interactive distance cues and personalized head-related transfer functions

   Michele Geronazzo, Federico Avanzini, and Federico Fontana

   Journal on Multimodal User Interfaces, Sep 2016

   Abs DOI

   This paper presents a novel spatial auditory display that combines a virtual environment based on a Digital Waveguide Mesh (DWM) model of a small tubular shape with a binaural rendering system with personalized head-related transfer functions (HRTFs) allowing interactive selection of absolute 3D spatial cues of direction as well as egocentric distance. The tube metaphor in particular minimizes loudness changes with distance, providing mainly direct-to-reverberant and spectral cues. The proposed display was assessed through a target-reaching task where participants explore a 2D virtual map with a pen tablet and hit a sound source (the target) using auditory information only; subjective time to hit and traveled distance were analyzed for three experiments. The first one aimed at assessing the proposed HRTF selection method for personalization and dimensionality of the reaching task, with particular attention to elevation perception; we showed that most subjects performed better when they had to reach a vertically unbounded (2D) rather then an elevated (3D) target. The second experiment analyzed interaction between the tube metaphor and HRTF showing a dominant effect of DWM model over binaural rendering. In the last experiment, participants using absolute distance cues from the tube model performed comparably well to when they could rely on more robust, although relative, intensity cues. These results suggest that participants made proficient use of both binaural and reverberation cues during the task, displayed as part of a coherent 3D sound model, in spite of the known complexity of use of both such cues. HRTF personalization was beneficial for participants who were able to perceive vertical dimension of a virtual sound. Further work is needed to add full physical consistency to the proposed auditory display.

2. Influence of voxelization on finite difference time domain simulations of head-related transfer functions

   Sebastian Prepeliță, Michele Geronazzo, Federico Avanzini, and 1 more author

   The Journal of the Acoustical Society of America, May 2016

   Abs DOI

   The scattering around the human pinna that is captured by the Head-Related Transfer Functions (HRTFs) is a complex problem that creates uncertainties in both acoustical measurements and simulations. Within the simulation framework of Finite Difference Time Domain(FDTD) with axis-aligned staircase boundaries resulting from a voxelization process, the voxelization-based uncertainty propagating in the HRTF-captured sound field is quantified for one solid and two surface voxelization algorithms. Simulated results utilizing a laser-scanned mesh of Knowles Electronics Manikin for Acoustic Research (KEMAR) show that in the context of complex geometries with local topology comparable to grid spacing such as the human pinna, the voxelization-related uncertainties in simulations emerge at lower frequencies than the generally used accuracy bandwidths. Numerical simulations show that the voxelization process induces both random error and algorithm-dependent bias in the simulated HRTF spectral features. Frequencies fr below which the random error is bounded by various dB thresholds are estimated and predicted. Particular shortcomings of the used voxelization algorithms are identified and the influence of the surface impedance on the induced errors is studied. Simulations are also validated against measurements.

3. Localization of self-generated synthetic footstep sounds on different walked-upon materials through headphones

   Luca Turchet, Simone Spagnol, Michele Geronazzo, and 1 more author

   Virtual Reality, Mar 2016

   Abs DOI

   This paper focuses on the localization of footstep sounds interactively generated during walking and provided through headphones. Three distinct experiments were conducted in a laboratory involving a pair of sandals enhanced with pressure sensors and a footstep synthesizer capable of simulating two typologies of surface materials: solid (e.g., wood) and aggregate (e.g., gravel). Different sound delivery methods (mono, stereo, binaural) as well as several surface materials, in the presence or absence of concurrent contextual auditory information provided as soundscapes, were evaluated in a vertical localization task. Results showed that solid surfaces were localized significantly farther from the walker’s feet than the aggregate ones. This effect was independent of the used rendering technique, of the presence of soundscapes, and of merely temporal or spectral attributes of sound. The effect is hypothesized to be due to a semantic conflict between auditory and haptic information such that the higher the semantic incongruence the greater the distance of the perceived sound source from the feet. The presented results contribute to the development of further knowledge toward a basis for the design of continuous multimodal feedback in virtual reality applications .

4. IJHCS

   Interactive spatial sonification for non-visual exploration of virtual maps

   Michele Geronazzo, Alberto Bedin, Luca Brayda, and 2 more authors

   International Journal of Human-Computer Studies, Jan 2016

   Abs DOI

   This paper presents a multimodal interactive system for non-visual (auditory-haptic) exploration of virtual maps. The system is able to display haptically the height profile of a map, through a tactile mouse. Moreover, spatial auditory information is provided in the form of virtual anchor sounds located in specific points of the map, and delivered through headphones using customized Head-Related Transfer Functions (HRTFs). The validity of the proposed approach is investigated through two experiments on non-visual exploration of virtual maps. The first experiment has a preliminary nature and is aimed at assessing the effectiveness and the complementarity of auditory and haptic information in a goal reaching task. The second experiment investigates the potential of the system in providing subjects with spatial knowledge: specifically in helping with the construction of a cognitive map depicting simple geometrical objects. Results from both experiments show that the proposed concept, design, and implementation allow to effectively exploit the complementary natures of the “proximal” haptic modality and the “distal” auditory modality. Implications for orientation & mobility (O&M) protocols for visually impaired subjects are discussed.

5. A motion based setup for peri-personal space estimation with virtual auditory displays

   Michele Geronazzo and Paola Cesari

   In Proc. 22nd ACM Symposium on Virtual Reality Software and Technology (VRST 2016), Nov 2016

   DOI
6. Acoustic selfies for extraction of external ear features in mobile audio augmented reality

   Michele Geronazzo, Jacopo Fantin, Giacomo Sorato, and 2 more authors

   In Proc. 22nd ACM Symposium on Virtual Reality Software and Technology (VRST 2016), Nov 2016

   DOI
7. The selfear project: a mobile application for low-cost pinna-related transfer function acquisition

   Michele Geronazzo, Jacopo Fantin, Giacomo Sorato, and 2 more authors

   In Proc. 13th Int. Conf. Sound and Music Computing (SMC 2016), Sep 2016
8. Selfie acustiche con il progetto selfear: un’applicazione mobile per l’acquisizione a basso costo di pinna-related transfer function

   Michele Geronazzo, Jacopo Fantin, Giacomo Sorato, and 2 more authors

   In Proc. XXI Colloquio di Informatica Musicale (XXI CIM), Sep 2016
9. Audio augmented reality headset: a product requirements research in today’s available technologies

   Guido Baldovino and Michele Geronazzo

   Aug 2016

2015

1. Absence of modulatory action on haptic height perception with musical pitch

   Michele Geronazzo, Federico Avanzini, and Massimo Grassi

   Front. Psychol., 2015

   Abs DOI

   Although acoustic frequency is not a spatial property of physical objects, in common language, pitch, i.e., the psychological correlated of frequency, is often labeled spatially (i.e., “high in pitch” or “low in pitch”). Pitch-height is known to modulate (and interact with) the response of participants when they are asked to judge spatial properties of non-auditory stimuli (e.g., visual) in a variety of behavioral tasks. In the current study we investigated whether the modulatory action of pitch-height extended to the haptic estimation of height of a virtual step.We implemented a HW/SW setup which is able to render virtual 3D objects (stair-steps) haptically through a PHANTOM device, and to provide real-time continuous auditory feedback depending on the user interaction with the object. The haptic exploration was associated with a sinusoidal tone whose pitch varied as a function of the interaction point’s height within (i) a narrower and (ii) a wider pitch range, or (iii) a random pitch variation acting as a control audio condition. Explorations were also performed with no sound (haptic only). Participants were instructed to explore the virtual step freely, and to communicate height estimation by opening their thumb and index finger to mimic the step riser height, or verbally by reporting the height in centimeters of the step riser. We analyzed the role of musical expertise by dividing participants into non musicians and musicians. Results showed no effects of musical pitch on high-realistic haptic feedback. Overall there is no difference between the two groups in the proposed multimodal conditions. Additionally, we observed a different haptic response distribution between musicians and non musicians when estimations of the auditory conditions are matched with estimations in the no sound condition.

2. The external ear acoustics: a mixed structural modeling approach in virtual auditory displays

   Michele Geronazzo

   J. of the Italian Society of Acoustics (RIA), 2015
3. Proceedings of the 2015 IEEE 2nd VR Workshop on Sonic Interactions for Virtual Environments (SIVE)

   Michele Geronazzo, Federico Avanzini, Stefania Serafin, and 3 more authors

   2015
4. Report on the in-vehicle auditory interactions workshop: taxonomy, challenges, and approaches.

   M. Jeon and al.

   In Proc. of the 7th Int. Conf. on Automotive User Interfaces and Interactive Vehicular Applications (AutoUI 2015), Sep 2015
5. Use of personalized binaural audio and interactive distance cues in an auditory goal-reaching task

   Michele Geronazzo, Federico Avanzini, and Federico Fontana

   In Proc. of the 21st Int. Conf. on Auditory Display (ICAD 2015), Jul 2015
6. Coloration metrics for headphone equalization

   Braxton Boren, Michele Geronazzo, Fabian Brinkmann, and 1 more author

   In Proc. of the 21st Int. Conf. on Auditory Display (ICAD 2015), Jul 2015
7. Evaluating vertical localization performance of 3D sound rendering models with a perceptual metric

   M. Geronazzo, A. Carraro, and F. Avanzini

   In 2015 IEEE 2nd VR Workshop on Sonic Interactions for Virtual Environments (SIVE), Mar 2015

   Abs DOI

   The head-related transfer functions (HRTFs) describe individual acoustic transformation that sound sources undergo due to human anatomy before arriving at the left and right tympanic membranes. The resulting spectral modifications are the main localization cues for elevation detection in space. In this paper, synthetic HRTF mod- els able to render the vertical spatial dimension in virtual auditory displays, are evaluated via auditory models. Perceptually-motivated metrics describe the output of 4 virtual experiments that numeri- cally simulate real listening experiments for 20 virtual subjects. The current implementation considers a limited set of parameters for a structural model of the pinna acting as a proof-of-concept of such approach. Accordingly, results confirm that the research framework is a flexible tool for systematic evaluation of different instances of structural model.

8. Personalization support for binaural headphone reproduction in web browsers

   Michele Geronazzo, Jari Kleimola, and Piotr Majdak

   In Proc. 1st Web Audio Conference, Jan 2015
9. Subjective evaluation of a low-order parametric filter model of the pinna for binaural sound rendering

   Sandro Scaiella, Simone Spagnol, Michele Geronazzo, and 1 more author

   In 22nd Int. Congress on Sound and Vibration (ICSV22), Jul 2015
10. HOBA – Hrtfs On-demand for Binaural Audio, https://github.com/hoba3d - a web framework for personalized 3D audio rendering

    Michele Geronazzo and Jari Kleimola

    2015
11. Sonic interaction in virtual environments

    Stefania Serafin, Rolf Nordahl, Amalia De Götzen, and 3 more authors

    In 2015 IEEE 2nd VR Workshop on Sonic Interactions for Virtual Environments (SIVE), Mar 2015

    DOI
12. L’acustica dell’orecchio esterno: un approccio a modelli strutturali misti per display uditivi virtuali

    Michele Geronazzo

    Rivista Italiana di Acustica, 2015
13. Informatica e Interdisciplinarità per l’Apprendimento Aumentato Attraverso Tecnologie Audio

    Michele Geronazzo

    Jul 2015

2014

1. Synthetic individual binaural audio delivery by pinna image processing

   Simone Spagnol, Michele Geronazzo, Davide Rocchesso, and 1 more author

   Int. J. of Pervasive Computing and Communications, 2014

   DOI
2. Proceedings of the XX Colloquium on Music Informatics

   Michele Geronazzo and Simone Spagnol

   Dec 2014
3. Phona: a public dataset of measured headphone transfer functions

   Braxton B. Boren, Michele Geronazzo, Piotr Majdak, and 1 more author

   In Proc. 137th Conv. Audio Eng. Society, Oct 2014

   Abs

   A dataset of measured headphone transfer functions (HpTFs), the Princeton Headphone Open Archive (PHOnA), is presented. Extensive studies of HpTFs have been conducted for the past twenty years, each requiring a separate set of measurements, but this data has not yet been publicly shared. PHOnA aggregates HpTFs from different laboratories, including measurements for multiple different headphones, subjects, and repositionings of headphones for each subject. The dataset uses the spatially...

4. Multimodal exploration of virtual objects with a spatialized anchor sound

   Michele Geronazzo, Alberto Bedin, Luca Brayda, and 1 more author

   In Proc. 55th Int. Conf. Audio Eng. Society, Spatial Audio, Aug 2014

   Abs

   A multimodal interactive system for audio-haptic integration is presented in this paper. Preliminary subjective tests with a virtual reality setup were conducted with the goal of interpreting cognitive mechanisms and improving performances in orientation & mobility protocols for visually impaired subjects, where spatial representations need to be developed using residual sensory channels. An object recognition experiment was performed in order to investigate the contribution of dynamic...

5. Enhancing vertical localization with image-guided selection of non-individual head-related transfer functions

   Michele Geronazzo, Simone Spagnol, Alberto Bedin, and 1 more author

   In IEEE Int. Conf. on Acoust. Speech Signal Process. (ICASSP 2014), May 2014

   Abs DOI

   A novel approach to the selection of generic head-related transfer functions (HRTFs) for binaural audio rendering through headphones is formalized and described in this paper. A reflection model applied to the user’s ear picture allows to extract the relevant anthropometric cues that are used for selecting two HRTF sets in a database fitting that user, whose localization performances are evaluated in a complete psychoacoustic experiment. The proposed selection increases the average elevation performances of 17% (with a peak of 34%) with respect to generic HRTFs from an anthropomorphic mannequin. It also significantly enhances externalization and reduces the number of up/down reversals.

6. Audio 3d e ancoraggio sonoro per l’esplorazione multimodale di ambienti virtuali

   Michele Geronazzo, Luca Brayda, Alberto Bedin, and 1 more author

   In Proc. XX Colloquium on Musical Informatics (XX CIM 2014), Nov 2014
7. Valutazione parametrica di un modello strutturale di orecchio esterno per il rendering binaurale del suono

   Sandro Scaiella, Simone Spagnol, Michele Geronazzo, and 1 more author

   In Proc. XX Colloquium on Musical Informatics (XX CIM 2014), Nov 2014
8. Un nuovo approccio a modelli strutturali misti per la sintesi e la personalizzazione di hrtf

   Michele Geronazzo, Simone Spagnol, and Federico Avanzini

   In Proc. 41st Convegno Nazionale Associazione Italiana di Acustica (41 AIA 2014), Jun 2014
9. SOFA – Spatially Oriented Format for Acoustics, http://www.sofaconventions.org/, headphone support for standardization

   Michele Geronazzo

   2014
10. BT-DEI HpIRs in http://padva.dei.unipd.it , a public headphone impulse response database

    Michele Geronazzo

    2014
11. Mixed structural models for 3D audio in virtual environments

    Michele Geronazzo

    Apr 2014

2013

1. On the relation between pinna reflection patterns and head-related transfer function features

   Simone Spagnol, Michele Geronazzo, and Federico Avanzini

   IEEE Transactions on Audio, Speech, and Language Processing, Mar 2013

   Abs DOI

   This paper studies the relationship between head-related transfer functions (HRTFs) and pinna reflection patterns in the frontal hemispace. A pre-processed database of HRTFs allows extraction of up to three spectral notches from each response taken in the median sagittal plane. Ray-tracing analysis performed on the obtained notches’ central frequencies is compared with a set of possible reflection surfaces directly recognizeable from the corresponding pinna picture. Results of such analysis are discussed in terms of the reflection coefficient sign, which is found to be most likely negative. Based on this finding, a model for real-time HRTF synthesis that allows to control separately the evolution of different acoustic phenomena such as head diffraction, ear resonances, and reflections is proposed through the design of distinct filter blocks. Parameters to be fed to the model are derived either from analysis or from specific anthropometric features of the subject. Finally, objective evaluations of reconstructed HRTFs in the chosen spatial range are performed through spectral distortion measurements.

2. Extraction of pinna features for customized binaural audio delivery on mobile devices

   Simone Spagnol, Michele Geronazzo, Davide Rocchesso, and 1 more author

   In Proc. 11th International Conference on Advances in Mobile Computing & Multimedia (MoMM’13), Dec 2013
3. Influence of auditory pitch on haptic estimation of spatial height

   Michele Geronazzo, Federico Avanzini, and Massimo Grassi

   In Proc. 10th International Symposium on Computer Music Multidisciplinary Research (CMMR’13), 2013

   Abs

   This paper presents an experiment aimed at assessing the influence of auditory feedback on haptic estimation of size. Experimental subjects were instructed to explore a virtual 3D object (a stair-step) with a haptic device, and to return a verbal estimate of the step riser height. Haptic exploration was accompanied with a real-time generated sinusoid whose pitch varied as a function of the interaction point’s height within two different ranges. Experimental results show that the haptic estimation is robust and accurate regardless the frequency range of the accompanying sound.

4. Automatic extraction of pinna edges for binaural audio customization

   Simone Spagnol, Davide Rocchesso, Michele Geronazzo, and 1 more author

   In Proc. IEEE Int. Work. Multi. Signal Process. (MMSP 2013), Oct 2013

   Abs DOI

   The contribution of the external ear to the head-related transfer function (HRTF) heavily depends on the listener’s unique anthropometry. In particular, the shape of the most prominent contours of the pinna defines the frequency location of the HRTF spectral notches along the elevation of the sound source. This paper addresses the issue of automatically estimating the location of pinna edges starting from a set of pictures produced by a multi-flash imaging device. A basic image processing algorithm designed to obtain the principal edges and their distance from the ear canal entrance is described. The effectiveness of the developed hardware and software is preliminarily evaluated on a small number of test subjects.

5. Mixed structural modeling of head-related transfer functions for customized binaural audio delivery

   Michele Geronazzo, Simone Spagnol, and Federico Avanzini

   In Proc. 18th Int. Conf. Digital Signal Process. (DSP 2013), Jul 2013

   Abs DOI

   A novel approach to the modeling of head-related transfer functions (HRTFs) for binaural audio rendering is formalized and described in this paper. Mixed structural modeling (MSM) can be seen as the generalization and extension of the structural modeling approach first defined by Brown and Duda back in 1998. Possible solutions for building partial HRTFs (pHRTFs) of the head, torso, and pinna of a specific listener are first described and then used in the construction of two possible mixed structural models of a KEMAR mannequin. Thanks to the flexibility of the MSM approach, an exponential number of solutions for building custom binaural audio displays can be considered and evaluated, the final aim of the process being the achievement of a HRTF model fully customizable by the listener.

6. A standardized repository of head-related and headphone impulse response data

   Michele Geronazzo, Fabrizio Granza, Simone Spagnol, and 1 more author

   In Proc. 134th Conv. Audio Eng. Society, 2013
7. A modular framework for the analysis and synthesis of head-related transfer functions

   Michele Geronazzo, Simone Spagnol, and Federico Avanzini

   In Proc. 134th Conv. Audio Eng. Society, May 2013

   Abs

   The paper gives an overview of a number of tools for the analysis and synthesis of head-related transfer functions (HRTFs) that we have developed in the past four years at the Department of Information Engineering, University of Padova, Italy. The main objective of our study in this context is the progressive development of a collection of algorithms for the construction of a totally synthetic personal HRTF set replacing both cumbersome and tedious individual HRTF measurements and the exploitation of inaccurate non-individual HRTF sets. Our research methodology is highlighted, along with the multiple possibilities of present and future research offered by such tools.

2012

1. Hearing distance: a low-cost model for near-field binaural effects

   Simone Spagnol, Michele Geronazzo, and Federico Avanzini

   In Proc. EUSIPCO 2012 Conf., Sep 2012

   Abs

   An extremely low-order filter model for source distance rendering in binaural reproduction is proposed in this paper. The main purpose of such model is to cheaply simulate the effect that source-listener distance has on the sound waves arriving at the ears in the near field, a region where the relation between sound pressure and distance is both highly frequency-dependent and nonlinear. The reference for the model is based on an analytical description of a spherical head response, appropriately filtered out so as to include distance-dependent patterns only. To this regard, the model is objectively seen to provide an excellent fit in the whole near field, despite its simplicity.

2. Employing spatial sonification of target motion in tracking exercises

   Simone Spagnol, Michele Geronazzo, Federico Avanzini, and 2 more authors

   In Proc. 9th Int. Conf. Sound and Music Computing (SMC 2012), Jul 2012

   Abs

   This paper presents the results of an experiment in which the effect of spatial sonification of a moving target on the user’s performance during the execution of basic tracking exercises was investigated. Our starting hypothesis is that a properly designed multimodal continuous feedback could be used to represent temporal and spatial information that can in turn improve performance and motor learning of simple target following tasks. Sixteen subjects were asked to track the horizontal movement of a circular visual target by controlling an input device with their hand. Two different continuous task-related auditory feedback modalities were considered, both simulating the sound of a rolling ball, the only difference between them being the presence or absence of binaural spatialization of the target’s position. Results demonstrate how spatial auditory feedback significantly decreases the average tracking error with respect to visual feedback alone, contrarily to monophonic feedback. It was thus found how spatial information provided through sound in addition to visual feedback helps subjects improving their performance.

3. Model-based customized binaural reproduction through headphones

   Michele Geronazzo, Simone Spagnol, Davide Rocchesso, and 1 more author

   In Proc. XIX Colloquio di Informatica Musicale (XIX CIM), Nov 2012

   Abs

   Generalized head-related transfer functions (HRTFs) represent a cheap and straightforward mean of providing 3D rendering in headphone reproduction. However, they are known to produce evident sound localization errors, including incorrect perception of elevation, front-back reversals, and lack of externalization, especially when head tracking is not utilized in the reproduction . Therefore, individual anthropometric features have a key role in characterizing HRTFs. On the other hand, HRTF measurements on a significant number of subjects are both expensive and inconvenient. This short paper briefly presents a structural HRTF model that, if properly rendered through a proposed hardware (wireless headphones augmented with motion and vision sensors), can be used for an efficient and immersive sound reproduction. Special care is reserved to the contribution of the external ear to the HRTF: data and results collected to date by the authors allow parametrization of the model according to individual anthropometric data, which in turn can be automatically estimated through straightforward image analysis. The proposed hardware and software can be used to render scenes with multiple audiovisual objects in a number of contexts such as computer games, cinema, edutainment, and many others.

2011

1. A head-related transfer function model for real-time customized 3-d sound rendering

   Michele Geronazzo, Simone Spagnol, and Federico Avanzini

   In Proc. 7th Int. Conf. on Signal Image Technology & Internet-Based Systems (SITIS 2011), Dec 2011

   Abs DOI

   This paper addresses the problem of modeling head-related transfer functions (HRTFs) for 3-D audio rendering in the front hemisphere. Following a structural approach, we build a model for real-time HRTF synthesis which allows to control separately the evolution of different acoustic phenomena such as head diffraction, ear resonances, and reflections through the design of distinct filter blocks. Parameters to be fed to the model are both derived from mean spectral features in a collection of measured HRTFs and anthropometric features of the specific subject (taken from a photograph of his/her outer ear), hence allowing model customization. Visual analysis of the synthesized HRTFs reveals a convincing correspondence between original and reconstructed spectral features in the chosen spatial range. Furthermore, a possible experimental setup for dynamic psychoacoustical evaluation of such model is depicted.

2. Customized 3D Sound for Innovative Interaction Design

   Michele Geronazzo, Simone Spagnol, and Federico Avanzini

   In Proc. SMC-HCI Work., CHItaly 2011 Conf., Sep 2011

   Abs

   This paper considers the impact of binaural 3D audio on several kinds of applications, classified according to their degree of body immersion and their own coordinate system deviation from a physical condition. A model for sound spatialization, which includes additional features with respect to existing systems, is introduced. A significant reduction of computational costs is allowed by model parametrization according to anthropometric information of the user and audio processing through low-order filters, thus resulting affordable for several kinds of devices. According to the following examination, this approach to 3D sound rendering can grant a transversal enrichment to the CHI research purposes, in reference to content creation and adaptation, resourceful delivery and augmented media presentation. In several contexts where personalized spatial sound reproduction is a central requirement, the quality of the immersive experience could only benefit from this sort of adaptable and modular system.

2010

1. Fitting pinna-related transfer functions to anthropometry for binaural sound rendering

   Simone Spagnol, Michele Geronazzo, and Federico Avanzini

   In Proc. IEEE Int. Work. Multi. Signal Process. (MMSP’10), Oct 2010

   Abs DOI

   This paper faces the general problem of modeling pinna-related transfer functions (PRTFs) for 3-D sound rendering. Following a structural approach, we aim at constructing a model for PRTF synthesis which allows to control separately the evolution of ear resonances and spectral notches through the design of two distinct filter blocks. Taking such model as endpoint, we propose a method based on the McAulay-Quatieri partial tracking algorithm to extract the frequencies of the most important spectral notches. Ray-tracing analysis performed on the so obtained tracks reveals a convincing correspondence between extracted frequencies and pinna geometry of a bunch of subjects.

2. Estimation and modeling of pinna-related transfer functions

   Michele Geronazzo, Simone Spagnol, and Federico Avanzini

   In Proc. of the 13th Int. Conference on Digital Audio Effects (DAFx-10), Sep 2010
3. Structural modeling of pinna-related transfer functions for 3-d sound rendering

   Simone Spagnol, Michele Geronazzo, and Federico Avanzini

   In Proc. XVIII Colloquio di Informatica Musicale (XVIII CIM), Oct 2010

   Abs

   This paper considers the general problem of modeling pinna-related transfer functions (PRTFs) for 3-D sound rendering. Following a structural approach, we present an algorithm for the decomposition of PRTFs into ear resonances and frequency notches due to reflections over pinna cavities and exploit it in order to deliver a method to extract the frequencies of the most important spectral notches. Ray-tracing analysis reveals a convincing correspondence between extracted frequencies and pinna cavities of a bunch of subjects. We then propose a model for PRTF synthesis which allows to control separately the evolution of resonances and spectral notches through the design of two distinct filter blocks. The resulting model is suitable for future integration into a structural head-related transfer function model, and for parametrization over anthropometrical measurements of a wide range of subjects.

4. Structural modeling of pinna-related transfer functions

   Simone Spagnol, Michele Geronazzo, and Federico Avanzini

   In Proc. 7th Int. Conf. Sound and Music Computing (SMC 2010), Jul 2010

   Abs

   This paper faces the general problem of modeling pinna-related transfer functions (PRTFs) for 3-D sound rendering. Following a structural modus operandi, we exploit an algorithm for the decomposition of PRTFs into ear resonances and frequency notches due to reflections over pinna cavities in order to deliver a method to extract the frequencies of the most important spectral notches. Ray-tracing analysis reveals a convincing correspondence between extracted frequencies and pinna cavities of a bunch of subjects. We then propose a model for PRTF synthesis which allows to control separately the evolution of resonances and spectral notches through the design of two distinct filter blocks. The resulting model is suitable for future integration into a structural head-related transfer function model, and for parametrization over anthropometrical measurements of a wide range of subjects.

2009

1. Modello strutturale di orecchio esterno per il rendering audio 3D

   Michele Geronazzo

   2009