Poznan, Poland & New York, NY (November 2, 2022)—The Music and Audio Research Lab at New York University and 3D audio recording solutions company Zylia are collaborating on the Spatial Sound Scene Description (S3D) project—an exploration of how computers can extract spatial information about the environment out of sound.
Such information, captured by Zylia’s higher-order ambisonics microphones, is potentially useful in areas such as media production, urban monitoring or security. If a person stands on a city sidewalk with their eyes closed and listens, they will hear the sounds of events happening around them: birds chirping, squirrels scurrying, people talking, doors opening, an ambulance speeding, a truck idling. In addition, they will also likely be able to perceive the location of each sound source, where it’s going, and how fast it’s moving. This rich information about the surrounding environment, if extracted by computers, could deliver a valuable description of the world.
The S3D project is funded by a grant from the National Science Foundation and has many real-world applications. For instance, hearing-impaired individuals could use them in their daily life to avoid dangerous situations in outdoor or domestic environments. Ecologists could use them to monitor and study wildlife more accurately. City agencies could better quantify traffic, construction and other urban activities. The collaborators note that what is important in all these use cases is that sound can include information about events that are not easily visible, such as sources that are small (for example, insects), far away (a distant jackhammer) or simply hidden behind another object (an incoming ambulance around a building’s corner).
Zylia Ambisonic Microphone – A Real-World Review
“We are very excited to pursue this collaboration with Zylia,” said Juan Pablo Bello, professor of music technology, computer science & engineering and director of the Music and Audio Research Lab (MARL). “There is great potential in the combination of Zylia’s cutting-edge technologies for spatial audio capture and the AI-driven solutions for acoustic scene understanding that we are developing as part of the S3D project.”
“We are delighted to be collaborating with Zylia on the S3D project,” added Agnieszka Roginska, music professor of music technology. “Together, we are significantly advancing the research in building computational models for spatial sound scene analysis and description.”
“We believe that 3D audio is a very important element in experiencing the ever-evolving modern world,” said Tomasz Żernicki, CEO of Zylia. “Scientists start to notice that also in a field of various research can highly benefit from the information contained in a sound, especially now, when the technology for capturing high-quality 3D audio is easily available.”
