Menlo Park, CA (February 23, 2017)—For the first time, Facebook engineers have revealed some of the technical details behind the platform’s new 360 audio encoding and rendering for virtual reality video in a blog post on its Code site.
In the post, software engineer Hans Fugal and Varun Nair, Facebook audio, VR, and co-founder of immersive and interactive audio technology company Two Big Ears, which Facebook acquired in 2016, describe the challenges and solutions behind the development of the Spatial Workstation tool suite. “New 360 audio encoding and rendering technology makes maintaining high-quality spatial audio throughout the pipeline—from editor to user—possible for large-scale consumption for the first time,” they write.
The April issue of PSN will feature an in-depth conversation with Fugal and Nair.
The post, “Spatial audio – bringing realistic sound to 360 video,” reveals the technical challenges encountered by the team in the search for a suitable audio codec to deliver a consistent experience across all viewing platforms.
The Spatial Workstation suite is based on what Facebook refers to as hybrid higher-order ambisonics, “an 8-channel system with rendering optimizations to incorporate the quality of higher-order ambisonics with fewer channels, ultimately saving on bandwidth.”
The Spatial Workstation can additionally output two channels of head-locked audio, enabling a static stereo track for music or voiceover, for example. “Rendering with hybrid higher-order ambisonics and head-locked audio simultaneously is a first for the industry,” write Fugal and Nair.
Facebook’s 360 audio solution is said to be very efficient: “The spatial audio renderer supports real-time experiences with less than half a millisecond of latency.”
Further, the encoder tool exports to multiple platforms. “The Rendering SDK is integrated across Facebook and Oculus video, which guarantees a unified experience from production to publication. That saves time and ensures that what you hear in production is what you publish,” the pair write.
Facebook Code
code.facebook.com
