Nvidia adds path tracing audio and real-time 360 video stitching to its VRWorks SDK

Nvidia has provided VR content creators with two new powerful tools as part of the VRWorks Software Development Kit (SDK). The Audio SDK allows path-tracing of sound waves in virtual environments, in real time, and is supported in the Unreal 4 engine by Epic. The 360 video SDK can stitch together videos from 8 4K cameras in realtime, allowing for live streaming of 360 degree stereoscopic video content.VRWorks Audio provides physically accurate audio in virtual environments, simulating the way sound waves would interact with objects, materials, surfaces and spaces in the real world. Conventionally, audio in VR experiences just use the position of the sound source, the direction, and the distance. However, in the real world, the audio interacts with the amount of open space, as well as the materials used in the spaces. A long narrow corridor, a large hall with plenty of curtains, and an open field, all have different acoustic properties. Additionally, different audio sources will sound different based on your location in the environment.The VRWorks Audio SDK allows for real-time simulation of all these different environments and the ways in which various materials interact with the audio. The paths of the audio are traced, similar to how ray tracing is done for light. The acoustic model of the environment is updated in real time, without a necessity to pre-bake an acoustic image of the scene. Audio effect filters are generated in real time, and applied to the audio source waveforms. There are a set of C-APIs for integration into any engine, and integration for the Unreal 4 engine is available on GitHub. You can see it in action in the demo video below.The VRWorks 360 Video SDK has been released in public beta. The mono version is already available, and the stereo version will also be made available. In the mono version, 360 videos can be viewed by the end user by swiping across the screen. The stereo version presents two video streams to the user, one for each eye, and requires VR headsets to be viewed. The SDK was demonstrated on the V1 pro camera system by Z Cam.Kinson Loo, CEO of Z CAM said "The fact that Nvidia manages to stitch 4K 360 stereoscopic video in real time, making livestreaming possible, changes the production pipeline and enables entirely new use cases in VR."VR content creators can download the VRWorks 360 Video and Audio SDKs from the Nvidia Site.