VR Experience To Simulate what Music Feels like to a Person that is Deaf

November 2017 - An article from the New York Times describes a recent collaboration with Rachel Kolb, who was born deaf and with a recent cochlear implant now has partial hearing, and the VR department of the journal to create a VR experience that explores how music might feel to someone with a hearing impairment. Inspired by Rachel’s stories and using her narration of her sensations of experiencing music as an adult for the first time, the team set out to create an inviting and innovative VR experience. James Merry, an animator for the production company Squint/Opera in London, spoke of his process for creating the animations for the VR visuals, “When I work with voice-over, I’m hardly ever able to work out what’s being said from the voice track alone. I can hear when something is being said, but not so much what is being said. So I use a combination of the audio track and the transcript with timestamps. The audio waveform, which I can see on the computer screen, helps me to sync things up.” James uses hearing aids and was inspired by his impairment to use a warm and loose hand-drawn style for the VR animations. In addition to James’ animations, the team utilized Lytro’s camera technology to create a 360 degree, seamless world. Users can navigate the world with the VR headset and experience the luscious hand-drawn animation and high-resolution video with a high degree of freedom. The New York Times’s VR experience is another example of virtual reality that is inclusive of the lived experience of people with disabilities, where their reality is included in the design of the virtual environment. Source: Maureen Towey, New York Times.