Adam Levenson‘s career has evolved from performing as a classically trained percussionist, then serving as a composer and sound designer in the burgeoning multimedia industry in the early 90’s—“before anyone knew what a video game sound designer even was,” as he put it—followed by sound and music director positions at Interplay, Electronic Arts, and Shiny Entertainment. During his tenure as Senior Director at Activision from 2006 - 2011, Adam established and managed both the Central Audio and Central Talent teams supporting work on major franchises. In 2011, he launched Levenson Artists Agency, providing representation to clients such as celebrity composers Hans Zimmer and Lorne Balfe. Now, Adam oversees the busy operations of SomaTone’s creative teams in Emeryville and Vancouver while focusing on leading the company’s growth and expansion into new arenas in his latest role as the COO of SomaTone Interactive.
In this latest Game Audio Artistry article, Adam examines how new sensorial technologies can take the immersive gameplay experience to heightened and even more engaging levels of player fascination and enjoyment.
Immersification noun \i-ˈmərs-ə-fə-ˈkā-shən\: the process of creating enhanced involvement in a particular activity
The Beginnings
In 1974, the blockbuster disaster film Earthquake wowed movie theater audiences with Sensurround effects. Low frequency sounds were pumped through big Cerwin-Vega subwoofers that sent vibrations through the seats during the film’s tremor scenes. It physically involved theatergoers in the drama and, like an amusement park ride, the result was thrilling. In some theaters, Sensurround effects sent pieces of ceiling plaster falling into the audience and shook seats in neighboring theaters showing The Godfather Part II. Sensurround was audience immersion on an epic scale, a bold experiment in producing a multi-sensorial experience.
Now, 40 years later, consumers of mobile interactive entertainment are also seeking deeper engagement. Giant leaps in display technology have provided intensely vivid visual experiences, but humans have five senses, and with the advent of sophisticated digital devices with built-in functionality to stimulate those sensations, it’s now up to content creators to take full advantage and rock audiences in their virtual theater seats.
Devices
The Apple iPad Air features dual microphones, Amazon’s Fire Phone sports dynamic perspective, and Samsung Galaxy smartphones include tactile feedback. These are just a few of the offerings in the development of recent mobile technologies that provide sensory engagement. Although both smell and taste are primary and essential senses, there isn’t much being done with olfactory and gustatory digital transmissions (putting experiments such as Smell-O-Vision and Nokia’s Scentsory Phone aside). Maybe that’s OK. But entertainment technologies and content designed to stimulate hearing and touch are engaging consumers, and a resurgence of virtual reality points to a trend towards greater immersification.
Hearing
Dolby has always been at the leading edge of sound technologies that improve sound quality and envelope the listener. Even on smartphones, Dolby Digital Plus delivers virtualized surround sound through headphones or even with built-in mobile device speakers. This opens the door for app and video game developers to provide a more engaging experience. Realizing the potential means being creative with music and sound implementation. That starts with developing rich audio environments, including layers of ambient sound, music, voice performances, and specific sound effects to make all the game mechanics come to life.
Despite recent advances, music in games generally plays in a linear manner, just like it did when Edison invented the phonograph cylinder in the 19th century. Amazingly, visionaries in music technology have been working on concepts in “real-time composition” since the 90’s. It has been called many names like computational music and algorithmic composition, and much of this research has been academic. With vastly increased processing power, new devices may soon present an opportunity to implement fully responsive music. Music adds an emotional dimension that makes us laugh out loud at funny scenes, jump out of our seats when surprised, or get choked up during sad dialog. How much more immersive would a Star Wars game experience be if the score could build and sting with each dramatic Jedi lightsaber hit?
Touch
On one hand, the virtuality of new smartphone and tablet interfaces promotes a whole new physical language of gestures. We can push, swipe, sweep, and slide. But on the other hand, we lose the satisfaction of getting real world physical feedback when we perform the gesture. We type, press buttons, turn pages, move objects, throw switches, but for the most part, we don’t feel anything. In our mobile entertainment, explosions, cars racing by, or feats of strength that we experience visually lack any sense of physicality. Haptics technology changes all of that. Haptic effects are touch or tactile feedback events produced by actuators (a kind of motor) integrated into our devices. Video game console controllers have used haptic type feedback, or rumble effects, for years. But with newer technologies from companies like Immersion, users feel customized force or resistance as they perform virtual activities. Sensing the impact of a soccer ball kick, the recoil of a gun, or the mechanical click of a button push brings the experience to life, providing a deep level of immersification.
Ultimate Immersification
Although VR still calls to mind images of disoriented people at 90’s tech conferences stumbling around in bulky headsets, advances in virtual reality technology have reinvigorated consumer interest. Earlier this year, Facebook acquired innovators Oculus VR for $2 billion – imagine simulated social networking, and the motivation for the acquisition becomes clear. By combining high-resolution imagery with high fidelity sound and haptics feedback, VR has the potential to offer practically total immersion. The potential applications for VR range from therapeutic uses, to military training, to entertainment media, and beyond. Although VR doesn’t currently have the portability and accessibility of more mobile technologies, the renewed interest and innovation in the field do reflect growing consumer demand for immersification of our digital experiences.
Contemporary immersion relies on rich creative content like engaging stories, great music, impactful sound, and believable physics. Developers and publishers that incorporate those elements into their entertainment media have already taken the first step towards immersion. New technologies allow for the completed, multi-sensorial experience without the inconvenience of falling ceiling plaster.
The thrilling part is, it all just keeps getting more and more immersive.
Comments
