Amon Tobin’s ISAM Uses LEDs & Virtual Mapping to Create 4D Show

On this particular Sunday, a certain electronic musician was not only playing at one of the venues, but was also unveiling the LA debut of an unprecedented visual performance that’s had bloggers, geeks and freaks alike in a tizzy for months. Brazilian electronic pioneer Amon Tobin would be performing songs from his new album, ISAM, from within a 3-dimensional structure that uses virtual mapping, LED fixtures, projection (and, admittedly, a lot of technology I don’t quite understand) to create an entirely new audio-visual experience.

On stage, the crowd was confronted with a giant, custom-built, three dimensional, geometric structures. A large cube in the middle housed Amon himself, as if he were piloting his own cyborg-like spaceship. For each intricately layered, hard-hitting song, there was an equally intricate visual theme that played out on the structure, making it look as if Amon was immersed within the music itself, and inexorably linking the audio and visual experiences. Without much technical knowledge, describing how this effect is achieved is difficult. One of my friends said it looked as if animation or illustration had come to life in front of you, or as if you were able to see things that were, simply put, not real.

Gizmodo called it “The Concert of the Future, Today” and CHARTattack said, “it was a live show one might have said could only be created and performed by a cybernetic organism,” where “groundbreaking modeling and mapping techniques were used to take the audience at the Metropolis on a trip to dozens of worlds. … it felt like one was transported into deep space or, alternately, inside the TRON grid, a game of Tetris, an M.C. Escher painting, the fiery pits of Mordor, an acid-coloured kaleidoscope, a Splinter Cell video game…”

The visual team responsible for this masterpiece consisted of “V Squared director Vello Virkhaus, designer and programmer Peter Sistrom and Leviathan chief scientist Matt Daly…[also] Bryant Place built a custom application that runs the visuals of the entire Amon Tobin show end to end. The application is made in TouchDesigner, Derivative’s visual programming environment, and does the projection mapping, video playback, Kinect response, realtime effects and more.”

Amazingly, the installation runs on only one projector and in a way, creates a four-dimensional experience, not just three. From a Derivative interview with the team, here’s what Vello had to say about it. “4D in mathematics is a very abstract concept in which this additional dimension is indistinguishable, yet acknowledged. This unknown to me relates to the pronounced visual effect the mapped structure of Tobin creates for the viewer. What is fascinating is that we give viewers an idea of what it might be like to see beyond 3D space, to see all points simultaneously for both the exterior and the virtual interior of the set. The combination of this mapped effect and Amon’s music produced some very intense emotional reactions from people.”

Of course, LED light fixtures are an integral part of this concept; at the very least the designers used them to create preliminary models for the project. “We put together an initial presentation with a set of story boards and written treatments of ideas, along with a reel of LED pixel mapping samples. Before this project, we had only done a few video mapping projects, but nothing of this scale or complexity. We had been primarily working with LED, and mapping in planar space.”

And, not surprisingly, when Derivative asked the team what they hope to work on next, almost all of them include LED lighting projects in their future plans.

“Bryant: Well one of the things I’m going to take to the field very soon is doing things beyond video and video mapping. I’m getting more into DMX and trying to control servos and send information to other devices rather than a video signal and integrating those two. Like doing some sort of LED video mapping but then on top of that sending information to lighting or other devices.

Matt: Definitely work on more experiential projects. We want to do bigger more integrated projects using everything from projection system technology and LED wall technology to integrated systems large multi-screen venues, digital signage integrated into building systems – that sort of range.”