Fracture IO

Fracture was an installation for the 2013 One Show in which a volumetric capture booth translated the poses of partygoers into generative sculptural forms animated in realtime. The underlying capture software, written in OpenFrameworks, used four Kinects to reconstruct a seamless high resolution mesh using algorithms from the open source Point Cloud Library. These meshes were fed into a second piece of software, written with Unity, that generated and animated the forms based on the topography of the poses.

Created with Jeff Crouse and MPC.

Digital Spaces @ NYU

Digital Spaces is a class co-taught with Tabor Robak at NYU's Interactive Telecommunications Program.

From the course description:

In this course we’ll explore Realtime 3D as a medium for artistic expression and develop our own interactive experiences that take place in virtual spaces. We’ll critically survey a broad selection of Realtime 3D works, from traditional video games to more experimental interactive experiences. What kind of stories and experiences would you create in an interactive virtual space? We’ll make spaces that are immersive, atmospheric, surprising and absorbing. To create our own worlds, we’ll be using Unity3D, a free full-featured 3D game engine.

Syllabus from Spring 2013

Chris Milk's The Treachery of Sanctuary

Conceived by Chris Milk, The Treachery of Sanctuary is a large-scale interactive triptych in which visitors power and control the installation, making their way through three transformative experiences of flight.

In the first panel, as a visitor raises their arms, their silhouette slowly begins to dissolve, arms first, into a swarm of crows. In the second panel, a flock of crows viciously attacks visitors, violently ripping away pieces of their silhouettes until only a pair of legs remains. Upon entering the third panel, visitors sprout an enormous pair of wings when they thrust their arms upward.

The Treachery of Sanctuary was commissioned for the Creators Project event in San Francisco, March 2012. A mixture of technologies including Kinect, OpenFrameworks and Unity3D were used in its execution. With Chris Milk, Ben Tricklebank, James George, Brian Chasalow and Charlie Whitney. All photographs by Bryan Derballa.

Video

Draw With Your Face

Draw With Your Face is a game for Google+ Hangouts created in collaboration with OKFocus. Some of the mechanics will be familiar to players of other drawing games: players take turns trying to get other players to guess a word with a drawing.

As the name suggests, Draw With Your Face adds a new twist: using face tracking, players create their drawings hands-free using just their face! It's fast-paced and funny. And fun!

Draw With Your Face

Development blog

Draw With Your Face video

Hello, Future

Live interactive visuals for Jamie Lidell's performance at the Saatchi & Saatchi 2011 New Directors Showcase.

Watch on YouTube

360frames times infinity

360frames times infinity is a photo booth installation built on top of the 3frames website. 8 webcams surround an enclosure in 360 degrees, generating bullet time-style animated gifs that are uploaded to 3frames. An accompanying projected visualization displays more than 1000 of the most recent uploads to 3frames as an infinitely scrolling starfield, updating in realtime to reveal the latest uploads as they happen.

Watch in HD on Vimeo

3frames

3frames is a simple, evolving website that allows visitors to create and share animated .gifs using their webcam.

http://3fram.es

Cosmogramma Fieldlines

Created in collaboration with Flying Lotus, Cosmogramma Fieldlines is an interactive interpretation of the cover artwork for Lotus' third full-length album, Cosmogramma. The downloadable app allows users to interact using their webcam or their mouse, creating an entrancing combination of sound and motion. Developed with OpenFrameworks.

Cosmogramma Fieldlines for Mac

Cosmogramma Fieldlines for Windows

Fieldlines at flying-lotus.com

The Moon is made of plops.gif

The lunar topographic data from NASA's Clementine LIDAR mission represents the surface of the Earth's moon in a set of more than 1 million points of latitude and longitude. Paul Robertson's plops animated gif seamlessly morphs between thirteen characters from Japanese anime and video games over the course of 72 frames. In The Moon is made of plops.gif, these two disparate data sets merge into an image simultaneously alien and familiar. Psychedelic colors and an undulating sea of pixel art characters take the form of the most familiar celestial body of the night sky.

Photo set on Flickr

Coverage on Gizmodo

Paul Robertson's plops post

World Series of 'Tubing

An augmented reality game show, players compete against one another by selecting and presenting their "best" YouTube video clips. The audience then choose their favorite videos with a cleverly conceived audience response system using laser pointers. The World Series of 'Tubing brings televised competitive sport and game show aesthetics into the realm of participatory theater, complete with live commentators and stellar motion graphics. By blurring the distinctions between performer, audience, and participant, 'Tubing builds upon the inherent qualities of shared culture as it exists online. In collaboration with Jeff Crouse.

Interview on Rocketboom

World Series of 'Tubing at the 2009 Performa Biennial

Generative Studies for The Root of the Root

For The Root of the Root show with Marius Watz and Paul Prudence, custom software written in Processing was used to generate images which were then burned into birch plywood using a laser cutter. Incredibly detailed line work and perceptual distortions invite the eye to discover its own path across geometric abstractions and sinuous magnetic fieldlines.

The Root of the Root photo set by Marius Watz on Flickr

The Root of the Root at Devotion Gallery

Spikey

A series of generative studies on the sharp edges of polygonal imagery. Custom software written using Processing and OpenFrameworks is used to generate still images, animations and laser-engraved woodcuts.

Horizon Zero video (with Champagne Valentine)

Spikey on Vimeo

YouCube

YouCube allows visitors to engage and play with the plethora of content on YouTube in a new way. Visitors can map six YouTube videos onto a cube that can be manipulated in 3D space. As the cube rotates around, the corresponding video sounds fade in and out of audible range. Once a YouCube is completed, the creator can share their cube with friends through a permalink or browse through the database of other YouCubes.

Visit YouCube

Video Documentation

Spore Skeletons

Spore Skeletons visualizes the skeletal structures of the most popular user-created creatures from the popular computer game Spore. In the featured “augmented reality” mode, using a printed marker and a webcam, visitors can see the skeletons realistically integrated into the room with them. Spore Skeletons won the first prize in the official Spore API contest.

Visit Spore Skeletons

Video Documentation

Chromedrones

Chromedrones is a web-based sound toy that was commissioned for Google’s Chrome Experiments website. All the graphics and animation are created using JavaScript and the Canvas tag, showcasing the Chrome browsers fast interpreting engine. A version of Chromedrones for iPhone is currently in development.

Launch Chromedrones

Visit Chromedrones at Chrome Experiments

Terre Natale

Terre Natale is an immersive installation featuring a 360 degree data-driven animation divided into five narratives concerning the movement of people around the world. Terre Natale was created with Diller Scofidio + Renfro, Laura Kurgan, and Mark Hansen. In collaboration with Jeremy Linzee, Robert Gerard Pietrusko, Stewart Smith, Michael Doherty, and Hans-Christoph Steiner.

AAAA Visuals

AAAA Visuals is a custom piece of software written for a performance by the Brucknerhaus Orchestra at the Sonic Embodiments evening of the 2008 Ars Electronica festival. Fluid and organic shapes and motion are created by manipulating forces and parameters in realtime with the orchestra. Created in collaboration with Aaron Koblin.

Video Documentation

House of Cards

Radiohead’s House of Cards video was shot entirely using several kinds of laser sensors. The resulting raw footage was a massive collection of points in 3D space which was used to create the video, an online 3D viewer and released open source on the Google Code website.

House of Cards 3D Viewer

Official House of Cards page at Google

Torrent Raiders

Torrent Raiders is a dynamic network visualization realized through the idioms and aesthetics of arcade-style video games. Driven in real-time by the activity of bit torrent swarms, Torrent Raiders takes place on the ad-hoc networks created by bit torrent users. Torrent Raiders playfully addresses issues of domestic surveillance and intellectual property by putting players in the role of a mercenary copyright enforcer, encouraging them to capture evidence against peers on torrents in order to collect bounties.

Visit the Torrent Raiders website

Digg Swarm

Digg Swarm is a look at the real-time activity of the popular social bookmarking website, Digg. Users are represented by yellow circles and stories on Digg are represented by larger circles containing text. As users digg stories, they both grow and created a growing web of connections that effects the overall layout. Created while working at Stamen Design.

Visit Digg Swarm