ZED Brings René Magritte’s Art to Life at San Francisco MOMA

Surrealist painter René Magritte made his mark mixing objects and scenes together in metaphysical ways. His works would show fruit hovering in front of a man’s face, canvases of the world behind the wall on which it hung, or artists painting on the very art they stood in to give themselves a companion.

The ZED camera also has a natural ability for mixing realities. So when the San Francisco Museum of Modern Art commissioned frog design to turn Magritte’s still paintings into an interactive exhibit, they incorporated ZED cameras into each of the room’s six augmented reality works.

Frog Design based each work on a specific Magritte painting or a visual theme his works frequently espouse. To make them interactive, they used the ZED to know when a person walked up to the display, track their movement, and extract the person’s image without the real-life background.

From there, they could mix the viewers themselves into the works of art in real-time, even as they moved around. Each “mix” mimics a well-known visual element from Magritte’s unique style; the moon from Les Mystères de l’Horizon floats above the visitor’s head, their silhouette in La Décalcomanie cuts a hole to a different world, and the tree trunks of Le Blanc Seing occlude the visitor’s image in a beautifully unnatural way.

After testing several sensors, frog chose the ZED primarily for its robust and easy-to-use SDK. With it, they extract point cloud data from the environment, apply their own filtering, use OpenCV to detect the 3D bounds of people, and apply image effects. They ran all demos on a Jetson TX2 embedded board running a custom openFrameworks build.

The team also benefited from the ZED’s high resolution and a lack of interference caused by having multiple sensors in the room. Thanks to its simple design, it was easy to conceal the ZEDs and ZED Minis into windowsills and other normal features where two small holes wouldn’t look out of place.

Magritte believed that an image is separate from the real thing it represents. His portrayal of objects as they couldn’t possibly appear was his way of proving that. But his work, while historically labeling him a surrealist, could arguably make him one of the world’s first mixed reality artists.

He once said that “to be a surrealist means barring from your mind all remembrance of what you have seen, and being always on the lookout for what has never been.” With modern tools like the ZED turning the impossible into reality, we can’t wait to see what the surrealists of today will create.

René Magritte: The Fifth Season is now open to visitors at the San Francisco Museum of Modern Art until October 28, 2018. Get tickets and more info here.