Want Tо Knоw What Virtual Realitу Might Becоme? Lооk Tо The Past


The Scottish scientist was one of those 19th-century characters with no real equivalent today. An ordained minister in the Church of Scotland, he took an early interest in astronomy and became for a time one of the world’s leading experts on the science of optics. He also harbored a great fondness for popular amusements, and at some point in the early part of the century, he began frequenting a theatrical horror show in the West End of London called the Phantasmagoria. He went in part as a debunker, a skeptic hoping to reveal the secret craft behind the spectacle. But he also sensed that something profound was lurking in the trickery. He suspected that the showmen were exploiting some intrinsic quirks in the human sensory system — perhaps, he hoped, rendering them more intelligible to the scientist. Brewster called the world of scientifically produced illusion “natural magic.”

The Phantasmagoria came to London in 1801, after a decade or two of development in Germany and France. Relying heavily on ghostly magic-lantern projections, the show submerged its patrons in a multisensory vault of dread and illusion — in contemporary terms, a cross between the immersive theater of “Sleep No More” and Disney’s Haunted Mansion. Shortly after its arrival, the success of the Phantasmagoria and a handful of similar shows set off a kind of entertainment version of the Cambrian explosion. Bizarre new species of illusion proliferated across the West End. The names themselves, with their strange Greek neologisms, suggest just how far the language strained to tout the novelty of the experiences. According to “The Shows of London,” by Richard Altick, a visitor to the city in the early to mid-1800s could enjoy a “novel mechanical and pictorial exhibition” called the Akolouthorama; a rival spook show called the Phantascopia; an exhibition called the Spectrographia, which promised “traditionary ghost work!”; an influential mechanical exhibition dubbed the Eidophusikon; and the Panstereomachia, “a picto-mechanical representation,” in the words of The Times of London.

Brewster himself was something of a natural magician. Right around the period he was studying the Phantasmagoria, he invented the kaleidoscope, which for a few years was the PlayStation of the late Georgian era. (Brewster barely made a penny from the device, as imitators quickly flooded the market with clones of his original idea.) Decades later, he invented the lenticular stereoscope, a hand-held technology that fools the eye into perceiving two distinct flat images as a single 3-D scene. This time around, Brewster managed to build a successful business selling his contraption, branded as a “Brewster Stereoscope.” Queen Victoria famously marveled at one during the Great Exhibition of 1851. Oliver Wendell Holmes published a paean to the stereoscope in The Atlantic, rhapsodizing over the new technology with an enthusiasm that wouldn’t have been out of place in an early issue of Mondo 2000 or Wired:

Oh, infinite volumes of poems that I treasure in this small library of glass and pasteboard! I creep over the vast features of Rameses, on the face of his rockhewn Nubian temple; I scale the huge mountain-crystal that calls itself the Pyramid of Cheops. I pace the length of the three Titanic stones of the wall of Baalbec — mightiest masses of quarried rock that man has lifted into the air. …

By the dawn of the 20th century, almost every species in the 19th-century genus of illusion was wiped off the map by a new form of “natural magic”: the cinema. The stereoscope, too, withered in the public imagination. (It lingered on as a child’s toy in the 20th century through the cheap plastic View-Master devices many of us enjoyed in grade school.) But then something strange happened: After a century of irrelevance, Brewster’s idea — putting stereoscopic goggles over your eyes to fool your mind into thinking you are gazing out on a three-dimensional world — turned out to have a second life.

This is how, a few months ago, I found myself holding an original 19th-century wooden stereoscope. I was visiting the headquarters of , a Los Angeles-based media company that produces stories in virtual reality, and that maintains a collection of stereograms from the heyday of the device. It’s easy to see why, because the family resemblance to today’s V.R. goggles is unmistakable; in fact, the stereoscope I held was literally forward-compatible with today’s smartphones. ’s chief executive and co-founder, Bryn Mooser, slid an iPhone into the slot behind the lenses, and as I pulled the contraption up to my eyes, I was transported to the banks of the Ganges River. It felt like a moment from some steampunk novel: a Victorian contraption conjuring a world through natural magic of a distinctly digital kind.

The moment suggested a tantalizing possibility: that the mass extinction of all those 19th-century spectacles, all those illusion palaces and contraptions, might itself prove to be a kind of mirage. What if those marvels of the past only went into a 100-year hibernation? Like all forms of new media, virtual reality is frequently described in terms of its immediate predecessors: movies or video games. But if we are trying to imagine the future of V.R., we may well have more to learn from the immersive shows of 19th-century illusionists than from some franchise superhero movie or Call of Duty. No doubt some future V.R. creators will figure out how to tell cinematic stories or make compelling game experiences. But if history is any guide, it’s very likely that we will eventually discover that V.R. is actually better at something else.

The question is: What is that something else?

We owe a great deal of the modern world to people doggedly trying to solve some high-minded problem: how to construct an internal-combustion engine, or how to manufacture vaccines in large quantities. But a surprising amount of modernity can trace its roots to another kind of activity: people mucking around with magic, toys, games. When human beings create and share experiences designed to delight or amaze, they often end up transforming society in more significant ways than people focused on more utilitarian concerns. Everyone knows the old saying “Necessity is the mother of invention,” but if you do a paternity test on many of the modern world’s most important ideas or institutions, you will find, invariably, that leisure and play were involved in the conception as well.

FIG. 1 An 1849 rendering of David Brewster’s stereoscope design.

From The Popular Science Monthly, Volume 21.

Indeed, the study of delight and amusement often turns out to be a way of predicting the future. The idea of a true global economy was first visible in the market for entirely frivolous spices like cinnamon and nutmeg; the first programmable machines — predating modern digital computers by a millennium — were automated toys and music boxes. The mathematics of probability theory, which now underpin everything from insurance to airplane design to clinical drug trials, were first developed to analyze dice games. When those Victorian technophiles stared through stereoscopes and projected themselves into distant worlds, they were also, unwittingly, projecting themselves into a future that wouldn’t appear for another century and a half.

Optical illusions occupy a special place in the history of play and wonder, straddling the border between parlor trickery and high . Until the invention of cinema in the late 19th century, which fooled the eye into seeing motion in a series of still images, the most famous and influential “trick of the eye” was the invention of linear perspective, generally credited to the architect Filippo Brunelleschi, though the fundamental rules that governed the technique were first outlined in the book “On Painting,” by Leon Battista Alberti, published in 1435. Technically speaking, linear perspective is nothing more than an optical illusion, but it is rightfully considered one of the most transformative innovations of the Renaissance.

For a brief period at the end of the 18th century, it seemed as though an Irish painter named Robert Barker had stumbled across an innovation of comparable significance. At some point in the mid-1780s, Barker took a stroll to the top of Calton Hill in Edinburgh. Standing near the current site of the Nelson Monument and gazing out over the city, Barker hit upon the idea of painting the entire 360-degree view by rotating a sequence of square frames around a fixed spot, sketching each part of the vista and then uniting them as a single wraparound image. (He had to invent a new technique to compensate for the visual distortions that appeared when painting on a concave surface.) Barker was granted a patent in 1787 for “an entire new Contrivance of Apparatus … for the Purpose of displaying Views of Nature at large.” At the suggestion of a “classical friend,” Barker hit upon a name for his creation, drawing on the Greek phrase for “all-encompassing view.” He called it the Panorama.

By 1793, Barker had constructed a six-story building near London’s Leicester Square, custom-designed for the exclusive purpose of displaying two separate Panoramas to crowds of paying spectators. The lead attraction was an immense vista of London encompassing 1,479 square feet. Barker ran advertisements that modestly suggested his technique was “the greatest improvement to the art of painting that has ever yet been discovered.” For a time, the bombast seemed warranted. The show was a runaway success. The king and queen requested an advance viewing, though Queen Charlotte later reported that the illusion made her dizzy.

Surveying the illusion artists of the early 19th century, a young Charles Dickens described the technology in terms that echo much of the recent enthusiasm over virtual reality and its potential:

It is a delightful characteristic of these times, that new and cheap means are continuously being devised, for conveying the results of actual experience to those who are unable to obtain such experiences for themselves; and to bring them within the reach of the people — emphatically of the people; for it is they at large who are addressed in these endeavours, and not exclusive audiences. … Some of the best results of actual travel are suggested by such means to those whose lot it is to stay at home. New worlds open out to them, beyond their little worlds, and widen their range of reflection, information, sympathy and interest. The more man knows of man, the better for the common brotherhood among us all.

Dickens saw the virtual explorations of the Panorama and the stereoscope as a way for human beings to extend the range of their perceptions — literally to see the world through the eyes of others. Today this sentiment has become a slogan of sorts for the V.R. medium. The director Chris Milk, in a widely circulated TED talk, called V.R. systems “empathy machines,” a description echoed by other V.R. auteurs. At the RYOT studios, a director named Angel Manuel Soto showed me a short V.R. film called “Bashir’s Dream,” which tells the story of a young Syrian boy who was paralyzed after being shot by a sniper. The film — though “film” is almost certainly the wrong word to describe it — shifts between live-action shots of Bashir’s wheeling his way through battle-scarred urban landscapes and an animated reconstruction of the shooting that nearly killed him. At the end, you are projected into a kind of dream landscape, as Bashir imagines escaping the twin prisons of a war zone and a wheelchair. The film is haunting, to be sure, but watching it, I couldn’t help wondering if the empathy it evoked was really all that different from the kind of enlarged perspective that great documentary films already provide.

My suspicion is that “empathy” will turn out to be the wrong word to describe what separates V.R. from the medium of film. Think, for example, about certain ways the first-person perspective of V.R. actually limits what we see. Humans have evolved a complex apparatus for detecting the emotional states of others by intuitively assessing the micromuscular movements of the face. The paradox of V.R. is that when you see the world through someone else’s eyes, you can’t actually see the person’s eyes. You can see what the person is seeing, but it’s much harder to grasp what he or she is feeling. A cinematic close-up conveys emotional depth far more effectively than a point-of-view shot in a 360-degree film can. What V.R. does provide is so new that we don’t really have a word for it: perceptual empathy, sensory immersion.

FIG. 2 A woodcut broadside for Paul de Philipsthal’s Phantasmagoria show, which opened in London in 1801.

From the Lewis Walpole Library at Yale University.

“Come on in, the jellyfish is ready for you.”

I had navigated my way through a warren of vacant hallways to meet with the British artist and V.R. creator Barnaby Steel, a founder of a group known as Marshmallow Laser Feast, which seems to oscillate between a commercial creative agency and an experimental art collective. When I finally found my way to the mysterious Room 530, in the rear of their Los Angeles offices, Steel welcomed me into an undecorated open-plan space filled with a dozen or so black workstations. Steel and his colleagues were working on a project for the V.R. music-experience company Redpill with a curious goal: to summon a shimmering oversize jellyfish that would adapt, in real time, to any music that you feed into the simulation. Each new song triggered a different set of jellyfish behaviors: flashes of color in its gelatinous body, tentacles pulsing with the rhythm. The contrast between the banal exterior of the setting and the psychedelic alternate reality being created there was almost comical, like stumbling upon an ayahuasca ceremony in some conference room from the set of “Office Space.”

Steel strapped the Oculus goggles to my head and placed two controllers in my hands. Inside the simulated world, I could lift my hands and see two ghostly images of them, as though contemplating a live X-ray of my own body. Looking up, I saw the jellyfish hovering above, bobbing softly in the blackness like an immense Portuguese man-of-war. The music part of the simulation wasn’t operational yet, so I could hear the real world around me as I gazed at the imaginary dancing creature. “You can smack the side of it,” Steel said, and I did; the jellyfish flinched, and a blast of purplish color rippled off from the point of impact.

I found myself trailing my fingers through the tentacles, watching them dance. At Steel’s encouragement, I pulled the whole creature over my head, as through grabbing the ends of a blanket, and suddenly I was inside the body, draped by tentacles on all sides. It was, by a wide margin, the most sensual encounter I have ever had with an invertebrate.

FIG. 3 An illustrated cross-section of the building housing Barker’s Panorama near Leicester Square.

From the British Museum.

“Our starting point when we first began experimenting with virtual reality was: What does it enable that wasn’t possible before?” Steel explained, after I emerged. “And then, what are we interested in? How can we expand our experience of reality?” The world that we experience is obviously limited by our senses, he pointed out, but many more potential senses exist than those we possess.

That line of thinking led not just to the jellyfish but to another V.R. installation called “In the Eyes of the Animal,” which Steel and his colleagues staged in a forest in Britain’s Lake District. Donning V.R. goggles, visitors entered a parallel version of the forest, where they could shift in and out of the perspectives of different creatures, from midges to frogs to owls. As you switched among the animals, the software simulated the unique perceptual tools of each organism. The midge, for instance, could detect carbon dioxide being exhaled in an ordinary human breath from hundreds of feet away. When you adopted the midge’s perspective, carbon-dioxide density was represented by swirling red points in the forest. “We tried to imagine the trees breathing,” Steel explained. “What it would look like if you could see the chemical composition of the air.”

A few decades ago, the philosopher Thomas Nagel published a classic essay on the question of consciousness, with what may be the greatest title in the history of philosophy: “What Is It Like to Be a Bat?” As a nod to Nagel’s essay, Steel and his colleagues are now working on an extension to “In the Eyes of the Animal” that tries to simulate sonar navigation. Nagel’s original question was almost a rhetorical one — the point was that humans simply can’t imagine what it is like to be a bat because our mode of consciousness is so different. But V.R. experiments like “In the Eyes of the Animal” at least hint at an answer to Nagel’s question, using a hybrid of advanced modern technology — Lidar scanners, advanced graphic cards — and the stereoscopic illusion that David Brewster first explored two centuries ago.

In September 1812, the Bavarian musician and inventor Johann Nepomuk Maelzel found himself in Russia, just in time to witness the legendary burning of Moscow that greeted Napoleon’s arrival in the city and would soon lead to his epic defeat there. The fire and the subsequent battle for Moscow would inspire many great works of art in the years that followed: Tolstoy’s “War and Peace”; Tchaikovsky’s “1812 Overture.” But one of the very first — and perhaps most original — attempts to represent this world-historic event was engineered by Maelzel within a year of Napoleon’s defeat, in the form of an animated diorama called “The Conflagration of Moscow.” Maelzel’s creation had its premiere in Vienna, but he would ultimately take it across Europe and North America, dazzling audiences for decades with his mesmerizing reconstruction of the great city on fire.

A detailed inventory of the show that toured the United States gives some sense of the scale of the production. Movable frames representing the buildings of Moscow — the Kremlin, church spires, castles — were designed to collapse or explode on cue. Behind the skyline, Maelzel hung a transparent painting that suggested a haze of smoke and fire; behind it another painting depicted other buildings in the distance ablaze, with a moon glowing in the night sky above the carnage. At the front of the stage, two bridges and a causeway carried more than 200 miniature Russian and French soldiers. Fire screens enabled actual flames to creep across the urban landscape without damaging any of the equipment.

The story conveyed by “The Conflagration of Moscow” was the least interesting thing about the show. Yes, events followed a preordained sequence on the stage: Napoleon’s army advanced; the Russians retreated; flames surged across the skyline. But the true appeal of the spectacle came from the sense of immersion, just as it had with Barker’s Panorama. You didn’t go to see these illusion shows because you wanted to follow the arc of a compelling character; you went because you wanted to be present in some stunning place, or time, or perspective, that would otherwise be impossible to inhabit.

FIG. 4 An illustration of a “magic lantern” from “Ars Magna Lucis et Umbrae,” by Athanasius Kircher (1646).

SSPL/Getty Images

Something similar is likely to happen with virtual reality. One thing that is striking about “In the Eyes of the Animal” and Steel’s musical jellyfish is that there is nothing narrative about the experiences. They’re powerful, evocative; they fill you with an undeniable sense of wonder. But they don’t tell you a story.

“This isn’t an evolution from cinema,” Bryn Mooser said after we put down the stereoscopes at the RYOT offices. “This isn’t storytelling.” If V.R. allows you to project yourself onto the deck of the Titanic, I suspect we won’t want the entire James Cameron-style back story about a dashing artist and his fleeting romance with a wealthy young woman facing an arranged marriage. We’ll just want to experience the sinking of the ship. Plot points will be a nuisance. A V.R. equivalent of “Jurassic Park” won’t bother with the relationship between John Hammond and his grandkids; we’ll just stroll through the grassy plain and gaze at the brontosaurus.

The most surprising twist in the evolution of V.R. may turn out to be the pace of the new medium. Quick cuts are an almost physical act of violence in V.R.; jumping from one perspective to another can create a literal sense of nausea. But more telling, perhaps, is the fact that people don’t want to move on to another experience once they’ve put the headset on. They want to linger. “I want to just put you in a field,” Mooser told me, “and you just do what you want to do in that field.”

“We weren’t expecting it really,” Steel says about the process of creating “In the Eyes of the Animal,” “but by the end of it, we were all thinking, If you’d had a stressful day, it’s super nice to just put it on and just explore. You always find new perspectives and angles. With these bigger trees, you’re scanning so much detail that you notice stuff in V.R. that you wouldn’t notice in real life. We’re stuck at five to six feet high, and so you rarely go down and put your chin on the floor. But in V.R., you just stick yourself down there, and you’re like: ‘I love it down here! Look at all these pine cones!’ ”

It’s possible, of course, that the next generation — acclimated to these merely stereoscopic explorations — will agitate for the medium to take on the kinetic qualities of the cinema: not just the intricacies of plot but the rapid cuts, the action sequences. But think of all those countless thousands of spectators who thrilled to the “natural magic” of Barker’s Panorama, standing in silent contemplation, traveling to a distant place for a few minutes before venturing back out into the chaos and smog of the great city. A similar sense of quiet contemplation may well be what differentiates V.R. from its immediate predecessors. In an age when action movies have acclimated our eyes to multiple cuts per second, and in which video games bombard us with nonstop carnage, there turns out to a surprisingly meditative quality to the world we inhabit with V.R. goggles on. This could well turn out to be the most magical trick of all: harnessing all this advanced technology to slow us down and make us wonder again.

  • Facebook
  • Twitter
  • Google+
  • Linkedin
  • Pinterest

Leave a Reply