#augmented reality

By Mike Shebanek, Senior Director, Accessibility

Yahoo’s Accessibility Lab is well known for creating hands-on interactions for software developers to experience how people with disabilities successfully use technology—when it’s well designed of course!  One of the experiences makes use of goggles with filtered lenses that simulate different types of vision loss, such as glaucoma and cataracts. Unfortunately, the lab is limited in the number of goggles we have available, and visitors to the lab can’t take the goggles home with them or back to their offices to experience vision loss in their own environments or to share this alternate reality with their colleagues. Thanks to Theia Immersive in the UK, that is about to change.

Theia has just introduced a new technology called Eyeware which is designed to enable architects, environmental engineers and space planners to explore how their designs and spaces might limit or enable people with disabilities. But by putting Virtual Reality and Augmented Reality to work, Eyeware can also create a real-time, immersive simulation that opens the door to simulating the many different and unique experiences of how people with low vision, color blindness and a variety of other eye conditions, see and navigate the world around them.

Using Virtual Reality, Theia Immersive can model a real environment, indoors or out and can simulate what it would be like to navigate with vision loss, or limited mobility. But even more impressively, they can make adjustments to the virtual environment in real time, to better understand how changes in lighting, signage, the use of color, and many other elements impact a user’s experience in that space. The system uses OmniDeck technology that drives the VR experience and enables a person to really (physically) walk through the virtual world, hear positional audio, and interact with their virtual environment. All of this can be captured, analyzed, and shared, also in real time, to an audience via TV monitors and computer screens.

The company has also created a personal, mobile version of their technology—Eyeware for iOSandEyeware for Android. This version uses Augmented Reality - the app runs on  your phone when mounted in the Google Cardboard viewer (about $10). Theia Immersive offers their own Cardboard viewer that has a cutout to uncover your phone’s built-in rear camera (you can cut a hole in your own Cardboard viewer to achieve the same purpose). You can choose various visual effects  to simulate different types of vision loss. As you look through the Cardboard googles, you’re seeing the real world around you, but through the eyes of someone that has vision loss.

Video of the Augmented Reality Eyeware app using a Cardboard Viewer

We are very excited about this new technology and its far-reaching potential and are working with Theia to bring Eyeware into the Yahoo Accessibility Lab. We think it can have a profound impact on our visitors and will be encouraging our product teams to “see it for themselves” and consider how they can better design their products to provide access to people with vision loss and other disabilities.

In the meantime, Theia is coming to San Francisco on May 18th in celebration of Global Accessibility Awareness Day, to an event hosted by the LightHouse for the Blind called “Eyeware – a New Virtual Reality and Augmented Reality Experience for Inclusive Design.” They’ll be demonstrating both the virtual reality and augmented reality versions of Eyeware. Yahoo is proud to be a sponsor of this event and invites you to attend.

Disrupting the Museum with Augmented Reality

In January I wrote about activist augmented reality art interventions on Wall Street and the way artists used AR technologies during the Occupy movement. Manifest.AR’s #arOCCUPYWALLSTREET installations challenged the financial establishment, and their subversive projects continue to address powerful institutions and broken systems. These open-source, low-cost, guerrilla art projects belong to a new form of contemporary activist art that exists outside the museum space. But they raise a compelling question: if AR is such a powerful activist tool outside the gallery, what can it achieve within the institutions of the art world?

Tamiko Thiel, ARt Critic Face Matrix (From the exhibition We AR in MoMA), 2010. Screenshot of the digital augmented reality intervention in MoMA, New York. Source.

Guerrilla museum interventions that use augmented reality were prefigured by decades of non-digital interventions into exhibition spaces like MoMA and the Venice Biennale. Throughout the second half of the twentieth century, artists used physical interventions to critically engage with collections, institutions, and the art world more broadly. Performances and physical artworks became vehicles to reveal and challenge institutional biases, practices, gaps in collections, and policies. In the 1960s, Yayoi Kusama staged a performance called Narcissus Garden at the Venice Biennale where, without official permission, she set up her own installation of mirrored balls on the lawn of the Giardini della Biennale and sold them to visitors (2). This guerrilla work “challenged the commercialization of the art system” (3) and its power to determine what is exhibited and where. Her simple act of invading the exhibition may seem harmless, but the authorities’ negative reaction (removing Kusama from the Biennale grounds) hints that it was actually quite subversive. It’s no small irony that now, when Kusama is part of the institutional canon of contemporary art, Narcissus Garden is exhibited by major museums around the world.

These twentieth-century works of Institutional Critique question and “disrupt” (4) institutions, revealing their hegemony within the art world and all of the power that comes with this. Andrea Fraser’s performance Museum Highlights: A Gallery Talk (where the artist pretended to be a tour guide and delivered a satyrical, critical script to visitors) calls attention to the way large, powerful museums are able to define “art” and the art historical canon for visitors through exhibitions, published materials, and tours offered by the museum. The AR artists known as Roma Europa Fake Factory describe how “defining what is real is an act of power. Being able to reinvent reality is an act of freedom” (5). The exclusive, symbolic space of the museum has long been a site of critique for artists, so what possibilities has AR opened up in this space?

Yayoi Kusama, Narcissus Garden, 1966. Installation of mirrored plastic balls at the 33rd Venice Bienniale.

Augmented reality technologies open up the possibilities for artists who want to stage critical museum interventions. The technology is cheap, Internet access is widely available, most museum visitors have a smartphone (6), and AR has no geographical limits. This limitlessness is one of the aspects of AR that is the most groundbreaking. “Every single place on Earth has coordinates that can be tracked technologically” (7), meaning that augments can be placed anywhere, regardless of physical accessibility – even in the Oval Office, one of the most impenetrable spaces in the world (8). Artists are able to claim space within institutions like MoMA and intervene in their collections, which was not possible before (or at least not possible for more than a short amount of time – in 2010 Maria Anwander “donated” a kiss to MoMA, but it was removed from the wall). With AR, “public space is… truly open” (9). An artist doesn’t have to go to MoMA or gain access to the Venice Biennale to intervene in it, as long as they have the right technology.

Andrea Fraser, Museum Highlights: A Gallery Talk, 1989. Film documentation of a performance. Philadelphia Museum of Art.

We AR in MoMA was an “uninvited AR event” planned by Manifest.AR members Mark Skwarek and Sander Veenhof. Along with several other artists, they placed new works in MoMA’s physical galleries and added a new AR “gallery” on top of the museum’s roof. The physical space of the gallery stays the same while the art changes, replacing MoMA’s collection with new works by artists like Skwarek and Veenhof that could be viewed through the LAYAR app. Tamiko Theil’s contribution, Art Critic Face Matrix, fills the space with the disembodied heads of an AR art critic who makes faces and “cast[s] its baleful eyes over the surrounding works [in MoMA’s permanent collection] and seem[s] to scream, “You call this ARt???”” (10). 

The works in We AR in MoMA are interventions into the canon of modern and contemporary art that is constructed by/represented in MoMA. By intervening in the space with their own (uninvited) works, Manifest.AR challenges the institution’s control over its space, its collection, and what its viewers experience. AR artworks benefit from existing online (where they can’t be removed by museum security), but they don’t only exist there. They are ‘really’ in MoMA in the sense that visitors can see them and walk around them, and they are geolocated inside the museum even if there are no viewers. The AR Manifesto states that “It is There and can be Found – if you Seek It” (11). In projects like We AR in MoMA, artists are doing something that Kusama and others interested in Institutional Critique could not do — they are breaking into the museum, placing a critical work, and staying there indefinitely.

Maria Anwander, The Kiss, 2007/2010. Lipstick and museum label applied to the wall. Documentation of the performance at MoMA, New York. Source.

Visitors to the Venice Biennale in 2011 could view the uninvited augmented reality Invisible Pavilion through their smartphones, which recalled Kusama’s original Narcissus Garden intervention. CONT3XT.NET’s Blemish appears as a constellation of black square dots overlaid on the real environment through the viewer’s phone camera. They describe how “The immaterial defect of form – a dead pixel – is inscribed in the auratic art spaces of the Venice Biennial. Barely perceptible for the viewers it is disguised as a loose arrangement of black squared errors which finally can be read as an abstract comment about the blemished context of art” (12). By this, they mean to ask viewers: who or what is inand who or what is out? Who and what is allowed or not allowed into these high art cultural spaces?

Like earlier works of Institutional Critique, AR interventions are a critical practice — the AR manifesto states: “With AR we install, revise, permeate, simulate, expose, decorate, crack, infest and unmask Public Institutions, Identities and Objects previously held by Elite Purveyors of Public and Artistic Policy in the so-called Physical Real.” In other words, they are challenging many of the same institutional problems as artists like Fraser and Kusama, including the status and power of museums to define art, their lack of transparency, and their exclusive control of the physical space of the gallery.

Gabriel Barcia-Colombo, White Light Afterlife (from MoMAR’s Hello, we’re from the internet), 2018. MoMA, New York. Source.

The open-source AR artists’ collective MoMAR (established by David Lobser and Damjan Pita) also bring digital artworks into museum collections that are physically closed off to most artists. Their guerrilla exhibitions Hello, We’re From the Internet andOpen to the Public (both located inside MoMA) are critical interventions into the canon of modern and contemporary art constructed by and represented in the museum’s modern art galleries. They challenge the institution’s control over “art,” its space, its collection, what its visitors experience, and what they take away from the museum.

InHello, We’re From the Internet, the artists overlaid or substituted the Pollock paintings on the wall with digital augments. In one work, the original painting is gone except for the frame, and instead visitors see into a recessed space (a cube within a cube) occupied by skeletons holding cell phones and the cryptic floating word “AGAIN.” It is meme-like, absurd, and vaguely funny. At the top of the screen there is a question mark that viewers can click to see the name of the artist (almost like a gallery label) and in the bottom left corner is an arbitrary score. The overall interface of the AR exhibition is designed to resemble what users see through Instagram or Snapchat’s in-app camera. Although this work is located at MoMA, it is not geolocated. Instead, it uses “the… iconic paintings [as] merely markers—points of reference telling the app where to display the guerrilla artists’ works” (13). MoMAR’s own works are elevated by their placement inside frames in the prestigious galleries of the Museum of Modern Art, while paintings by the canonical artist Jackson Pollock may as well be QR codes. It is a powerful co-optation of the institution’s symbolic power. Furthermore, Hello, We’re From the Internet andOpen to the Public challenged the museum’s control and transformed its galleries into an open, collective, and interactive space that no longer serves the institution’s interests.

Erin Ko, La Barrera (from MoMAR’s Open to the Public), 2019. MoMA, New York. Source.

MoMAR is explicitly critical of large, powerful institutions and their ability to decide what is “art” and who is an “artist” by including or excluding them from their collection. In doing so, they add value to certain media, certain works of art, and certain artists — all determined by the institution. The artists’ statement on MoMAR’s website (which is itself overlaid onto the official MoMA site) reads:

“If we are to understand that art is the great measure of our culture we must also acknowledge it is owned, valued and defined by ‘the elite.’ We must also recognize then that the term “open to the public” is not an invitation, but a declaration of values. Values that are not our own. And so it has remained for 335 years. Until now” (15)

The title Open to the Public explicitly references MoMAR’s desire to “open” these institutions by sourcing public augmented reality art to put into the museum without authorization. Real-world artworks in the collection are replaced with nonsensical, meme-like, and sometimes ‘cheap’-looking augments that contrast the “high art” displayed in MoMA’s galleries. However, in appropriating the frame and location of the original paintings, the AR works contextually appear to be “fine art” as well. This critical action thus makes us question how we understand art and who defines it — what is the role of the context and the institution in defining art and creating the canon? What are the implications of this?

Scott Garner, Number 1B, (from MoMAR’s Hello, We’re From the Internet), 2018. MoMA, New York. Source.

Entering these spaces critiques and challenges the museum’s power to decide who gets in and who stays out. At the same time, it also validates the invading artist’s work and may ultimately benefit their career. “It is certainly a victory that a group of artists – by using alternative methodological approaches to what are the structures of the capitalistic system, is able to enter into that very capitalistic system in order to become institutionalized and perhaps – in the near future – be able to make money in order to make art” (16). Potentially, these AR interventions might erode MoMA’s power and authority to define “art.” Interventions by MoMAR, Manifest.AR, and CONT3XT.NET, among others, might erode these institutions’ power and authority to define “art.” But at the same time they may also canonize AR as “high art” itself – a gradual move from ‘outsider’ to ‘insider’ that mirror’s Kusama’s Narcissus Garden.

Screenshot of “MoMAR” superimposed over “MoMA” on the exterior view of the museum, as seen on their augmented reality app. MoMA, New York. Source.

Notes

1. Simona Lodi, “Spatial Narratives in Art,” in Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium, ed. Vladimir Geroimenko (New York: Springer, 2014), 282.

3. Lanfranco Aceti, "Not Here, Not There: An Analysis Of An International Collaboration To Survey Augmented Reality Art,” Leonardo19, no. 2 (2013): 6.

4. Aceti, "Not Here, Not There,” 7.

5. Lodi, “Spatial Narratives in Art,” 286.

6. Aceti, "Not Here, Not There,” 8.

7. Lodi, “Spatial Narratives in Art,” 283.

8. Mark Skwarek, “Augmented Reality Activism,” in Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium, ed. Vladimir Geroimenko (New York: Springer, 2014), 22.

9. Quoted in Rinehart, 9.

12. quoted in Lodi, “Spatial Narratives in Art,” 287.

16. Aceti, "Not Here, Not There,” 7.

ff7ogzine:

Do you like moogles? Postcards? Both? You’re in luck! This amazing augmented reality postcard by @kleptoyade has the cutest animations–you can even feed the moogle kupo nuts! Donate here to access, kupo (^=˃ᆺ˂) https://tiltify.com/@ff7ogzine/ff7ogzine/

Get my AR mog house postcard (and so many other awesome goodies!) in the FF7 OG Zine digital merch pack!

The postcard plays an animation when you view it through your phone!

Available from just $1, and the proceeds go toward saving our planet!

The next evolution in art

Setting the gold standard

Welcome To The Matrix

A little ASMR ♾