Subverting the Cellphone Zeitgeist

Smartphones get a bad rap, and rightly so. We are a society of noses glued to screens. We breathe in the spaces between push notifications. In the evenings, seated across from friends and family, we infinite scroll and half-listen.

But it doesn’t have to be this way. Imagine a future in which our mobile devices don’t come in the way of being truly present. Imagine a future in which these devices actually enable the creation of meaningful shared experiences. A year ago, we took a stab at prototyping this future.

Designing for the Opposite of Immersion

Our chosen interaction medium was augmented reality (AR), an interface that places digital objects in people’s physical surroundings. At the project’s inception in early 2017, AR was an emerging technology that was largely restricted to single-user interactions. In our work, we wanted to push the envelope on AR and make it a social experience for colocated human users. In order to do this, our game needed to empower our users to be less focused on their devices and more attuned to their partners and the physical world around them. In other words, we would have to design the opposite of an immersive experience.

We conducted several brainstorms using the round robin brainstorming technique [link], and we generated nine different game concepts. We playtested these concepts in low fidelity and scored each against our chosen design goals. Eventually, we landed on a concept called Brick. Brick is a collaborative game in which two players work together to fll in a pattern of empty slots in a grid, using AR bricks that are scattered all over the room. Some of the bricks need to be collected individually, while others must be collected by both players working together. To win, the players must fll in the entire pattern before time runs out. At some point during the game, a bomb appears in the pattern, and the bomb must be defused collaboratively within seven seconds. If the bomb isn’t defused in time, it sets the players back by knocking of adjacent bricks in the pattern. Here are some photos of Brick’s low-fidelity prototypes.

A New Paradigm for Augmented Reality

In the early months of 2018, when Brick was being developed, the ecosystem of AR games offered limited opportunities for a truly multiplayer experience. At the time, multiplayer AR games typically reduced to single-player experiences with an asynchronous communication capability across multiple devices. We wanted to push the envelope by using AR to connect people not just with their own physical environments but also with each other. To do so, we developed a framework for shared-world AR, in which players inhabit a shared, real-time augmented environment and can engage in synchronous and collaborative interactions with other players.

For a two-player shared-world AR system such as Brick, we prototyped and iterated on five categories of interactions (see Figure 10): single-player interactions, which play out between individual users and the AR interface; intrapersonal interactions, which involve an individual user’s internal deliberations and experiences; multiplayer interactions, which play out between multiple users and the AR interface; interpersonal interactions, which involve multiple users but not the interface; and environmental interactions, which involve users and their physical environment.

Findings

Design for simplicity. Our fndings indicate that phone-based mental models of interaction overrode physical models early in the game. For example, one participant said, "My first instinct was to tap on things. I didn’t know that I had to approach things." Even after learning how to play, players needed ongoing technologically mediated cues to move their bodies.

Design for physical proximity. When players are colocated, a shared-world AR system ofers numerous opportunities for physical proximity and contact. Brick’s design forced players to stand close to each other during gameplay, and some of our playtesters went so far as to expect collaboration to be associated with physical proximity. For example, one player said, "I can hold [a collaborative brick], and as soon as you’re holding it as well, that’s when we pick it up."

Design for exertion. Our fndings indicate that designers should account for exertion and fatigue while creating room-scale shared-world AR experiences. In the case of Brick, many players experienced a spike in heart rate while collecting bricks from around the room. While designing specifc interactions in AR, both gross-motor involvement and frequency of repetition may be used to intentionally control difculty and fatigue.

Design for the ecosystem. In shared-world AR, single-player and multiplayer interactions had intrapersonal, interpersonal, and environmental implications. For example, the apparently single-player interaction of collecting an individual brick involves intrapersonal cognition, in which a player comprehends the gestures necessary to pick up, carry, and drop of the brick; interpersonal conversation, as when players help each other locate bricks during gameplay; and environmental awareness, since players navigate the physical space as they move.