A location-based AR storytelling game built with Google Geospatial API.

Players take on the role of Selena, who's trying to recover lost memories by repairing corrupted photos on her old phone.

Project Type : Augmented Reality / Location-Based Game

My Role: Game Design / 3D Design / Game Programming

Frame by Frame
AR Game

Date : Jan. 2023 Collaborator: Jingyuan Li

Concept

< In this game, you play as Selena, who finds her old phone with corrupted photos inside. You'll need to go to the locations where the photos were taken and match them with the real-world environment to repair the photos and retrieve your lost memories. The truth unfolds as you collect different memory fragments and discover hidden information .>

Player Experience

01 / Entering the game, the player will find two apps—photo gallery and map—on the home screen

02 / In the photo gallery, there are three corrupted photos, each of which was taken at a different location in New York

03 / By tapping one of the photos, the player can see instructions on how to repair the photo and retrieve the related memory

04 / In the map app, the player will see their real-time location as well as three markers

05 / As the player taps on each of the markers, an information page appears, showing where the corrupted photos were taken

06 / When the player gets close to the places in real life, a pop-up notification will appear

07 / The player needs to match the translusent photo with the real-world environment to repair it. When succeeded, Selena's lost memories will appear as 3d models

08 / The player can interact with the 3d objects and discover hidden information that helps reveal the story

09 / The corrupted photos will be repaired after the player completed the tasks

10 / By tapping each photo, the player can check the items they collected and Selena's notes on it

11 / After repairing all of the photos, a system message will show up. The player can decide whether or not to erase the memories again

Development Process

We selected Brooklyn Bridge, Bryant Park, and Rockefeller Center as our three interaction locations since they are famous tourist attractions in New York. This fits well with our story, where characters Selena and Leon were visiting New York for the first time.

At these three locations, we recorded the latitude, longitude, altitude, and rotation quaternion of the phone when taking a picture at a specific location and angle.

Using this data, we placed geospatial anchors at the given latitude, longitude, and altitude using Google's Geospatial API. When players take a photo, Google’s Visual Positioning System (VPS) will determine the precise location of a user’s device. if the player is close enough to the anchor and holding their phone at a similar angle, their photo will be considered "matched" with its location in real life.

We also placed 3d models using anchors so that when the matching process is completed, the player will see the characters appear in front of them.