As of the time of this writing, statues and memorials of Confederates, slave-owning politicians, explorers, and even U.S. Presidents are being taken down from public places around the country either formally by city officials or informally by demonstrators. The scenes of crowds tearing down these statues are reminiscent of German crowds hammering the Berlin Wall in 1989, Ukrainians pulling down statues of Lenin in 2014, and even depictions of Americans tearing down symbols of the British monarchy at the founding of the country. This has gotten me wondering about how mixed reality technologies can be used to create new digital monuments when physical ones aren’t yet possible, recontextualize controversial monuments that can’t or won’t be removed, and potentially offer problematic monuments a digital afterlife to teach a more accurate version of history.
Mixed reality (frequently referred to as augmented reality) offers a way to challenge and recontextualize the monuments around us. It uses digital devices, geo-location, 3D graphics, and object recognition software to superimpose digital content onto the real world. This content is typically viewed through a smartphone or tablet, but augmented reality glasses and contact lenses could become the norm in the future. This ability to use a device’s location, orientation, and camera to place digital objects in amongst physical objects allows designers to create a layer over reality for users to interact with. As Movers and Shakers NYC’s CEO Glenn Cantave explained in a TED Talk from 2018, augmented reality affords people the ability to place new digital monuments in real physical spaces without the need for approval from governments or private landowners. So unlike physical illegally-placed monuments by activists, digital monuments can’t be taken down by authorities (at least in many countries). Movers and Shakers also took part in an unsuccessful effort to have the statue of Christopher Columbus removed from Columbus Circle in Manhattan. Movers and Shakers created an augmented reality exhibit to highlight Columbus’s history of enslavement and genocide. This augmented reality experience was part of a museum exhibit and not located at the site of the statue, but it easily could have been.
Overlaying physical structures with symbols that reframe the structure’s meaning isn’t new though. Polish artist, Krzysztof Wodiczko, used a powerful projector in 1985 to display a swastika on the façade of South Africa House in London to protest the system of apartheid that was still in practice there at that time. Since then, many activists have used projectors that use the sides of buildings and other physical structures as screens for political messages. Light projectors were also used to create holograms of the Buddhas of Bamiyan that had been destroyed by the Taliban in the positions where the real cliff carvings once stood.
But placing digital monuments in physical spaces does not just have to be the work of activists. Governments or private institutions looking to install monuments on their properties can also turn to augmented reality to help in the design process. Just as IKEA now lets you digitally place furniture in your home to see how it would look before you buy it, institutions could let their constituents view candidates for monument designs as part of a participatory design process. People could view different monument proposals on the spot where they would go, and they could vote for which one they liked the most.
Augmented reality can digitally restore monuments that have been removed or destroyed while also providing users with valuable context about why the monuments were built in the first place. All monuments and memorials are built to express the values of their builders, and they work to shape the worldviews of those who see them. Removing or replacing public displays don’t necessarily erase the stories they tell, but it does change people’s relationships with those stories. Audio narration, video clips, data visualization, and still images can be brought in to do more than just display a 3D model of a structure that is not really there. What type of content the developers want to display will depend on what they want to say about the recreated piece or who it was meant to honor. In this case, monuments go beyond their subjects to become cultural artifacts of their builders as well.
We might wonder why we would want to repurpose monuments that represent values or interpretations of history that we now reject. Allied forces invading Nazi Germany certainly destroyed Nazi imagery, but post-war governments also preserved Nazi constructions like parts of the Atlantic Wall and Auschwitz concentration camps to serve as memorials and educational resources. After 1989, Germans did not pulverize the Berlin Wall completely ever though they enthusiastically rejected the Soviet-imposed government that built it. Part of it has been maintained as a historical site, and large segments of the wall have been sent all over the world. Preserving cultural artifacts of even evil political regimes with proper educational resources can be a way to unmake the distorted view of reality they once helped to make.
Augmented reality tools are not limited to creating just a simple model like a statue. Historic buildings that might be gone can be rebuilt in digital form on the spot where they used to stand. Whole neighborhoods could be overlaid digitally with a smartphone or tablet. One could imagine recreating the neighborhood of Greenwood in Tulsa for locals and tourists to see what the district was like before its destruction in 1921. In fact, the event has already been captured in a virtual reality experience.
City governments or private actors could post markers like QR codes at the locations to direct users to interact with these AR experiences. If there is a city-wide effort to create these kinds of mixed reality historical interest points, schools or libraries could create scavenger hunts to encourage students to visit different locations around the city. Connecting digitally-augmented locations can not only provide a city with a cohesive sense of identity and history, it can highlight the shared history of different cities. I recently watched a video featuring Professor of Art History, Renée Ater, who discussed her research on slavery memorials around the world. One example, the Middle Passage Monument, was sunk several hundred miles off the coast of New York City, and there were plans to submerge replicas of this monument at various other locations along the Atlantic basin that had been parts of the Atlantic slave trade. The project was prohibitively expensive, but digital mixed reality projects connecting several cities could be significantly cheaper. It would also be easy to track public engagement with digital monuments, which would allow participants in each city to feel connected with all of the people in the other cities.
Perhaps the biggest advantage to augmented reality monuments is that people have a choice to engage with them. When there was a groundswell to remove Confederate statues in the mid-2010s, one proposal was to expand the sites around the statues by building new memorials to the victims of slavery and Jim Crow and attaching plaques explaining that the older monuments were built in order to reinforce the Lost Cause myth as a justification for segregation. While these proposals would help people engage with the troubling history in a new way, it ignored the fact that one of the reasons people wanted the statues gone was because they just wanted to be able to go on about their day without constant reminders of their oppression. Augmented reality, however, allows people to pick and choose whether they want to engage with that history. Of course, the advantage of giving people a choice whether to engage or not is also a disadvantage because it requires an active decision to participate as opposed to stumbling upon a physical structure. This means that it will necessarily reach a smaller audience.
Augmented reality also has some important technical limitations right now as well. While WebXR is quickly making it possible to have augmented reality experiences through the web browsers on our phones, many phones are not compatible with this technology. This means that for now, augmented reality experiences need to be available as a downloadable app to reach the widest possible audience. Having to get the app and letting it take up precious real estate in a phone’s storage is another hurdle for getting an audience to engage with your augmented reality experience.Still, the technology is advancing rapidly, and people will gradually adopt more powerful devices that will enable richer experiences. Pokemon GO, for example, recently debuted a new feature that provides for more immersive experiences by using object recognition software to embed the pokemon in your physical environment in a more convincing way. When the app first launched in 2016, the augmented reality feature would superimpose a pokemon in the middle of the phone’s camera view. Two years ago, the company shared a demo of a much more sophisticated augmented reality technology that would recognize ground planes and physical objects allowing Pikachu to run along the ground and even behind physical objects. It will be exciting to see how this kind of technology will be used to create rich educational experiences out in the real world in the years to come.