Beyond VR: How Mixed Reality and Spatial Computing Are Rewriting the Museum Visit
Introduction
During the COVID-19 pandemic, many museums offered virtual tours and VR exhibits to connect with remote audiences. These efforts showed that digital tools can keep museums relevant even when physical visits aren’t possible. Now, as visitors return on-site, the focus is shifting beyond VR to approaches that blend the digital and physical. Mixed Reality (MR) and spatial computing lead this shift, making museum visits more interactive and personalised.
Unlike Virtual Reality (VR) – which puts you in a completely virtual environment, MR and spatial computing overlay digital content onto the real museum space. Visitors still see the actual gallery and artefacts around them, with added digital elements on top. The goal isn’t to replace physical exhibits, but to enhance them with context-aware digital overlays. This article explains these concepts and explores how they’re rewriting the museum experience in practice.
From VR to XR: Key Terms Explained
Virtual Reality (VR): A fully digital environment. Wearing a VR headset, you are transported somewhere else entirely and cannot see the real museum space.
Augmented Reality (AR): Digital content layered on the real world, usually via a smartphone or tablet. For example, pointing your phone at a painting might make an animation or extra information appear on the screen.
Mixed Reality (MR): Advanced AR using see-through devices (like Microsoft HoloLens or Apple’s Vision Pro) that integrate virtual objects into the real environment. MR objects appear fixed in place and can interact with real surfaces, imagine a holographic historical figure standing next to an artifact as you move around it.
Spatial Computing: Technology that lets computers understand and interact with 3D space. It underpins AR/MR by tracking the user’s position and surroundings so that digital content can be contextually placed in a museum (aware of which room or exhibit you’re in).
In short, AR and MR (often collectively called XR for “extended reality”) let museums combine digital storytelling with physical exhibits. Next, we’ll see why this is so promising beyond what basic VR can offer.
Why Move Beyond VR?
AR and MR have clear advantages for on-site use in museums:
Visitors Stay Present: AR/MR add to the real gallery instead of pulling you into a separate virtual world. The experience stays grounded in the museum – you can view exhibits and fellow visitors while also engaging with digital enhancements.
Shared Experiences: Museum visits are social, but a VR user is isolated behind a headset. With AR and MR, multiple people can share an experience together. For example, a family can jointly watch an AR animation unfold over an artifact on their own devices, keeping the visit collaborative2.
Lower Barriers: AR often runs on smartphones or tablets visitors already carry, so museums don’t always need expensive headsets or special setups2. Even MR headsets, while currently pricey, let users move freely through exhibits without some of the physical constraints of older VR systems.
In-Context Learning: AR/MR tie information directly to objects and spaces. A visitor might look at a statue through a tablet and see it “restored” to its original colors virtually, right on the statue. This helps people instantly understand context or visualise the past in ways a static text panel cannot1.
Physical-Digital Blend: Rather than replacing real artefacts with digital ones, AR/MR combine them. The real object remains the centrepiece, while digital content provides additional layers – an animation showing how a tool was used, a map illuminating where an artefact came from, etc. This enriches the story without losing the authenticity of physical objects.
Mixed Reality in Action: Museum Examples
Many museums have begun piloting AR and MR on their gallery floors. For example:
Natural History Museum (London): In 2024 the NHM launched Visions of Nature, an MR experience using Microsoft HoloLens 2 headsets. Visitors could walk around a gallery and encounter holographic animals and environmental scenes woven with the real exhibits. A virtual guide narrated future Earth scenarios as digital creatures roamed the physical room, bringing an educational climate story to life right in the museum.
Cleveland Museum of Art: For the 2021 exhibition Revealing Krishna, the museum offered an MR tour via HoloLens. Visitors saw 3D holograms and animations superimposed around a sculpture of Krishna, virtually “traveling” to the artefact’s original site in Cambodia and witnessing scenes from its history, all while standing in front of the real object.
National Museum of Singapore: The museum’s Story of the Forest installation uses AR to animate 19th-century illustrations from its collection. Visitors with a smartphone app see birds, animals, and plants from the artwork come to life as digital creatures in the gallery. They can “catch” these virtual animals by finding them (like a Pokémon Go-style scavenger hunt) and then learn details about each species. This interactive approach turned a traditional exhibit into a playful educational adventure.
Other examples: The de Young Museum in San Francisco set up an AR station for virtually trying on historical costumes, and the National Gallery in London projected famous paintings onto city streets via AR to engage the public beyond the museum’s walls.
Benefits for Museums and Visitors
When thoughtfully implemented, mixed reality can offer several benefits:
Deeper Engagement and Learning: Visually rich, interactive content captures attention better than static displays. Studies suggest well-designed AR/MR experiences can boost visitors’ enjoyment and understanding of exhibits.
Appeal to Tech-Savvy Audiences: Featuring AR or holograms can make museums more attractive to younger, tech-oriented visitors. It offers a fresh twist on museum-going that counters the “dusty museum” stereotype, helping museums connect with the digital generation.
Accessibility and Preservation: AR apps can provide on-demand translations, audio descriptions, or sign-language support, making exhibits more inclusive. Digital augmentation also allows some experiences to extend beyond museum walls (for instance, an AR exhibit accessible remotely), and lets visitors engage with delicate or distant objects via realistic 3D models – preserving originals while expanding access.
Curatorial Insights: Digital experiences can yield data on what content visitors engage with most, helping museums understand audience interests. This feedback can inform exhibit planning and highlight which stories or objects resonate strongly.
Challenges and Considerations
Introducing AR/MR in a museum setting comes with a few challenges:
Infrastructure and Maintenance: Quality AR/MR content can be costly to produce, and the hardware (devices, software, Wi-Fi) needs upkeep. Plan for upfront development costs and ongoing maintenance, including software updates, device charging/cleaning, and tech support. Partnerships or grants can help offset costs, but long-term sustainability must be considered.
User Experience: If the interface is confusing or buggy, visitors may give up quickly. The experience should be intuitive with clear instructions. Brief on-site demos can help, and staff or volunteers should be on hand to assist with setup (for example, helping someone launch the app or adjust a headset).
Content Over Gimmick: Every digital feature should serve a purpose. It’s important that AR/MR elements add real value or enjoyment aligned with exhibit goals – not just tech for tech’s sake. Involving educators and curators in design can ensure the focus stays on meaningful content rather than gimmicks.
Visitor Comfort and Safety: Visitors using AR (looking through screens or wearing headsets) must be able to do so safely. Museums may need to mark AR-friendly areas and keep pathways clear. Shared devices should be cleaned between uses for hygiene. And because not everyone will use AR/MR, all essential information should still be available in traditional formats so no visitor is left out.
Conclusion
Mixed reality and spatial computing are beginning to rewrite the museum visit, adding new dimensions to how stories are told. These technologies allow museums to keep the essence of the in-person experience, the excitement of seeing authentic objects, while layering on interactive, immersive elements that appeal to contemporary audiences. A hologram or AR animation will never replace a real artifact, but it can augment the encounter, making it more informative, engaging, and memorable.
Museums around the world are already exploring creative ways to use AR and MR, from bringing paintings to life on city streets to adding holographic guides in gallery halls. The key is to use these tools in service of storytelling and learning. When done right, mixed reality experiences can deepen understanding, attract new visitors, and re-energise museum spaces, all without sacrificing the integrity of the traditional museum visit.
