ZenFri and Durham College develop app to dig deep into museum collections and archives

A creative collaboration between ZenFri and the Mixed Reality Capture Studio is re-imagining the museum experience and leveraging technology to expand the stories curators share with visitors. 

ZenFri is a creative arts and technology studio focused on innovative storytelling, incorporating augmented reality, virtual reality, film, games and more. They are working with the Mixed Reality Capture (MRC) Studio at Durham College to develop Artefacta, a mobile application that allows users to interact with museum content on their own device. 

“Every museum has a little bit of text on the wall that’s geared towards a general audience, but they often have huge archives of stories that they just have no space to tell,” explains Corey King, ZenFri studio head and co-founder.  

But technology presents an opportunity to break those space limitations allowing users to explore a museum’s archive and collections further based on their own interests and curiosity. 

“Now, the text on the wall can become an access point to deeper conversations about that content based on documents that the curators approve,” said King. “It’s about trying to give the museums more power and more voice.” 

MRC Studio senior manager Jason Hunter said museum visitors can use Artefacta to customize their experience. 

“They can scan an item and perhaps watch a documentary about it or link to a story about how it was cleaned up or when it was excavated,” he said. “There are all kinds of different pieces of media that can go along with that – photo and video are sort of the two traditional ones you might think of, but then it goes beyond that to 3D scans of objects or site recreations.” 

The app is focused on accessibility. 

“What Artefacta does is it makes it so that that a museum’s text can be translated into a lot of different languages, it makes so that that text can be spoken,” said King. “If you have difficulty reading or you’re a young kid, you can just turn it on so that every piece of text you look at gets read to you. And you can ask questions back and that’s where we dive into the archival material.” 

Hunter said people often think of gaming in connection to extended reality (XR), but Artefacta is proof of broader possibilities. 

“We’re really proud of this project because it takes the grassroots framework of what XR development is and puts it into a completely different context, which is education and cultural preservation,” he said. “At MRC, we’re interested in exploring how XR content can be used in sectors where people aren’t traditionally thinking of using it.” 

Artefacta uses AI and data on localized computers on the museum’s premises which means the museum’s data is protected. 

“The knowledge of the curators isn’t being fed to a large language model and the data remains in-house,” said Hunter. 

He sees the collaboration with ZenFri as a great example of an applied research success story where a product was brought from the development phase to a protype to a pilot project that is ready to be launched. The next step is testing Artefacta in Canadian museums and MRC will continue to support ZenFri as they move to commercialize the product. 

King described the process of working with the MRC Studio as seamless. 

“They felt like an extension of my team in a real way,” he said. “I talked to them like my team, I set priorities, I reviewed things and it felt very natural. What they actually delivered was more than what I was anticipating they would deliver.”