Multimodal Representations of Cultural Heritage
​​Key Activities:​
-
Metadata & Ontologies: We are creating systems to organize and connect different types of data, enabling seamless transformation across formats.
-
Multimodal & Multisensory Representation: Research is being conducted on how to best represent CH using multiple senses (e.g., sight, sound, touch) for a more immersive experience.
-
Co-design & Participation: We use participatory methods, involving the community in designing multisensory representations to ensure inclusivity and relevance.
-
Aesthetic & Experiential Validation: We gather feedback and requirements through co-design methods to validate how well the representations evoke emotional and aesthetic responses.​
-
Complementary communication methods for inclusive multimodal access: We are developing complementary a haptic communication systems, HaptiVerse, to work alongside visual and auditory modalities and support meaningful access to cultural content for people with dual sensory impairments.​
​
Our Virtual Exhibition
​
We have been working on the design of a virtual exhibition to incorporate the multimodal representations of cultural assets. Users are able to experience 2D and 3D representations of statues and paintings and get engaged in multisensory interactions with the space and the artefacts. Therefore, users have the opportunity to interact through their vision, hearing, and haptic senses.
Our haptic communication system "HaptiVerse"
​
We have been developing a comprehensive system for haptic communication that enables meaning-bearing tactile patterns, to be designed, sent, and received either in person or remotely. HaptiVerse supports multisensory access to cultural heritage by exchanging information in haptic format alongside visual and audio media — offering a more inclusive experience, especially for individuals with sensory impairments such as deafblindness. The system includes a design tool, called HaptiDesigner; a library of reusable haptic signs, called Haptiotek; modular wearable or actuated‑device support, HaptiMesh and HaptiWear; and functionality for real-time or asynchronous use, expanding the ways in which one can receive information and interact with cultural objects.
Get in touch with us if you want to access the Haptiverse! (contact@muse-it.eu)
We want to extend cultural experiences to all senses, and in particular beyond visual and auditory senses. Our goal is to develop advanced ways of representing cultural heritage that make it more accessible and inclusive. By combining different sensory experiences (visual, audio, haptics) and layers of information, we aim to create richer, more engaging interactions with both tangible and intangible heritage.
This means that cultural objects such as works of art, crafts, heritage sites or historical archives can be experienced virtually, both by sight (images, videos or texts), hearing (sound, music or audio description) and touch (through haptic technologies).
Here’s how we’re working toward this goal: