Menu
Berlin, Kulturforum & Gallery of Old Masters, 7-11 July 2025
Event title: Please Touch! Towards an AI-based inclusive, multi-sensory experience of the Museum. A Virtual Journey into Inclusive Culture
Host: Stiftung Preußischer Kulturbesitz – Staatliche Museen zu Berlin (SMB PK) – Kunstgewerbemuseum and Gemäldegalerie | Museum of Decorative Arts and Gallery of Old Masters
Venue & date: Berlin, 7–11 July 2025.
Objective: The Berlin Shift pilot was implemented in close alignment with the KPIs defined for the Shift tools. Its central ambition was to expand the exhibition experience into an inclusive, multisensory journey. Landscape and cityscape paintings are reinterpreted as immersive sound environments. Paintings from the Gallery of Old Masters and the National Gallery Berlin are automatically deciphered and described using complementary information drawn from curated archival and online sources. Image-to-text-to-speech technologies make artworks more accessible for visually impaired persons (VIPs) and for the wider public. Finally, the Shift XR Accessibility Framework should integrate the various tools through a tactile interface. Here, visitors, especially blind and visually impaired visitors, were able to feel and explore a virtual 3D museum with the help of a headset and a data glove. Touchable hotspots link to descriptive texts and audio narrations, thereby enhancing both accessibility and immersion.
In order to test these tools with the intended target groups, as well as with colleagues and museum professionals, and to gain an impression of user response, a three-part process with distinct formats was implemented:
Pilot XR Accessibility Workshop
The Shift Pilot “CH – Visitor Journeys Without Sensing Boundaries,” showcased the integration of diverse shift tools through the Shift XR Accessibility Framework as showcased in Figure 3. Equipped with a headset and data glove, visitors entered an extended reality museum landscape, where they could explore Berlin’s cultural treasures through a multisensory lens. Objects that remain locked behind glass cases—untouchable and immovable—became tactile, movable, and even audible within the virtual museum. Under the theme “Please Touch! Shift | tactile, auditory, adaptive,” the experience opened an inclusive new dimension of perception, allowing visitors to engage with icons such as the bust of Nefertiti, the Pergamon Altar, and the Celtic gold helmet from the National Museums in Berlin.
The complex testing of the XR Accessibility Platform, supervised by professional staff, took place in the small theatre of a semicircular demo room in Berlin’s Gemäldegalerie. The testing was conducted in one-on-one sessions, which were accompanied and formatted using standardized evaluation methods.
Around 60 participants from three visitor groups were carefully selected for this pilot segment: blind and visually impaired people, the general museum audience, and cultural experts and curators. Each person was guided through the XR experience for about 45 minutes. While the participants moved through the XR museum world with headsets and data gloves, their actions were simultaneously projected onto the wall. This allowed the people accompanying them and other visitors to actively participate in the experience. Afterwards, a comprehensive survey was conducted using standardized questionnaires to extensively evaluate the actual quality of the experience and visitor feedback.
Number and Distribution of Attendees: Approximately 60 selected participants tested the XR applications and completed questionnaires. These participants were evenly distributed across three groups:
In addition, a further 60–80 individuals followed the event via live projection and expressed interest in this innovative approach to cultural education. For this latter group, no formal evaluation was carried out.
Tools being tested and evaluated: Haptic Glove, Image-to-Text, Image-to-Video, Image-to-Soundscape, Text-to-Speech; Audio Narrative, 3D Segmentation.
Ordinary Feedback
Shift Demo Showroom
To showcase the full range of Shift tools in action, a Shift Showroom was hosted in the Gallery of Old Masters from 9–11 July 2025, located directly adjacent to the aforementioned Accessibility Framework Workshop room. The event offered a variety of formats designed to engage visitor’s hands-on in different ways.
Exhibition
PLEASE TOUCH / SHIFT | tactile, auditory, adaptive. Towards an AI-based inclusive, multi-sensory experience of the Museum. New projects for virtually open up collections to all your senses.
The exhibition featured six large-scale reproductions of paintings mounted on panels, allowing visitors to engage with them at their own pace. To complement the visual experience, tablets provided access to Shift-generated content, including audio and text descriptions, narrations, and videos created with AI. The invitation for the poster is presented in Figure 4. In addition, QR codes placed on the works enabled visitors to access this information — as well as entertainment features — directly on their own mobile devices, in detail:
1. Hans Holbein (the younger), The merchant George Gisze (1497-1562), oakwood, 97.5 x 86.2 cm, London, 1532, Staatliche Museen zu Berlin, Gemäldegalerie, Ident. No.: 586
2. Arnold Böcklin, Isle of the Dead, oil on panel, 80 x 150 cm, 1883, Staatliche Museen zu Berlin, Alte Nationalgalerie, Ident. No.: NG 2/80.
3. Karl Friedrich Schinkel, Gothic Church on a Rock by the Sea, Oil on canvas, 71 x 98 cm, 1815, Staatliche Museen zu Berlin, Alte Nationalgalerie, Ident. No.: NG 2/80.
4. Gottlieb Schick, Portrait of Heinrike Dannecker, oil on canvas, 119 x 100 cm, 1802, Staatliche Museen zu Berlin, Alte Nationalgalerie, Ident. No.: A II 840.l
5. Max Liebermann, Garden Restaurant by the Havel. Nikolskoe, Oil on Canvas, 71.5 x 86.5 cm, 1916, Staatliche Museen zu Berlin, Alte Nationalgalerie, Ident. Ident. No.: NG 4/94.
6. Gustave Courbet, The Wave, oil on canvas, 112 x 144 cm, 1869/1870, Staatliche Museen zu Berlin, Alte Nationalgalerie, Ident. No.: A I 967.
The visitors showroom experience is presented in Figure 5.
Hands-On and lecture programme
Throughout the event, numerous presentations and demonstrations of the Shift tools were projected onto a large screen in the same room as the exhibited paintings by representatives from various Shift partners as presented in Figure 6. These specialists were on hand at all times to answer questions about the technology and, more importantly, to provide hands-on support to interested visitors. The programme covered all aspects of the project, from the integrated Shift platform to issues of authenticity and credibility of AI-generated content in the CH sector.
Programme
WEDNESDAY July 9
10:00-12:00 Visual Storytelling. From Image to Video- Balkan Art in Motion
Krishna Chandramouli | Queen Mary University London
14:00-15:30 Stories Continued: Visual Extensions of Still Images. Paintings in
dialogue
Krishna Chandramouli | Queen Mary University London
18:30-20:00 From Image to text to Speech – An AI journey to Inclusiveness of Art
Beatrice Aretz, Chair of Health Informatics, TUM School of Medicine &
Dionyssos Kounadis-Bastian, audEERING GmbH-Agile Robots SE
THURSDAY July 10
10:00-12:00 SHIFT platform – Practical Use of AI-Based Tools in the Collections
Razvan Purcarea, SIMAVI Software Imagination & Vision
14:00-15:30 From Image to text to Speech – An AI journey to Inclusiveness of Art
Beatrice Aretz, Chair of Health Informatics, TUM School of Medicine &
Dionyssos Kounadis-Bastian, audEERING GmbH-Agile Robots SE
16:00-18:00 SHIFT platform – Practical Use of AI-Based Tools in the Collections
Razvan Purcarea, SIMAVI Software Imagination & Vision
FRIDAY July 11
10:00-12:00 Please Touch! The inclusive Museum.
Milena Milošević Micić, Director
The Homeland Museum of Knjaževac & Balkan Museum Network
14:00-15:30 Landscape to Soundscape (first results and future perspectives)
Andreas Bienert, former Digital Collections and documentation, SMB
iconclass_earconclass_SMB_Bienert.pdf
16:00-18:00 The Inclusive Museum – first results and predictions
Reiner Delgado & Moritz Maier, Deutscher Blinden- und
Sehbehindertenverband e.V. Berlin
Shift Showroom / Hands-On and lecture programme
The audience and number of visitors to the events at the SHIFT Showroom July 9-11, 2025:
A significant proportion — approximately 30% of the roughly 100 visitors over the three-day event — comprised invited individuals and their accompanying persons who participated in testing the XR Accessibility Framework. An additional 15% consisted of museum staff and colleagues, primarily working in the field of cultural education. Thus, about half of all visitors and participants across the three showroom events fell into the category of general museum visitors, a group that was not further subdivided.
Event formats: Exhibition, Lectures, Hands-On Demo, Mobile Devices.
Tools demonstrated: Image-to-Text, Image-to-Video, Image-to-Soundscape, Text-to-Speech; Audio Narrative, 3D Segmentation, AI-based narration, Foreground/Background Segmentation, Image to Video Transformation, Shift-platform.
Feedback: Overall, the audience’s assessment of the technologies presented was overwhelmingly positive, especially with regard to the potential of multimodal communication. In particular multisensory experience options were clearly confirmed, while criticism and reservations are mainly directed at the quality of the performance achieved so far: both the auditory and haptic implementation were often rated as still insufficient.
The positive response to the showroom proves that SHIFT solutions can move from prototype to operational use – from 3D segmentation by Massive Dynamic Sweden and VR integration by FORTH, to TTS narratives and curatorial soundscapes – offering museums a credible path to deliver accessible, multi-sensory, user-centred experiences. The next step is continuous refinement based on audience feedback and integration into the curatorial and educational cycles of museums across the European network.
Shift Opening Event: Tactile, Auditory, Adaptive
The pilot activity was promoted through a digital poster which was actively promoted through social media and is presented in Figure 9.
AI-based tools in museums and collections. Digital techniques for inclusive and multi-sensory access to cultural heritage
Venue: Staatliche Museen zu Berlin – Preußischer Kulturbesitz, Kulturforum, July 9, 2025 | 5 p.m. to 10 p.m.
Objective: To promote the inclusive transformation of museums with virtualised exhibitions and AI-based support, an opening event for the SHIFT SHOWROOM was held on Wednesday, 9 September 2025, from 5 p.m. to 10 p.m. at the Kulturforum. For this event, a collaboration was arranged with the museum project ‘Virtual Couture,’ whose temporary exhibition was coinciding at the Kunstgewerbemuseum. The opening featured several lectures and presentations on the virtualisation of museum work and the innovative possibilities of inclusive, creative, diverse and adaptive communication. Afterwards, the audience was explicitly invited to experience the various Shift tools of multimodal communication and multisensory perception in a joint ‘hands-on’ session. The pictures from the welcome event is presented in Figure 10.
Programme [EN]
17:00 Welcome – Sibylle Hoiman, Director of the Berlin Museum of Decorative Arts
SHIFT & ViCo. Two digital perspectives on art and fashion in
museums.
17:20 Introduction – Andreas Bienert, Digital collections and documentation, SMB
A new member of the team? How AI can help enrich the museum
experience and leave no one behind.
17:40 Presentation – Sabine de Günther, FB Informationswissenschaften FH-Potsdam
ViCo Fashion 3D – digitized, animated, and interpreted
18:00 Presentation – Katerina Valakou, RaD Engineer, ICS-Forth, Greece
Moritz Maier, Projektkoordination SHIFT, DBSV, Berlin
Polyphonic images – dialogues without barriers
18:30 Hands-on / Get-together
Hands-on SHIFT and guided tour of the KGM and SHIFT showroom
under the rotunda of the Gemäldegalerie.
Grant Agreement Number: 101060660
Project Full Title: MetamorphoSis of cultural Heritage Into augmented hypermedia assets For enhanced accessibiliTy and inclusion
Project Acronym: SHIFT
Topic: HORIZON-CL2-2021-HERITAGE-01
Type of action: HORIZON-RIA
Granting authority: European Research Executive Agency (REA)
Start date of the project: 1 October 2022
Duration: 36 months
EU Contribution: 3 527 250.00 Euro
“Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Executive Agency (REA). Neither the European Union nor the granting authority can be held responsible for them.”