About the project

About the project.

The SHIFT consortium brings together 13 leading research and industrial organisations and SMEs, coordinated by SIMAVI (Romania), with the vision to strengthen the impact of cultural heritage assets. It will increase the appeal of historical artefacts, improving their accessibility and usability for everyone through better content representation, enriched user experiences, inclusive design, training and more engaging business models.

General overview.

SHIFT: MetamorphoSis of cultural Heritage Into augmented hypermedia assets For enhanced accessibiliTy and inclusion supports the adoption of digital transformation strategies and the uptake of tools within the creative and cultural industries (CCI), where progress has been lagging.

Objectives.

The project is strategically conceived to deliver, by advancing beyond the state-of-the-art, a set of loosely coupled technological tools that offers cultural heritage institutions (CHI) the impetus to stimulate growth and communicate new experiences to all citizens (including people with disabilities) by embracing the latest technological innovations in

artificial intelligence
machine learning
multi-modal data processing
digital content transformation methodologies
semantic representation
linguistic analysis of historical records
use of haptics interfaces

Work packages.

WP1

SHIFT user requirements analysis for Cultural Heritage accessibility and inclusion

Objectives

To study and formalise the stakeholder requirements on cultural curation, accessibility, inclusion and storytelling
To identify the IT infrastructure commonly available among stakeholders to support SHIFT component installation
To develop usability evaluation guidelines and acceptability metrics
To document content distribution processes from end-user organisations to technology providers
To gather, curate, edit, publish UGC (or professional) contents at any scale

WP2

Tools for authoring augmented hypermedia and gesture recognition for enriched accessibility

Objectives

To design and implement novel deep-learning architectures for foreground and background segmentation from CH archives

To investigate physics-informed constraints in training deep-learning algorithms for foreground object motion sequence generation

To develop automated motion sequence generator for foreground object using GANs
To recognition actions from video sequence(s) for generating audio captioning

WP3

Audio engineering framework and multi-modal stimuli transformation

Objectives

To devise comprehensive textual representation algorithms based on novel NLP approaches of cultural heritage assets based on big volumes of descriptions and semantics

To develop a novel text/video to speech tool able to convey the visual / textual stimuli of the source to emotional augmented audio streams

To model temporal evolution of language resources for enriched cultural content curation
To implement haptic interaction with 3D digital tangible and intangible cultural heritage assets, augmenting the Users

Experience (UX) with new interaction paradigms that can be used in situ or remotely

To provide a comprehensive intuitive and accessible for all multimodal storytelling of cultural heritage assets

WP4

Cross-modality curation of cultural content using inclusion by design

To define a set of content pre-processing features and implement the toolbox for metadata extraction
To develop novel approaches for multimedia curation based on association by design
To implement standardised APIs for stakeholder system integration with SHIFT repository
To develop metadata interoperability formats based on association by design for long-term preservation of cultural assets

WP5

SHIFT architecture design, integration, validation, and demonstration

Objectives

To design and develop a valid architectural system and implement a functional platform and define integration patterns
To ensure full functional integration based on the architectural design
To implement a back-end system that currently supports “content management services” and user management for all actors
To perform platform and tools wide validation with specific end-user pilots

WP6

Dissemination and communication

Discover whether the system is right for the market and find bottlenecks and possible oversights. Ensure extensive adoption by stakeholders and the public by disseminating results and outcomes in an effective manner. To develop a sustainable business for the platform, the multimedia video clip and storytelling products

Grow our contributor communities and welcome new audiences to join our brand storytelling, by encouraging them to share their experiences though photos, videos and live video feeds
Deliver authentic, immersive, and relevant content through interactive galleries, social activities, promotional contests and more

WP7

Project coordination

The project management encompasses contractual issues, technical, administrative, finance, communication and knowledge management inside the project, and external relationships between the project and the EC. The Project Coordinator SIMAVI will lead the overall project management including the system integration and technical deployment in the cloud, but it will be assisted by the members of the Steering Committee and the WP leaders. The project coordination aims to guarantee a timely implementation of the respective activities and achievements of the project objectives and the quality of the project deliverables. The objective of this WP is to ensure that the project achieves its goal within the budget and scheduled deadlines.