Workpackages

In this work package, the audiovisual content analysis components will be created. That way, relevant metadata such as speech transcripts, audio and visual concepts and structural information can be generated. The technology will analyze audiovisual files from different sources (e.g. data repositories from broadcasters and news agencies, content from social media platforms, and their associated metadata) in order to extract scene and human-centered activity semantic concepts. The respective analysis tools will allow for the extraction of acoustic cues (presence of speech, presence of music, audio event detection, speaker segmentation and diarization) and automatically produce enhanced transcriptions.

This work package deals with the analysis of textual input from Social Media platforms and the Web. The text analysis will be performed by existing technology that will extract names, locations, organizations, dates, event, relations, topics and sentiments. Different approaches (rule-based linguistics, (Bayes) classifiers, maximum entropy models, hidden Markov models) will be chosen to achieve results of highest quality. The languages covered are English, German, Spanish and French. The main goal of the Social Media Analysis is to enrich the EUMSSI platform knowledge base with social media documents. The components will analyze tweets, Facebook pages, documents from the blogosphere and comments from the official accounts of Deutsche Welle including its official YouTube channels.

The audio, image and video, social media and text analysis components from WP3 and WP4 will deliver heterogeneous (text-based) data streams that describe the content of the media resources and user activity around these resources. The goal of this work package is to ensure the availability of the data and to provide a framework for the data management and workflow. This includes the processing, cross-linking and enrichment of data that results from the analysis processes. The resulting data is  represented in a semantic framework. The framework will allow for semantic reasoning that provides the basis for content-based and collaborative  recommendation strategies that will be used for adaptive content interpretation in WP6.

The goal of this work package is to explore multimodal content-based recommendation techniques and use them in the implementation of the two use cases: a contextualization tool for news editors and a multimodal and collaborative content-based recommendation system for end-users.

The two pilot demonstrations will run in a real-world environment in order to validate the EUMSSI platform by the two addressed user groups, media professionals and TV viewers using second screens. The results gathered in the validation phase will be taken to further improve EUMSSI and its single components.

The dissemination strategy foresees the definition and execution of a multi-channel communication plan to carry out diverse and novel activities and create compelling events that live-up to the wide industry scope of the project and the challenging environment it tackles.

The tasks dedicated to exploitation will analyze and monitor the potential market and business opportunities of EUMSSI, and prepare the route map for its future use.