A multimedia content search engine thanks to AI and Google Cloud Platform

The classification of multimedia content is an area that lends itself to the use of an Artificial Intelligence engine and to the introduction of machine learning techniques. The computing capabilities possible thanks to the use of cloud infrastructures also make the processing independent of the size, and type, of the information to be managed. Therefore, the management of huge multimedia catalogs typical of the broadcasting market becomes feasible and effective.

The client that contacted Lutech for this major multimedia content classification project is one of the largest European media and television content providers with a huge satellite and digital network infrastructure.
The company has also always been naturally suited to assessing new technologies and is constantly looking for processes that have a positive impact on its audience and, consequently, on its business.

The primary need expressed by the broadcaster concerned the tagging of its entire multimedia catalog in order to extract a set of content-based information to be made available to a search and correlation engine for use by both its consumers and internal operators.

Customizing the proposals for the end customer and offering the best possible customer experience once again lies at the heart of this innovative project.

While current technologies allow the analysis of a text, or of voice content, in a relatively simple way, the same cannot be said for contents such as images or videos. In particular, the request was to be able to tag a video based on the objects displayed, celebrities, topics and other features included in the frames.

The solution identified by Lutech involved using GCP Vertex AI as the platform, from which to develop a custom engine for analysis and classification of video content. In particular, it was a question of retrieving various video analysis services including object detection and speech-to-text based on technologies provided by Google platform.

In addition to this set of application tools, Lutech's development team added custom models for use cases not covered by the platform. 
The project was implemented through a web application whose engine analyzes videos by launching a sequence of processes in parallel, in order to increase processing speed. 

For this purpose, the APIs (Application Programming Interfaces) already available on the Google Cloud platform were used. These include Speech-to-text, Vision APIs, Video Intelligence algorithms and APIs for the analysis of Natural Processing, with applications on use cases related to object extraction, face recognition, and content detection, to name just a few.

For this purpose, APIs (Application Programming Interfaces) already available in the GCP Vertex AI platform were involved. These included Cloud Speech API, Cloud Video Intelligence API, and Cloud Natural Language API, with applications on use cases related to object extraction, face recognition, and content detection, to name a few.

Upon the rollout of Lutech’s project, the customer gained an application system with a high level of accuracy, based on cloud platform algorithms regularly fine-tuned by Google's team. The system performance was also very high thanks to the complete scalability of the components used and, moreover, given the nature of the algorithms used, the opportunity for continuous learning arises.

Ultimately, the broadcaster can now enjoy previously unheard of processing speeds, and its customers can make use of a content search engine which is unmatched amongst Italian multimedia operators.