Concept-based Video Search with the PicSOM Multimedia Retrieval System

Reference:

Ville Viitaniemi, Mats Sjöberg, Markus Koskela, and Jorma Laaksonen. Concept-based video search with the PicSOM multimedia retrieval system. Technical Report TKK-ICS-R39, Aalto University School of Science and Technology, Department of Information and Computer Science, Espoo, Finland, December 2010.

Abstract:

In this report we describe the structure of the PicSOM multimedia retrieval system and elaborate on its automatic concept detection and video search subsystems. We evaluate several alternative techniques for implementing these two components of the PicSOM system in a comprehensive series of experiments employing the large-scale setups of the TRECVID video retrieval evaluation campaigns of 2008 and 2009. Based on the results of the experiments, we conclude that fusion-based shot-wise visual analysis together with N-gram temporal concept-wise post-processing make an efficient combination of techniques for automatic semantic concept detection from video material. It has previously been shown that semantic concepts are very beneficial for video search. Here we investigate the lexical and visual-example-based selection of concepts for search queries, concluding that both selection methods are successful in matching useful concepts. Finally, we show that the performance of the PicSOM system has improved since the TRECVID 2008 evaluation and now compares very well with the state-of-the-art in concept detection and video search.

Suggested BibTeX entry:

@techreport{TKK-ICS-R39,
    address = {Espoo, Finland},
    author = {Ville Viitaniemi and Mats Sj{\"o}berg and Markus Koskela and Jorma Laaksonen},
    institution = {Aalto University School of Science and Technology, Department of Information and Computer Science},
    month = {December},
    number = {TKK-ICS-R39},
    title = {Concept-based Video Search with the {PicSOM} Multimedia Retrieval System},
    type = {Technical Report},
    year = {2010},
}