Joint categorization of queries and clips for web-based video search

Building a video search engine on the Web is a very challenging problem. Compared with web page search, video search has its unique characteristics (such as high volume of data for each video, existence of multi-modal information including meta-data, visual content, audio, closed caption, etc). In this paper, we investigate some promising approaches to boosting the search relevance of a large scale video search engine on the Web. The contribution of our work is three-fold. (1) We developed a specialized video categorization framework which combines multiple classifiers based on different modalities. (2) By learning users' querying history and clicking log, we proposed an automatic query profile generation technique and applied the profile to query categorization. (3) A highly scalable system was developed, which integrates the online query categorization and offline video categorization. Naive Bayes with mixture of multinomials, Maximum Entropy, and Support Vector Machine categorization methods and the profile learn-ing technique were evaluated on a large scale set of video data on the Web. The evaluation of the developed system and user study has indicated that the joint categorization of queries and video data boosts the video search relevance and user search experience. The high efficiency of our approaches is also demonstrated by the good responsiveness of the system for the video search engine on the Web.