UB at GeoCLEF 2006

This paper summarizes the work done at the State University of New York at Buffalo (UB) in the GeoCLEF 2006 track. The approach presented uses pure IR techniques (indexing of single word terms as well as word bigrams, and automatic retrieval feedback) to try to improve performance of queries with geographical references. The main purpose of this work is to identify the strengths and shortcomings of this approach so that it serves as basis for future development of a geographical reference extraction system. We submitted four runs to the monolingual English task, 2 automatic runs and two manual runs, using the title and description fields of the topics. Our official results are above the median system (auto=0.2344 MAP, manual=0.2445 MAP). We also present an unofficial run that uses title description and narrative which shows a 10% improvement in results with respect to our baseline runs. Our manual runs were prepared by creating a Boolean query based on the topic description and manually adding terms that are consulted from geographical resources available on the web. Although the average performance of the manual run is comparable to the automatic runs, a query by query analysis shows significant differences among individual queries. In general, we got significant improvements (more that 10% average precision) in 8 of the 25 queries. However, we also noticed that 5 queries in the manual runs perform significantly below the automatic runs.