Cross-Lingual Image Search on the Web

Most people locate images on the Web by query- ing image search engines such as Google's. The images are tagged by the words in their "vicin- ity", which limits the ability of a searcher to re- trieve them. Although images are universal, an En- glish searcher will fail to find images tagged in Chi- nese, and a Spanish searcher will fail to find im- ages tagged in English. Cross-lingual homonyms cause problems as well. For example, the Hungar- ian word for tooth is 'fog', which makes it chal- lenging for Hungarian searchers to find images of teeth on the Web. To solve these and related problems we introduce the PANIMAGES cross-lingual image search en- gine.1 PANIMAGES enables searchers to trans- late and disambiguate their queries before send- ing them to Google. PANIMAGES utilizes sev- eral machine-readable dictionaries composed into a graph that records relationships between words and word senses across different languages. The graph enables PANIMAGES to infer translations by com- posing information garnered from multiple dictio- naries. PANIMAGES can translate over 437,000 words and supports queries in 50 languages. Our experiments show that for queries in languages with a limited Web presence (ranging from Dutch or Norwegian to Lithuanian or Telugu), PANIMAGES increases the number of correct images by 75% in the first 15 pages (270 results), while increasing precision by 27%.