Click-Search: Supporting Information Search with Crowd-Powered Image-to-keyword Query Formulation

Information search is a common yet important task in everyday work and life. It remains challenging how to help users search for information or things they don’t know how to express with words. Also, even when people know how to express, the cognitive cost required to retrieve the concepts and formulate the queries can be excessive. In this paper, we present Click-Search, a search user interface that allows people to indicate their search intents by merely selecting and cropping segments of pictures. The system automatically converts cropped image segments to keywords based on known semantic labels at the level of image pixels that are generated by crowdsourced image tagging. Through a user study, we showed that Click-Search could support object-finding activities effectively with a satisfactory user experience.