More than 300 researchers gathered at the 2013 International Brain-Computer Interface (BCI) Meeting to discuss current practice and future goals for BCI research and development. The authors organized the Virtual Users' Forum at the meeting to provide the BCI community with feedback from users. We report on the Virtual Users' Forum, including initial results from ongoing research being conducted by 2 BCI groups. Online surveys and in-person interviews were used to solicit feedback from people with disabilities who are expert and novice BCI users. For the Virtual Users' Forum, their responses were organized into 4 major themes: current (non-BCI) communication methods, experiences with BCI research, challenges of current BCIs, and future BCI developments. Two authors with severe disabilities gave presentations during the Virtual Users' Forum, and their comments are integrated with the other results. While participants' hopes for BCIs of the future remain high, their comments about available systems mirror those made by consumers about conventional assistive technology. They reflect concerns about reliability (eg, typing accuracy/speed), utility (eg, applications and the desire for real-time interactions), ease of use (eg, portability and system setup), and support (eg, technical support and caregiver training). People with disabilities, as target users of BCI systems, can provide valuable feedback and input on the development of BCI as an assistive technology. To this end, participatory action research should be considered as a valuable methodology for future BCI research.

We present a lightweight tool to compare the relevance ranking provided by a search engine to the relevance as actually judged by the user performing the query. Using the tool, we conducted a user study with two different versions of the search engine for a large corporate web site with more than 1.8 million pages, and with the popular search engine GoogleTM. Our tool provides an inexpensive and efficient way to do this comparison, and can be easily extended to any search engine that provides an API. Relevance feedback from actual users can be used to assess precision and recall of a search engine’s retrieval algorithms and, perhaps more importantly, to tune its relevance ranking algorithms to better match user needs. We found the tool to be quite effective at comparing different versions of the same search engine, and for benchmarking by comparing against a standard.

One factor that may affect whether users of technical support Web sites can rapidly find information relevant to their needs is the quality of the summary of documents returned as the result of search queries. This paper reports on two studies that were part of an effort to create high-quality machine-generated summaries for the presentation of search results for technical support documents. The initial study asked experts to compose document summaries. The results of the first study were used to guide the development of heuristics for generating programmatic summaries that were tested in the second study, which was a user evaluation that compared the effectiveness of four types of document summaries for search purposes: programmatic summaries based on selective sentence extraction using knowledge of the semantic structure of documents, a term-hits-in-context (THIC) summary, the current summaries on the company's live site, and document titles alone. This comparison sought to determine the techniques most likely to help users find information, hence increasing customer goal attainment and satisfaction. The implications of our results for summarizing technical support documents for search are discussed.

In this paper we present what we have learned while working with a large distributed marketing organization in order to design and develop ContactPoint, a personalized content delivery system and collaborative tool. It can be very difficult for members of large organizations to stay informed about what is happening in different parts of the organization. ContactPoint allows members of the organization to subscribe to one or more information sources, receive information alerts on their desktop, easily identify subject matter specialists, and collaborate with those specialists.

This paper describes the evaluation of a natural language dialog-based navigation system (HappyAssistant) that helps users access e-commerce sites to find relevant information about products and services. The prototype system leverages technologies in natural language processing and human-computer interaction to create a faster and more intuitive way of interacting with websites, especially for less experienced users. The result of a comparative study shows that users prefer the natural language-enabled navigation two to one over the menu driven navigation. In addition, the study confirmed the efficiency of using natural language dialog in terms of the number of clicks and the amount of time required to obtain the relevant information. In the case study, as compared to the menu driven system, the average number of clicks used in the natural language system was reduced by 63.2% and the average time was reduced by 33.3%.

With the emergence of e-commerce, websites must accommodate both customer needs and business requirements. Menu driven navigation and keyword search provided by most commercial sites have tremendous limitations. There is no way to balance the current needs and intentions of a user with the business requirements of the site. Often, as a result, users are overwhelmed and frustrated by the lengthy interaction, because it’s hard to precisely describe their intentions, e.g. buying "dark pants without cuffs". The solution lies, in our opinion, in centering electronic commerce websites around natural language and multimodal dialog. This claim is supported by results of a recent study we performed, and which we present in this paper.

The video illustrates how a customer logged in from home over a single phone line to a Web-based Internet banking self-service can invoke human assistance on demand using a customer care technology called CLIVE [1]. Once connected, the customer and the customer service representative (CSR) can speak with each other, interact with the contents of a shared Web page, and maintain awareness of each other’s location on the Web page.