We present an interactive search assistance agent that integrates search and recommendation-based experiences into a novel unified presentation. The agent guides the user toward more desirable inventory, while also satisfying the constraints that the user constructs with the help of the agent. We built a prototype which supports this mechanism by connecting to the search functionality on eBay. We have also created the popularity knowledge-base of the agent by aggregating data from billions of user transactions. The accompanying video demonstrates the resulting prototype in this context.

In this paper, we propose a new method for addressing post-purchase recommendations for a dynamic marketplace. The proposed method uses the transactional data as the primary data source to mine co-purchase relationships. The item listings from the transactional data are mapped to their static `cluster' representation and a cluster-cluster directed graph is generated. Clusters have explicit definitions and thus it allows us to compute content similarity between any two nodes in the cluster-cluster graph. A large marketplace will have a long tail with respect to the demand (purchase) of the items. It is a well-known problem that pure collaborative filtering systems will be unable to provide relevant recommendations for the long tail. One of the important features of our method is in addressing the issue of sparse transactional data. In addition to computing cluster-cluster relationships, we also compute category-category relationships. When our system does not have sufficient data to compute related clusters for a given cluster using the cluster-cluster graph, we use category-category graph, to first find related categories for a given cluster. We show experimental A/B test results showing significant improvement over a previously reported system that solves the same complex problem.

We present a new algorithm for recommending alternatives to a given item in an e-commerce setting. Our algorithm is an incremental improvement over an earlier system, which recommends similar items by first assigning the input item to clusters and then selecting best quality items within those clusters. The original algorithm does not consider the recent context and our new algorithm improves the earlier system by personalizing the recommendations to user intentions. The system measures user intention using the recent queries, which are used to determine the level of abstraction in similarity and relative importance of similarity dimensions. We show that user engagement increases when recommended item titles share more terms with most recent queries. Moreover, the new algorithm increases query coverage without sacrificing input item similarity and item quality.

We are proposing a new similarity based recommendation system for large-scale dynamic marketplaces. Our solution consists of an offline process, which generates long-term cluster definitions grouping short-lived item listings, and an online system, which utilizes these clusters to first focus on important similarity dimensions and next conducts a trade-off between further similarity and other quality factors such as seller trustworthiness. Our system generates these clusters from several hundred millions of item listings using a large Hadoop map-reduce based system. The clusters are learned using user queries as the main information source and therefore biased towards how users conceptually group items. Our system is deployed on several eBay sites in large-scale and has increased user-engagement and business metrics compared to the previous system. We show that utilizing user queries helps capturing similarity better. We also present experiments demonstrating that adapting the ranking function, which controls the trade-off between similarity and quality, to a specific context improves recommendation performance.

Automatic transfer of learned knowledge from one task or domain to another offers great potential to simplify and expedite the construction and deployment of intelligent systems. In practice however, there are many barriers to achieving this goal. In this article, we present a prototype system for the real-world context of transferring knowledge of American football from video observation to control in a game simulator. We trace an example play from the raw video through execution and adaptation in the simulator, highlighting the system's component algorithms along with issues of complexity, generality, and scale. We then conclude with a discussion of the implications of this work for other applications, along with several possible improvements.

Transfer of learned knowledge from one task to another offers an opportunity to reduce development cost of knowledgebased systems by reusing existing knowledge in novel situations. However, minor differences in the initial and target environments can reduce the effectiveness of the system substantially. In previous work, we presented a system that acquired procedural knowledge of American football from video footage, and then applied it to controlling players in a simulated environment. In this paper, we extend that system by adding the ability to adapt the transferred procedures to better fit the simulator. We show that even when the transferred structural knowledge provides a quality starting point for performance in the game environment, a simple parameter optimization technique can significantly improve its performance and utility.

We describe a new approach for learning procedural knowledge represented as teleoreactive logic programs using relational behavior traces as input. This representation organizes task decomposition skills hierarchically and associate explicitly defined goals with them. Our approach integrates analytical learning with inductive generalization in order to learn these skills. The analytical component predicts the goal dependencies in a successful solution and generates a teleoreactive logic program that can solve similar problems by determining the structure of the skill hierarchy and skill applicability conditions (preconditions), which may be overgeneral. The inductive component experiments with these skills on new problems and uses the data collected in this process to refine the preconditions. Our system achieves this by converting the data collected during the problem solving experiments into the positive and negative examples of preconditions that can be learned with a standard Inductive Logic Programming system. We show that this conversion uses one of the main commitments of teleoreactive logic programs: associating all skills with explicitly defined goals. We claim that our approach uses less expert effort compared to a purely inductive approach and performs better compared to a purely analytical approach.

Using domain knowledge to speed up learning is widely accepted but theory revision of such knowledge continues to use general syntactic operators. Using such operators for theory revision of teleoreactive logic programs is especially expensive in which proof of a top-level goal involves playing a game. In such contexts, one should have the option to complement general theory revision with domain-specific knowledge. Using American football as an example, we use Icarus’ multi-agent teleoreactive logic programming ability to encode a coach agent whose concepts correspond to faults recognized in execution of the play and whose skills correspond to making repairs in the goals of the player agents. Our results show effective learning using as few as twenty examples. We also show that structural changes made by such revision can produce performance gains that cannot be matched by doing only numeric optimization.

In this paper, we present an approach to transfer that involves analogical mapping of symbols across different domains. We relate this mechanism to Icarus, a theory of the human cognitive architecture. Our system can transfer skills across domains hypothesizing maps between representations, improving performance in novel domains. Unlike previous approaches to analogical transfer, our method uses an explanatory analysis that compares how well a new domain theory explains previous solutions under different mapping hypotheses. We present experimental evidence that the new mechanism improves transfer over Icarus' basic learning processes. Moreover, we argue that the same features which distinguish Icarus from other architectures support representation mapping in a natural way and operate synergistically with it. These features enable our analogy system to translate a map among concepts into a map between skills, and to support transfer even if two domains are only partially analogous. We also discuss our system's relation to other work on analogy and outline directions for future research.

Abstract : Developing autonomous agents for computer games is often a lengthy and expensive undertaking that requires manual encoding of detailed and complex knowledge. In this paper we show how to acquire hierarchical skills for controlling a team of simulated football players by observing video of college football play. We then demonstrate the results in the Rush 2008 football simulator, showing that the learned skills have high fidelity with respect to the observed video and are robust to changes in the environment. Finally, we conclude with discussions of this work and of possible improvements.

In this paper we introduce a probabilistic relational framework for automated human behavior modeling. This is achieved by observing a human successfully achieving given goals. The proposed modeling algorithm uses commonsense background knowledge analytically to facilitate modeling of complicated human behavior. At the same time it is grounded in physical observations and takes noisy sensory data as input. The separation of domain axioms and modeling algorithm provides a framework which is easily applicable across domains. The acquired model is represented in the form of probabilistic hierarchical task network which provides flexible models applicable to similar tasks.

This paper describes a system for learning domain-dependent knowledge in the form of goal-indexed Hierarchical Task Networks (HTNs). DLIGHT is a goal-directed incremental learning algorithm which observes solution traces and generates rules for solving problems. One of the main challenges in learning this kind of knowledge is determining a good level of generality. Analytical methods, such as explanation-based macro-operator learning, construct very specific structures that guarantee a successful execution when applicable but generalize poorly to new problems. Previous goal-directed learning approaches produce hierarchical rules with more relaxed preconditions, but the learned knowledge suffers from over-generality. Our approach builds on one such approach but it strikes a better balance between generality and specificity. This is done by carrying out a goal-dependency analysis to determine the structure of the hierarchy and precondition of each rule to follow the successful solutions more closely while maintaining generality. We hypothesize that this algorithm produces HTNs that generalize well and can solve problems efficiently. We evaluate the system's behavior experimentally in several planning scenarios and conclude with related work and future research paths.

Developing computer game agents is often a lengthy and expensive undertaking. Detailed domain knowledge and decision-making procedures must be encoded into the agent to achieve realistic behavior. In this paper, we simplify this process by using the ICARUS cognitive architecture to construct game agents. The system acquires structured, high fidelity methods for agents that utilize a vocabulary of concepts familiar to game experts. We demonstrate our approach by first acquiring behaviors for football agents from video footage of college football games, and then applying the agents in a football simulator.

Knowledge bases play an important role in many forms of artificial intelligence research. A simple approach to producing such knowledge is as a database of ground literals. However, this method is neither compact nor computationally tractable for learning or performance systems to use. In this paper, we present a statistical method for incremental learning of a hierarchically structured, first-order knowledge base. Our approach uses both rules and ground facts to construct succinct rules that generalize the ground literals. We demonstrate that our approach is computationally efficient and scales well to domains with many relations.

Important advances in automated planning have been made recently, especially with the development of domainconfigurable planning systems. These planners use a domainindependent search engine for planning, but they have also the ability to exploit domain-specific planning knowledge. Examples of such planners include the well-known TLPLAN (Bacchus & Kabanza 2000), TALPLANNER (Kvarnstr¨ om & Doherty 2001), and SHOP2 (Nau et al. 2003). One challenge for domain-configurable planners is that they require a domain expert to provide planning knowledge to the system. When this knowledge is not accurate, complete, poorly expressed, the performance of these planners diminishes considerably and very quickly, even in simple planning benchmarks. In this paper, we present a preliminary report on our research aimed to mitigate this issue by combining the use of domain-specific knowledge and domainindependent heuristic search. We describe H2O (short for Hierarchical Heuristic Ordered planner), a new Hierarchical Task-Network (HTN) planning algorithm that can heuristically select the best task decompositions by using domainindependent state-based heuristics. Our experiments in the DARPA Transfer Learning Program demonstrated the potentialities of H2O: given HTNs generated by a machine-learning system, which were much less optimal than an expert would encode, H2O was able to solve problems that SHOP2 could not.

Transfer is the ability to employ knowledge acquired in one task to improve performance in another. We study transfer in the context of the ICARUS cognitive architecture, which supplies diverse capabilities for execution, inference, planning, and learning. We report on an extension to ICARUS called representation mapping that transfers structured skills and concepts between disparate tasks that may not even be expressed with the same symbol set. We show that representation mapping is naturally integrated into ICARUS' cognitive processing loop, resulting in a system that addresses a qualitatively new class of problems by considering the relevance of past experience to current goals.

In this dissertation, we investigate learning by observation , a machine learning approach to create cognitive agents automatically by observing the task-performance behavior of human experts. We argue that the most important challenge of learning by observation is that the internal reasoning of the expert is not available to the learner. As a solution, we propose a framework that uses multiple complex knowledge sources to model the expert more accurately. We describe a relational learning by observation framework that uses expert behavior traces and expert goal annotations as the primary input, interprets them in the context of background knowledge, inductively finds patterns in similar expert decisions, and creates an agent program. The background knowledge used to interpret the expert behavior does not only include task and domain knowledge, but also domain independent learning by observation knowledge that models the fixed mental mechanisms of the expert. We explore two learning approaches. In learning from behavior performances approach, the main source of information used in learning is behavior traces of expert recorded during actual task performance. In the learning from diagrammatic behavior specifications approach, the expert specifies behavior using a graphical representation, abstractly depicting the critical situations for the desired behavior. This provides the expert with additional modes of interaction with the learning system; simplifying the learning task at the expense of more expert effort. Both of these approaches are uniformly represented in relational learning by observation framework. Our framework maps "learning an agent program" problem on to multiple learning problems that can be represented in a "supervised concept learning" setting. The acquired procedural knowledge is partitioned into a hierarchy of goals and it is represented with first order rules. Using an inductive logic programming (ILP) learning component allows our system to combine complex knowledge from multiple sources. These sources include the behavior traces, which are temporally changing relational situations, the expert goal annotations, which are hierarchically organized and provide structured information, and background knowledge, which is represented as relational facts and first order rules. Our learning by observation framework needs to store large amounts of behavior data and access it efficiently during learning. We propose an episodic database as a solution, which is an extension of Prolog that improves Prolog by providing efficient and power mechanisms to store and query relational temporal information. We evaluated our framework using both artificially created examples and behavior observation traces generated by AI agents. We developed a general methodology to test relational learning by observation. Our methodology is based on first using a hand-coded agent program as the expert, and then comparing the decision making knowledge of the expert and learned agent programs on observed situations.