Deep neural networks have achieved remarkable performance across a variety of applications. However, their decision-making processes are opaque. In contrast, k-nearest neighbor (k-NN) provides interpretable predictions by relying on similar cases, but it lacks important capabilities of neural networks. The neural network k-nearest neighbor (NN-kNN) model is designed to bridge this gap, combining the benefits of neural networks with the instance-based interpretability of k-NN. However, the initial formulation of NN-kNN had limitations including scalability issues, reliance on surfacelevel features, and an excessive number of parameters. This paper improves NN-kNN by enhancing its scalability, parameter efficiency, ease of integration with feature extractors, and training simplicity. An evaluation of the revised architecture for image and language classification tasks illustrates its promise as a flexible and interpretable method.

Manual knowledge acquisition of case retrieval features is expensive and may be infeasible for cases containing hard-to-characterize data such as images. Deep learning (DL) methods excel at extracting useful feature information from raw data, making them appealing for learning feature information. Previous work has demonstrated the promise of integrated systems for case-based image classification, using a deep neural network to generate features which are then used for case retrieval, resulting in classifications that can be explained in terms of prior cases. However, the accuracy of the combined system may lag behind that of the original DL model. In response, our previous work proposed MultiNet, a method using ensembles for localized feature extraction. Multi-Net improved performance, but experiments showed limitations of its design. This paper presents Deep Ensemble Feature Extraction for Retrieval (DEFER), a feature-extraction-based classification approach aimed at addressing those issues. To increase accuracy, DEFER adds a discriminator to focus retrieval within each replica and weighted voting based on confidence in its class prediction, grounded in nearest-neighbor retrieval. In experiments for image classification, DEFER outperforms analogous DL-only and DL-case-based systems, supporting that its approach can improve performance.

Case-based prediction (CBP) methods do not learn a model of the target decision function but instead perform an inference process that depends on two similarity measures and a reference case base. This paper proposes a strategy, called EnergyCompress, to learn an effective case base by selecting relevant cases from an initial set. Use of EnergyCompress decreases CBP inference time, through case base compression, and also increases prediction performance, for a wide variety of CBP algorithms. EnergyCompress relies on the proposition of a general formulation of the CBP task in the framework of energy-based models, which leads to a new and valuable characterization of the notion of competence in case-based reasoning, in particular at the source case level. Extensive experimental results on 18 benchmark datasets comparing EnergyCompress to 5 reference algorithms for case base maintenance support the benefit of the proposed strategy.

The cost and difficulty of acquiring case features motivates interest in machine learning for feature acquisition. For computer vision domains, manual feature extraction has proven infeasible, but previous studies have shown the effectiveness of extracting features from deep neural models for case-based classification. Such approaches have generally been based on training the network for stand-alone classification accuracy, under the assumption that effective classification reflects high quality network features. However, it is not clear that the features best suited to network processing will be best for CBR. In response, this paper proposes refining previous network feature extraction approaches by adapting network training to reflect the goal of using network features for CBR. Specifically, it proposes augmenting conventional crossentropy loss with a proxy term that reflects how the CBR system will use extracted features for similarity assessment. To this end, we investigate using Pairwise Distance, Cosine Similarity, and Sinkhorn Divergence as proxy functions within a triplet loss training framework. Evaluations on the benchmark image classification datasets MNIST, Animals with Attributes 2, and CIFAR-10 support the effectiveness of this method, with an integrated case-based classification system using the extracted features outperforming the feature extraction network applied end-to-end as well as integrated models developed in our previous research.

This short paper presents preliminary research on the Case-Enhanced Vision Transformer (CEViT), a similarity measurement method aimed at improving the explainability of similarity assessments for image data. Initial experimental results suggest that integrating CEViT into k-Nearest Neighbor (k-NN) classification yields classification accuracy comparable to state-of-the-art computer vision models, while adding capabilities for illustrating differences between classes. CEViT explanations can be influenced by prior cases, to illustrate aspects of similarity relevant to those cases.

High-quality indices are essential for accurate retrieval in case-based reasoning. However, in some domains, indexing knowledge may be incomplete, unavailable, or unfeasible to obtain by knowledge acquisition, making knowledge-light machine learning methods an appealing alternative for generating indexing features. In response, previous work has developed promising methods for extracting indexing features from deep neural networks trained on case data. However, it has also underlined that CBR using features extracted from a deep neural network achieves low accuracy in domains for which the network itself has low accuracy when trained from scratch. This is a special concern for CBR feature extraction because the ability of CBR to reason successfully in “small-data" domains has been seen as one of its benefits. This paper reports on work investigating the hypothesis that transfer learning may help decrease the data requirements for index extraction. Specifically, it examines how model pretraining affects the quality of extracted indexing features for case-based classification, measured by the performance of a case-based classifier using those features for retrieval. Experimental results suggest that using a pretrained deep learning model for feature extraction can improve classification accuracy and consistency compared to using similar models trained from scratch. An unexpected and intriguing result is that the case-based classifier using extracted features outperformed analogous deep learning classifiers for the tested dataset.

Systems based on Large Language Models (LLMs), such as ChatGPT, have impressive performance but also well-known issues with erroneous output. Retrieval Augmented Generation (RAG), which typically presents the LLM with text snippets of additional knowledge retrieved from an external knowledge base, is a popular method for increasing LLM accuracy. This paper presents initial studies exploring augmenting LLMs with cases rather than snippets and prompting LLMs towards performing case-based reasoning. The studies consider four possible scenarios, exploring the potential benefit of LLMs performing different subparts of the CBR process: (1) a scenario in which the LLM is prompted to adapt a presented case, (2) a scenario in which the LLM is first prompted to perform similarity assessment to select a case from a set of candidates, and then to adapt the selected case, (3) a scenario in which the LLM is prompted to select the two most similar cases to a problem and generate an adapted/combined solution in light of both, and (4) a scenario in which the LLM selects the nearest neighbor and nearest unlike neighbor and generates an adapted/combined solution based on both. Results of tests using Llama and ChatGPT are encouraging for the accuracy benefits of providing LLMs with cases and raise questions for future study.

Recent research brings the strengths of neural networks to bear on CBR tasks such as similarity assessment and case adaptation. This paper further advances this direction by implementing both retrieval and adaptation as a single neural network. Such an approach has multiple goals: From the perspective of CBR, it enables harmonizing the interaction between feature extraction, retrieval/similarity assessment, and case adaptation through end-to-end training. From the perspective of neural networks, a neural network implementing CBR processes ceases to be a black box and provides the natural interpretability of CBR. As a first step towards this goal, this paper presents neural network based k-nearest neighbor (NN-kNN), a network architecture that can be interpreted as a k-NN method. Unlike other network architectures, NN-kNN’s decisions can be fully explained in terms of surface features, feature/case weights and nearest neighbors. It can be trained or fine-tuned using existing neural network methods. This study illustrates its feasibility and examines its strengths and limitations. The approach is evaluated for classification and regression tasks comparing NN-kNN, a standard neural network, and k-NN models using state-of-the-art distance metric learning algorithms. In these tests, NN-kNN achieves equal or less error when compared to the other models, while being fully interpretable as a k-NN method. The study also considered the limitations of NN-kNN and future directions to alleviate them.

This paper revisits the notion of case base competence in the light of recent advances in the modeling of analogical reasoning, based on the idea of similarity transfer from a situation space to an outcome space. For that we consider the CoAT indicator, that measures the compatibility between two similarity measures on a case base, and use it to define an intrinsic measure of competence of a case base with respect to a reference set. Initial experimental results show that the proposed competence measure correlates with the performance of the CoAT prediction algorithm. In fact, our preliminary results seem to indicate that, under some initial conditions, our competence based model can fit any classification boundary. We then revisit the notions of case competence and locality, and show that some source cases may degrade the overall case base competence while others may improve it, and that a given source case may have disparate influence on different regions of the case space

A summary of Roger Schank's career might initially appear fairly typical for an eminent academic. Following a PhD in linguistics at the University of Texas at Austin in 1969, Roger held faculty positions in linguistics and computer science at Stanford, computer science and psychology at Yale, and computer science and education at Northwestern. He served terms as chair of computer science at both Yale and Northwestern. After Northwestern, he was Chief Educational Officer for Carnegie Mellon's Silicon Valley campus. He authored over 30 books spanning AI, cognitive science, psychology and education. He advised nearly 50 PhD students. He was a Fellow of AAAI. But the hundreds of people who worked or interacted with Roger over the years know there was nothing typical about him. Roger was a force of nature. He questioned everything, especially (and gleefully) focusing on topics that were supposed to be canon. He came in, broke things apart, and built new things in their place. In linguistics, he rejected the Chomskyian approach to divorce the study of language from the study of meaning, with his seminal work on semantic primitives. In AI, where language processing focused on the propositions, he argued for the importance of much larger memory structures such as scripts and plans, and for memory processes, such as remindings, for modeling understanding. He argued for examples, that is, cases, rather than logical rules, for modeling human reasoning. Much of his work elicited initial pushback, which then transitioned to wary toleration, and finally arrived at such widespread acceptance that now his ideas are often assumed without attribution. Roger relished debate, and engaged avidly in ongoing discourse on the issues he studied. Where many labs have weekly “discussions” or “chats”, Roger fashioned weekly “Friday fights” and an “Indefensible position” seminar. One facet of these was the Socratic investigation of complex topics; another was as a crucible for the courage to make bold claims and the skills to distill, defend, and question them. He questioned loudly. But under the disputative bearing, to those who knew and worked with him he had abundant loyalty and good will. He was an explorer of the mind and of the world, an astute observer of humans and human nature: an intuitive psychologist. He had a knack for identifying key questions, always noticing customs, behaviors, and anomalies to explain, gathering data and categorizing to generate theories. His travels and knowledge of wine and food were a rich source of examples for his work and camaraderie. He did things in a big way, from academic passions like studying how language and the mind work and how people learn, to personal passions like food and football. Many stories about Roger occur at restaurants because meals were events. Many fans watch weekend football, but Roger created a room with half a dozen separate TVs, most with picture in picture, to monitor a dozen games simultaneously. Roger believed strongly in developing communities, not only in research labs and departments, but at the national and international level. He founded new fields in order to create lasting communities of like minds, in cognitive science and education. His studies of human memory led to launching the field of case-based reasoning, holding its 31st international conference this year. He was a co-founder of the field of Cognitive Science, the Cognitive Science Society, and the journal Cognitive Science. His PhD students at Stanford, Yale, and Northwestern, along with many developers, artists, and content creators at his companies and the Institute for the Learning Sciences will attest to the communities and cultural traditions he established over and over. For example, it is not unusual to hold a party when a PhD student finishes their dissertation, but Roger's parties had an extra. Every student with a PhD (prior and current) had to present a talent (real, imagined, feigned, or facetious) to the group, culminating with the new PhD revealing their secret talent. This ritual created a human connection among students beyond the academic. As his passion shifted to education, he led establishment of the field of the Learning Sciences, combining education, cognitive science, and computer science. In 1989 at Northwestern, he formed the Institute for the Learning Sciences and created the first Learning Sciences MS and PhD programs in the School of Education. He pioneered the study of stories in AI, spurring development of innovative story-based educational environments. He was himself a great story-teller; he used stories to connect, illuminate, and educate, both in his personal life and in his books. Finally, Roger cared deeply about impact. Long before the startup culture arose as a way for faculty to commercialize their ideas, Roger started companies because he believed that changes in how people should interact with technology, and how education should be done, would mainly happen through the business world. While at Yale, he started Cognitive Systems for building knowledge-rich intelligent systems, and Compu-Teach for building K-12 educational systems for personal computers. While at Northwestern he started Cognitive Arts and later Socratic Arts to develop learn-by-doing systems for training and education. In 2000 he left academics to focus on nothing less than transforming education with systems to support hands-on project-based teaching. His website lists some of the topics he cared about: “Making school less miserable for kids”, “fixing corporate training”, “getting the right information to people at the right time” “building the right kind of artificial intelligence systems”, “empowering people to develop effective learning experiences”, and, always and most importantly, “understanding how the human mind works”. Roger will be missed, but he also will still be here: his work continues to have a persistent and profound impact on a range of fields that are focused on the scientific study of the human mind. The authors declare that there is no conflict. Richard Granger received his Bachelor's and Ph.D. from MIT and Yale. He is a Professor at Dartmouth with joint appointments in the Department of Psychological and Brain Sciences, the Thayer School of Engineering, and the Cognitive Science Program; he directs Dartmouth's interdisciplinary Brain Engineering Laboratory (brainengineering.org), with publications and patents ranging from computation to cognition to basic neuroscience. He advises multiple technology corporations and government research agencies, is co-inventor of FDA-approved devices and drugs in clinical trials, and has been the principal architect of a series of advanced computational systems for military, commercial, and medical applications. David Leake received his PhD from Yale University. He is a Professor of Computer Science in the Luddy School at Indiana University (IU), where he served as Executive Associate Dean from 2012−2020, and a member of the IU Cognitive Science Program. His current research areas include case-based reasoning, explanation, and neuro-symbolic AI. He has authored/edited over 200 publications. He is Editor in Chief Emeritus of AI Magazine after 17 years as Editor in Chief. In 2014 he received the AAAI Distinguished Service Award. He is a Senior Member of AAAI. Christopher K. Riesbeck is an Associate Professor of Computer Science at Northwestern University, a Fellow of AAAI, the director of the CS Master's program, and co-director of the Center for Computer Science and the Learning Sciences.

Case-based reasoning (CBR) is a problem-solving and learning methodology that applies records of past experiences, captured as cases, to solve new problems. The performance of CBR depends on retrieving cases relevant to each new problem that the reasoner encounters. In real-world applications, the distribution of problems can change over time, which can cause an issue for the competence and efficiency of CBR systems. This paper proposes addressing this issue through predictive case discovery , which involves predicting cases expected to be useful for future problems to acquire them in advance. It presents an overview of case discovery for problem-distribution drift, including the challenges involved, proposed strategies, and future research directions. It illustrates with a case study evaluating a clustering-based case discovery strategy in a path planning domain across four scenarios: no drift, non-cyclical drift, cyclical drift, and drift from obsolescence.

This chapter introduces three exploratory studies that assessed the capabilities of artificial intelligence (AI) through standardised education tests designed for humans. The first two studies, conducted in 2016 and 2021/22, asked experts to evaluate AI's performance on the literacy and numeracy tests of the OECD's Survey of Adult Skills (PIAAC). The third study collected expert judgements of whether AI can solve science questions from the OECD's Programme for International Student Assessment (PISA). The studies aimed to refine the assessment framework for eliciting expert knowledge on AI using established educational assessments. They explored different test formats, response methodologies and rating instructions, along with two distinct assessment approaches. A "behavioural approach" used in the PIAAC studies emphasised smaller expert groups engaging in discussions, and a "mathematical approach" adopted in the PISA study relied more heavily on quantitative data from a larger expert pool. This chapter presents the results of the studies and discusses the advantages and disadvantages of their methodological approaches.

The capability of systems based on large language models (LLMs), such as ChatGPT, to generate human-like text has captured the attention of the public and the scientific community. It has prompted both predictions that systems such as ChatGPT will transform AI and enumerations of system problems with hopes of solving them by scale and training. This position paper argues that both over-optimistic views and disppointments reflect misconceptions of the fundamental nature of LLMs as language models. As such, they are statistical models of language production and fluency, with associated strengths and limitations; they are not-and should not be expected to be-knowledge models of the world, nor do they reflect the core role of language beyond the statistics: communication. The paper argues that realizing that role will require driving LLMs with symbolic systems based on goals, facts, reasoning, and memory.

For many years, symbolic paradigms have dominated AI, but the impressive accomplishments of neural approaches have made them a new dominant paradigm. However, each of these paradigms has distinct advantages. This talk presents a hybrid paradigm that integrates case-based reasoning (CBR) with network methods to achieve the benefits of both. CBR is a knowledge-based reasoning and learning methodology inspired by human cognition that adapts prior cases—records of prior experiences, to solve new problems. The CBR process supports efficient knowledge-based reasoning and the reuse of structured or unstructured solutions; it is naturally interpretable, can learn from few examples, and provides inertia-free lazy learning. However, the success of CBR depends on similarity and case-adaptation knowledge, which may be hard to acquire. This talk presents opportunities for combining case-based reasoning with neural networks to leverage both paradigms.

Classification accuracy for case-based classifiers depends critically on the features used for case retrieval. Feature extraction from deep learning classifier models has proven a useful method for generating case-based classifier features, especially for domains in which manual feature engineering is costly or difficult. Previous work has explored how the quality of extracted features is influenced by structural choices such as the number of features extracted and the location/depth of extraction. This paper investigates how feature quality is influenced by another factor: the choice of the network model itself. We consider a selection of deep learning models for a computer vision classification task and test the accuracy of a case-based classifier using features extracted from them, both as the sole feature source and in combination with a supplementary set of knowledge-engineered features. Results suggest that feature quality reflects a trade-off between model complexity and training data requirements and provide lessons for the selection of deep learning architectures for feature extraction to support case-based classification.

Integrations of case-based reasoning (CBR) with neural approaches are appealing because of their complimentary characteristics. This chapter presents research on neuro-symbolic integrations to support CBR, to reduce knowledge engineering and improve performance for CBR systems. It summarizes three strands of research: First, on extracting features for case retrieval from deep neural networks to use in concert with expert-generated features, second, on applying neural networks to learn to adapt the solutions of retrieved cases to fit new situations, and third, on harmonizing similarity learning with case adaptation learning, in order to focus retrieval on adaptable cases. It summarizes strengths, weaknesses and tradeoffs of these approaches, and points to future challenges for neuro-CBR integrations.

From the early days of case-based reasoning research, the ability of CBR systems to explain their decisions in terms of past cases has been seen as an important advantage. However, there have been few studies on the factors affecting the effectiveness of explaining CBR decisions by cases. This paper presents results from a human subjects study that examined how alternative retrieval processes (one-shot or conversational) and case presentation approaches affect the perceived goodness of case-based explanations for explaining system behavior, their convincingness, and the trust they engender. The study corroborates that cases are well received as explanations, with some benefit for providing information to support similarity comparison, and suggests that elucidating the retrieval process has little effect on explanatory effectiveness.