Our goal is to lay a foundation for a model of high level cognition in humans that respects biological constraints. To do so, we integrate realistic neural models of attention, memory and control into an active information foraging system comprising many of the component neural systems and strategies in published models of high-level active vision. The direction of attentional focus requires integration of bottom-up and top-down attention. We base our modeling methodology for both individual components and the overall system operation on structured hierarchical Bayesian models. These models are built upon computationally tractable, neurally implementable sampling approximations to inference and control.

To investigate the process of reasoning with an interactive diagram, we recorded eye movements and mouse clicks of 28 users as they investigated social relationships in a 313-node network diagram. The MEgraph application used to display this network combines techniques such as topological range searching and motion highlighting to enable interactive exploration of complex network diagrams. Long-term memory encoding was assessed with a surprise recall protocol one week later, with and without lightweight visual history traces. Frequent video-game players relied more on peripheral vision, moving their gaze less often. History support was also associated with more efficient visual strategies. History traces improved users' ability to reconstruct prior work on retest.

We describe interactions between kinetic (moving) and static information displays. We have implemented "moxel" kinetic displays in a classic discovery platform with many standard information visualization and analytic tools, and experimented with interactions between them. Moxels, which generalize pixels, are an advanced, moving, form of iconographic display of the kind first developed in static form by Pickett and White (1966). As with the static graphic icons of those early displays, moxels provide a way of mapping together in one image multiple data variables, but with potentially more potency with in-place motion. We show examples of how the two kinds of displays have been integrated, and discuss issues with the integration of dynamic and static visualizations in a single environment. We discuss several interaction paradigms between them including linked brushing, multiple selections, and operations on selected regions.

One means of quickly improving the performance of a system is to add a small amount of intelligence. Systems often already contain sufficient data such that by adding the ability to reason about this data a bit, the data becomes knowledge. The system can act on that knowledge and its performance can improve in a measurable manner, often dramatically. This incremental approach to software improvement mimics the evolution of biological organisms. In this paper we describe how we were able to improve the performance of a large-scale logistics analysis system by adding the ability to remember its previous computations, and to modify its behavior based on this recall.

Intelligence analysis requires detecting and ex- ploiting patterns hidden in complex data. When the critical aspects of a data set can be effec- tively visually presented, displays become pow- erful tools by harnessing the pattern-recognition capabilities of human vision. To this end, shape, color, and interactive techniques are widely util- ized in intelligence displays. Unfortunately, the volume and complexity of intelligence data has outstripped our ability to visualize that data. Un- der the ARDA GI2Vis program we have over- come this limit with a broad new class of visu- alization techniques based on the innate human ability to perceive motion. These "kinetic" visu- alizations encode attributes of data using motion, and use motion to query for and highlight pat- terns in intelligence data. By conveying complex data directly to the pre-attentive perceptual sys- tem, these displays require less time and cogni- tive effort to detect patterns than purely static displays. In this work we describe kinetic dis- plays for network analysis, detection of patterns and correlations across multiple displays of geo- spatially registered event data, and visualization of structures in high-dimensional data sets such as imagery-derived intelligence. We also present the theoretical basis for this work, and briefly describe the results of published experiments demonstrating the utility of kinetic displays.