A fundamental look at models and intelligence: abstract of keynote

Comprehensible and analyzable models are central to building confidence in cyber-physical systems. Hybrid systems theories, interface theories, formal semantics, concurrent models of computation, component models, and ontologies all augment classical software engineering techniques such as object-oriented design to catch errors and to make systems more modular and composable. However, as systems integrate self-adaptive capabilities and learning algorithms, keeping models comprehensible and analyzable gets challenging. Every model lives within a modeling framework, ideally giving semantics to the model, and many modeling frameworks have been developed that enable rigorous analysis and proof of properties. But every such modeling framework is an imperfect mirror of reality. An engineered system operating in the physical world may or may not accurately reflect behaviors predicted by a model, and the model may not reflect behaviors that are critical to correct operation of the system. Software in a cyber-physical system, for example, has timing properties that are rarely represented in formal models. As artificial intelligence gets more widely used, the problem gets worse, with predictability and explainability seemingly evaporating. In this keynote talk, I examine the strengths and limitations in the use of models. I will show that two very different classes of models are used in practice, classes that I call "scientific models" and "engineering models" [1]. These two classes have complementary properties, and many misuses of models stem from confusion about which class is being used. Scientific models of intelligent systems are very different from engineering models. I will argue that rigorous and analyzable engineering models are useful even in the face of uncertainty and can be used to regulate adaptation and circumscribe the behaviors of learning-based systems.