Detecting and Adapting to Users’ Cognitive and Affective State to Develop Intelligent Musical Interfaces

In musical instrument interfaces, such as piano keyboards, the player’s communication channels may be limited by the expressivity and resolution of input devices, the expressivity of relevant body parts, and human attention bottlenecks. In this chapter, we consider intelligent musical interfaces that can measure cognitive or affective states implicitly in real-time to allow musically appropriate adaptations by the system without conscious effort on the part of the user. This chapter focuses on two specific areas in music where the detection of cognitive and affective states has been applied to interaction design for music: musical learning (including learning instruments or pieces of music) and musical creativity (including composing and improvisation). The motivation, theory, and technological basis for work of this kind are discussed. Relevant existing work is considered. The design and evaluation of two systems of this kind for musical learning and musical creativity implemented by the authors is presented and critiqued.