This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, providing an overview of AI assistants, their technical foundations and potential range of applications. It then explores questions around AI value alignment, well-being, safety and malicious uses. Extending the circle of inquiry further, we next consider the relationship between advanced AI assistants and individual users in more detail, exploring topics such as manipulation and persuasion, anthropomorphism, appropriate relationships, trust and privacy. With this analysis in place, we consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants. Finally, we conclude by providing a range of recommendations for researchers, developers, policymakers and public stakeholders.

Interfaces increasingly mimic human social behaviours. Beyond prototypical examples like chatbots, basic automated systems like app notifications or self-checkout machines likewise address or 'talk to' people in person-like ways. Whilst early evidence suggests social cues can enhance user experience, we lack a good understanding of when, and why, their use in interaction design may be inappropriate. We combined a qualitative survey (n=80) with experience sampling, interview, and workshop studies (n=11) to understand people's attitudes and preferences regarding how a range of automated systems talk to/at them. We thematically analysed examples of phrasings or conduct our participants disliked, their reasons, and how they would prefer to be treated instead. One category of inappropriate use we identified is when social design elements are used to manipulate user behaviour. We distinguish four such tactics: 'agents' playing on users' emotions (e.g., guilt-tripping, coaxing), being pushy, mothering users, or being passive-aggressive. Another category regards pragmatics: personal or situational factors that can make even a seemingly helpful or friendly message come across as rude, tactless, invasive, etc. These include contextual insensitivity (e.g., embarrassing users in public); expressing clearly false personalised care; or treating a user in ways they find misaligned with the system's role or the nature of their relationship. We discuss these inappropriate uses in terms of an emerging 'social' class of dark and anti-patterns. From participant suggestions, we offer recommendations for improving how interfaces treat people in interaction, including broader normative reflections on treating users respectfully.

Digital self-control tools (DSCTs) help people control their time and attention on digital devices, using interventions like distraction blocking or usage tracking. Most studies of DSCTs’ effectiveness have focused on whether a single intervention reduces time spent on a single device. In reality, people may require combinations of DSCTs to achieve more subjective goals across multiple devices. We studied how DSCTs can address individual needs of university students (n = 280), using a workshop where students reflect on their goals before exploring relevant tools. At 1-3 month follow-ups, 95% of respondents still used at least one type of DSCT, typically applied across multiple devices, and there was substantial variation in the tool combinations chosen. We observed a large increase in self-reported digital self-control, suggesting that providing a space to articulate goals and self-select appropriate DSCTs is a powerful way to support people who struggle to self-regulate digital device use.

With the growing popularity of dialogue agents based on large language models (LLMs), urgent attention has been drawn to finding ways to ensure their behaviour is ethical and appropriate. These are largely interpreted in terms of the 'HHH' criteria: making outputs more helpful and honest, and avoiding harmful (biased, toxic, or inaccurate) statements. Whilst this semantic focus is useful from the perspective of viewing LLM agents as mere mediums for information, it fails to account for pragmatic factors that can make the same utterance seem more or less offensive or tactless in different social situations. We propose an approach to ethics that is more centred on relational and situational factors, exploring what it means for a system, as a social actor, to treat an individual respectfully in a (series of) interaction(s). Our work anticipates a set of largely unexplored risks at the level of situated interaction, and offers practical suggestions to help LLM technologies behave as 'good' social actors and treat people respectfully.

The development of increasingly agentic and human-like AI assistants, capable of performing a wide range of tasks on user's behalf over time, has sparked heightened interest in the nature and bounds of human interactions with AI. Such systems may indeed ground a transition from task-oriented interactions with AI, at discrete time intervals, to ongoing relationships - where users develop a deeper sense of connection with and attachment to the technology. This paper investigates what it means for relationships between users and advanced AI assistants to be appropriate and proposes a new framework to evaluate both users' relationships with AI and developers' design choices. We first provide an account of advanced AI assistants, motivating the question of appropriate relationships by exploring several distinctive features of this technology. These include anthropomorphic cues and the longevity of interactions with users, increased AI agency, generality and context ambiguity, and the forms and depth of dependence the relationship could engender. Drawing upon various ethical traditions, we then consider a series of values, including benefit, flourishing, autonomy and care, that characterise appropriate human interpersonal relationships. These values guide our analysis of how the distinctive features of AI assistants may give rise to inappropriate relationships with users. Specifically, we discuss a set of concrete risks arising from user-AI assistant relationships that: (1) cause direct emotional or physical harm to users, (2) limit opportunities for user personal development, (3) exploit user emotional dependence, and (4) generate material dependencies without adequate commitment to user needs. We conclude with a set of recommendations to address these risks.

Recent years have seen a surge in applications and technologies aimed at motivating users to achieve personal goals and improve their wellbeing. However, these often fail to promote long-term behaviour change, and sometimes even backfire. We consider how self-determination theory (SDT), a metatheory of human motivation and wellbeing, can help explain why such technologies fail, and how they may better help users internalise the motivation behind their goals and make enduring changes in their behaviour. In this work, we systematically reviewed 15 papers in the ACM Digital Library that apply SDT to the design of behaviour change technologies (BCTs). We identified 50 suggestions for design features in BCTs, grounded in SDT, that researchers have applied to enhance user motivation. However, we find that SDT is often leveraged to optimise engagement with the technology itself rather than with the targeted behaviour change per se. When interpreted through the lens of SDT, the implication is that BCTs may fail to cultivate sustained changes in behaviour, as users' motivation depends on their enjoyment of the intervention, which may wane over time. An underexplored opportunity remains for designers to leverage SDT to support users to internalise the ultimate goals and value of certain behaviour changes, enhancing their motivation to sustain these changes in the long term.

With the growing popularity of conversational agents based on large language models (LLMs), we need to ensure their behaviour is ethical and appropriate. Work in this area largely centres around the 'HHH' criteria: making outputs more helpful and honest, and avoiding harmful (biased, toxic, or inaccurate) statements. Whilst this semantic focus is useful when viewing LLM agents as mere mediums or output-generating systems, it fails to account for pragmatic factors that can make the same speech act seem more or less tactless or inconsiderate in different social situations. With the push towards agentic AI, wherein systems become increasingly proactive in chasing goals and performing actions in the world, considering the pragmatics of interaction becomes essential. We propose an interactional approach to ethics that is centred on relational and situational factors. We explore what it means for a system, as a social actor, to treat an individual respectfully in a (series of) interaction(s). Our work anticipates a set of largely unexplored risks at the level of situated social interaction, and offers practical suggestions to help agentic LLM technologies treat people well.

The increasing sophistication of NLP models has renewed optimism regarding machines achieving a full human-like command of natural language. Whilst work in NLP/NLU may have made great strides in that direction, the lack of conceptual clarity in how 'understanding' is used in this and other disciplines have made it difficult to discern how close we actually are. A critical, interdisciplinary review of current approaches and remaining challenges is yet to be carried out. Beyond linguistic knowledge, this requires considering our species-specific capabilities to categorize, memorize, label and communicate our (sufficiently similar) embodied and situated experiences. Moreover, gauging the practical constraints requires critically analyzing the technical capabilities of current models, as well as deeper philosophical reflection on theoretical possibilities and limitations. In this paper, I unite all of these perspectives -- the philosophical, cognitive-linguistic, and technical -- to unpack the challenges involved in approaching true (human-like) language understanding. By unpacking the theoretical assumptions inherent in current approaches, I hope to illustrate how far we actually are from achieving this goal, if indeed it is the goal.

Recent hype surrounding the increasing sophistication of language processing models has renewed optimism regarding machines achieving a human-like command of natural language. Research in the area of natural language understanding (NLU) in artificial intelligence claims to have been making great strides in this area, however, the lack of conceptual clarity/consistency in how 'understanding' is used in this and other disciplines makes it difficult to discern how close we actually are. In this interdisciplinary research thesis, I integrate insights from cognitive science/psychology, philosophy of mind, and cognitive linguistics, and evaluate it against a critical review of current approaches in NLU to explore the basic requirements--and remaining challenges--for developing artificially intelligent systems with human-like capacities for language use and comprehension.

This article considers the remaining hindrances for natural language processing technologies in achieving open and natural (human-like) interaction between humans and computers. Although artificially intelligent (AI) systems have been making great strides in this field, particularly with the development of deep learning architectures that carry surface-level statistical methods to greater levels of sophistication, these systems are yet incapable of deep semantic analysis, reliable translation, and generating rich answers to open-ended questions. I consider how the process may be facilitated from our side, first, by altering some of our existing language conventions (which may occur naturally) if we are to proceed with statistical approaches, and secondly, by considering possibilities in using a formalised artificial language as an auxiliary medium, as it may avoid many of the inherent ambiguities and irregularities that make natural language difficult to process using rule-based methods. As current systems have been predominantly English-based, I argue that a formal auxiliary language would not only be a simpler and more reliable medium for computer processing, but may also offer a more neutral, easy-to-learn lingua franca for uniting people from different linguistic backgrounds with none necessarily having the upper hand.