Monitoring and maintaining user engagement in human-chatbot interactions is challenging. Researchers often use cues observed in the interactions as indicators to infer engagement. However, evaluation of these cues is lacking. In this study, we collected an inventory of potential textual engagements cues from the literature, including linguistic features, utterance features, and interaction features. These cues were subsequently used to annotate a dataset of 291 user-chatbot interactions, and we examined which of these cues predicted self-reported user engagement. Our results show that engagement can indeed be recognized at the level of individual utterances. Notably, words indicating cognitive thinking processes and motivational utterances were strong indicators of engagement. An overall negative tone could also predict engagement, highlighting the importance of nuanced interpretation and contextual awareness of user utterances. Our findings demonstrated initial feasibility of recognizing utterance-level cues and using them to infer user engagement, although further validation is needed across different content-domains.

Currently, leaderboards are often used to evaluate natural language processing (NLP) systems and in particular large language models. In this paper we argue why we should step away from leaderboards and follow a more inclusive approach both in developing as well as in evaluating models. The focus of evaluation should be on the complete context in which the system operates. To accomplish this, researchers should take an inclusive approach and take note of developments in multiple scientific fields (from NLP to communication science).

Healthcare services and products are rapidly changing due to the development of new technologies, offering relevant solutions to improve patient outcomes. Patient-Generated Health Data and knowledge-sharing across the European Union (EU) has a great potential of making healthcare provision more effective and efficient by putting the patient at the centre of the healthcare process. While such initiatives have been taken before, a uniting and overarching approach is still missing. The EU-funded IMPROVE project will develop an evidence-based and actual framework to effectively leverage the added value of people-centred integrated healthcare solutions, using predominantly PROMs, PPI, PREMs, and other Patient-Generated Health Data (PGHD). As a result, the project facilitates the effective and efficient implementation of Value-Based Healthcare across the EU by putting the patient central in the healthcare process.

Background: Cigarette smoking poses a major public health risk. Chatbots may serve as an accessible and useful tool to promotecessation due to their high accessibility and potential in facilitating long-term personalized interactions. To increase effectivenessand acceptability, there remains a need to identify and evaluate counseling strategies for these chatbots, an aspect that has notbeen comprehensively addressed in previous research. Objective: This study aims to identify effective counseling strategies for such chatbots to support smoking cessation. In addition,we sought to gain insights into smokers'expectations of and experiences with the chatbot. Methods: This mixed methods study incorporated a web-based experiment and semistructured interviews. Smokers (N=229)interacted with either a motivational interviewing (MI)-style (n=112, 48.9%) or a confrontational counseling-style (n=117,51.1%) chatbot. Both cessation-related (ie, intention to quit and self-efficacy) and user experience-related outcomes (ie, engagement,therapeutic alliance, perceived empathy, and interaction satisfaction) were assessed. Semistructured interviews were conductedwith 16 participants, 8 (50%) from each condition, and data were analyzed using thematic analysis. Results: Results from a multivariate ANOVA showed that participants had a significantly higher overall rating for the MI (vsconfrontational counseling) chatbot. Follow-up discriminant analysis revealed that the better perception of the MI chatbot wasmostly explained by the user experience-related outcomes, with cessation-related outcomes playing a lesser role. Exploratoryanalyses indicated that smokers in both conditions reported increased intention to quit and self-efficacy after the chatbot interaction.Interview findings illustrated several constructs (eg, affective attitude and engagement) explaining people's previous expectationsand timely and retrospective experience with the chatbot. Conclusions: The results confirmed that chatbots are a promising tool in motivating smoking cessation and the use of MI canimprove user experience. We did not find extra support for MI to motivate cessation and have discussed possible reasons. Smokersexpressed both relational and instrumental needs in the quitting process. Implications for future research and practice are discussed.

Chatbots have several affordances that may stimulate self-disclosure and help-seeking by people with mental health problems, such as accessibility, anonymity, and convenience. This study aims to examine the effect of reciprocal self-disclosure by chatbots on friendship formation between humans and chatbots. For this, a Wizard of Oz experiment with two conditions was conducted. Participants (N = 86) were randomly assigned to either the condition with or without reciprocal self-disclosure by the chatbot. Our preliminary findings revealed that there was no significant difference between the conditions regarding feelings of friendship. However, the results showed that people trusted the chatbot in the reciprocal self-disclosure condition more. Furthermore, people perceived the chatbot in the reciprocal self-disclosure condition as more empathetic. These first results suggest that in developing affective social chatbots it would be important to implement reciprocal self-disclosure as it stimulates perceptions of trust and empathy.

This study aimed to investigate if and how chatbots can increase smokers’ intention to quit, specifically looking into the effectiveness of two communication styles (i.e., motivational interviewing (MI) and confrontational counseling (CC)) and the moderating role of individual differences (i.e., need for autonomy and perceived self-efficacy) that may affect smokers’ experience with the chatbot. In an online between-subjects experiment (N = 233), smoking participants were assigned to interact with either a MI chatbot (n = 121) or a CC chatbot (n = 112) for one 8-min session. Their need for autonomy and perceived self-efficacy were measured, as well as their satisfaction with the conversation and pre- and post-test intention to quit smoking. No significant effects of different communication styles were found regarding the outcomes, nor did the need for autonomy moderate these results. However, the effect of MI on user satisfaction was more profound among smokers with higher self-efficacy, and a positive effect of self-efficacy on user satisfaction appeared. Additionally, interacting with the chatbots about one’s smoking behavior significantly increased participants’ intention to quit, regardless of its communication style. As such, this study sheds light on the potential of conversational chatbots for smoking cessation interventions, as well as pathways for future research.

Introduction Conversational agents (CAs; computer programs that use artificial intelligence to simulate a conversation with users through natural language) have evolved considerably in recent years to support healthcare by providing autonomous, interactive, and accessible services, making them potentially useful for supporting smoking cessation. We performed a systematic review and meta-analysis to provide an overarching evaluation of their effectiveness and acceptability to inform future development and adoption. Aims and Methods PsycInfo, Web of Science, ACM Digital Library, IEEE Xplore, Medline, EMBASE, Communication and Mass Media Complete, and CINAHL Complete were searched for studies examining the use of CAs for smoking cessation. Data from eligible studies were extracted and used for random-effects meta-analyses. Results The search yielded 1245 publications with 13 studies eligible for systematic review (total N = 8236) and six studies for random-effects meta-analyses. All studies reported positive effects on cessation-related outcomes. A meta-analysis with randomized controlled trials reporting on abstinence yielded a sample-weighted odds ratio of 1.66 (95% CI = 1.33% to 2.07%, p < .001), favoring CAs over comparison groups. A narrative synthesis of all included studies showed overall high acceptability, while some barriers were identified from user feedback. Overall, included studies were diverse in design with mixed quality, and evidence of publication bias was identified. A lack of theoretical foundations was noted, as well as a clear need for relational communication in future designs. Conclusions The effectiveness and acceptability of CAs for smoking cessation are promising. However, standardization of reporting and designing of the agents is warranted for a more comprehensive evaluation. Implications This is the first systematic review to provide insight into the use of CAs to support smoking cessation. Our findings demonstrated initial promise in the effectiveness and user acceptability of these agents. We also identified a lack of theoretical and methodological limitations to improve future study design and intervention delivery.

We present HyLECA, an open-source framework designed for the development of long-term engaging controlled conversational agents. HyLECA’s dialogue manager employs a hybrid architecture, combining rule-based methods for controlled dialogue flows with retrieval-based and generation-based approaches to enhance the utterance variability and flexibility. The motivation behind HyLECA lies in enhancing user engagement and enjoyment in task-oriented chatbots by leveraging the natural language generation capabilities of open-domain large language models within the confines of predetermined dialogue flows. Moreover, we discuss the technical capabilities, potential applications, relevance, and adaptability of the system. Lastly, we report preliminary findings from integrating state-of-the-art large language models in simulating a conversation centred on smoking cessation.

Digital health interventions for sexual health promotion have evolved considerably alongside innovations in technology. Despite these efforts, studies have shown that they do not consistently result in the desired sexual health outcomes. This could be attributed to low levels of user engagement, which can hinder digital health intervention effectiveness, as users do not engage with the system enough to be exposed to the intervention components. It has been suggested that conversational agents (automated two-way communication systems e.g. Alexa) have the potential to overcome the limitations of prior systems and promote user engagement through the increased interactivity offered by bidirectional, natural language-based interactions. The present review, therefore, provides an overview of the effectiveness and user acceptability of conversational agents for sexual health promotion. A systematic search of seven databases provided 4534 records, and after screening, 31 articles were included in this review. A narrative synthesis of results was conducted for effectiveness and acceptability outcomes, with the former supplemented by a meta-analysis conducted on a subset of studies. Findings provide preliminary support for the effectiveness of conversational agents for promoting sexual health, particularly treatment adherence. These conversational agents were found to be easy to use and useful, and importantly, resulted in high levels of satisfaction, use and intentions to reuse, whereas user evaluations regarding the quality of information left room for improvement. The results can inform subsequent efforts to design and evaluate these interventions, and offer insight into additional user experience constructs identified outside of current technology acceptance models, which can be incorporated into future theoretical developments.

Background Cigarette smoking poses a major threat to public health. While cessation support provided by healthcare professionals is effective, its use remains low. Chatbots have the potential to serve as a useful addition. The objective of this study is to explore the possibility of using a motivational interviewing style chatbot to enhance engagement, therapeutic alliance, and perceived empathy in the context of smoking cessation. Methods A preregistered web-based experiment was conducted in which smokers ( n = 153) were randomly assigned to either the motivational interviewing (MI)-style chatbot condition ( n = 78) or the neutral chatbot condition ( n = 75) and interacted with the chatbot in two sessions. In the assessment session, typical intake questions in smoking cessation interventions were administered by the chatbot, such as smoking history, nicotine dependence level, and intention to quit. In the feedback session, the chatbot provided personalized normative feedback and discussed with participants potential reasons to quit. Engagement with the chatbot, therapeutic alliance, and perceived empathy were the primary outcomes and were assessed after both sessions. Secondary outcomes were motivation to quit and perceived communication competence and were assessed after the two sessions. Results No significant effects of the experimental manipulation (MI-style or neutral chatbot) were found on engagement, therapeutic alliance, or perceived empathy. A significant increase in therapeutic alliance over two sessions emerged in both conditions, with participants reporting significantly increased motivation to quit. The chatbot was perceived as highly competent, and communication competence was positively associated with engagement, therapeutic alliance, and perceived empathy. Conclusion The results of this preregistered study suggest that talking with a chatbot about smoking cessation can help to motivate smokers to quit and that the effect of conversation has the potential to build up over time. We did not find support for an extra motivating effect of the MI-style chatbot, for which we discuss possible reasons. These findings highlight the promise of using chatbots to motivate smoking cessation. Implications for future research are discussed.

Online medical consultation has become increasingly popular, while little is known about what features of such service can impact users’ emotions and behaviours. This study looked into the language features of online text-based medical consultation. Specifically, the aim of this paper was to examine the effects of vague language (i.e., non-specific, imprecise language) on health-related uncertainty, and its affective and behavioural consequences, while considering individual differences in regulatory focus. A between-subject (vague language vs. precise language vs. control condition) web-based experiment was conducted (N = 249), where participants in the experimental groups read virtual doctor-patient conversations where the doctor used either vague or precise language. Results showed that vague language induced more uncertainty than precise language (p = .010); such uncertainty was appraised as a danger (r = .18, p = .004) but not an opportunity (r = .01, p = .932), and subsequently led to negative emotions (r = .45, p < .001). No effects were found on behavioural outcomes, and there was no moderation from regulatory focus. The results suggest that online healthcare providers should refrain from using vague language in communication with patients to avoid eliciting uncertainty and subsequent negative feelings. Future research is needed to further examine the behavioural effects of uncertainty and explore factors that could foster the appraisal of opportunity.