“Take Nothing on Its Look”: Revealing Users’ Expectations and Experiences in Social Human–Robot Interaction Jessica Lindblom, Julia Rosén, Maurice Lamb, Erik Billing ACM Transactions on Human Robot Interaction, 2026 The use of social robots in many sectors of society is predicted to progressively increase. Therefore, exploring how expectations play a role in and change users’ experiences when interacting with these robots over time is necessary. From an interpretative and insight-driven approach, our aim was to explore how humans experience in-person interactions with the social robot Pepper, which was equipped with the OpenAI GPT-3 language model. Qualitative data from 62 video recordings of the interactions with Pepper and post-test interviews were collected from 31 participants. An experiential reflexive thematic analysis was applied. The main findings include various levels of interaction quality, different interaction strategies, and elements influencing the users’ expectations and experiences, which were synthesized into a coherent framework. It appears that the participants adapted their interaction strategies based on their expectations and the perceived capability of the robot, which influenced their experiences. This reveals that positive user experience is not solely determined by interaction quality, showing the interplay among these aspects when interacting with a social robot. To conclude, our findings underscore the intricate nature of the role of user expectations and experiences in social human–robot interaction. The work adds complementary qualitative approaches to the Human–Robot Interaction community to provide additional insights on interacting with social robots.
Efficacy and effectiveness of robot-assisted therapy for autism spectrum disorder: From lab to reality Daniel David, Paul Baxter, Tony Belpaeme, Erik Billing, Haibin Cai, et al. Science Robotics, 2025 The use of social robots in therapy for children with autism has been explored for more than 20 years, but there still is limited clinical evidence. The work presented here provides a systematic approach to evaluating both efficacy and effectiveness, bridging the gap between theory and practice by targeting joint attention, imitation, and turn-taking as core developmental mechanisms that can make a difference in autism interventions. We present two randomized clinical trials with different robot-assisted therapy implementations aimed at young children. The first is an efficacy trial ( n = 69; mean age = 4.4 years) showing that 12 biweekly sessions of in-clinic robot-assisted therapy achieve equivalent outcomes to conventional treatment but with a significant increase in the patients’ engagement. The second trial ( n = 63; mean age = 5.9 years) evaluates the effectiveness in real-world settings by substituting the clinical setup with a simpler one for use in schools or homes. Over the course of a modest dosage of five sessions, we show equivalent outcomes to standard treatment. Both efficacy and effectiveness trials lend further credibility to the beneficial role that social robots can play in autism therapy while also highlighting the potential advantages of portable and cost-effective setups.
Previous Experience Matters: An in-Person Investigation of Expectations in Human–Robot Interaction Julia Rosén, Jessica Lindblom, Maurice Lamb, Erik Billing International Journal of Social Robotics, 2024 The human–robot interaction (HRI) field goes beyond the mere technical aspects of developing robots, often investigating how humans perceive robots. Human perceptions and behavior are determined, in part, by expectations. Given the impact of expectations on behavior, it is important to understand what expectations individuals bring into HRI settings and how those expectations may affect their interactions with the robot over time. For many people, social robots are not a common part of their experiences, thus any expectations they have of social robots are likely shaped by other sources. As a result, individual expectations coming into HRI settings may be highly variable. Although there has been some recent interest in expectations within the field, there is an overall lack of empirical investigation into its impacts on HRI, especially in-person robot interactions. To this end, a within-subject in-person study ($$N=31$$ N = 31 ) was performed where participants were instructed to engage in open conversation with the social robot Pepper during two 2.5 min sessions. The robot was equipped with a custom dialogue system based on the GPT-3 large language model, allowing autonomous responses to verbal input. Participants’ affective changes towards the robot were assessed using three questionnaires, NARS, RAS, commonly used in HRI studies, and Closeness, based on the IOS scale. In addition to the three standard questionnaires, a custom question was administered to capture participants’ views on robot capabilities. All measures were collected three times, before the interaction with the robot, after the first interaction with the robot, and after the second interaction with the robot. Results revealed that participants to large degrees stayed with the expectations they had coming into the study, and in contrast to our hypothesis, none of the measured scales moved towards a common mean. Moreover, previous experience with robots was revealed to be a major factor of how participants experienced the robot in the study. These results could be interpreted as implying that expectations of robots are to large degrees decided before interactions with the robot, and that these expectations do not necessarily change as a result of the interaction. Results reveal a strong connection to how expectations are studied in social psychology and human-human interaction, underpinning its relevance for HRI research.
Language Models for Human-Robot Interaction Erik Billing, Julia Rosén, Maurice Lamb ACM IEEE International Conference on Human Robot Interaction, 2023 Recent advances in large scale language models have significantly changed the landscape of automatic dialogue systems and chatbots. We believe that these models also have a great potential for changing the way we interact with robots. Here, we present the first integration of the OpenAI GPT-3 language model for the Aldebaran Pepper and Nao robots. The present work transforms the text-based API of GPT-3 into an open verbal dialogue with the robots. The system will be presented live during the HRI2023 conference and the source code of this integration is shared with the hope that it will serve the community in designing and evaluating new dialogue systems for robots.
How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers Sara Mahmoud, Erik Billing, Henrik Svensson, Serge Thill Frontiers in Artificial Intelligence, 2023 Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning—just as in human learning—as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task.
Understanding Eye-Tracking in Virtual Reality Ceur Workshop Proceedings, 2022
The Social Robot Expectation Gap Evaluation Framework Julia Rosén, Jessica Lindblom, Erik Billing Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2022
Action similarity judgment based on kinematic primitives Vipul Nair, Paul Hemeren, Alessia Vignolo, Nicoletta Noceti, Elena Nicora, et al. ICDL Epirob 2020 10th IEEE International Conference on Development and Learning and Epigenetic Robotics, 2020
Social Robots in Therapy and Care Daniel Hernandez Garcia, Pablo G. Esteban, Hee Rin Lee, Marta Romeo, Emmanuel Senft, et al. ACM IEEE International Conference on Human Robot Interaction, 2019
