1 Introduction

Due to current changes in domestic structures and working environments, a reduction in informal care and a higher demand for formal care solutions will take place in the upcoming years [1]. Furthermore, demographic change increases this event even further. In 2018, 95.000 people in Austria were cared for in residential homes or nursing homes [2] and it is to be expected, that by 2030 the number of people in Austria over the age of 85 will be increased by 50%. Longer life expectancy and the rise of multimorbidity lead to the fact that by 2030 the number of people in need of care in residential homes will increase by 33% implicating a higher demand for formal care and nursing staff. Demand forecasts show that by 2030 about 30.000 additional professional caregivers will be needed in Austria as compared to 2019 [3]. Moreover, the requirements for nursing have changed. More elderly people have to be cared for and their personal resources have to be supported to allow a more autonomous and self-determined life for as long as possible [4]. These needs come with high challenges for caring staff. Especially lack of time resources hinders caregivers from meeting the requirements of high-quality care [5].

Therefore, socially-assistive robots (SAR) are a promising resource when it comes to challenging and changing care settings. SARs can be used as an information tool which can help to obtain and promote cognitive skills in elderly people [6]. Moreover, socially interacting robots can contribute to a higher quality of life by alleviating loneliness and encouraging more social interaction [7]. While developing technical solutions, mainly young and middle-aged people are included in the design processes and the needs of elderly people remain unconsidered. To address the special cognitive and physical needs of this target group, the elderly have to be included in research and development [8]. Even though researchers are already putting this recommendation into practice, vulnerable groups – like older adults or people with a cognitive impairment such as dementia – are still underrepresented in research processes. Also, vulnerable groups are currently often only addressed indirectly, for example via relatives in terms of their perception or reaction regarding SARs [9].

To evaluate the interaction between the SAR and a human being the behaviour towards the robot has to be observed [10]. By using a questionnaire based on the „Unified Theory of Acceptance and Use of Technology“ (UTAUT), technical acceptance can be raised [11]. This approach involves vulnerable groups directly in the research process and gives an expression of how the interaction with the robot correlates with technical acceptance.

1.1 Socially-Assistive Robots (SARs)

SARs are a combination of assistive and social-interactive robots [12]. Through verbal and non-verbal communication, they can interact with people and create an emotional connection with the person using the robot [13]. In contrast to other robot types such as service robots (e.g. “Care-O-bot” [14]) SARs typically are barely able to provide physical support but provide support indirectly by facilitating a socially enriched form of often human-like interaction.

1.1.1 James

The robot “James”Footnote 1 was used in this study. James is a socially-assistive robot developed, to support people in daily business like a butler. It moves autonomously and can be controlled over speech commands or via tablet. After the spoken command “OK James”, it can answer questions or give information e.g. regarding the weather [15]. By touching the screen placed on the robot’s “head”, videos or games can be started and calls can be made [16]. In spring 2020, during the global spread of the Coronavirus, 60 James were used in elderly care homes in Belgium to help the residents stay in touch with relatives and friends and the events of everyday life [17]. So far, there exists no scientific evidence that emphasizes the effects of James. Therefore, it is highly important to deepen the research regarding clinical effects of SARs in general and the used platform “James” in particular.

1.2 Technology Acceptance Models

The actual use of new technology is influenced by various factors, and several models that evaluate factors of influence are known. In the field of robotics, the “Technology Acceptance Model” (TAM) and the “Unified Theory of Acceptance and USE of Technology” (UTAUT) are used predominantly [9]. The given models lack the assumption that technology can be perceived as a social being and are not geared towards specific target groups. Therefore, Heerink et al. [18] created the “Almere Model” based on four constructs taken from UTAUT and two from TAM. In total, the model consists of 13 constructs, each of which is assigned a definition [18] (Fig. 1).

Fig. 1
figure 1

Own figure; Almere Model according to Heerink et al. [18]

Full size image

2 Aim

Currently, most of the user research on acceptance of SARs is done using self-reported measurements such as interviews, focus groups or surveys. Only few studies have applied observational measures, that focused on the reaction of elderly users with or without cognitive impairments on a robot. Most often, such observational studies have examined the reaction on the robotic seal Paro, which provides emotional support by mimicking a seal without involving further movement, speech, or higher levels of interaction [19,20,21]. Here studies have shown that the introduction of Paro to care settings has led to an increase in social interaction between users and their caregivers [22, 23]. However, the features and intended use-cases of Paro are not comparable to the robot James, which intends to provide functional support and motivate users to undertake physical and cognitive training. One recent study observed different engagement dimensions of elderly users with dementia with the service robot MARIO, which is more similar to James regarding its functionality [24]. While the authors scored the human-robot-interaction quantitatively, we intend to qualitatively describe observed reactions of the elderly users towards the robot. Due to the high costs and efforts involved in conducting the studies, few long-term real-life studies with companion-type SAR and their interaction with groups of people with mild cognitive impairments have been conducted and published to date [25].

This study enhances the body of research in the field of human-robot interaction by observing reactions to SARs and interactions between SARs and users and draw conclusions on their acceptance. To this end, users were recruited from a particularly vulnerable target group, partly having cognitive disorders. Likewise, the study was conducted in the real environment of nursing homes to increase the ecologic validity of the results.

3 Methods

3.1 Design and Participants

To investigate the human-robot interaction, Bethel and Murphy [26] describe four methods: self-assessment, observational or behavioural measures, psychophysiology measurements, and task performance metrics. The most used method is self-assessment. A disadvantage of this method is that participants often cannot answer questions adequately, so no valid statements are provided to the research [26]. Kienzler et al. [27] determined by their research, that positive expression can be interpreted as an approval of the system. Therefore, in this study, observational and behavioural measures were used to focus on the interaction and reaction of a person to a SAR. Johnson and Christensen [28] define observation as “watching the behavioural patterns of people in certain situations to obtain information about the phenomenon of interest”. Furthermore, the social life of a person can be raised over observable remarks, such as facial expressions and gestures [29]. Therefore, facial expressions, gestures, and gazes as well as verbal comments were closely monitored.

Observation measures should take place in a client’s familiar environment to understand the acting behaviour and actions of the participant [30]. Seven observations were conducted between October 2019 and December 2019, and twelve participants in two geriatric nursing homes in Vienna (A) have been included. Each observational session lasted approximately one hour. Between the sessions were five to fourteen days. A “Montreal Cognitive Assessment” (MoCa) was carried out on potentially interested people to assess their dementia impairment. Being able to understand the targets and actions of the study, a MoCa-Score between 19 and 26, as well as the ability to move around with or without aids were carried out as inclusion criteria for participating. The MoCa is an assessment for dementia impairments. A score between 18 and 25 marks a mild cognitive impairment. A score more than 26 designates normal cognition. Nursing staff specifically approached potential residents. Finally, twelve participants from two residential homes could be included in the study. Half of the participants (n = 6) though had a MoCa Score over 26 but were included after discussion in the project team. Ten participants were women and two were men; they were aged from 80 to 96 (Mean: 86.08, SD: 5.47).

3.2 Data Generation

The interaction with the robot was video recorded by a GoPro with no additional technical equipment. For an unbiased view of the situation during the observation measure, no structured guideline is recommended [29] and hence was not used. For the duration of the observation, only events that cannot be captured on video were documented. Furthermore, information about the time and place of the observation, participants, the duration of the observation, and a brief description of the setting was noted. Later, while analyzing the footage, a protocol describing the behaviour of the participants during the individual actions with James was written down. The evaluation procedure is described in detail in chapter “Data Analysis”.

Observation settings were different in the participating nursing homes. The arrangement of the observation setting in each nursing home is shown in Figs. 2 and 3. Four intervention sessions with observations were conducted in Nursing Home A and three in Nursing Home B. In Nursing Home A, the sessions took place within the ward area, so that other residents, which were not involved in the study, as well as staff members were present. This led to minor disturbing factors such as ambient noise. In Nursing Home B, the session was conducted outside the ward. Hence, more space was available, and there was less interference from other people surrounding the scene.

Fig. 2
figure 2

Setting in Nursing Home A

Full size image
Fig. 3
figure 3

Setting in Nursing Home B

Full size image

3.3 Observation Setting

The host of the sessions was a member of the research team with a background in occupational therapy, who introduced the SAR “James” to the participants. Afterwards, James introduced itself and its applications by showing the features that could be used. James was speaking with a male voice, with eyes displayed on the screen, while talking. James played a joyful song, approached each participant, and offered a piece of chocolate, that was placed in a small bag on his front. In every session with James, participants were invited to solve tasks regarding memory function, attention, and concentration in interaction with “James”. Moreover, the SAR was moving around, played videos of physical exercises via its screen to improve flexibility and relaxation, and residents were encouraged to take part (Fig. 4). James had several games to play, single-user or with a partner, such as classical “Memory”, “Four in a row” (Fig. 5) or other cognitive games like “I pack my basket and take … with me”. Throughout these meetings, participants had the chance to operate the robot themselves and interact with it, with support from the host if needed.

Fig. 4
figure 4

Study participants performing physical exercises with James

Full size image
Fig. 5
figure 5

Study participants playing the game “4 in a row” with James

Full size image

3.4 Data Analysis

Videos from seven interaction sessions were recorded. As a second step, the recordings were transcribed by using MAXQDAFootnote 2. Behaviours of the participants, such as facial expressions, gestures, and verbal statements, were identified and aligned with the observation protocol. Verbal comments were transcribed verbatim. For anonymization participants were allocated a number (P1-P12) and caregivers were referred to as “CG”. Emerging data were analysed according to Mayring’s content analyses [31], including deductive and inductive approaches. Six categories from “Video Coding – Incorporation Overserved Emotion” (VC-IOE) were used deductively on the material to assess activity and efficiency of the intervention [32]. Data that could not be classified into one of these categories were treated inductively and assigned to newly created categories (Table 1). The category “other emotions” was used as a subcategory associated with the dimension “emotion” in the VC-IOE. Next, observations of each category were linked to the constructs of the Almere Model [18], to draw conclusions on technology acceptance factors. The video analysis was conducted by a single researcher.

Table 1 Deductive and inductive Categories in Data Analysis
Full size table

3.5 Ethics

The ethical advisory boards of each trial site gave ethical approval. All participants were informed comprehensively about the study and its details, including the information that they are observed and video recorded. They (or their legal representative) gave their approval over written informed consent. The participants were informed that they can quit at any time without giving reasons and that the interaction with the robot is voluntary.

4 Results

In the following, the results emerging from the data analysis along the categories of the VC-IOE plus the inductively created sub-category “unsteadiness and defensiveness” are described in detail. The category “other emotions” is included under the heading “emotions”.

4.1 Emotions

All participants showed positive reactions to the SAR “James” while interacting with it. Some of them smiled when they were greeted by James, others were smiling or laughing throughout the group activities or when they were playing a game or watching videos on “James”. Three participants acknowledged praise from the robot with a smile. Some situations caused negative emotions that were expressed by pulling eyebrows together or pressing lips together. These reactions occurred mainly when the display on the screen could not be recognised or when the participants were solving cognitive tasks. Some participants showed neutral emotions throughout the physical exercises or while watching videos on screen. Six participants reacted neutrally when they were welcomed by James. Others raised their eyebrows, opened their eyes widely and/or changed their position/view when James started interacting: James makes a sound, P1 says: “oh, now.!”, the eyebrows go up and the upper body is leaned forward [24.10.(1); Pos.67.

4.2 Visual Reaction

All participants made constant eye contact with James when they were greeted by it. Also, during other activities (e.g. videos, games) participants stayed focused on James’ screen. Interruption to the reaction and interaction was mainly caused by disturbing surrounding noises or other people who crossed the scene: James is moving away from P3, someone crosses the room, P3 looks at the person and then back to James [24.10.(1); Pos.69].

4.3 Verbal Comments

Seven participants made positive expressions while looking at and/or interacting with the SAR – such as praise, greeting, and/or thanking: “P3 smiles at James and says Good morning” [31.10.2019(1);Pos.4]. Or while James moved around offering sweets from his basket: “P12 takes a candy and says while showing James the candy: Thank you very much, I enjoy that” [29.11.2019(1) Pos. 27].

There was one only specific negative comment to James, throughout a conversation of three residents: “Who is getting on your nerves? P9 points at James and says “the computer, this James” [09.12.2019(2); Pos.59–63]. Sporadic statements were registered which show a defensiveness towards James: P1 and P2 playing Memory; the host wants to integrate P5 and move her nearer towards James. P5: “No, I am not pressing, I don’t touch it” and makes a swiping motion with her hand [6.11.2019(1); Pos. 133].

Throughout the physical exercises, participants did not make any verbal comments; cognitive tasks sometimes stayed verbally uncommented.

4.4 Behavioural Patterns

During the observations, the handling of the robot was consistently adequate and can be interpreted as “positive”. Participants tapped James’ screen fast and without hesitation: The host suggests playing a game together and says:” You can play a concentration game if you want”. P3 looks at James and taps on the screen [6.11.2019(2); Pos. 28–29]. This scene was observed throughout playing games and while operating with the robot autonomously. In one situation a participant tried to move the screen towards another person. This intention was stopped before being carried out by the host, as moving the SAR physically would have led to the need for technical recalibration. No approach toward James was observed when physical activities were conducted and/or when the robot was out of reach for the participants.

4.5 Group Behaviour

Playing games together with James and carrying out cognitive tasks led to interaction between the participants. In some cases, residents started a conversation about a video they had seen on James’ screen: James plays the video “14 little cats”. P11 says towards the others: “I’ve always had cats.” P9 replies astonished: ”Really?” P8 to P11:” me too” [04.12.2019(2); Pos.11–13]. Some participants showed no signs of group behaviour during the interaction with James.

4.6 Agitation

Only one participant showed signs of agitation within the first session with the SAR. Agitation became visible through repetitive movement of the fingers and hand as well as the motion of the tongue on the lower lip. These signs of agitation did not occur in the following sessions.

4.7 Unsteadiness and Defensiveness

In cases when James did not show an immediate reaction while operating with it, unsteadiness among the participants occurred. Therefore, the host was addressed whether there went something wrong in carrying out the given task: P9 wants to ask James what time it is. P9:” Ok, James. What’s the time, please?” No reaction from James. P9 says:” Maybe, it doesn’t like my voice” [29.11.2019(2); Pos.101–107]. Some participants appeared defensive towards the robot in terms of touching or operating it. One person tried to use the hand of someone else to touch the screen.

The following action of one participant shows that there has to be build up trust in the robot: James is announcing the current time. P11 looks at the wristwatch. The host points out: “Instant control.” P11 laughs:” We do not trust each other yet” [29.11.2019(2); Pos.93–94].

5 Discussion

This study aimed to raise the reactions and interactions toward the SAR to draw conclusions on the acceptance of the system. The results are processed based on the research questions and conclusions on technical acceptance are drawn in the following (Table 2).

Table 2 Observation categories corresponding to Almere constructs
Full size table

5.1 Interaction

The majority of the participants took up contact and interacted with James in the study settings. Mostly they held visual contact with the robot throughout the interaction and treated it in a respectful way like greeting it, making fun and giving praise. Moreover, they started a tactile interaction with the SAR by touching its screen when operating with James. Only a few dismissive statements in special situations were recorded. Continued eye contact is a sign of openness and interest and positive verbal comments indicate a positive attitude [33]. Therefore, the observations in this study suggest an interest in the robot among the participants. An intention to touch is crucial to a successful human-robot interaction [34].

5.2 Reaction

Emotional reactions to the robot were observed over facial expressions like signs of pleasure or anger as well as gestures of rejection. During study observation, mainly positive emotions were expressed and recorded. Examples of positive emotions were smiling or laughing and, in some cases, clapping hands. Impairments of the view, light reflections on the screen, or struggles while solving cognitive tasks were triggers for negative emotions. In some situations when James did not react as intended insecure behaviour could be observed. Czaja et al. [35] stated, that new technologies can lead to a feeling of anxiety in older persons. Observed gestures of rejection could be a sign of anxiety towards the robot.

5.3 Acceptance

The classification of the collected observation data along with the Almere Model according to Heerink et al. [18] allows conclusions to the acceptance of the robot (see Fig. 1). Participants showed signs of surprise when the SAR welcomed them and kept visual attendance throughout the session. They greeted, waved, or spread their arms for a hug towards the robot, said “thank you”, gave praise to the SAR or were even joking with it. According to the Almere Model, those reactions are assigned to the construct “social presence”. Most participants reacted positively surprised or were enjoyed, which points to the construct of “perceived enjoyment” that further leads to “intention to use” followed by actual “use”. Consequently, this might be a sign to accept the new technology. Situations, that led to a loss of visual attendance mainly were caused by distractions in the surrounding, but also without an observable trigger. This might be a sign of a lack of “perceived sociability” of the SAR or a lack of interest, which can be interpreted as a negative “attitude towards the technology”.

The interaction with the SAR also triggered group behaviour between the participants and initiated conversations between them, which might contribute to a positive “attitude towards the technology”. This is in line with a scoping review, that concluded that SARs can act as a social facilitator [36].

Aggravation was caused mainly by visual impairments or overstraining situations, which can influence the acceptance of the system negatively. Emotions, verbal comments and signs of unsteadiness and defensiveness were observed with respect to such overstraining situations. According to the Almere Model, those observations can be classified as “perceived ease of use”, as well as “perceived adaptiveness”, which both affect the “perceived usefulness” and “intention to use”. In this case “facilitating conditions” must be taken into consideration. Therefore, participants need assistance from a host or should receive support from other participants (“social influence”) to reach the point of „use“. Uncertainty in operation, lack of “trust”, and signs of “anxiety” were also observed, which can influence the “perceived ease of use” and “perceived usefulness”. Anxiety can reduce the sense of self-efficacy and trigger a rejection of the system. Therefore, technical solutions should lead to successful experiences [35]. In this study, the majority of the participants operated the robot despite signs of uncertainty and subsequently showed joy. The participants interacted with the SAR by talking to it and touching it throughout the different games and exercises. Most users did not show any signs of hesitation or anxiety, which is a sign of “perceived ease of use”. Still, this has to be cautiously interpreted, as a host (“facilitating condition”) was present all the time and supported the participants, when necessary. Uncertainties in operation might be prevented with appropriate training and by getting routine in handling.

Still, in some situations, participants showed apparent rejection towards the robot or commented negatively on it, while conducting the study. This might be interpreted as a lack of technology acceptance, based on their “attitude towards the technology” and a lack of “perceived ease of use” and “perceived usefulness”. Also, users showed agitation towards the robot only in the first session, but not in consecutive sessions. This could be a sign of initial excitement wearing off, which could negatively influence the acceptance over a longer duration of use, which was not part of this study.

The mainly positive reactions to the SAR indicate a positive attitude towards the system, for most of the participants. This is in line with previous studies that mainly used self-assessment to investigate user acceptance [37, 38]. These earlier results can now be backed up by means of another research method focusing not on self-reported measures but on measurements of behaviour and emotion. According to the Almere Model, “social influence” has an impact on the attitude towards the SAR. Support in the use of the SAR by a facilitator or by another participant can influence the use in a positive way, which leads to an “intention to use”, and subsequent “use” of the system.

5.4 Limitations

A relatively small number of participants from two residential homes could be recruited due to time resources. Some difficulties arose in this study, regarding the inclusion criteria of the participants. Interested and suitable people did not have the predefined MoCa-Score. To obtain a suitable sample size within the limited research resources, participants who had a MoCa-Score above 26 were also included in the study. The so-called “Hawthorne Effect” states that the behaviour of the participants may be different because they have been under observation, which could cause another bias. Behaviour tends to be more positive in observation situations [39]. Another influence on behaviour could be the presence of caregivers in one of the observation settings who are important persons to the participants. According to Venkatesh et al. [40], the presence of related persons could also have an impact on the behaviours of the participants. Further, the conclusions, that were drawn from the observations on the acceptance of the SAR based on the Almere Model, should be interpreted with caution, and validated in further studies, as only subjective measures were used in this study. A mixed approach of observations with other indices as stress measures based on urine or saliva, as done by Nomura et al. [41] might give a more objective representation of the effect the SAR has on the participants, but also comes along with a more complex and obstructive study design and ethical considerations. Finally, the video analysis was conducted by a single researcher, introducing the potential for subjective bias in the interpretation of the data. This could affect the reliability and generalizability of the study’s findings, as the observations may be influenced by the individual’s preconceptions or specific focus areas.

6 Conclusions

The approach of using observations of emotions and behaviour of elderly people with mild cognitive impairments could be successfully implemented within a real-life scenario-based interaction study with a SAR in elderly care residences. Mainly upbeat emotions and interaction with the SAR were found indicating a positive attitude towards the specific SAR. The results correspond well with earlier studies that facilitated self-reported measurements such as interviews and focus groups with SARs and hence strengthen the scientific body of research and position that elderly people, even those with mild cognitive impairments react well when engaged by a SAR. Similar studies over a longer duration should be carried out to investigate the long-term effects of SAR interaction on emotion and behaviour.