Designing for usability: development and evaluation of a portable minimally-actuated haptic hand and forearm trainer for unsupervised stroke rehabilitation

1 Introduction

Stroke is one of the main contributors to disability worldwide and its impact on society is expected to further increase in the future with aging populations (Feigin et al., 2022). After stroke, the loss of upper-limb functions such as grasping and fine manipulation is particularly prevalent (Lai et al., 2002; Kwakkel et al., 2003; Zbytniewska-Mégret et al., 2023) and affects the autonomy and quality of life of patients (Mercier et al., 2001).

To maximize therapy outcomes, patients should undergo an intense and high-dosage (i.e., large number of repetitions and long overall training duration) neurorehabilitation program (Kwakkel et al., 2004; Schneider et al., 2016; Ward et al., 2019; Tollár et al., 2021). Unfortunately, the dosage and intensity of current interventions that rely on one-to-one interactions between therapists and patients are often considerably lower than recommended due to organizational constraints and limited resources in the healthcare system (Camillieri, 2019). This situation is expected to be further aggravated in the near future by the increasing financial pressure in healthcare and the global clinical staff shortage (Haakenstad et al., 2022). Group therapy and home rehabilitation are two approaches that could mitigate this societal challenge. Group therapy can be as effective as dose-matched individual therapy (Renner et al., 2016), while home-based rehabilitation provides the flexibility of location and can maintain therapy dosage following clinical discharge if exercises are frequently performed according to plan and executed correctly (Hackett et al., 2002; Cramer et al., 2019; Chi et al., 2020). It has been suggested that sustained training at home might even be a prerequisite for minimizing subsequent losses in patients' quality of life (Tollár et al., 2023). Thus, a paradigm shift is needed from conventional on-site labor-intensive motor rehabilitation to minimally supervised motor rehabilitation at the patient's home.

However, both group therapy and at-home rehabilitation require adequate tools that deliver high-dosage training and ensure patient engagement in minimally supervised (group therapy) or unsupervised environments (home rehabilitation). Sensorized or robotic devices in combination with interactive gamified exercises are promising candidates to drive this paradigm shift (Chen et al., 2019; Handelzalts et al., 2021; Lambercy et al., 2021; Forbrigger et al., 2023a). The effectiveness of robot-based group therapy in providing high dosage high-intensity therapy has already been demonstrated (e.g., Hesse et al., 2014), while the feasibility of sensorized or robotic devices to deliver high dosage in self-guided therapies at home has been largely endorsed, (e.g., Sivan et al., 2014; Wittmann et al., 2016; Hyakutake et al., 2019; McCabe et al., 2019; Rozevink et al., 2021). Notably, there is even evidence that technology-based rehabilitation programs can outperform conventional (i.e., non-interactive exercises according to paper instructions) home rehabilitation (Wilson et al., 2021; Swanson et al., 2023).

Among current technological solutions for unsupervised home rehabilitation, we can find non-actuated devices like the Armeo^®Senso (Hocoma AG, Switzerland), the FitMi (Flint Rehab, USA) or MERLIN (Guillén-Climent et al., 2021). These sensorized devices typically track the patient's movements using sensors such as inertial measurement units (IMUs) or sensorized wheels (e.g., rotary encoders), allowing the patients' movements to be used as inputs for gamified exercises on a tablet, computer, or smartphone. Some devices, like the Gripable (GripAble Limited, United Kingdom), Pablo^® (TyroMotion GmbH, Austria), or the NeuroBall^TM (Neurofenix, USA) additionally feature sensors to detect grip strength. While non-actuated devices have shown their feasibility to deliver high dosage in self-guided therapies at home (Wittmann et al., 2016; Rozevink et al., 2021), they are limited in their capabilities to actively support or resist patients' hand movements. This is overcome with actuated robotic devices such as the PoRi, a compact hand-held device with one actuated degree of freedom (DoF) for grasping that includes haptic feedback vibro-tactile actuators (Wolf et al., 2022). Other examples include the hCAAR, a robotic device for planar movements in the transversal plane (Sivan et al., 2014), or the Motus Hand, a commercial device for wrist flexion/extension (Wolf et al., 2015). Yet, while all these solutions seemed to be well suited for minimally supervised or unsupervised training, except for PoRi, they mostly target motor functions and neglect the training of somatosensory functions.

The execution of skillful movements relies on the integration of meaningful sensory information such as touch and proprioception (Scott, 2004; Pettypiece et al., 2010) and the provision of such information during training is therefore highly recommended (Bolognini et al., 2016; Handelzalts et al., 2021). In robotic training, such sensory information can be provided through haptic rendering—i.e., the generation of physical forces from interactions with tangible virtual objects (Gassert and Dietz, 2018). Although multiple robotic rehabilitation devices have specifically been developed to address this (e.g., Metzger et al. 2011; Fong et al. 2017; Rätz et al. 2021a), they are mostly intended for clinical rehabilitation. To our best knowledge, the recent ReHandyBot (Articares Pte Ltd, Singapore)—a commercial device based on the haptic tabletop device HandyBot (Ranzani et al., 2023) with two DoF, i.e., grasping and pronosupination—is currently the only commercial portable upper-limb device intended for home use which was explicitly designed to also address sensory deficits.

Commercial devices also remain costly, limiting their adoption both for group therapy and at-home rehabilitation. Cost-effectiveness was listed as one of the driving reasons for not recommending robot-assisted neurorehabilitation in adult post-stroke training by the United Kingdom National Institute for Health and Care Excellence guidelines in October 2023 (NICE, 2023). These costs were not only associated with the device purchase (first investment) but also with maintaining the equipment, the staff time for setting up the machine for each use, and time to teach the patient how to use it. Further, it was noted that machines were only used in a small subset of patients and so could not be used at their full capacity, increasing the cost per use and so overall intervention costs. We thus think that there is a further need for low-cost, versatile, intuitive, and highly portable hand rehabilitation devices that provide meaningful haptic feedback for use in minimally supervised or unsupervised settings.

Here, we present the design and results from a first usability test of our second prototype of a portable and low-cost haptic hand trainer based on a novel compliant shell mechanism. The device offers two degrees of freedom: An actuated one for grasping as well as haptic rendering, and a passive one for pronosupination movements. Meeting the stringent criteria for home rehabilitation devices is challenging (Chen et al., 2019; Forbrigger et al., 2023b). It has been shown that usability and users' perceptions of assistive devices and rehabilitation technology for home use substantially influence their long-term utilization (Biddiss and Chau, 2007; Sivan et al., 2014; Sugawara et al., 2018; Ciortea et al., 2021). Therefore, we co-created the novel portable device with clinical personnel following a human-centered design approach to ensure efficient and goal-oriented development. For this purpose, we followed four phases when designing our solution: (i) Understand the context of use; (ii) Specify patients' requirements; (iii) Design the solution; and (iv) Evaluate against requirements. Insights from the first two phases are published in Rätz et al. (2021b) and Van Damme et al. (2022). Here we report on the two later phases. We embraced a mixed-method approach to evaluate the usability of our invention, where quantitative methods such as questionnaires were combined with qualitative approaches like interviews. Hereby, the use of standardized questionnaires was advocated as the results become more meaningful and comparable (Meyer et al., 2021). We included supplementary techniques like video recordings and eye-tracking to help gain further insights into the root causes of usability issues (Goldberg and Wichansky, 2003; Schaarup et al., 2015; Maramba et al., 2019). For a comprehensive online guide on usability assessment methods, see Meyer et al. (2023).

The rest of the paper is organized as follows: We first present the development of the second prototype of the novel portable hand trainer as well as the design of an accompanying rehabilitation game including the computation of interaction forces with virtual game objects. This development is the continuation of a concept that we proposed in Van Damme et al. (2022). We then introduce the setup and methodology of a usability experiment with 13 healthy participants, including three therapists, in a simulated unsupervised scenario. Finally, we present the results and discuss their implications for further developments and studies in patients' homes. This paper evaluates our design choices and offers other device and rehabilitation game developers detailed insights into our learnings.

2 Materials and methods

2.1 Device development

2.1.1 Requirements

The first prototype of our portable hand trainer was developed and patented in 2022 (Van Damme et al., 2022; Rätz et al., 2023). The core idea of this first prototype was a U-shaped compliant shell that is grasped with the entire hand—i.e., enclosed with fingers, thumb and palm—and could allow an extremely simple and inherently safe mechanical human-device interaction even in case of improper setup of the user's hand. This shell design mimics a natural large-diameter power grasp, i.e., simultaneous flexion or extension of all fingers with abducted thumb. This particular grasp was selected as it represents one of the most frequently employed hand movements in activities of daily living (ADL) (Bullock et al., 2013) and is effectively trained in clinical rehabilitation (Pandian et al., 2012). Importantly, our first prototype drastically minimized the risk of skin getting pinched in gaps between moving parts regardless of the exact hand proportions of the user, making it an excellent candidate for home rehabilitation. Finally, it also featured a highly-backdrivable transmission and offered good mechanical transparency, allowing for open-loop impedance control to achieve fine haptic rendering. While the initial prototype served as a preliminary proof of concept, a more elaborate version was clearly needed for a first usability study.

For the second generation of the portable hand trainer presented in this study, we defined the following improvements based on first evaluation tests, informal discussions with therapists of the Department of Neurology, University Hospital Bern, Switzerland, and the literature (e.g., Lu et al., 2011; Akbari et al., 2021; Li et al., 2021; Rätz et al., 2021b): i) The device must be aesthetically pleasing and should look like a medical device. This includes integrating the electronics (except for the power supply and emergency stop) into the device housing. ii) A passive DoF shall be added for pronosupination movements. iii) Although training was feasible for various hand sizes with the first prototype, we found that shorter fingers would benefit from smaller shell sizes. In addition, we also aimed to increase the range of motion of fingers and thumb during grasping. iv) The device must be safe for the ultimate goal is to make it in real unsupervised training. v) The device should remain as compact and portable as possible.

2.1.2 Shell design

The desired bending behavior of the shell during grasping—i.e., following the natural movement of the thumb and the fingers—is achieved by anchoring the shell center part to the device while moving the shell ends on circular paths. For this, the shell ends are connected to a thumb and finger lever (Figure 1). These levers are coupled and actuated through a transmission (see Section 2.1.3).

We decided to design three shell sizes (small, medium, large) for this study's device, using the hand measurements of Garrett (1971). The 5th percentile female hand was considered a small hand, the 95th percentile male hand a large one, and the average of both a medium-sized hand. It is important to note that when closing the hand, the arc length of the inner (i.e., palmar) side of the hand shortens—as easily visible by the skin creases beneath the finger joints—while the thin shell can be assumed to maintain the same arc length when bent. This results in a sliding motion of the fingers along the shell when closing the hand. However, in the first prototype, we found that this sliding is imperceptible to the user and does not pose a problem. In this second prototype, we used this knowledge in the design of three size-specific shell geometries. We designed the size-specific shells such that their ends approximately align with the fingertips (or thumb tip respectively) when the hand is extended (Figure 3). With this and the aforementioned tendency of the fingers to slide backwards on the shell, we know that the fingertips will not collide when the shell is closed. The shell height is the same for the three sizes.

The lengths of the levers that move the shell ends and the locations of their respective center of motion were defined in an iterative process such that (i) the shell ends move along a natural fingertip path, and (ii) there is no collision between mechanical parts. For a quick change of the shells, each shell is mounted with a combination of removable rods at its ends and dowel pins at its center. Thereby, all three shells share a common center fixation, resulting in alignment of the shell center areas that support the thenar web space (i.e., the part between thumb and fingers). This allowed us to use the same wrist fixation for the three shell sizes.

2.1.3 Transmission and control

The device is actuated by an electric DC motor. The transmission is divided into a spur gear stage and a synchronous belt transmission stage. Hereby, the synchronous belt has two functions: First, it amplifies the motor torque, and second, it couples the thumb and the finger movements. Figure 1 shows the mechanism. Note the different diameters of the finger (d₅) and thumb actuation (d₄) pulleys, which allows us to take into account the different ranges of motion of the thumb and the fingers. The arc lengths of the finger and thumb paths are denoted s_f and s_t respectively, and are computed in Eq. (1) with d₁, d₂, d₃, d₄, and d₅ being the effective pulley/gear diameters, r_f and r_t the lever lengths (see Section 2.1.2), and θ_m being the angular displacement of the motor shaft.

Applying the principle of virtual work in a static condition, we can thus compute the required motor torque τ_m with Eq. (2), using the partial derivatives of s_f and s_t with respect to the angular position of the motor shaft θ_m and the thumb and fingertip forces F_t and F_f (i.e., the forces at the shell ends, along the circular path of the shell ends):

If we assume that the thumb and finger forces are equal and given by F = F_t = F_f, the motor torque is given by Eq. (3). Note, that this assumption does not necessarily strictly hold during use, as the additional hand-device contacts at the wrist fixation and thenar web space might result in a statically over-constrained situation where unequal forces to thumb and fingers could be applied. However, this assumption is required for the control of the one-DoF shell actuation.

Because the thumb and fingertip movements are coupled, we need to compute their combined displacement s and the speed ṡ with Eq. (4):

The desired rendered force F is computed with Eq. (5), depending on the desired visco-elastic characteristics (viscosity B and stiffness K) of the virtual object/environment the participant may interact with.

We increased the device transparency by compensating the inherent restitution force of the printed shell to allow the fingers to move freely when not in contact with virtual objects. For each shell size, we identified the inherent spring constant and offset by applying a constant motor torque in steps of 3 mNm and measuring the resulting angular motor position (and thus the shell deflection). The compensation torque τ_{m, c} was then computed as the linear regression of the collected data points (see Figure 2). This leads to a final motor torque in Eq. (6).

2.1.4 Final prototype

The final prototype (Figure 3) consists of a housing in which the transmission and electronic components are placed, a wrist rest, a button with integrated status LED, a DC motor, and the shell. The final prototype has a length of 210 mm, a width of 160 mm, and a height of 150 mm. The range of motion for finger and thumb levers (Figure 1) are approximately 80° and 40°, respectively. The majority of the device was 3D-printed in FDA-approved polylactic acid (PLA) plastic, while a few structural parts were printed in carbon-reinforced PLA or machined out of aluminum or stainless steel (e.g., shafts of the pulleys, vertical support rods at the ends of the shells). This results in a weight of 1030 g without the external power supply and emergency stops. For this study, a right-handed version was manufactured. The cost of one prototype unit without external emergency stop buttons is currently slightly below 1,000 CHF (approx. 1,000 EUR).

Since the endpoints of the three differently sized shells move on different paths, each shell comes with a distinct cover for the frontal upper part of the device, which is fixated on the device with magnets (front cover in Figure 3A). A shell can be exchanged in a few seconds by first pulling it vertically off the device (the vertical support rods can either stay in the shell or be removed separately), then exchanging the front cover and inserting the new shell (see video in Supplementary Material).

The DC motor with an optical encoder (3272CR and IER3 with 4096 pulses, Faulhaber, Germany) is placed vertically outside the main housing and inside the shell, as this allows the most efficient use of space. The resulting maximal continuous force at the fingertips depends on the shell size, 15.7 N (small), 13.6 N (medium), and 11.9 N (large). The device is powered by an external medical-grade 12 V power supply and can be equipped with two emergency stop buttons in series (one for the user and one for a therapist or experimenter). An ESP-32 microcontroller (Espressif Systems, China) with an Escon Module 50/5 motor driver (Maxon, Switzerland) is used to control the device. The embedded control software was written in C++ and is based on the open-source FreeRTOS™ (Amazon, USA) real-time operating kernel, allowing for appropriate scheduling and prioritization of tasks. One of the two cores of the microprocessor runs a dedicated thread for the haptic rendering and motor control at 1 kHz. Games or rehabilitation exercises are executed on a host computer, with which the device communicates through a USB serial connection.

To enable pronosupination movements (i.e., tilting of the device, Figure 3B), the edges of the bottom of the device were rounded. This allows smooth tilting of the device up to 15° to each side, however, the device can easily be tilted more while leaning on its edge. Because of the remaining flat part in the center, the device is still stable when positioned on a flat surface. The tilting angle is measured with an LSM6DSOX (Adafruit, USA) inertial measurement unit (IMU). A wrist strap was added to fix the user's hand in position and to facilitate pronosupination movements. The size of the wrist strap is adjustable via hook and loop and can be opened with a magnetic lock (Fidlock GmbH, Germany). Another strap was attached onto the distal position on the shell to fixate the fingertips (Figure 3A). This strap can be opened with a hook and loop fixation, however, this is not necessarily required as the fingers can simply be sled in.

Before use, the device needs to go through a short calibration step. Upon a three-second press on the device button, a calibration is performed by slowly closing the shell with a proportional-integral (PI) velocity controller and detecting the sudden increase in controller output when the mechanical limit is hit. This calibration step is required to obtain a mapping from the motor shaft angle—measured by an incremental encoder—to the shell endpoint positions, i.e., the opening of the shell. The shell restitution compensation is executed according to the shell size, which must be provided through the host computer prior to the initialization. After completion of the calibration routine, the status LED turns on.

2.2 Serious game

2.2.1 Requirements

We complemented our device with a rehabilitation exercise in the form of a serious game. A preliminary version of the game is described in Van Damme et al. (2022). Amongst other literature (e.g., Burke et al., 2009; Lohse et al., 2013; Li et al., 2021), we oriented ourselves during the development in the results of a survey that we performed among clinical personnel (Rätz et al., 2021b). We specified the following requirements for the game in this study: i) Ensure that the device can effectively showcase its available movements—i.e., power grasp and pronosupination movements. ii) The task in the game should resemble ADL, while still being entertaining. iii) To motivate participants, we aimed to incorporate a challenge-based component, e.g., limited game life, time constraints, and a scoring system. iv) The users shall be encouraged to actively extend their fingers during the exercise. v) The game should contain sub-tasks with different degrees of difficulty. In Rätz et al. (2021b), we found that adjustability of difficulty was desired by clinical personnel. We decided to abstain from adjustable settings in this study to keep the main focus on the evaluation of the device. vi) Provide meaningful, congruent, and diverse haptic feedback during interactions with virtual objects in the game.

2.2.2 Game design

A screenshot of the designed serious game that satisfies the aforementioned requirements is shown in Figure 4. The game was developed in Unity3D (Unity Technologies, USA) and represents a cocktail bar with four differently colored liquid dispensers, each one having a glass beneath it. The goal is to fill the glasses by grasping the liquid dispensers with a virtual hand avatar by skillfully squeezing the shell. If the liquid dispensers are grasped too strongly, the liquid starts to spill, which results in lost game life, indicated with a life bar located at the top left of the screen. If the liquid dispensers are not squeezed strongly enough, no or only very little liquid will come out. A timer on the upper right corner of the screen indicates the remaining time. If glasses are overfilled and liquid flows over, the life bar also decreases. For each filled-up glass, the text “Full” appears and a point is added to the user's score, shown on the right of the screen.

The game simulates grasping by mimicking the sensation of physically squeezing a liquid dispenser made of a visco-elastic material. Each liquid dispenser has different characteristics (i.e., stiffness K and damping B in Eq. 5), reflected in the haptic rendering. When the user squeezes the shell, the virtual hand avatar first moves forward to the liquid dispenser in front of it, and, when further squeezed, the liquid dispenser is grasped. The variable s₀ from Eq. 5 is defined such that the rendering of the virtual wall starts at this point. Importantly, each liquid dispenser possesses different characteristics, i.e., different pairs of stiffness and damping (K∈{0.6, 0.1, 0.3, 0.01} N/mm and B∈{0.005, 0.001, 0.005, 0} Ns/mm), empirically selected and corresponding to the dispensers from left to right. Notably, the displayed behavior of the liquid matches the haptic rendering, e.g., a liquid dispenser with higher impedance (i.e., higher values of K and B) contains a sticky, viscous liquid, while a lower impedance indicates a runny liquid.

In the first phase, each one of the four glasses needs to be filled once. To move the virtual hand to grasp different dispensers, the fingers need to be extended, and the device tilted—i.e., performing a pronosupination movement. When the IMU detects tilting of more than 5°, the hand avatar moves one step (i.e., one liquid dispenser) in the corresponding tilting direction. After keeping the device tilted for 0.8 s, the avatar continues moving to the next position and so forth, until the device tilting angle is below 5° again. These values were defined through preliminary testing by the developers. To switch position again after a liquid dispenser has been grasped, the hand must be opened again, necessitating active finger extension as specified in the requirements. Once the first four glasses are filled, glasses start to appear randomly. If the life bar is empty, the score is reset to zero and the first phase starts again.

2.3 Usability evaluation

2.3.1 Participants

A total of 13 healthy participants took part in the usability evaluation of our haptic device (six male, six female, and one non-binary; ages between 21 and 64 years; twelve right-handed and one left-handed). Of the 13 participants, three were neurorehabilitation physiotherapists from Rijndam Rehabilitation Center, Rotterdam, the Netherlands. The other ten participants (referred to as non-expert participants in this study) were healthy adults recruited through word of mouth at the Delft University of Technology, Delft, the Netherlands. Following, we refer to participants by pseudonyms T1–T3 (therapists) and N1–N10 (non-expert participants). All participants were naive to the experiment and haptic device. The study was approved by the Human Research Ethics Committee (HREC) of the Delft University of Technology (Application ID: 2216).

2.3.2 Experimental setup and procedure

An unsupervised rehabilitation scenario was reproduced in two different locations. For the non-expert participants, we performed the experiment in a room (approximately 10 m × 5 m) with a table (2.5 m × 1 m) and a height-adjustable chair with backrest. On top of the table, we placed the hand trainer, a laptop, and one emergency stop button. The hand trainer was always placed on the right side beside the laptop, while the emergency stop button was placed on the left side in an easily reachable position. The entire setup was facing a short wall of the room. For the physiotherapists, the experiment was performed at the rehabilitation center in an office (approximately 6 m × 5 m) with a similar setup. A physiotherapist, who was familiar with the device but not involved in its development, led the experiment. One of the device developers was also around to provide support in case of technical difficulties.

The experiment started with obtaining the participants' written consent. Their hand size was then measured and a shell size—i.e., small, medium, large—was suggested by the experimenter according to a predefined size correspondence table. However, participants could switch to a different size after trying the recommended size if desired. The swapping of the shell was performed by the experimenter and is not part of the usability evaluation because in a real-life setting, this would be performed by the therapist and the patient would receive a device where the correct shell is already installed. Five participants felt the most comfortable with the small shell, while the other eight chose the medium size.

After selecting the shell size, participants were equipped with eye-tracking glasses (Tobii Pro Glasses 2, Tobii, Sweden). They were allowed to wear the eye-tracking glasses on top of their prescription glasses. The eye-tracking glasses were calibrated for each user following the manufacturer's guidelines for optimal performance. Participants whose calibration could not be performed successfully due to their prescription glasses were removed from the analysis. The glasses recorded a video of the participant's point of view and a sequence of gaze points (i.e., where they were looking). In addition to the eye-tracking glasses, the experiment was recorded with a video camera, allowing us to measure setup times and identify practical and technical issues after the experiment.

The participants were then invited to sit on the chair and follow the instructions on the laptop screen. They were asked to seek the help of the experimenters only in case of emergency or if they could not continue by themselves. In the case of the non-expert participants, the experimenters moved behind a movable wall equipped with a second emergency stop button but did not leave the room to ensure the participants' safety. At the rehabilitation center, the experimenters positioned themselves diagonally behind the physical therapists to stay out of their line of sight and simulate the minimally supervised scenario while still ensuring the participants' safety with the second emergency stop.

Participants were then asked to follow the instructions presented on the laptop screen through a series of slides related to the device setup, play of the game, and device doffing. The slides related to the device setup instructions included how to turn on the device, how to don the hand, the game instructions, and how to use the emergency button. The device could be turned on by pressing the device button (Figure 3) for at least three seconds. The participants could move to the next instruction slide with a short press of the same device button. After the instructions related to the device setup, a new slide prompted participants to play the game for five minutes. The remaining gaming time was displayed in the upper right corner during the game. When the time was up, a new slide with instructions on turning off the device and releasing the hand appeared. The entire set of instruction slides can be found in the Supplementary material. After the experiment, participants were asked to complete several questionnaires (see Section 2.3.3) and invited to share their experiences in a semi-structured interview. The audio of the interview was recorded for later analysis.

2.3.3 Outcome measures

We defined a variety of quantitative and qualitative outcome measures to assess the usability of the device as well as the participants' motivation and workload. First, the lead experimenter manually recorded the set-up time, i.e., the time required to turn on the device, donning, and doffing. We also noted the number of issues that occurred during the experiment. Hereby we categorized between practical issues (e.g., when the participant visibly misunderstood the instructions or did not know how to proceed) and technical issues (e.g., issues related to the device or the game). In each case, we further noted whether intervention from the experimenters was required to continue the experiment. In cases where the experimenter did not have a clear view of the participant, the recorded video was consulted ad hoc.

We assessed the participants' subjective perception of the system's usability with two questionnaires. We selected the Post-Study Usability Questionnaire (PSSUQ) (Lewis, 2002) for the entire system (i.e., game and device). It consists of 16 seven-point Likert-style items and is divided into three subscales: System Usefulness (i.e., satisfaction, simplicity, and comfort), Information Quality (i.e., if and how relevant information is presented), and Interface Quality (i.e., interaction with the device and game). For an isolated assessment of the device, we additionally employed the shorter System Usability Scale (SUS) questionnaire (Brooke, 1996), which consists of ten five-point Likert-style items. We chose the PSSUQ for the entire system as it exhibits finer granularity and the SUS for the isolated assessment of the device since the PSSUQ contains questions that only make sense in the presence of a software or information component.

The fact that the cognitive capabilities of stroke patients are often affected (e.g., see Mercier et al., 2001) motivated us to also investigate the mental load of our participants when using the system. We utilized the raw NASA Task Load Index (RTLX) (Hart, 2006), a widely used questionnaire in usability testing (Meyer et al., 2021). The RTLX assesses six individual domains, namely the mental, physical, and temporal (i.e., perceived time pressure) demand, the perceived performance, effort (i.e., the effort needed to achieve the performance), and the level of frustration. Each domain is assessed through a single 21-point Likert-style item, whereby zero reflects “very low” (or “perfect” in the performance item) and 20 “very high” (or “failure” in the performance item).

Since motivation is known to be a strong driver of effort and participation in robotic training of stroke patients (Sivan et al., 2014), we also included items from the Interest/Enjoyment and the Perceived Competence subscales of the Intrinsic Motivation Inventory (IMI) (McAuley et al., 1989). All questionnaire scores were normalized to a range from 0 to 100 for a more straightforward interpretation of the results. The PSSUQ and IMI subscale scores for each participant were computed by taking the arithmetic average of the corresponding items, and the overall scores of the PSSUQ and SUS were averaged for all items.

We employed the recorded eye-tracking data, i.e., the participants' points of view and accompanying gaze points, to identify the time participants spent looking at different elements of the experimental setup while playing the game. In the context of usability, the proportion of time spent looking at an element (gaze point rate) may reflect the importance of that element or could indicate difficulties in understanding an element (Jacob and Karn, 2003). This was achieved by counting the number of gaze points per participant landing on six different rectangular areas of interest (AOIs, Figure 5), representing elements of the experimental setup: the device, emergency stop, game (i.e., dispensers and glasses), life bar, score, and remaining time. The number of gaze points landing on the different AOIs was determined per participant from the eye-tracking videos using the AOI tool of the Tobii Pro Lab software (version 1.217, Tobii, Sweden). The AOIs were manually adjusted for keyframes, i.e., individual frames of the videos, at the beginning and end of head movements to ensure that the AOIs were accurately placed on top of their corresponding element. The AOIs' positions and sizes were then linearly interpolated between keyframes. We normalized the number of gaze points per participant and AOI n_AOI over the total number of gaze points per participant n_total to remove the effect of unequal dataset sizes between participants (${\widehat{n}}_{AOI}=n_{AOI}/n_{total}$). The gaze point rates ${\widehat{n}}_{AOI}$ were multiplied by the time spent playing the game (300 s) to calculate the total time participants looked at each of the AOIs.

Finally, we gathered qualitative data through open-ended questions (see Supplementary material) in semi-structured interviews. These questions served as initial prompts to guide the discussion, though the experimenters were free to ask follow-up questions, allowing them to explore topics that seemed particularly important to the individual participant. The audio recordings of the semi-structured interviews were transcribed locally on a computer with a custom software pipeline written in Python. First, a diarization (i.e., partitioning of the audio into segments according to the speaker) was performed with simple-diarizer (Simple Diarizer, 2023) using the xvec model and spectral clustering. The verbatim transcription was then performed based on faster-whisper (Faster Whisper, 2023), which is a re-implementation of the automatic speech recognition Whisper (Radford et al., 2023). We employed the pretrained medium size model. Afterwards, the transcriptions were manually checked and corrected analogous to the audio recordings. A thematic analysis was then performed to determine the principal themes (i.e., recurring patterns, opinions, and ideas) that emerged from the interviews. This methodology involves a systematic examination of the data, wherein text segments are designated descriptive labels known as codes. These codes with the accompanying text segments are then categorized into cohesive themes, which are subsequently summarized and reported. For a comprehensive description of the procedure, please refer to Braun and Clarke (2008).

3 Results

All 13 participants except participant N10 completed all steps of the experiment. The experiment with this participant was ended prematurely by the experimenters when the participant was playing the game due to technical problems with the device (see Table 1); however, participant N10 completed the rest of the experiment (i.e., questionnaires and interview) according to the protocol.

3.1 Setup time, technical issues and practical issues

The setup time measurements are depicted in Figure 6. The overall median time (first quartile, third quartile) that participants spent with the device setup was 58 (47, 63) s. In particular, turning on the device took 6 (5, 10) s, while the subsequent donning took 41 (33, 53) s. Finally, the doffing was again relatively quick, with a duration of 7 (3, 8) s.

The encountered technical and practical issues are summarized in Table 1. Overall, ten practical and four technical issues were observed. With five occurrences, the most observed issue was participants not properly using the magnetic wrist strap. One technical error (N10) led to the ending of the experiment for safety reasons since the technical root cause for this event was unknown at that time.

3.2 Questionnaires

The normalized scores of the questionnaires are summarized in Figure 7. Because of the ordinal nature of the results from the various questionnaires (Sullivan and Artino, 2013), we represent the central tendency using the median with first and third quartiles.

Regarding the usability questionnaires, the PSSUQ questionnaire, which was applied to the entire system, achieved an overall rating of 70.2 (65.6, 85.6) out of 100. Hereby, the System Usefulness subscale scored the highest with 83.3 (69.4, 83.3), followed by the Information Quality with 73.3 (50.0, 90.0) and Interface Quality with 66.7 (50.0, 83.3). The isolated device usability rating from the SUS achieved a score of 77.5 (72.5, 82.5). SUS values of 50.9–71.4, 71.4–85.5, and 85.5–90.9 correspond to OK–good, good–excellent, and excellent–best-imaginable usability, respectively according to Bangor et al. (2009). Note that previous studies have shown that the PSSUQ and the SUS questionnaires are highly correlated (Vlachogianni and Tselios, 2023).

The assessment with the RTLX showed a mental demand of 25.0 (15.0, 45.0), a physical demand of 25.0 (15.0, 55.0), and a temporal demand of 20.0 (5.0, 45.0). Furthermore, it revealed that participants rated their performance with 70.0 (55.0, 80.0), which they achieved with a perceived effort of 45.0 (30.0, 55.0). Hereby, they rated their frustration level as 20.0 (15.0, 30.0) out of 100. In general, low values of the RTLX items indicate a low workload, except for the performance item, where a high value indicates good perceived performance.

Finally, regarding motivation, the overall IMI Interest/Enjoyment subscale score reached 64.3 (57.1, 76.2) out of 100 and the Perceived Competence subscale reached a score of 63.9 (58.3, 69.4). High scores in the IMI subscales relate to high enjoyment and high perceived competence, respectively.

3.3 Gaze point rates per AOI

The results of the gaze point rate per AOI are shown in Figure 8. Two of the eye-tracking datasets were removed due to failed calibration procedures (N1, T2), one caused by technical issues with the data (faulty battery, N7), and one because of the premature termination of playing the game (N10), leaving nine out of 13 datasets. The screen area with the cocktail glasses and dispensers (i.e., the game AOI) obtained the highest normalized hit rate with 87.0% (4.3%) (average and standard deviation). Notably, participants T1 and N5 spent a considerable amount of time looking at the life bar (12.2 s and 7.1 s), while participants N3 and N8 spent more time looking at the device when compared to their peers (5.0 s and 3.5 s, respectively). Overall, the hit rates of 0.42% (0.57%) on the device and 0.03% (0.07%) on the emergency stop with resulting average duration of only 1.27 s and 0.097 s, respectively, were low in comparison with other AOIs.

3.4 Semi-structured interviews

The thematic analysis led to the classification of 495 quotations, resulting in the assignment of 86 codes, which we then organized into seven groups: General Impressions, Pronosupination Movements, Instructions, Game, Comfort, Grasping with Haptic Rendering, and Application & Clinical Use. Following, we present the main findings for each group with examples of supporting participant statements.

3.4.1 General impressions

The participants liked the sleek and simplistic design of the device. The majority of the participants appreciated having only one button for all functions, as it simplified the user experience and reduced the need to remember multiple buttons. One participant expressed concerns about accidentally turning the device off.

“It's quite portable, it's looking sleek, it has nice curves” (T1)

“I think it's very simple so that's great.” (N6)

“I like that there's only one button, because it's just easy” (N4)

The weight and size of the device was generally considered acceptable. Some suggested making it slightly lighter, while others thought it provided stability.

“I think it's nice that it's heavy when you have to move it, because then you really feel that it's rolling through.” (N4)

3.4.2 Pronosupination movements

Seven participants mentioned that the device tilting action to move between dispensers felt clunky and less responsive than expected. They struggled with the step-by-step movement of the hand avatar when tilting and were unsure if the hand needed to stay tilted to move multiple dispenser positions.

“And the turning to the left and right was very... It was taking steps. I thought it was more fluid, but it was taking steps.” (T2)

Furthermore, some participants stated that the tilting felt counter-intuitive at first as the design itself did not look as it was supposed to be tilted.

“It didn't feel very intuitive when I was moving it left and right. Because I would imagine if it's a device that's supposed to rotate it would have something at the bottom that's not flat.” (N6)

3.4.3 Instructions

The reported feelings about the setup and game instructions were mixed. While some participants complimented the simplicity, seven participants mentioned that the instructions were not clear enough and raised concerns about the cognitive load, especially for users with potential cognitive impairments. They recommended simplifying the instructions, making them less information-dense. Furthermore, it was repeatedly suggested that step-by-step video demonstrations or looping animations might be more informative and easier to follow.

“Very clear. And concise. Yeah no it was clear.” (N5)

“I think it's more understandable if I see a 5-minute video and see this is the procedure, then there is no need to read something.” (N8)

In particular, for the magnetic wrist strap, the participants wished to obtain more detailed information about the exact opening mechanism. Several participants were initially confused about the magnetic mechanism of the wrist strap. Some did not realize that it could be opened and instead released the adjacent hook and loop. It was also mentioned that the color coding of the parts could be improved (e.g., finger strap), and should be chosen more carefully to represent their respective importance during the setup. For example, the wrist strap locking mechanism should be visually more highlighted than the finger strap adjustment as it is required to be opened every time during setup, while the finger strap only needs to be adjusted occasionally.

“The only problem I had was with the wrist strap. It says open the lock which I interpreted as just open the hook and loop.” (N8)

“Yes, but there's a red strap here so at first I was just like this because I read quickly and I didn't really understand [...] maybe this [finger strap red part] shouldn't be highlighted more than this [wrist fixation].” (N5)

Participant T2 mentioned having read only a little bit of the instructions, and Participant N8 admitted clicking through the instructions, without following them.

3.4.4 Game

The game was generally perceived as fun and enjoyable to play for the given time. Although some participants struggled to some extent with the pronosupination movements to move the virtual hand sideways, the game appeared to be intuitive for most participants.

“The game, yes, it was funny. I wouldn't play it for hours, of course, but I think it's intuitive and fun.” (N5)

Five participants reported that the concept of the life bar was not fully understood or that the life bar was not even noticed for the majority of the time. It was suggested to make the life bar visually more dominant or to explain it better during the instructions.

“The position of the bar needs to be closer to what's happening. Or there needs to be some visual connection.” (N6)

Yet, few participants noted that the game was boring or could quickly become boring. In this context, some participants expressed their disappointment that the score was not saved and that there was no high score they could beat. It was suggested that a more competitive setting—even if it is just beating one's own score—would increase their motivation and interest in the game. Furthermore, more levels with increased difficulty would help to maintain motivation during longer sessions. The timer was mostly appreciated as a motivational element, although one participant perceived it as stressful.

“It also wasn't really clear to me what my previous score was, so what score should I beat? Because it was a fun game to play, I would like to be competitive.” (N1)

“Just shortly doing it is okay but playing it longer will be very boring for me.” (T2)

3.4.5 Comfort

Participants found the device generally comfortable and safe. Nevertheless, concerns about the wrist position and angle during prolonged use were raised, especially for persons with a paretic upper limb. Due to the height of the device, the hand was in an elevated position with respect to the elbow, resulting in a slight ulnar abduction.

“It is quite comfortable. I was like in a relaxing pose. It was not stressing my hand, it is also very smooth and it is not too tight.” (N2)

“The position of my wrist was a little bit uncomfortable, I think because it was elevated from the table.” (N8)

Two participants found the finger strap adjustment slightly finicky due to the limited space to attach the hook and loop on the shell. One participant desired to have the finger strap in a more proximal position. One participant pointed out that the thumb's position was somewhat unclear, and three suggested that a thumb strap might be helpful during extension movements.

3.4.6 Grasping with haptic rendering

Participants generally found the grasping motion easy to perform. Most of them appreciated the realistic grasping sensation and how the haptic feedback correlated with their actions.

“At the beginning I was looking at the device to see where my fingers were, but at some point, I was just not looking anymore because of the haptic feedback. It was nice.” (N5)

“Really cool, how the grasping really works nicely with the feedback, it really felt like I had some nice feedback, yeah it worked well” (N1)

However, a few also reported that the visuals played a predominant role in their interaction and expressed the need for more prominent and informative haptic feedback.

“I don't know how much I would have been able to tell the difference without the visual aid because I don't know if like my brain was so sensitive to what's happening with my hand. I think those visuals were super important.” (N6)

“I did not feel that a lot. I saw a lot with the drops, but I did not feel very different things.” (T3)

3.4.7 Application and clinical use

All participants stated that they would feel comfortable using the device themselves in an unsupervised environment in the hypothetical scenario of undergoing upper-limb rehabilitation. Two out of the three participating therapists noted that they would use it with their patients, while one was not sure yet. The therapists saw potential applications either in early rehabilitation, group therapy, or home rehabilitation—in particular for patients with reduced tactile or proprioceptive sensibility.

“I think when they have sensibility problems it's very difficult to give the right force to hold a glass or something. So people do that or it's too loose and it falls. So I think with this device you can maybe learn a little bit more and normally we do that with grabbing things. So I think it can be useful for that kind of problems.” (T3)

One therapist noted that stroke patients might benefit from adjustable assistance during the exercise. One mentioned that an initial assessment of patients' range of motion and available grasping force could be used to adjust the device and the game. Moreover, therapists highlighted the importance of variation during the rehabilitation training and suggested increasing the number of available exercises/games.

4 Discussion

4.1 We evolved our concept into a safe, aesthetic, and functional prototype

We developed a minimally-actuated device to meet the need for cost-effective haptic upper-limb training devices for minimally supervised or unsupervised neurorehabilitation. We realized a device that is inherently safe, suitable for a variety of hand sizes, and that can provide meaningful haptic feedback during the grasping of virtual objects by combining a compliant shell design with highly back-drivable actuation. We refined the device's appearance, and also added a passive DoF for wrist pronosupination, a movement highly recommended by therapists (Rätz et al., 2021b), by allowing the entire device to be tilted around its longitudinal axis. The combination of passive and active degrees of freedom is in line with the recommendations of Forbrigger et al. (2023a), who suggested this concept to reduce cost while still providing high functionality. We thus satisfied all the required device improvements that we defined based on the first concept (see Section 2.1.1).

Our novel hand trainer is complemented by a serious game that challenges users to fill virtual cocktail glasses using simulated liquid dispensers with different haptic behaviors, highlighting the haptic capabilities of our device. Thereby, the difficulty of successfully filling the glass without spilling any liquid depends on the simulated liquid and varies across the different dispensers. The task mimics a scenario akin to ADL, as it requires precise grasping, force dosing and timing to succeed. Moreover, the game promotes finger extension, as users must open their hand before switching between liquid dispensers using pronosupination movements.

When compared to the state of the art—represented by similar devices like the PoRi (Wolf et al., 2022) or the ReHandyBot (Articares Pte Ltd, Singapore)—our innovation exhibits a distinct advantageous combination of portability, intrinsic safety, and setup simplicity. Functional differences are that the PoRi is more lightweight and can be freely moved in space by patients with advanced proximal upper-limb functions, while our device sits stably on a surface, making it also accessible for more impaired patients. The ReHandyBot, already available on the market, offers actuated pronosupination, although at the cost of increased complexity. While other studies consider devices of more than 50 kg still portable (e.g., Sivan et al., 2014), we agree with Lu et al. (2011) that a portable device should be compact and lightweight enough to be easily transported to patients' homes—preferably by patients themselves—and low-cost. The affordability of our device is enabled by the combination of one active with one passive DoF and a readily available low-cost microcontroller and IMU. Moreover, most parts could be manufactured from technical plastics as we demonstrated by the mostly 3D-printed prototype. Currently, the main cost-driving elements are the high-end electric motor and motor driver, as they make up for more than 50% of the device's price.

To evaluate our design, we performed a usability study in a simulated unsupervised environment with 13 healthy participants, of whom three were physiotherapists from Rijndam Rehabilitation, Rotterdam, the Netherlands. This experience allowed us to gain valuable insights and information to note what needs to be dropped, added, kept, and improved in the following design iteration.

4.2 Lessons learned from the usability evaluation

4.2.1 Our device requires less than one minute to set up

The overall median set-up time—including turning on, donning, and doffing—remained below one minute. This is five times lower than the maximum setup time of robotic devices that therapists are willing to spend in inpatient rehabilitation (Rätz et al., 2021b). While the requirements in terms of setup time for home rehabilitation remain to be investigated, if we assume that they are of similar magnitude as those in a clinical setting, we feel confident that our device setup time is acceptable for home rehabilitation users. The very short doffing times observed once the participants understood how the straps work, already indicate that it is likely that our device could be donned and doffed even faster with more experience. Yet, it remains to be evaluated how stroke survivors—especially those suffering from spasticity and not being able to extend their fingers—will be able to accomplish the device setup.

4.2.2 Overall, our haptic device is perceived as highly usable and intuitive

The entire system, i.e., taking into account the device and game, achieved an overall median PSSUQ rating of 70.2 out of 100, indicating good usability, while the isolated device usability rating from the SUS achieved a score of 77.5, considered to correspond to good—excellent usability based on the ranges defined in Bangor et al. (2009). These values are in line with those from other studies of devices for similar applications. For example, the HandyBot was attributed a SUS score of 76.3 and 85.0 for the device itself and the GUI respectively (Ranzani et al., 2023). The user interface of the ReHapticKnob was rated 85.0 and two accompanying haptic games with 76.3 and 68.8 (Ranzani et al., 2021). The MERLIN device scored 71.9 in a home rehabilitation feasibility study (Guillén-Climent et al., 2021). Lastly, a SUS score of 77.5 was reported for the GripAble device in a usability study with Parkinson's disease patients (Saric et al., 2022).

The semi-structured interviews allowed us to gain a deep insight into participants' opinions. In general, the device was considered user-friendly and participants highlighted that the device looked sleek, portable and simple, thereby endorsing the overall concept. Interestingly, participants almost did not mention the shell during the interviews, suggesting that the interaction appeared to be natural and intuitive. This is supported by the results from the eye-tracking data, which show that participants did not look much at the device itself while playing the game. It seems that it was not necessary to often look at the device after donning it, indicating a generally intuitive and seamless human-device interaction. Importantly, the gaze rate on the emergency stop was marginal, possibly reflecting that participants felt safe during playing or indicating a high level of involvement in the game.

The results from the RTLX questionnaire, which reflect the participant's perceived workload during the experiment, seem to endorse the idea that the system was perceived as intuitive. With a median score of 20 and no data point higher than 30, the frustration level of the participants appears acceptable given that they used the device the first time. Furthermore, the median score of the mental, physical, and time demands were lower than 25, although with a larger dispersion. Yet, while lower values of the RTLX are preferable (except the inverted Performance item) for rehabilitation device interfaces (Ranzani et al., 2021), it can not be generally stated that mental, physical, and temporal demand, as well as effort, should be as low as possible for games or exercises. On the contrary, for example, to achieve a high exercise intensity, a larger (perceived) effort is typically desirable (Eston et al., 1987; Church et al., 2021). To promote neuroplasticity—which is the ultimate goal of this device—the performance should be high enough to keep the user motivated, but low enough to provide room for improvement (Guadagnoli and Lee, 2004). The perceived median performance score of 70 in combination with the perceived effort score of 45 indicates that the difficulty might have been appropriate for the skill level of the healthy participants. This is supported by the perceived competence subscale of the IMI, which is in line with the RTLX perceived performance item.

4.2.3 We should invest in game personalization

While some participants reported that they loved the game, some found it very boring. This seems to be reflected in the resulting score of the Enjoyment/Interest subscale of the IMI (64.3 over 100), which indicates a good but not high median intrinsic motivation of the participants during the experiment (Reynolds, 2007). As a comparison, the MERLIN device scored 85.7 in a home rehabilitation setting over a duration of a few weeks (Guillén-Climent et al., 2021). We presume that our study's lower score might be explained by the varying interests of the participants in the game, potentially influencing their intrinsic motivation. The diversity of participants' feelings and opinions not only highlights the need to improve our game further but also shows that multiple, different games would be a necessity for an at-home study with patients. A collection of interesting and diverse games is a prerequisite for successful home rehabilitation. Indeed, it has been observed that the usage times of robotic devices at home are low when the patients reported a lack of complexity and enjoyment in the games (Sivan et al., 2014). In particular for rehabilitation with stroke patients, it will be important to provide difficulty levels that are tailored to each patient's abilities (Colombo et al., 2007).

Moreover, multiple participants pointed out that a more elaborate scoring system (e.g., personal high score) could increase their motivation. Both the interviews and eye-tracking showed varying utilization and understanding of the life bar (that reflects performance), for which we see two reasons: i) Although the life bar was indicated in the instruction slides, we did not explicitly mention how it works. The time of five minutes might have been too short for some participants to implicitly learn the relation between life bar and spilling. ii) The interviews revealed that some participants did not notice the life bar.

4.2.4 There is room for improvement in the wrist fixation and the passive pronosupination degree of freedom

Twelve out of the thirteen participants were able to perform the setup of the device and play the game with no or minimal intervention. Yet, we identified a few practical and technical issues. With regard to practical issues, i.e., those related to misconception or incorrect manipulation, we found that five of the eleven occurrences stemmed from participants having difficulties with the magnetic wrist lock. While this specific practical issue did not require the intervention of the experimenters, it points to a usability issue. This was not expected, as this part was indeed designed to facilitate the setup. This issue is supported by the comments gathered from the semi-structured interviews that pointed out that the instructions regarding the wrist fixation might have not been clear enough.

The pronosupination passive DoF also gathered the attention of participants. We did not only find that it caused one of the practical issues, but multiple participants reported that the movements were not straightforward. One reason could be that the rounding of the bottom edges of the device is uniform along its length—i.e., cylindrical with the center flat part. This is in contrast to literature that describes pronosupination movements as rolling movements of a cone, with its center being the elbow (Kapandji, 1982). Thus, the rolling of the device might not correspond to natural, physiological pronosupination. Another reason could be that the flat bottom that we designed for stability seemed actually to have discouraged users from tilting the device. The pronosupination issue might have been further aggravated by the wrist position, which could become uncomfortable for prolonged use, according to the interviews. Indeed, the wrist is in a slight ulnar abducted position due to the elevated hand position with respect to the elbow.

4.2.5 The haptic rendering is generally well perceived

Participants generally appreciated the realistic haptic sensation and how the haptic feedback correlated with their grasping actions. Yet, a few participants mentioned that they did not consciously notice or use the haptic feedback. For this, we suggest four possible explanations: i) The haptic forces were not strong enough. ii) The haptic feedback worked well and was very coherent with the game. Therefore, participants did not actually notice that the haptic rendering forces were generated artificially. iii) Participants might have confounded the expected inherent springiness of the shell with the haptic rendering. iv) Participants subconsciously noticed the haptic feedback but did not perceive it as informative as they might have relied on the visual feedback as for example suggested by the answers of N1 and N6.

Although it is likely that some participants have mistaken the haptic rendering for the inherent compliance of the shell, points (i), (ii) and (iv) would require further investigation to be confirmed or disproved, for example in a within-subject study with haptic and non-haptic conditions. We can, however, comment on point (i): It is indeed possible that stiffness and damping values might have been chosen too low. A stiffness of 2 N/mm is required for an object to be perceived as stiff (Massie and Salisbury, 1994), while our stiffest object was only 0.6 N/mm. The chosen values were thought to well represent the deformable dispensers. However, it might have been beneficial to choose higher values or at least to accentuate the impact when touching a dispenser (e.g., with more distinct values of the K and B gains or vibratory cues).

4.2.6 Instructions are of critical importance for devices in minimally supervised environments

The other practical issues only occurred once and included instances where participants either did not adhere to the provided instructions or manipulated the device too early/late. We also noted confusion related to the game, in particular to the life bar and the pronosupination movements. This could indicate that parts of the instructions might not have been clear. This is supported by several statements from the semi-structured interviews. Indeed, we believe that unclear instructions were the main reason behind some of the low scores in the Information Quality subscale of the PSSUQ results. The score dispersion of the Information Quality is the highest among the PSSUQ subscales, showing that participants' perceptions of this aspect were very diverse, i.e., some were completely satisfied with the provided information, while others desired improvements in the provided information.

This brings us to an important learning for device development for unsupervised settings: The instructions are equally important as the device and the exercise themselves. In hindsight, we must acknowledge that we focused on the device and game during the development. This calls for the need to include other stakeholders in all design phases, such as cognitive psychologists.

4.2.7 The device could benefit from improvements to make it more robust

Although technical issues may be unfortunate at first sight—for example, for participant N10, who was not able to play the games for the full five minutes—they are an inherent aspect of early testing and a valuable opportunity for improving the device. The particular incident with N10 was most likely caused by slippage of the large gear pulley (pulley with diameter d₂ in Figure 1) on its axle due to insufficient clamping. This caused a misalignment of the motor encoder. The other technical issues necessitate further reliability testing of the software and the implementation of online error-checking routines and appropriate measures. For example, a failed calibration can easily be detected by driving the shell along its entire range of motion and comparing the resulting distance with the expected distance.

4.3 Study limitations

Our study has some limitations and shortcomings. First, we did not include stroke patients in this first usability study. While the inclusion of stroke patients in usability evaluations is undeniably important, the involvement of non-expert participants and therapists can also contribute indispensable insights in the early stages of device development. Following the double diamond design process model, after the first phases of discover and define, the iterative phases of design and deliver start, where new designs are created and evaluated by end users (Design Council, 2005). Ideally, the patients should be included in all these phases. Yet, the bureaucratic work required to involve patients in testing is long and tedious, requiring approvals from the local ethics committees every time a modification/improvement is made, which slows down the design process. Therefore, intermediate evaluation steps with healthy non-expert participants and therapists serving as proxies allow already assessing basic functionality, general user experience, and initial usability challenges that might not be exclusive to stroke patients. This helps to detect usability problems early, thus allowing faster convergence to more appropriate solutions, saving time, and reducing the burden on patients.

Second, an inherent drawback of our experimental design is that the usability of the device itself might have been confounded with the quality of the instructions. It has been shown that there is a positive significant correlation between the quality of user instructions and perceived product quality (Gök et al., 2019). Therefore, unclear instructions might have aggravated the perception of usability issues. However, in the case of this study, our set of rather minimalist instructions might actually have helped to extract the maximum amount of information from the experiment.

Third, the findings of our study could be limited by the participants' awareness of the experimenters' presence as the unsupervised scenario was only simulated. While this setup allowed intervention for practical or technical issues, it could have affected the participants' behavior when compared to a fully unsupervised setting.

4.4 Next steps in our human-centered design approach

In this first usability study, we gathered valuable information, recommendations, and points for improvements to be exploited in the next design iteration. In short, we plan to work on: i) Adapt the bottom of the device and the wrist fixation to facilitate the pronosupination movements while guaranteeing physiological positioning of the wrist. ii) Develop more games with different difficulty levels, and include an improved scoring system (e.g., personal high score). iii) Accentuate the haptic rendering to provide better noticeable variations between different game objects. This might include the implementation of more advanced techniques to further promote sensorimotor learning, such as haptic error modulation (Marchal-Crespo et al., 2019; Basalp et al., 2021). iv) Change the modality of instructions: Instead of slides, we will explore the use of video instructions. Moreover, we might perform checks to see if the user performed the correct action before continuing to the subsequent one. v) Further increase the portability of our system by removing the emergency stop buttons, potentially replacing the external power supply with a battery, and switching to wireless communication. This step will also necessitate making the device more robust and reliable. vi) Integrate an absolute encoder and automatic detection of the installed shell size to avoid the currently necessary calibration sequence. vii) Implement an assessment routine that allows to determine the user's range of motion and grasping force. viii) Further lower the cost of the device, for example by replacing the motor and motor driver with a lower-cost solution or the redesign of complicated components. On this note, the general robustness might also be further improved in prospect of potential future large-scale studies.

Gathering patient feedback—potentially also in a longitudinal study—will be our main focus after realizing the aforementioned improvements. The combination of group therapy with home rehabilitation (where patients use the exact same device) has been suggested as a promising way of efficiently increasing therapy dosage (McCabe et al., 2019) and could present a suitable use case for the next round of usability testing.

With respect to the possible commercialization of the device, the distribution and support will become key factors that need to be considered. Moreover, we will investigate various financial models to ensure the economic viability of such a relatively low-cost device once it is ready for commercialization. It has been shown that innovations with potentially high societal impact but lower economic value—e.g., medical low-cost devices such as the one presented in this study—are notoriously difficult to obtain investments (Allers et al., 2023). Thus, we must ensure that our device is not only low-cost but, first of all, cost-efficient, i.e., it must not only hold its therapeutic premise but also provide an economic benefit to the health care system and investors.

4.5 Conclusion

We presented the second iteration of a novel minimally-actuated haptic hand trainer for minimally supervised and unsupervised rehabilitation of patients with acquired brain injury, as well as an accompanying serious game. The introduction of a novel compliant shell mechanism allowed us to design a device that is simple and provides intuitive and intrinsically safe physical human-device interaction.

Following a human-centered iterative development approach, we performed a thorough analysis of the prototype's usability with therapists and healthy non-expert users. In a simulated unsupervised scenario, we asked the participants to set up the device and play a game based on a set of written instructions. Our mixed-method approach allowed us to gain insights into usability issues of our prototype. While the testing showed good overall usability of the device and the game, we identified various areas of improvement, such as the wrist fixation, the pronosupination movements, and instructions.

Our prototype shows promise for use in both minimally supervised therapy and unsupervised home rehabilitation. We are looking forward to further improving our device to deploy it with neurological patients and contribute to the democratization of robotic rehabilitation in order to improve the quality of life of especially vulnerable patients.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by Human Research Ethics Committee (HREC) of the Delft University of Technology (Application ID: 2216). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

RR: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Visualization, Writing – original draft, Writing – review & editing. AR: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing. NC-G: Conceptualization, Data curation, Formal analysis, Investigation, Writing – review & editing. GR: Conceptualization, Methodology, Project administration, Resources, Supervision, Validation, Writing – review & editing. LM-C: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by the Swiss National Science Foundation through the Grant PP00P2163800, the Dutch Research Council (NWO, VIDI Grant Nr. 18934), and the Convergence Flagship Human Mobility Center.

Acknowledgments

The authors would like to acknowledge the highly valued contribution of Jonas Kober during the mechatronic development of the presented device. We are also grateful for the help of Alberto Garzás Villar with the game development. Furthermore, we would like to thank the therapists from the Department of Neurology, University Hospital Bern, Switzerland for their feedback during the development of the game and the device and the therapists from the Rijndam Rehabilitation Center, Rotterdam, the Netherlands, for participating in the usability experiment. Finally, the authors highly appreciate the efforts of Katie Poggensee in proofreading the manuscript.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor JF-L declared a past co-authorship with the author LM-C.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot.2024.1351700/full#supplementary-material

References