1 Introduction

1.1 Problem and significance

Neurotraumas (e.g., stroke, spinal cord injury, and traumatic brain injury) affect millions of people annually and are among the leading causes of death and disability [1,2,3]. Affected individuals are frequently limited in mobility and must regain critical skills for independence and performing activities of daily living [4]. It is estimated that 50 percent of spinal cord injury cases involve upper extremity dysfunction [3], and about 65 percent of stroke survivors cannot use their affected hand for up to six months after the injury [5]. Thus, it is crucial to identify and deploy new forms of physical therapy after neurotrauma that will rehabilitate functional abilities effectively and quickly. Several classes of pathologies can benefit from novel therapies to rehabilitate motor function, including systemic diseases (e.g., Parkinson’s Disease [6]) and developmental disabilities (e.g., Autism Spectrum Disorder [7]). However, rehabilitation after neurotrauma is a special case to consider [8] as functional or cognitive deficits generally improve with sufficient time and proper management, yet negative sequelae from mild neurotraumas can persist as lifelong impairments [9]. This paper aims to contextualize how novel therapies with computerized interfaces, such as virtual reality, can specifically be optimized with cognitively-centered approach elements (e.g., augmented sensory feedback for motor learning, adapting training based on monitored psychophysiological measures) for persons rehabilitating upper-extremity motor function after neurotraumas. However, several of the proposed considerations can be reasonably extended to other pathologies benefitting from physical therapies to improve motor function.

1.2 Virtual reality motor rehabilitation – why it works, how to optimize

Computerized interfaces are increasingly prevalent for rehabilitation, given their features of programmability and flexibility to customize approaches for each user to achieve greater efficiency and engagement. Virtual reality is highly viable in motor rehabilitation since it can facilitate task-oriented movements while augmenting sensory activation. Furthermore, virtual reality paradigms can be programmed to accommodate various levels of movement ability and include gamification elements for greater engagement [10]. Previous studies have demonstrated benefits with virtual reality rehabilitation alone and when combined with traditional therapies [4, 11]. The apparent advantages of virtual reality approaches include greater motivation, engagement, and convenient repetition for the high-fidelity practice of functional movements [11]. However, empirical research has yet to demonstrate precisely how virtual reality mechanistically generates positive outcomes. While improved motor function is the primary metric of interest, psychophysiological factors such as motivation and engagement are crucial in facilitating effective motor rehabilitation. Thus, it is critical to consider how cognitive elements may be explicitly incorporated to optimize virtual reality motor rehabilitation. Although virtual reality therapies have already demonstrated promising results, more empirical research must be conducted to understand and leverage their underlying mechanisms.

It is unclear how programmable virtual reality elements (e.g., task type and complexity, environment, delivery of sensory guidance) can generate better motor control. Recent studies examining virtual reality for motor rehabilitation have explored factors of convenience [12], adaptability [13, 14], quality of motion control [15], and level of immersion[16, 17]. However, it is not evident how we can further optimize the design and deployment of virtual reality protocols for greater functional benefit beyond the motivation to undergo more training repetitions. Ultimately, it would be invaluable to understand how specific modifications to virtual reality design and training elements produce cognitive connections that directly support improved motor outcomes. Such findings would elucidate the benefits of virtual reality beyond motivation. Furthermore, these findings would incentivize the development of intelligent virtual reality approaches that may accelerate positive motor outcomes.

Despite the plethora of options to customize computerized interfaces, virtual reality rehabilitation approaches mainly rely on colorful, gamified displays to incentivize more practice repetitions. Advanced principles for motor learning and motor control are still not standardly considered in designing and deploying virtual reality protocols [18]. However, virtual reality environments allow for creating various complex functional tasks, providing sensory guidance cues, and real-time monitoring of user variables to adapt training. Furthermore, virtual reality platforms can provide highly stimulating visual, auditory, and haptic interfaces to guide users to perform complex motor functions more effectively. Thus, virtual reality environments can readily facilitate the greater cognitive engagement necessary to accelerate improvements in motor function further.

Recent literature reviews have specifically suggested how various VR-supported exercise therapies can effectively improve motor rehabilitation outcomes for upper extremity function [19,20,21] after stroke. Recent studies have also suggested how VR therapies produce measurable changes in neural activity, as measured by fMRI [22], and can be coupled with emerging approaches, e.g., action observation therapy [23], whereby patients observe purposeful action to be subsequently imitated while engaging multiple sensory modalities.

1.3 Objective of review paper

The primary objective of this paper is to review the literature relevant to identifying and exploiting cognitive-based approaches to motor rehabilitation with computerized interfaces for persons with neurotraumas. Furthermore, this review paper focuses on rehabilitating upper-extremity functions, given their broad use in daily activities, less stereotypical nature than common lower-extremity functions (e.g., gait), and common focus in rehabilitation after severe neurotraumas. Through relevant literature, this paper will suggest ways to maximize the potential of upper-extremity rehabilitation after neurotraumas through cognitive-level modulation via computerized feedback.

1.4 Literature review search methods

Through library resources available at Stevens Institute of Technology, we searched for articles and book chapters describing relevant studies associated with virtual reality rehabilitation in the following databases and related search engines: PubMed (MEDLINE), Web of Science, Scopus, ScienceDirect, SpringerLink, Wiley Online Library (Cochrane Library), and Google Scholar. Our search was restricted to articles between 1980 and 2023. Our primary search terms, pursued independently and in combinations included: “virtual reality,” “motor rehabilitation,” “stroke,” “spinal cord injury,” “traumatic brain injury,” “neurotraumas,” “augmented sensory feedback,” “multimodal feedback,” “intermittent feedback,” “terminal feedback,” “sense of agency,” “cognition,” “cognitive load,” “motivation,” “attention,” “reaction time,” “memory,” “upper extremity function.” Additionally, we reference our own recently published or submitted works. Given this is an “opinion/perspective” review paper attempting to contextualize new, in some ways non-traditional, approaches to virtual reality rehabilitation, no statistical meta-analyses were applied to the review results. Rather we summarize works emanating from the above literature searches and make recommendations of how elements of these studies may be innovatively applied to motor rehabilitation paradigms using computerized interfaces such as virtual reality.

1.5 Organization of review paper

This paper will first review training approaches for upper-extremity motor rehabilitation in Sect. 2, especially those employing virtual reality, as established by previous studies. Next, and more critically, in Sects. 3 through 5, we will review and relate literature from motor control learning and psychology to suggest cognitive bases to optimize virtual reality motor rehabilitation. The primary areas of discussion include: 1) how activating sensory modalities through computerized interfaces for feedback can be leveraged to accelerate motor rehabilitation outcomes, 2) how features in augmented sensory guidance can optimize motor outcomes, and 3) how certain psychophysiological factors should be further considered in the development of new virtual reality rehabilitation protocols.

2 Upper-extremity motor rehabilitation

2.1 Traditional rehabilitation of upper-extremity function after neurotraumas and the role of motor learning

Conventional rehabilitation strategies use exercises to improve the motor skills needed to perform daily activities. A physical or occupational therapist will supervise and guide rehabilitative practices for people with motor impairments [4, 24]. These professionals implement repetitive task training to reformulate neuromotor connections and to increase strength, range of motion, and coordination [3]. Cervical-level spinal cord injury and stroke can result in upper-extremity paresis that compromises the ability to reach and grasp [7]. Conventional therapies for rehabilitating upper-extremity function after spinal cord injury typically include joint exercises that facilitate greater strength, dexterity, and range of motion [24, 25]. Stroke rehabilitation typically centers on functional task practice [26], adjusting difficulty levels for each person. This training entails rigorous practices that improve motor skills transferrable to functional tasks. Persons can also receive task-specific training [27, 28]. For eligible persons with hemiparesis, therapists may incorporate constraint-induced movement therapy to compel more engagement of the affected side [29]. Unfortunately, less than one-third of stroke or spinal cord injury survivors receive outpatient rehabilitation [30]. Participation in regular rehabilitation is challenged by the effort and time involved in physical therapy. Conventional rehabilitation can frustrate patients due to its tedious and repetitive nature [4]. Ancillary factors that reduce outpatient treatment are lack of access to rehabilitation centers, family/caregiver support, and the financial resources to pursue regular physical therapy. Thus, rehabilitation methods must be designed to be highly efficient, whereby participants can achieve functional gains with fewer repetitions or exposures to therapy.

Since the primary goal of upper extremity rehabilitation following a spinal cord injury or stroke is to regain motor control, most rehabilitation paradigms implicitly incorporate motor learning. Motor learning involves the development of intrinsic mechanisms, such as neuromuscular control, to repeat a movement independently [31]. This motor training objective naturally relates to reformulating neural connections following neurotraumas for motor recovery [32, 33]. Practical training for motor tasks is often executed in two phases. The first phase is guided training, where feedback is provided in real-time or immediately after task completion. The second phase is retention, where guidance feedback is removed, and participants must perform the task independently and ecologically [34]. High performance during training ensures the participant's capability to do a given motor task well; however, only with high performance during retention is there demonstrable development of intrinsic mechanisms. Transfer tests can further indicate long-term learning whereby a task is presented during a retention test different from training but still leveraging the motor skills practiced during training [34]. Traditional physical therapy approaches aim for the successful transfer of skills through the development of motor skills during training to facilitate improved performance of activities of daily living. However, guidance during conventional rehabilitative training does not typically consider the presentation of augmented feedback intended to maximize retention outcomes.

There are inherent theoretical trade-offs between training and retention repetitions of movement. For example, the guidance hypothesis suggests that higher reliance on feedback during training can negatively affect motor learning retention [35, 36]. This phenomenon has been evaluated by altering the frequency of feedback trials as motor learning improves. Physical therapy methods could better cognitively engage participants during rehabilitation if programmable and customizable tools are enacted that leverage motor learning processes and make motor rehabilitation more efficient.

Many traditional rehabilitation exercises do not precisely simulate daily activities but focus on developing capabilities (e.g., strength, dexterity) that transfer to functional tasks. On the other hand, virtual reality paradigms can provide environments and tasks that have high fidelity, on physical and cognitive levels, to the user in simulating activities of daily living [11]. Consequently, patients may achieve more direct gains in relevant functions more quickly. Furthermore, virtual reality rehabilitation tasks can be more motivating to perform, easier to adjust on the fly, and convenient to repeat [37]. Through programmability, virtual reality tasks can be readily gamified or uniquely immersive to incentivize participation, and they can be customized on more granular levels in terms of adapting difficulty or modifying the task at hand [38, 39]. Such approaches can reduce the onus on clinicians being another human-in-the-loop to customize the training regime and focus more on supporting, guiding, and directing the patients at a high level. Furthermore, virtual reality can be used effectively in isolation or in combination with traditional therapy to maximize outcomes [11, 40, 41]. Thus, although conventional rehabilitation alone is effective, it is limited in its scope compared to virtual reality to customize training for greater motivation, efficiency, and convenience [42]. Ultimately, participation in motor rehabilitation therapies relies on users feeling cognitively engaged and experiencing more reliable gains in motor outcomes. Computerized interfaces, such as virtual reality, can further customize treatments to accommodate current levels of function and conduct training regimes that accelerate functional gains.

2.2 Virtual reality for motor rehabilitation

Advanced rehabilitative methods increasingly utilize computerized interfaces such as robotics and virtual reality to provide enhanced sensory feedback and gamification [43, 44]. Virtual reality rehabilitation is an attractive alternative to conventional therapy due to customizability and contextual incentives (e.g., in-game rewards) that foster greater motivation and engagement [45]. Programmability features allow finer adjustments of difficulty levels for each user and better simulation of activities of daily living for functional fidelity [11]. Virtual reality rehabilitation is also increasingly prevalent due to its commercially available and affordable technologies for home practice that can supplement the work done with a physical therapist [12]. Various virtual reality approaches have been utilized for persons with either spinal cord injury [4, 33, 41, 46] or stroke [47,48,49,50,51,52,53,54,55,56,57]. These studies demonstrate the effectiveness of virtual reality paradigms in improving motor function through methods that motivate greater participation in therapy. The primary objective of virtual reality therapies is to show improvement in functional capabilities. Brosnan et al. (2009) demonstrated how virtual reality therapy addressed motor deficits after stroke patients by encouraging the use of the hemiplegic side of the body while also increasing satisfaction, motivation, and interest in physical therapy [52].

Although virtual reality therapy has positively affected motor rehabilitation, it is unclear whether it is more effective than conventional therapy when controlling for dosage. A fair comparison can only be made when the therapy dose is similar in duration and intensity. Supplementing traditional rehabilitation methods with virtual reality therapy conclusively improved functional outcomes [4, 53]; however, the mechanism of improvement is likely attributable to increased dosage. Indeed, patients report greater enjoyment of therapy with virtual reality as it encourages greater participation in rehabilitative practices [54]. However, we assert there is still a missed opportunity in that virtual reality therapies could be better designed to incorporate sensory feedback and psychophysiological factors to outperform conventional approaches further. Therefore, it is necessary to consider further and investigate the incorporation of augmented guidance to facilitate motor learning and psychophysiological factors to verify neural engagement in virtual reality rehabilitation (Fig. 1).

Fig. 1
figure 1

Intelligent incorporation of augmented sensory feedback and psychophysiological factors to a motor rehabilitation regimen potentially addresses shortcomings of conventional therapies relating to motor learning efficiency and neural engagement. A flow chart with boxes and arrows. Box #1 is labeled “Reality Motor Rehabilitation” and has an arrow pointing down to Box #2 that is labeled “Designing, Incorporating, and Evaluating the Cognitive Basis of VR protocol”. Box #2 points down to box #3 which is labeled “Improved Motor Function for Activities of Daily Living”. Coming off of the left and right sides of box #2 are dashed arrows labeled “Augmented Sensory Feedback” and “psychophysiological factors” respectively that point down to two boxes that outline methods to use those tools in virtual reality rehabilitation. These boxes have dashed arrows that point down to box #3

Full size image

3 Activation of sensory modalities for motor rehabilitation

3.1 Visual feedback in virtual reality

Visual feedback is the most valuable and exploitable sensory modality in virtual reality. Visual feedback provides cues about task performance, such as body position or muscle activity, either in real-time or immediately following task completion. Visual feedback guides spatial positioning during movement tasks more effectively than audio or haptic feedback [55, 56]. Supplementary visual cues in virtual reality can improve the performance of isometric and dynamic exercises [57], and explicit visual cues about one’s spatial position can immediately reduce errors in a movement trajectory [58], including with virtual mirrors [59]. However, when virtual reality paradigms enhance visual presentations for immersion and gamification, the user’s capacity to receive additional visual guidance to improve performance should be carefully considered. Task-irrelevant immersive elements, such as extra objects within view or visual rewards for a more gamified context, can detract from the task-relevant intention (e.g., memory) of a visual stimulus used for guidance [60]. How visual feedback is best presented can also depend on participant experience and task complexity [61].

For persons recovering from neurotraumas, learning control schemes for brain-computer interfaces can be highly predicated on visual feedback, contingent on high attention [62]. Although this example is for an assistive device system, the ability to improve control, even if independent and voluntary, over a computerized interface such as virtual reality is highly prevalent for demonstrating functional gains for rehabilitation paradigms. Furthermore, there is growing evidence to suggest that if key neurocircuits are therapeutically reactivated with appropriate sensory feedback, neurological functions (e.g., limb movement, locomotion, etc.) can be better reanimated after neurotrauma [63, 64].

3.2 Multimodal sensory feedback for guiding motor tasks

Multimodal feedback entails the provision of augmented guidance cues through multiple sensory modalities (e.g., visual, auditory, and haptic) concurrently to enhance motor task performance and rehabilitative benefits [57] (Fig. 2). During physical rehabilitation, multimodal feedback, when adding audio or haptic cues to visual feedback, enhances complex motor learning compared to unimodal sensory feedback [65]. Furthermore, visual-audio and visual-haptic forms of multimodal feedback are especially beneficial for increasing perceptional accuracy and spatiotemporal learning [57], leading considerations to optimize motor practices within virtual reality environments. We consider visual-audio and visual-haptic pairings in greater detail in the following sub-sections.

Fig. 2
figure 2

Flow diagram depicting rehabilitation approaches with computerized interfaces (e.g., virtual reality) and utilizing augmented sensory feedback to guide improved motor performance. A flow chart with boxes and arrows. A person participating in virtual reality rehabilitation undergoes Computerized Practice of Motor Control Tasks (primary measures) that are Motion-based (kinematics) and Force-based (loads, EMG) and use sensory feedback to provide guidance for improved motor function. Examples of motor control tasks with computerized interfaces are depicted. Sensory feedback includes either visual feedback or multimodal feedback

Full size image

In general, multimodal sensory feedback has only been employed anecdotally for neuromotor rehabilitation [57]. While sensory-driven platforms have been developed to recover function, like post-stroke goal-directed reaching [66], there is a clear opportunity for deeper investigation when employing highly customizable and programmable platforms like virtual reality. The notion of multimodal approaches to neuromotor intervention is identifying multiple pathways in which to elicit more neural activation during training. Multimodal neural activation with augmented sensory feedback [67] through virtual reality [68] is a potentially promising pathway that has not been well tapped for rehabilitation after neurotraumas.

3.2.1 Adding haptic cues to visual feedback

Haptic feedback broadly encompasses any sensation related to touch. Examples include changes in applied forces, pressure, vibration, or temperature to relay signal information (e.g., amplitude and frequency [69].) related to task performance or the surrounding environment. Haptic feedback can be repulsive or attractive in cueing error magnitudes [70]. Haptic feedback is also effective in encoding supplementary information (e.g., force interactions) that facilitates user integration with movement-restoration devices, such as motor-actuated prostheses [71, 72]. In these cases, visual feedback is fundamental to informing the user about functional performance, and haptic feedback concurrently provides accompanying task information (e.g., errors, forces). Receiving augmented cues about task-related performance (e.g., encoding errors in performance) in real-time entails explicit feedback from which users constantly modulate their actions according to this feedback to guide performance [73].

The effectiveness of adding explicit haptic (e.g., vibration, forces, imposed motions felt proprioceptively, etc.) feedback to visual cues for virtual reality motor rehabilitation tasks will depend on factors ancillary to the training. These factors include task type, task complexity, user functional abilities, and user experiences with exercise activities. The challenge with adding explicit haptic feedback stems from possible cognitive overloading, especially if tasks are sufficiently simple to be mastered with visual feedback alone. For example, Hasson and Manczurowsky (2015) determined that vibration did not improve skill acquisition with a simple upper-extremity task if presented independently or with visual feedback [74]. Their results concluded that vibrotactile feedback was detrimental when participants had difficulty integrating the haptic cues with the viewed virtual avatar. Thus, to maximize potential rehabilitative benefits with explicit haptic feedback, the specific characteristics of the task and user must be carefully considered.

Alternatively, implicit haptic feedback, which is not directly guiding task performance, is intended to impact user engagement and actions in motor rehabilitation. Within virtual reality environments, implicit haptic feedback is typically employed for greater immersion or better simulation of physical interactions. Occasionally, haptic feedback is provided without real-world fidelity but simply for greater user engagement through sensory substitution. In this approach, one sensory modality (e.g., pressure) is translated into stimuli for another sensory modality (e.g., vibration magnitude). While not as ecologically valid, such approaches can foster greater immersion toward improved training outcomes [58].

3.2.2 Adding audio cues to visual feedback

Auditory feedback involves converting data produced by user activity [75] into additional sound cues (sonification) readily provided through virtual reality interfaces or wearable systems. As with haptic feedback, audio feedback can be readily coupled with augmented visual inputs used as an implicit or explicit learning tool [76]. With implicit learning, real-time sonification is provided independently of any reference. Explicit learning would entail the sonification of performance errors between the user’s movement and a target. Ultimately, audio feedback is only helpful for motor learning if the sound cues are consistent and timely [77]. Thus, any technical limitations that delay processing time can significantly limit the effectiveness of audio-based approaches for motor rehabilitation.

Augmented auditory feedback receives less attention than visual and haptic feedback with motor rehabilitation approaches, given its potential as a unique cognitive distractor [78]. However, the influence of auditory feedback on motor learning should not be discounted when designing virtual reality rehabilitation platforms. For example, in clinical stroke populations, it has been shown that audio feedback can make rehabilitation exercises more engaging for the participants, resulting in improved mobility and reduced reports of pain [79]. In addition, melodic sonification is proven to increase retention rates [80]. Positive emotional responses can emanate from sounds, especially music. Still, if the sound is not employed naturally for the motor task or the specific user, it may generate adverse motor responses [81]. These results motivate the inclusion of audio feedback in motor rehabilitation programs; however, careful considerations should be made in how auditory feedback is incorporated.

Virtual reality rehabilitation often includes auditory and visual components, and studies have directly compared the two modalities. Using functional magnetic resonance imaging, auditory and visual learning are shown to activate different brain areas when subjects perform the learned task without the stimulus present [82]. This result suggests that the design of a sensory-based motor learning method should consider the location of neurological damage. Concurrent audio and visual feedback can also improve the subject’s engagement and result in greater margins of improvement than either feedback modality individually [83]. While audio can be a helpful tool to enhance virtual reality rehabilitation methods, the feedback protocol should carefully consider customizing sound cues based on individual-level responses to accelerate motor learning trajectories [84,85,86].

4 Features of augmented sensory feedback used for training guidance

When utilizing programmable computerized interfaces for rehabilitation, there are several options for presenting augmented sensory feedback to those with neurotraumas. Persons with neurotraumas may have limitations in carrying a particular cognitive load [87, 88], which would dictate what levels of augmented sensory guidance would be optimal. On the other hand, additional sensory cues may be helpful to elicit greater arousal and attentiveness during training in the presence of neurotraumas [89]. While virtual reality can generate intricate and highly interactive environments with various sensory cues, it is unclear what additional information should be provided to enhance the learning of motor tasks. Since virtual reality readily allows personification elements with virtual avatars, the natural question is: what additional, sensory-based guidance can and should be provided to accelerate functional motor outcomes?

4.1 Simple versus complex feedback

Another feature to consider in augmented guidance is the presented feedback signals' complexity. Typically, feedback complexity will follow the complexity of the motor task [61]. For example, Wulf and Shea (2002) defined simple tasks as those capably learned in a single session or involving only a single degree of freedom. Conversely, complex tasks have multiple degrees of freedom, requiring numerous training sessions to master, and are more ecologically valid [61]. In prescribing visual feedback complexity, simple feedback provides a single DOF or performance variable. In contrast, complex feedback provides two or more streams of information [58]. Sanford et al. (2020) identified a trend in which concurrent complex feedback can be more valuable than simple feedback when provided in body-representative display modes [58]. Virtual reality may amplify these results, whereby elaborate virtual avatars can enhance embodiment [90] and sense of presence [91]. Complex feedback may be used even for relatively simple tasks if the body segments are constrained, e.g., squat maneuver with feet creating kinematic closed-chain to the ground [58, 92]. In this way, even a multi-segmental motion will effectively follow a single degree-of-freedom (e.g., squat depth). However, tracking multiple segments concurrently with complex feedback would still be highly feasible if not beneficial to performing the movement [58]. Given the complex nature of upper-extremity tasks (e.g., reach-to-grasp), complex feedback is expected to be most effective. Virtual reality training of complex tasks often displays a target image of a total body linkage or hand, with multiple segments and joint DOFs indicated against an entire user-generated body position. In this way, complex and representative feedback modes are again naturally coupled. Importantly, providing inappropriately complex feedback can cause cognitive overload and likely involves information irrelevant to the task or desired skill [93].

4.2 Abstract versus representative feedback

Previous studies with computerized interfaces have shown abstract feedback, displaying movements against targets as line plots or bar graphs [94] with no body-discernable features. Abstract feedback is often associated with simple tasks [61]. Representative—also known as natural— feedback uses virtual avatars or mirrors of the participant's body position and is related to complex feedback [57, 58, 92]. Abstract feedback is typically used with simple tasks since it requires only one performance variable to track, and a single trace readily represents it. Representative feedback is more associated with complex movement tasks [61], as displaying multiple performance variables concurrently as disjointed lines or graphs is neither sustainable nor informative. When developing virtual reality rehabilitation paradigms, displaying performance errors representatively and across multiple degrees of freedom (i.e., for complex function) can immediately improve motor performance [58]. An example of abstract versus representative feedback for visual guidance of two-legged squat kinematics [58, 92] is shown in Fig. 3. With abstract feedback, the participant is shown sinusoidal trajectories to track using dynamic traces that change with the angular motions of the participant’s body segments while performing the squat. With representative feedback, those same angular trajectories are displayed in real-time with a sagittal-plane view of two overlaying dynamic stick-figures, one that moves according to the participant’s squat motions and the other adhering to the target trajectories the participant should match.

Fig. 3
figure 3

Example depiction of abstract (LEFT) and representative (RIGHT) visual feedback to guide participants performing a squat task that adheres to target segmental (torso, thigh, shank) trajectories. A figure with example panels whereby person performs squat with two types of visual guidance (abstract, representative). Both types of guidance provide the same segment angular trajectories to track, but abstract feedback presents the target trajectories as sinusoids while representative feedback presents the target trajectories with a side-view (sagittal plane) of a dynamic stick-figure that approximately depicts a body

Full size image

Complex-representative visual feedback involves concurrently displaying multiple performance variables projected onto a multi-segment avatar the user can embody, including virtual reality interfaces driven by myoelectric commands [95]. For example, Blana et al. (2016) developed a virtual reality prosthesis training system, integrating motion capture and EMG control to display a transparent guide arm against an opaque avatar actively controlled by participants [96]. This approach facilitated relatively fast training for users to complete 3D-reaching tasks without adverse effects. Complex-representative feedback can exist in the first- or third-person perspective. Perez-Marcos et al. (2017) conducted a pilot study with stroke patients to demonstrate the beneficial effects of virtual reality rehabilitation on upper extremity function and range of movement with multiple tasks projected in either first- or third-person displays [97].

Although abstract feedback is classically associated with simple tasks, abstract motor tasking coupled with cross-training elements can induce positive cognitive outcomes [98]. Aoyagi et al. (2019) showed that adding weights during a figure-8 tracing task with upper-extremity motions increased the perception of agency. Few studies have evaluated abstract feedback within virtual reality immersive environments due to the natural match between complex-representative feedback and body avatars. However, if intelligently designed to feel natural and complementary to the virtual avatars being controlled, with or without a sense of embodiment, simple and abstract feedback to depict target motion paths may still be effective in training better motor performance, even within sophisticated virtual environments.

Integration of abstract and representative cues is readily done in virtual reality. A previous study merged non-motor abstract cues (e.g., words, color changes) with representative guidance of motor actions in the form of various hand gestures (e.g., wrist deviations, hand pronation/supination, different grip configurations, etc.) [99]. As another example, color changes as abstract cues can highlight target areas of interest during full-body movements primarily guided by complex-representative feedback [100]. While participants utilized complex-representative feedback to match target body positions, the color changes successfully reinforced performance errors. Hybrid feedback approaches that cognitively engage users without distraction during rehabilitative training should be further investigated for their potential to improve motor outcomes.

4.3 Concurrent versus terminal feedback

Another essential feature to consider with augmented guidance in virtual reality motor rehabilitation is the timing of the feedback. Concurrent feedback is provided in real-time to guide the user toward target positions while performing the movement simultaneously [57]. Terminal feedback is provided immediately after completing the movement to summarily indicate performance errors about the previous movement in preparation for the next training repetition [57]. Both concurrent and terminal feedback can present identically and be similarly identified along particular feedback features (i.e., degree of complexity, rate of intermittency, level of body representation). The key difference is receiving real-time feedback while moving (concurrent) versus observing a replay of the feedback that would have been observed in real-time but after movement completion (terminal). Motor rehabilitation may incorporate concurrent and terminal feedback simultaneously [101] or transition from concurrent to terminal over a long period as the participant's motor learning improves [102]. Concurrent feedback is most beneficial in the early stages of motor learning when the person is relatively naïve to the task, making more significant adjustments and more considerable gains in performance [102]. Terminal feedback becomes beneficial in the latter stages of motor learning when only finer adjustments are made while improving long-term learning [102]. Thus, the optimal timing in providing feedback will be dictated by where one is in the motor rehabilitation learning cycle.

4.4 Continuous versus reduced frequency (intermittent) feedback

This section will discuss feedback frequency, or intermittency, as a fourth feature in designing optimal virtual reality rehabilitation protocols with augmented guidance. As referenced previously (Sect. 2.1), participants of motor learning protocols can accelerate the development of independent capabilities by reducing feedback frequency over an extended course of training [35, 36]. This approach aims to reduce the reliance on sensory feedback to support the higher performance of movements. The three primary methods to employ reduced frequency training are bandwidth [103,104,105], faded [106], and self-paced [107,108,109]. All three methods have been evaluated positively with terminal feedback, but only bandwidth feedback has been proven effective when coupled with concurrent feedback [92, 110]. For intermittent feedback training based on bandwidth performance, feedback is only provided when the participant’s performance errors (e.g., in tracking a target trajectory) exceed a pre-set error magnitude (i.e., error band) [92, 111]. Figure 4 presents example time-stamped trajectory tracking with continuous versus intermittent (bandwidth) feedback. Thus, there is an implicit reward to perform well, i.e., maintaining low error levels, by observing a removal (disappearance) of feedback, which also promotes greater reliance on intrinsic mechanisms. In a study not employing virtual reality, bandwidth feedback demonstrated greater potential for learning than continuous feedback for the two-legged squat exercise [92]. Further research is necessary to determine the boundaries of feasibility and optimality in employing concurrent bandwidth feedback with virtual reality applications, especially for ensuring better long-term outcomes.

Fig. 4
figure 4

Example of continuous versus intermittent visual feedback to guide tracking of a movement trajectory. Continuous feedback is always present while intermittent feedback will appear and disappear according to some criterion (e.g., error < threshold (band), e.g., 5% of average error). A figure showing continuous feedback versus intermittent feedback at select time-instances. Intermittent feedback disappears when subject position error is less than bandwidth threshold of 5% of average error for that subject as marked by the stars

Full size image

5 Psychophysiological factors in motor rehabilitation

Although any form of motor training inherently accesses cognitive resources, most conventional methods for physical therapy do not strategically consider psychophysiological factors in motor rehabilitation processes. Advanced rehabilitative methodologies employ computerized interfaces, including robotics [112] and virtual reality [113], for improved motivation and motor performance. However, as previously mentioned, if the dosage is similar, the added benefits of computerized rehabilitation approaches compared to traditional therapies become more negligible [114]. Thus, it remains uncertain whether current methods with virtual reality induce the neural engagement necessary to improve motor outcomes efficiently. As discussed, computerized rehabilitation offers several design options to engage users, including gamification [115], customization [116], and augmented feedback [117]. However, how we can best assess whether specific virtual reality rehabilitation designs will result in desirable neural engagement is unclear. This section discusses potential psychophysiological factors that should be monitored to gauge engagement in improving motor performance (Fig. 5).

Fig. 5
figure 5

Psychophysiological factors should be monitored and assessed during virtual reality therapies to ensure optimal levels of neural engagement in training motor function. A flow chart with boxes and arrows. The words “Virtual Reality Rehabilitation” point to a circle labeled “psychophysiological factors to assess user engagement”. On the edge of the circle are boxes labeled “motivation”, “sense of agency”, “attention”, “cognitive load”, and “memory”. The circle points to the words “Improved Motor Function”

Full size image

Psychophysiological measures must be monitored for those with neurotraumas to assess bodily functions that directly impact health and quality of life. For example, spinal cord injury can result in various autonomic dysfunctions that increase mortality from cardiovascular and respiratory disease [118]. Thus, persons with neurotraumas may have unique responses to variations in training guidance that are readily measurable from skin-surface recordings, especially at the brain (electroencephalography) in reflecting a host of cognitive processes (e.g., load [119], attention [120]) affecting motor performance. In addition to physiological signals, survey measures can indicate critical cognitive states for persons with neurotraumas. Surveys have been extensively used to assess perceptions of utility and motivation to participate in rehabilitative therapies, especially novel approaches such as virtual reality [121]. Ultimately, generating positive perceptions from user perspectives can determine clinical acceptance of rehabilitation approaches [122]. Explicitly and formally considering such measures with rehabilitation paradigms may be crucial to ensure greater therapeutic effectiveness.

5.1 Motivation

As mentioned, motivation is a (if not the) critical user-perception measure to assess the potential success of virtual reality rehabilitation programs [11]. More specifically, motivational factors can influence the speed of movement initiation and execution [123]. Customization within virtual reality to capability levels can also increase motivation with motor learning [18]. Motivation can increase when the exercises seem applicable to daily activities [124]. Mouatt et al. (2020) demonstrated that immersive environments that manipulate ‘real’ competition enhanced participant motivation [42]. Therefore, motivation should be systematically considered in the development of virtual reality methods, leading to the inclusion of more goal or task-oriented, competitive, and transferable elements.

5.2 Sense of agency

The sense of agency is the neural perception of the true authorship of voluntary action and its related consequences [125]. In general, the higher agency one has, the better movement control one perceives; it is therefore intuitive to consider greater agency as a basis for improved functional performance [126]. Previous virtual reality studies have investigated the effect of modified visual feedback on agency and indicate that enhanced feedback in virtual reality can improve agency [127]. Other studies have shown that variations in control, as perceived visually, can co-modulate agency with motor performance at motion [128] and force [129] levels. In gamified environments with instrumented wearables, informing users about the successful accomplishment of motor tasks can also facilitate positive correlations in agency and performance [130, 131].

An implicit measure of agency that can be directly coupled to motor actions is based on the phenomenon of intentional binding [132]. Intentional binding refers to the perception of a compressed time interval between a voluntary action and related sensory consequence [132]. Sensory cues, including visual and audio feedback, are often provided to subjects while performing functional movement tasks such as reaching and grasping [133]. As such, these cues can be readily integrated within virtual reality applications (as color changes or sound) for action-outcome events to assess intentional binding.

5.3 Attention and reaction time

Attention is a well-established measure of cognitive engagement during activity [134]. Attention is often measured in virtual reality headsets with eye-tracking capabilities that analyze the focus of attention [135] or saccadic times in response to visual stimuli [136]. Attention demonstrates the ability to screen out irrelevant stimuli and focus on information directly related to the given task [137]. Attentional focus directly impacts movement performance and efficiency [34]. Rehabilitation methods can be designed to improve attention in persons with neuromuscular pathologies [138]. Virtual reality can provide specific stimuli to add or remove distractions intended to test attention [139]. Highly customized virtual reality systems can significantly improve attention compared to conventional methods [47]. However, virtual reality designs centered on improving attention may not similarly achieve gains in executive function for persons with brain injury [140]. Thus, optimal protocols with virtual reality may need to balance the evaluation of psychophysiological factors like attention while pursuing gains in motor function. Reaction time is a crucial indicator of cognitive processing during motor performance [141] and can be a surrogate measure for attention levels [142]. Previously, studies have assessed reaction time as an indicator of sensorimotor coordination and performance [143]. Virtual reality approaches may be effectively pursued to measure and improve motor reaction times.

5.4 Cognitive load and working memory

Cognitive load fundamentally infers the amount of information that working memory can hold at a given instance of activity [144]. Working memory is the cognitive system that stores information in advance for utilization in complex tasks. Since working memory relates to information processing, learning, and problem-solving, it is a variable well-posed to leverage motor control principles in virtual reality rehabilitation [145]. Virtual reality interactions are proven to improve real-world performance through memory-level therapies [146]. Furthermore, virtual reality motor rehabilitation with proven neuroplasticity improvements will enhance working memory [147]. Thus, cognitive loading can be highly sensitive to variations in virtual reality protocols for motor rehabilitation, including the level of immersion [148]. Ultimately, a desirable range of cognitive loading should be experienced by users undergoing motor rehabilitation with virtual reality. Cognitive overload occurs when there is too much information or too many tasks to execute or learn simultaneously, resulting in an inability to process this information [144] productively. Cognitive loading is crucial for optimizing the effects of augmented feedback methods since cognitive overload can inhibit motor learning [115]. Thus, participants can risk cognitive overload with even simple tasks if augmented feedback is overwhelming. However, cognitive underload must also be avoided, typically with adjustments in task challenge, to mitigate possible user disengagement during motor task practice [149].

Achieving target levels of cognitive load could be the key to improving motor rehabilitation efficiency. For example, one study demonstrated that virtual reality training with specific cognitive load levels could significantly improve walking function for chronic stroke participants [150]. Virtual reality methods can vary task or feedback guidance complexity to modulate cognitive loading levels for optimal neural engagement. Ideally, virtual reality methods should also improve working memory to expand cognitive load capacity for movement tasks further. After boundaries of cognitive capabilities are established, virtual reality methods should maintain users within cognitive loading ranges that maximize post-training motor outcomes.

6 Conclusions

We assert that leveraging augmented sensory feedback and psychophysiological factors during virtual reality rehabilitation may be the key to unlocking the full potential of virtual reality motor rehabilitation after neurotraumas. Augmented sensory guidance accelerates motor learning with visual, auditory, and haptic cues presented individually or in combination. Feedback features such as timing, complexity, intermittency, and level of body representation may be specified to optimize virtual reality rehabilitation at subject- and task-specific levels. Additionally, incorporating psychophysiological factors into virtual reality motor paradigms ensures consistent and desirable levels of neural engagement of patients within their rehabilitation regimen. An array of psychophysiological factors (e.g., motivation, agency, attention, cognitive loading) may be monitored and assessed using advanced technologies. Furthermore, these factors can be subsequently manipulated to optimal levels within virtual reality protocols that intelligently adjust guidance levels, task type and difficulty, and immersion for each user.