Skip to main content

Virtual reality and motor imagery for early post-stroke rehabilitation

Abstract

Background

Motor impairment is a common consequence of stroke causing difficulty in independent movement. The first month of post-stroke rehabilitation is the most effective period for recovery. Movement imagination, known as motor imagery, in combination with virtual reality may provide a way for stroke patients with severe motor disabilities to begin rehabilitation.

Methods

The aim of this study is to verify whether motor imagery and virtual reality help to activate stroke patients’ motor cortex. 16 acute/subacute (< 6 months) stroke patients participated in this study. All participants performed motor imagery of basketball shooting which involved the following tasks: listening to audio instruction only, watching a basketball shooting animation in 3D with audio, and also performing motor imagery afterwards. Electroencephalogram (EEG) was recorded for analysis of motor-related features of the brain such as power spectral analysis in the \(\alpha\) and \(\beta\) frequency bands and spectral entropy. 18 EEG channels over the motor cortex were used for all stroke patients.

Results

All results are normalised relative to all tasks for each participant. The power spectral densities peak near the \(\alpha\) band for all participants and also the \(\beta\) band for some participants. Tasks with instructions during motor imagery generally show greater power spectral peaks. The p-values of the Wilcoxon signed-rank test for band power comparison from the 18 EEG channels between different pairs of tasks show a 0.01 significance of rejecting the band powers being the same for most tasks done by stroke subjects. The motor cortex of most stroke patients is more active when virtual reality is involved during motor imagery as indicated by their respective scalp maps of band power and spectral entropy.

Conclusion

The resulting activation of stroke patient’s motor cortices in this study reveals evidence that it is induced by imagination of movement and virtual reality supports motor imagery. The framework of the current study also provides an efficient way to investigate motor imagery and virtual reality during post-stroke rehabilitation.

Background

Stroke is a brain lesion which generally causes disability and even death [1, 2]. Motor impairment is a common consequence of stroke affecting stroke patients’ ability to live independently [3, 4]. The major mechanism behind post-stroke recovery is neuroplasticity which rewires the neural network of the brain [5, 6]. Early motor rehabilitation is essential to effectively restore motor function of stroke patients because neuroplasticity is most active within the first month post-stroke [2, 6, 7]. Initially, mobility of stroke patients is minimal; often they are not able to instigate any movement [2, 8]. There are six Brunnstrom motor recovery stages (BMRS) which describe different levels of mobility[2]. Stroke patients who cannot initiate any movement from affected body parts are classified in stage 1 of the BMRS [2]. Conventional post-stroke rehabilitation that relies on physical movement may be ineffective during the early stage of post-stroke motor recovery because stroke patients are often severely paralysed and unable to participate in physical rehabilitation [2, 8, 9].

Motor imagery (MI) is the mental representation of a body movement [6]. In MI, a patient is required to mentally rehearse a movement without its physical execution [3, 4]. It was suggested that MI could promote recovery of the lesioned brain areas using functional and other neuronal networks; hence, MI appears to be an effective alternative therapy for early post-stroke motor rehabilitation [5, 7]. However, MI requires training and may be challenging particularly for stroke patients [10].

It was shown that observing an action may activate the motor cortex and promote motor learning; thus, facilitating neural recovery [3, 11]. This is due to the mirror neurons being activated during both action execution and observation [3, 11]. The mirror neuron system assists the observer to imitate an observed action; hence, there may be an overlap between action observation (AO) and the process of performing a physical movement [3, 11]. It was also reported that AO via virtual reality (VR) technology could assist stroke patients to focus on MI tasks by visually simulating real movements within an immersive environment, minimising distractions from the surroundings, thus, potentially reducing the difficulty of conventional MI [10, 12]. VR technology has also been shown to assist stroke patients in a minimally conscious state to perform MI [9]. As a result, combining MI and AO for performing the same movement may enhance activation of the motor cortex and facilitate motor recovery of stroke patients, especially in stage 1 of the BMRS [4, 5, 10]. Despite there being positive evidence of VR-assisted MI in post-stroke rehabilitation, the experimental protocol of different studies is not standardised and involves various VR machineries [8, 10, 12]. The findings of different VR-MI studies are not conclusive though promising.

Physiological measure of MI recorded by electroencephalogram (EEG) provides a relatively accessible and objective way to measure brain signals induced by MI with a high temporal resolution [10, 12,13,14,15,16]. In this study, we apply filters as well as both EEG spectra and entropy analyses to investigate whether MI and VR may help to activate the brain areas responsible for motor functions; thus, potentially promoting motor recovery.

Results

The 18 EEG channels covering the motor brain areas shown in Fig. 5 are considered in computing the periodograms and band powers for stroke patients as movement processes mainly involve the motor cortex [8, 10]. Figure 1 shows all subjects’ epoch-averaged periodograms normalised with respect to the subjects’ own experimental tasks in this study: VOICE, MI after VOICE, VR+MI, and MI after VR, with 1 = the maximum and −1 = the minimum. Table 1 presents the p-values of the Wilcoxon signed-rank test for comparing the \(\alpha\) and \(\beta\) band powers associated with different pairs of tasks performed by the stroke patients. Figures 2 and 3 are, respectively, the normalised \(\alpha\) and \(\beta\) band power scalp maps relative to all classes in Experiment 1 for stroke patients 2 and 3 and in Experiment 2 for stroke patients 8 and 10. Figure 4 shows the spectral entropy scalp maps normalised to all classes in Experiments 1 and 2, respectively.

Table 1 Results of the Wilcoxon signed-ranked test between different tasks in the \(\alpha\) and \(\beta\) frequency bands from the 18 EEG channels on the motor brain areas of stroke subjects

Discussion

The experiments of this study aim to guide participants to mentally perform a basketball shooting movement which they cannot perform physically. Bimanual basketball shooting is selected as the MI task to mimic a sport activity involving the upper limbs. Mentally performing a sport involving both hands may actively promote both hemispheres of the brain to be activated, maximising brain activity of the motor cortex, especially for non-experts [3, 6, 17,18,19,20,21]. An upper limb MI is chosen because a larger area of the motor cortex is activated to control upper limbs, thus has been shown to be more effective in motor function recovery than that of lower limbs [3, 6, 22]. The basketball shooting instruction provided in the current study prompts participants to activate their motor cortex by gradually guiding them to imagine the movement in a few steps. Stroke patients have suffered brain damage, so movement instruction should be relatively straightforward and simulate physical movement as much as possible to make the MI task practical [6, 9, 23,24,25,26,27]. A sport exercise shown via video is used to induce a sense of embodiment and self-esteem from the stroke patients by attempting to trigger the neural pathways for motor processes through the patients’ imagination of performing a physical task that appears to be impossible [6, 8, 17, 18, 21, 28,29,30]. The activation intensity distribution of the brain can be studied by power spectral density (PSD), band power and spectral entropy of the EEG data that have been preprocessed [8, 31,32,33].

Power spectral density

Activation of the motor cortex is expected to induce signal peaks predominantly in the alpha (8–12 Hz) and beta (13–30 Hz) frequency bands as they correspond to motor-related processes [10, 33,34,35,36,37,38]. Periodograms illustrate how each subject’s spectral power distribution changes across different frequencies. A larger magnitude of intensity of spectral power peaks in the periodograms indicates a greater brain activation at the corresponding frequencies. For stroke subjects, there are peaks in the \(\alpha\) and \(\beta\) frequency bands for all of the tasks. A power peak in either the \(\alpha\) or the \(\beta\) band detected from the motor cortex can itself be used as an indicator of motor-related processes [10, 33,34,35,36,37,38]. There is a peak in the 20–25 Hz range within the \(\beta\) band in all classes as illustrated in Fig. 1a–p. The spectral power peaks in the \(\alpha\) band are greater for MI after VR than those of other classes for subjects 8, 9, 13 and 14, indicating VR assistance for MI. In Experiment 2, there is no assistance provided to subjects performing MI after VR for approximately 5 min; hence, distractions and fatigue may affect some subjects’ MI ability. MI after VOICE shows a distinctively greater peak in the 20–25Hz range than that of both VOICE and MI after VR for subjects 11, 15 and 16, as shown in Fig. 1k, o and p.

VR+MI in Experiment 1 provided visual and audio instructions in 3D while stroke patients perform MI. There is a more prominent peak in 20–25 Hz for VR+MI and MI after VR than that of other tasks for subjects 3 and 4 as shown in Fig. 1c and d. VOICE may only be helpful for subjects 5–7 from Experiment 1 as illustrated by their \(PSD'\) peaks in the \(\beta\) band. MI after VOICE shows no assistance for half of the subjects in Experiment 1 as indicated by its broad and flat power spectrum as illustrated in Fig. 1a–g. This may again due to attention deficiency when no cues for MI are given at all.

Fig. 1
figure 1

Periodograms for each of the subjects from Experiment 1 (ag) and Experiment 2 (hp) showing their normalised power spectral densities (\(PSD'\)) across frequencies 1 to 40 Hz. The legend for all plots is shown at the bottom right of this figure

Band power

Band powers provide an overall representation of PSD patterns. Greater \(\alpha\) and \(\beta\) band powers correspond to more intense motor-related brain activation[10, 33, 34]. The Wilcoxon signed-rank test is non-parametric which is ideal for comparing two conditions for the same participants without assuming normality of samples[39]. The results indicate that there is a 0.01 significance of rejecting the null hypothesis that VR+MI (or MI after VR) and MI after VOICE have the same band powers for almost all stroke subjects; however, VOICE and MI after VOICE have more similar band powers. Similarly, VR+MI and MI after VR also have more similar band powers as indicated by their p-values not reaching the 0.01 significance level.

Stroke patients 2, 3, 8 and 10 are illustrative of the types of responses seen in all patients. A higher value of band power (red) corresponds to more intense brain activation. The other stroke patients from Experiment 1 show similar brain activity to stroke patients 2 and 3. Most Experiment 1 stroke patients’ brain activities are similar to stroke patient 2’s, with the motor cortex being less active during MI+VR and MI after VR, as shown in the first row of Figs. 2 and 3, and Additional file 1: Figures S1 and S2. This could be attributed to some stroke patients performing visual imagery (VI) because of their lack of understanding of MI, therefore decreasing activity in the motor cortex. Stroke patient 2 may not be able to focus on MI even with VR assistance; whereas, the motor cortex of stroke patient 3 is more active during MI after VR and MI after VOICE in Experiment 1. Performing MI after VR is present appears to help stroke patient 3 to activate the motor cortex near channel C4 as illustrated by more intense \(\alpha\) band power.

Stroke patients 8 and 10 share similarities with the other stroke patients from Experiment 2. The motor cortex of stroke patient 8 during MI after VR in Experiment 2 is overall more activated in the \(\alpha\) and \(\beta\) bands compared to other classes without VR as shown in Figs. 2 and 3. Stroke patient 10 does not reflect any assistance of VR in MI induced brain activation in the motor cortex compared to MI without VR as shown in Figs. 2 and 3.

Stroke patients 3 and 8 are illustrative of most stroke patients’ brain activities in the \(\alpha\) and \(\beta\) bands, achieving maximum band power for conditions involving VR; whereas, a small number of stroke patients have similar band power scalp maps as stroke patient 10, not activating the motor cortex during MI after VR as illustrated in Additional file 1: Figures S1–S4.

Fig. 2
figure 2

\(\alpha\) band power scalp maps of stroke patients 2, 3, 8 and 10 (top row to bottom row) showing the intensity variations normalised relative to all classes from Experiments 1 and 2, respectively. Red is 1 = maximum; blue is −1 = minimum

Fig. 3
figure 3

\(\beta\) band power scalp maps of stroke patients 2, 3, 8 and 10 (top row to bottom row) showing the intensity variations normalised relative to all classes from Experiments 1 and 2, respectively. Red is 1 = maximum; blue is −1 = minimum

Fig. 4
figure 4

Spectral entropy scalp maps of stroke patients 2, 3, 8 and 10 (top row to bottom row) for frequencies 1–40 Hz showing the intensity variations normalised relative to classes from Experiments 1 and 2, respectively. Red is 1 = maximum; blue is – 1 = minimum

Spectral entropy

Entropy in a biological process measures the complexity of a physiological signal [40,41,42,43,44,45]. Brain activation is associated with peaks in the \(\alpha\) and \(\beta\) band of a power spectrum; hence, spectral entropy is used to study brain activation by analysing power spectral patterns. A power spectrum with a pattern identical to that of a single frequency component such as a sinusoid has the smallest spectral entropy [31, 32]. On the contrary, a flat power spectrum having all frequency component s with equal power like that of white noise corresponds to the greatest spectral entropy [31, 32]. A higher value of spectral entropy represents a more uniform and flatter power spectrum distribution [31, 46, 47]. Spectral entropy is a measure of the regularity of a power spectrum which should be interpreted together with spectral power analysis. A lower value of spectral entropy (blue) is associated with more intense brain activity if the associated band power is closer to maximum.

Most stroke patients’ brain patterns are similar to stroke patients 3 and 8 having the lowest spectral entropy values for VR-assisted MI; whereas, some stroke patients are similar to stroke patient 10 with higher spectral entropy values for VR-assisted MI as shown in Additional file 1: Figures S5 and S6. VR potentially assists most stroke patients to activate their motor cortex during MI, but is ineffective for some stroke patients as indicated by Figs. 2, 3 and 4.

Conclusions

MI may activate the motor brain areas of stroke patients as deduced by their normalised PSD, band power and spectral entropy computed in this study. The \(PSD'\) peaks, maximum band power and minimum spectral entropy are present in the motor-related \(\alpha\) and \(\beta\) frequency bands for most of the 16 (acute/subacute) stroke patients’ motor cortices during VR-assisted MI indicating more intense brain activation than that of MI alone. The p-values of the Wilcoxon signed-rank test associated with \(\alpha\) and \(\beta\) band powers between the conditions in Experiments 1 and 2 of this study, respectively, achieve a 0.01 significance for most stroke patients indicating that MI tasks involving VR and without VR do not have the same brain activation pattern. VR is potentially an effective tool for assisting MI performance. MI in combination with VR could be particularly beneficial for stroke patients without other rehabilitative options because of their severe motor impairment. Future research may investigate the effects of fatigue and sensory distractions during shorter or longer MI experiments [26, 27, 48,49,50,51,52]. Nonetheless, the results and framework of this study are useful for future work which may provide new insights in the applicability of MI and VR in stroke rehabilitation.

Methods

Experimental protocol

Participants

Experiments on stroke patients were conducted at Jiaxing 2nd hospital in China in April and July 2021 [53]. These experiments were approved by the Ethics Committee of Jiaxing 2nd Hospital Rehabilitation Centre in accordance with the Declaration of Helsinki. All stroke patients gave informed consent before participating in the study. The experiment could be terminated whenever the participants felt unwell with symptoms such as nausea. An initial assessment of upper limb mobility of stroke patients based on the BMRS and the mini-mental state examination (MMSE) for cognitive function were performed by medical doctors. All participants’ demographics are shown in Table 2 according to the following enrolment criteria:

  1. i

    Subjects were over 18 years old.

  2. ii

    Subjects were in stage I, II or III of the Brunnstrom stages of stroke recovery.

  3. iii

    Subjects have normal vital signs and with sufficient vision and hearing to follow instructions as determined by medical doctors using the MMSE assessment with a minimum threshold score of 10 [9, 54, 55].

Only adult stroke patients with stable vitals were recruited by clinicians to minimise unforeseen paediatric medical complications. MI is most beneficial for stroke patients with no or minimal physical movement which persists through the first 3 BMRS. Most stroke patients in this study achieve either a MMSE score corresponding to normal cognitive function, i.e. above 25 or even 30, the maximum. Only stroke patients 12 and 15 have a MMSE score in the range 10 to 19 indicating potentially moderate, but not severe, cognitive impairment [9, 54, 55].

Table 2 Demographic information for the (Experiments 1 and 2) stroke participants
Fig. 5
figure 5

Diagram showing the electrode position distribution for stroke subjects

Fig. 6
figure 6

Flowcharts showing the schematic of the experiments outlining the tasks performed by the subjects, with each trial lasting 1 min in duration. There are four tasks in Experiment 1: VOICE, MI after VOICE, VR+MI and MI after VR. There are three tasks in Experiment 2: VOICE, MI after VOICE and MI after VR

Fig. 7
figure 7

Flowchart showing the timing of the video for the VR+MI task. ad are four cues, the colours of the cues are consistent with the corresponding colours of the times at which they occur in the experimental process

Data collection

A g.HIamp (from g.tec, Austria) with 80 wet electrode channels arranged in the standard international 10–10 configuration at a sampling rate of 1200 Hz were used to record EEG data from all stroke patients. Figure 5 shows the following 18 EEG channels used for this study covering the motor brain areas: FC1, FC2, FC3, FC4, FC5, FC6, C1, C2, C3, C4, C5, C6, CP1, CP2, CP3, CP4, CP5, and CP6. The reference channel is attached to the left earlobe and Cz is the ground channel.

The EEG collected from each task of the experiment were saved as 1-min files where each file corresponded to 1 trial for stroke patients. The number of trials for stroke patients is shown in Experiments 1 and 2 of Fig. 6.

Experimental design

Subjects wore the EEG devices and were lying down on a bed during the experiments. The current study consists of 2 experiments as shown in Fig. 6 illustrating the periods of EEG measurement and their associated experimental tasks. The break period is 1 day for Experiment 1 and 15–20 min for Experiments 2. The (Mandarin Chinese) audio and video instructions, designed by Shantou University using Assembly-CSharp API from Unity[56], were given in a few steps as shown in Fig. 7. For tasks involving VR (in 3D), subjects wore a head-mounted display centred in a VR environment made by HTC VIVE PRO EYE with the helmet-mounted display (HMD) of 1440\(\times\)1600 resolution per eye and 110\(^{\circ }\)Field of view. Further details of the respective tasks of the experiments are provided in the following:

VOICE The subject wears headphones and listens to the voice instructions (in Chinese) which describes a sequence of movements for the purpose of shooting a basketball with both hands, i.e. hands reaching and holding the ball, lifting the ball, increasing arm strength, then shooting the ball. The voice instruction is played three times, each time lasts for 1 min and it is the same audio used in the video instruction from cues (a) to (d) shown in Fig. 7. Three trials were performed for all experiments.

MI after VOICE The subject was asked to imagine the movement associated with the voice instruction immediately after the end of the voice instruction. All experiments had 2 trials performed.

VR+MI In addition to the voice guidance, the subject wore the head-mounted display and observed the animation corresponding to the voice instruction for shooting a basketball while imagining the associated movement simultaneously. Figure 7 illustrates the timeline of the video instruction with cues showing the general structure of the whole video. Firstly, there is a text instruction which lasts for 1 min. Secondly, a 3-2-1 countdown appears, then the video is played in sync with the voice instruction having another 3-2-1 countdown before shooting the basketball from cues (a) to (d) of Fig. 7. Finally, a congratulatory message follows the basketball shooting. There were 3 trials for VR+MI as only text instruction was presented in the first minute of EEG recording as shown in Experiment 1 of Fig 6.

MI after VR In this task, the subject performs MI of shooting the basketball after watching the corresponding 3D video from the head-mounted display. Each minute of EEG recording is considered 1 trial as depicted in Fig. 6.

Data preprocessing

Figure 8 shows the general procedure for processing EEG data in the current study. Raw EEG is extracted as input data. At the beginning of each trial, the EEG recordings contain noise interference caused by the machinery or other sources; furthermore, trial recordings do not contain exactly the same number of samples. As a result, only samples from the first \(6{th}\) second to the \(51{st}\) second of each trial are considered for noise removal and consistency. The data are shaped as 1-s epochs \(\times\) channels \(\times\) samples for efficiency [33].

The samples for stroke patients are downsampled from 1200 Hz to 200 Hz to reduce computational complexity as usual human neural activities and the damping effects of the skull are at frequencies less than 200 Hz [57, 58]. A \(5{th}\) order Butterworth bandpass filter removes non-motor-related frequency components of the signal outside 1 to 40 Hz as this range includes all relevant frequencies for motor processes [59,60,61]. The flatness response of the Butterworth filter is suitable for preserving the desired frequency range and eliminating irrelevant parts of the signal such as power-line at 50 or 60 Hz [60, 62].

Fig. 8
figure 8

Pipeline for EEG data preprocessing (green) and analysis (blue) applied in this study

Wavelet filter

The automatic tunable artifact removal (ATAR) algorithm is designed to remove artefacts of a signal that does not rely on expert knowledge or manual identification of noisy EEG components like ICA [63]. The ATAR algorithm provides a relatively objective way to remove artefacts from the EEG data by utilising the EEG signal’s intrinsic parameters [63, 64]. The EEG signal is first deconstructed into sets of wavelet coefficients by applying a discrete wavelet transform (DWT) using the Daubechies wavelet 4 (db4) [61, 63, 64]. The decomposition level is 3 for stroke patients to produce coefficients that are approximately associated with the frequency range: 1 to 35 Hz. db4 is relatively smooth which is effective for detecting EEG variations [64]. A linear attenuation filter is applied to either remove or adjust wavelet coefficients that are large according to the following threshold functions [63, 64].

$$\begin{aligned} \psi _A\left( r,k_1,k_2\right) = {\left\{ \begin{array}{ll} f_B(r), &{} \text {if } f_B(r) \ge k_1 \\ k_1, &{} \text {otherwise} \end{array}\right. } \end{aligned}$$
(1)

and \(\psi _B = 2\psi _A\),

$$\begin{aligned} f_B(r) = k_2e^{B\left( \frac{100r}{2k_2}\right) }, \end{aligned}$$
(2)

where r is the interquartile range of \(\omega\). B=0.1 is the attenuation constant (steepness) between 0 and 1. A higher value of \(\beta\) makes the ATAR algorithm more aggressive in artefact removal. \(\beta\)=0.1, the default setting, is close to 0 which prevents loss of signal. \(k_1=8\) Hz and \(k_2=35\) Hz are the lower and upper frequency bounds to narrow the motor-related components, respectively [61, 63, 64]. The interquartile range of wavelet coefficients, r, applied in the threshold function effectively reduce the outliers outside of r and retain the core features of the signal [61, 64]. The linear attenuation filter function is given by

$$\begin{aligned} \lambda _a\left( \omega ,r,k_1,k_2\right) = {\left\{ \begin{array}{ll} \omega , &{} |\omega |\le \psi _A \\ {{\,\textrm{sgn}\,}}(\omega )\psi _A\left( 1-\frac{|\omega |-\psi _A}{\psi _B-\psi _A}\right) , &{} \psi _A< |\omega |\le \psi _B \\ 0, &{} \text {otherwise}, \\ \end{array}\right. } \end{aligned}$$
(3)

where \({{\,\textrm{sgn}\,}}(\cdot )\) is the signum function. Finally, the filtered wavelet coefficients are used to reconstruct the signal by using the inverse wavelet transform.

The attenuation filter function, \(\lambda _\alpha\), involves the lower and upper bounds on the threshold value: k1 and k2, respectively[63, 64]. \(\beta\) is the attenuation constant (steepness). k1=8 Hz and k2=35 Hz. r is the interquartile range of wavelet coefficients[61, 63, 64]. Finally, the first set of the wavelet coefficients is chosen to reconstruct the signal as it is best corresponded with the frequency range: 1 to 40 Hz [61, 63, 64].

Data analysis

Periodograms of the stroke patients are, respectively, averaged over all epochs for each of the different experimental tasks and are normalised by the minimum–maximum feature scaling function

$$\begin{aligned} x'=2\frac{x-x_{min}}{x_{max}-x_{min}}-1 , \end{aligned}$$
(4)

with respect to the tasks shown in Fig. 6: VOICE, MI after VOICE, VR+MI, and MI after VR, with 1 = the maximum and – 1 = the minimum. Power spectral densities of the preprocessed signal across frequencies is computed by Welch’s method using a Hamming window with zero padding to smooth the output. Simpson’s rule was used to calculate the band powers in the \(\alpha\) and \(\beta\) bands by summing the PSD in the respective frequency range [65]. The Wilcoxon signed-rank test was used to compare different pairs of tasks performed by stroke patients with the associated p-values given in Table 1 for the \(\alpha\) and \(\beta\) bands. The Python MNE library is used to compute the scalp maps for the \(\alpha\) and \(\beta\) band powers of each stroke patient [66]. The band powers are normalised by Eq. 4, the minimum–maximum feature scaling function, where band powers from all tasks shown in Fig. 6 are considered for each experiment, respectively.

The Python package: antropy is used to compute spectral entropy [67]. Spectral entropy utilises Shannon entropy and the signal’s power spectrum to compute the regularity of the time series corresponding to the uniformity of power spectrum distribution as shown in Eq. 5 [31, 32, 41, 46, 68]:

$$\begin{aligned} H_{spec}=-\sum _{f_{0}}^{f_{n}}{\hat{p}}(f)log_{2}({\hat{p}}(f)); {\hat{p}}(f)=\frac{p(f)}{\sum _{f_{0}}^{f_{n}}p(f)}, \end{aligned}$$
(5)

where p(f) is the power spectral density; \({\hat{p}}(f)\) is the normalised power spectral density; \(f_{0}\) and \(f_{n}\) are, respectively, the first and last frequencies of the integrated frequency range; the logarithmic base is 2 and the spectral entropy is in units of bits. The Python MNE library and Eq. 4 are applied to compute the normalised spectral entropy relative to the experimental tasks specified in Fig. 6 for each stroke patient.

Availability of data and materials

The dataset of the current study may be requested from the corresponding authors.

Abbreviations

AO:

Action observation

API:

Application programming interface

ATAR:

Automatic tunable artifacts removal algorithm

BMRS:

Brunnstrom motor recovery stages

3D:

3 Dimensional

db4:

Daubechies wavelet 4

DWT:

Discrete wavelet transform

EEG:

Electroencephalogram

ICA:

Individual component analysis

MI:

Motor imagery

MMSE:

Mini-mental state examination

PSD:

Power spectral density

VI:

Visual imagery

VR:

Virtual reality

References

  1. Intercollegiate Stroke Working Party (ICSWP). National Clinical Guideline for Stroke. Bowen A, James M, Young G, editors. London, United Kingdom: Royal College of Physicians; 2016.

  2. Brunnstrom S. In: movement therapy in hemiplegia: a neurophysiological approach. New York, NY: Harper & Row; 1970. p. 113–22.

    Google Scholar 

  3. Hanakawa T. Organizing motor imageries. Neurosci Res. 2016;104:56–63. https://doi.org/10.1016/j.neures.2015.11.003.

    Article  Google Scholar 

  4. Chholak P, Niso G, Maksimenko VA, Kurkin SA, Frolov NS, Pitsik EN, et al. Visual and kinesthetic modes affect motor imagery classification in untrained subjects. Sci Rep. 2019;9(1):1–12.

    Article  Google Scholar 

  5. Munzert J, Lorey B, Zentgraf K. Cognitive motor processes: the role of motor imagery in the study of motor representations. Brain Res Rev. 2009;60(2):306–26.

    Article  Google Scholar 

  6. Choy CS, Cloherty SL, Pirogova E, Fang Q. Virtual reality assisted motor imagery for early post-stroke recovery: a review. IEEE Rev Biomed Eng. 2022;16:487–98.

    Article  Google Scholar 

  7. Wilkins KB, Dewald JPA, Yao J. Intervention-induced changes in neural connectivity during motor preparation may affect cortical activity at motor execution. Sci Rep. 2020;10(1):1–13.

    Article  Google Scholar 

  8. Mane R, Chouhan T, Guan C. BCI for stroke rehabilitation: motor and beyond. J Neural Eng. 2020;17(4): 041001.

    Article  Google Scholar 

  9. Maggio MG, Naro A, La Rosa G, Cambria A, Lauria P, Billeri L, et al. Virtual reality based cognitive rehabilitation in minimally conscious state: a case report with EEG findings and systematic literature review. Brain Sci. 2020;10(7):1–16.

    Article  Google Scholar 

  10. Vourvopoulos A, Jorge C, Abreu R, Figueiredo P, Fernandes JC, Bermúdez i Badia S. Efficacy and brain imaging correlates of an immersive motor imagery BCI-driven VR system for upper limb motor rehabilitation: a clinical case report. Front Human Neurosci. 2019;13:1–17.

    Article  Google Scholar 

  11. Filipe M, Pedro P, Tânia C, João B. Brain activity during virtual and real dart throwing tasks in patients with stroke: a pilot study. Front Human Neurosci. 2016. https://doi.org/10.3389/conf.fnhum.2016.220.00079.

    Article  Google Scholar 

  12. Cervera MA, Soekadar SR, Ushiba J, Millán JdR, Liu M, Birbaumer N, et al. Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis. Ann Clin Transl Neurol. 2018;5(5):651–63.

    Article  Google Scholar 

  13. Zhang C, Liu C, Zhang X, Almpanidis G. An up-to-date comparison of state-of-the-art classification algorithms. Expert Syst Appl. 2017;82:128–50. https://doi.org/10.1016/j.eswa.2017.04.003.

    Article  Google Scholar 

  14. Gu X, Cao Z, Jolfaei A, Xu P, Wu D, Jung TP, et al. EEG-Based brain-computer interfaces (BCIs): a survey of recent studies on signal sensing technologies and computational intelligence approaches and their applications. IEEE/ACM Trans Comput Biol Bioinform. 2021;18(5):1645–66.

    Article  Google Scholar 

  15. Hsu WY. EEG-based motor imagery classification using neuro-fuzzy prediction and wavelet fractal features. J Neurosci Methods. 2010;189(2):295–302. https://doi.org/10.1016/j.jneumeth.2010.03.030.

    Article  Google Scholar 

  16. Gerloff C, Bushara K, Sailer A, Wassermann EM, Chen R, Matsuoka T, et al. Multimodal imaging of brain reorganization in motor areas of the contralesional hemisphere of well recovered patients after capsular stroke. Brain. 2006;129(3):791–808.

    Article  Google Scholar 

  17. Moran A, Guillot A, MacIntyre T, Collet C. Re-imagining motor imagery: building bridges between cognitive neuroscience and sport psychology. British J Psychol. 2012;103(2):224–47.

    Article  Google Scholar 

  18. Wriessnegger SC, Unterhauser K, Bauernfeind G. Limb Preference and skill level dependence during the imagery of a whole-body movement: a functional near infrared spectroscopy study. Front Human Neurosci. 2022;16:1–12.

    Article  Google Scholar 

  19. Kemlin C, Moulton E, Samson Y, Rosso C. Do motor imagery performances depend on the side of the lesion at the acute stage of stroke? Front Human Neurosci. 2016;10:1–10.

    Article  Google Scholar 

  20. Xie F, Xu L, Long Z, Yao L, Wu X. Functional connectivity alteration after real-time fMRI motor imagery training through self-regulation of activities of the right premotor cortex. BMC Neurosci. 2015;16(1):1–11.

    Article  Google Scholar 

  21. Calabrò RS, Cerasa A, Ciancarelli I, Pignolo L, Tonin P, Iosa M, et al. The arrival of the metaverse in neurorehabilitation: fact, fake or vision? Biomedicines. 2022;10(10):1–12.

    Article  Google Scholar 

  22. Nunez PL, Srinivasan R, et al. Electric fields of the brain: the neurophysics of EEG. Oxford, USA: Oxford University Press; 2006.

    Book  Google Scholar 

  23. Khan MA, Das R, Iversen HK, Puthusserypady S. Review on motor imagery based BCI systems for upper limb post-stroke neurorehabilitation: from designing to application. Comput Biol Med. 2020;123: 103843. https://doi.org/10.1016/j.compbiomed.2020.103843.

    Article  Google Scholar 

  24. Khan MA, Saibene M, Das R, Brunner I, Puthusserypady S. Emergence of flexible technology in developing advanced systems for post-stroke rehabilitation: a comprehensive review. J Neural Eng. 2021;18(6): 061003.

    Article  Google Scholar 

  25. García Carrasco D, Aboitiz Cantalapiedra J. Effectiveness of motor imagery or mental practice in functional recovery after stroke: a systematic review. Neurol. 2016;31(1):43–52. https://doi.org/10.1016/j.nrleng.2013.02.008.

    Article  Google Scholar 

  26. Oostra KM, Oomen A, Vanderstraeten G, Vingerhoets G. Influence of motor imagery training on gait rehabilitation in sub-acute stroke: a randomized controlled trial. J Rehabil Med. 2015;47(3):204–9.

    Article  Google Scholar 

  27. Santos-Couto-Paz CC, Teixeira-Salmela LF, Tierra-Criollo CJ. The addition of functional task-oriented mental practice to conventional physical therapy improves motor skills in daily functions after stroke. Brazilian J Phys Ther. 2013;17(6):564–71.

    Article  Google Scholar 

  28. Saposnik G, Levin M. Virtual reality in stroke rehabilitation: a meta-analysis and implications for clinicians. Stroke. 2011;42(5):1380–6.

    Article  Google Scholar 

  29. de Vries S, Tepper M, Otten B, Mulder T. Recovery of motor imagery ability in stroke patients. Rehabil Res Pract. 2011;2011:1–9.

    Google Scholar 

  30. Daprati E, Nico D, Duval S, Lacquaniti F. Different motor imagery modes following brain damage; 2010.

  31. Yu X, Mei Z, Chen C, Chen W. Ranking power spectra: a proof of concept. Entropy. 2019;21(11):1–12.

    Article  MathSciNet  Google Scholar 

  32. Inouye T, Shinosaki K, Sakamoto H, Toi S, Ukai S, Iyama A, et al. Quantification of EEG irregularity by use of the entropy of the power spectrum. Electroencephalogr Clin Neurophysiol. 1991;79(3):204–10.

    Article  Google Scholar 

  33. Ma X, Qiu S, He H. Multi-channel EEG recording during motor imagery of different joints from the same limb. Sci Data. 2020;7(1):1–9. https://doi.org/10.1038/s41597-020-0535-2.

    Article  Google Scholar 

  34. Newson JJ, Thiagarajan TC. EEG frequency bands in psychiatric disorders: a review of resting state studies. Front Human Neurosci. 2019;12(January):1–24.

    Google Scholar 

  35. Darvas F, Scherer R, Ojemann JG, Rao RP, Miller KJ, Sorensen LB. High gamma mapping using EEG. NeuroImage. 2010;49(1):930–8. https://doi.org/10.1016/j.neuroimage.2009.08.041.

    Article  Google Scholar 

  36. Smith MM, Weaver KE, Grabowski TJ, Rao RPN, Darvas F. Non-invasive detection of high gamma band activity during motor imagery. Front Human Neurosci. 2014;8:817.

    Article  Google Scholar 

  37. Cagnan H, Mallet N, Moll CKE, Gulberti A, Holt AB, Westphal M, et al. Temporal evolution of beta bursts in the parkinsonian cortical and basal ganglia network. Proc Natl Acad Sci United States of America. 2019;116(32):16095–104.

    Article  Google Scholar 

  38. Urigüen JA, Garcia-Zapirain B. EEG artifact removal - State-of-the-art and guidelines. J Neural Eng. 2015;12(3): 031001.

    Article  Google Scholar 

  39. Rey D, Neuhäuser M. Wilcoxon-signed-rank test. Berlin, Heidelberg: Springer Berlin Heidelberg; 2011. p. 1658–9.

    Google Scholar 

  40. Ge D, Srinivasan N, Krishnan SM. Nonlinear analysis of EEG signals at different mental states. BioMed Eng Online. 2002;12:1–12.

    Google Scholar 

  41. Mu Z, Hu J, Min J. Driver fatigue detection system using electroencephalography signals based on combined entropy features. Appl Sci (Switzerland). 2017;7(2):150.

    Google Scholar 

  42. Richman JS, Moorman JR. Physiological time-series analysis using approximate entropy and sample entropy maturity in premature infants Physiological time-series analysis using approximate entropy and sample entropy. Am J Physiol Heart Circ Physiol. 2000;278:H2039–49.

    Article  Google Scholar 

  43. Pincus SM. Approximate entropy as a measure of system complexity. Proc Natl Acad Sci United States of America. 1991;88(6):2297–301.

    Article  MATH  MathSciNet  Google Scholar 

  44. Al-Shargie F, Tariq U, Babiloni F, Al-Nashash H. Cognitive vigilance enhancement using audio stimulation of pure tone at 250 Hz. IEEE Access. 2021;9:22955–70.

    Article  Google Scholar 

  45. Chen W, Wang Z, Xie H, Yu W. Characterization of surface EMG signal based on fuzzy entropy. IEEE Trans Neural Syst Rehabil Eng. 2007;15(2):266–72.

    Article  Google Scholar 

  46. Tian Y, Zhang H, Xu W, Zhang H, Yang L, Zheng S, et al. Spectral entropy can predict changes of working memory performance reduced by short-time training in the delayed-match-to-sample task. Front Human Neurosci. 2017;11(August):1–12.

    Google Scholar 

  47. Llanos F, Alexander JM, Stilp CE, Kluender KR. Power spectral entropy as an information-theoretic correlate of manner of articulation in American English. J Acoust Soc Am. 2017;141(2):EL127–33. https://doi.org/10.1121/1.4976109.

    Article  Google Scholar 

  48. Guillot A. Neurophysiological Foundations and Practical Applications of Motor Imagery. In: The Cambridge Handbook of the Imagination; 2020. p. 207–226.

  49. Guillot A, Collet C. Duration of mentally simulated movement: a review. J Motor Behav. 2005;37(1):10–20.

    Article  Google Scholar 

  50. Wang X, Wang H, Xiong X, Sun C, Zhu B, Xu Y, et al. Motor imagery training after stroke increases slow-5 oscillations and functional connectivity in the ipsilesional inferior parietal lobule. Neurorehabil Neural Repair. 2020;34(4):321–32.

    Article  Google Scholar 

  51. Decety J, Jeannerod M. Mentally simulated movements in virtual reality: does Fitt’s law hold in motor imagery? Behav Brain Res. 1995;72(1–2):127–34.

    Article  Google Scholar 

  52. Iwatsuki K, Hoshiyama M, Oyama S, Yoneda H, Shimoda S, Hirata H. Electroencephalographic functional connectivity with the tacit learning system prosthetic hand: a case series using motor imagery. Front Synaptic Neurosci. 2020;12(February):1–7.

    Google Scholar 

  53. Huang J, Lin M, Fu J, Sun Y, Fang Q. An Immersive Motor Imagery Training System for Post-Stroke Rehabilitation Combining VR and EMG-based Real-Time Feedback. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2021; p. 7590–7593.

  54. Vertesi A, Lever JA, Molloy DW, Sanderson B, Tuttle I, Pokoradi L, et al. Standardized mini-mental state examination. Use and interpretation. Can Fam Phys. 2001;47:2018–23.

    Google Scholar 

  55. Kim KS, Lee SJ, Suh JC. Numerical simulation of the vortical flow around an oscillating circular cylinder. Proc Int Offshore Polar Eng Con. 2005;2005:162–7.

    Google Scholar 

  56. Technologies U. Unity - Scripting API;. Available from: https://docs.unity3d.com/ScriptReference/.

  57. Beaulieu-Laroche L, Brown NJ, Hansen M, Toloza EHS, Sharma J, Williams ZM, et al. Allometric rules for mammalian cortical layer 5 neuron biophysics. Nature. 2021;600(7888):274–8.

    Article  Google Scholar 

  58. Muthukumaraswamy SD. High-frequency brain activity and muscle artifacts in MEG/EEG: a review and recommendations. Front Human Neurosci. 2013;7:1–11.

    Article  Google Scholar 

  59. Vourvopoulos A, Pardo OM, Lefebvre S, Neureither M, Saldana D, Jahng E, et al. Effects of a brain-computer interface with virtual reality (VR) neurofeedback: a pilot study in chronic stroke patients. Front Human Neurosci. 2019;13:1–17.

    Article  Google Scholar 

  60. Rogasch NC, Sullivan C, Thomson RH, Rose NS, Bailey NW, Fitzgerald PB, et al. Analysing concurrent transcranial magnetic stimulation and electroencephalographic data: a review and introduction to the open-source TESA software. NeuroImage. 2016;2017(147):934–51.

    Google Scholar 

  61. Wang XW, Nie D, Lu BL. Emotional state classification from EEG data using machine learning approach. Neurocomputing. 2014;129:94–106. https://doi.org/10.1016/j.neucom.2013.06.046.

    Article  Google Scholar 

  62. Butterworth S. On the theory of filter amplifiers. Exp Wirel Wirel Eng. 1930;7:536–41.

    Google Scholar 

  63. Bajaj N, Requena Carrión J, Bellotti F, Berta R, De Gloria A. Automatic and tunable algorithm for EEG artifact removal using wavelet decomposition with applications in predictive modeling during auditory tasks. Biomed Signal Process Control. 2020;55: 101624. https://doi.org/10.1016/j.bspc.2019.101624.

    Article  Google Scholar 

  64. Subasi A. EEG signal classification using wavelet feature extraction and a mixture of expert model. Expert Syst Appl. 2007;32(4):1084–93.

    Article  Google Scholar 

  65. Hahn BH, Valentine DT. Introduction to Numerical Methods. In: Essential MATLAB for Engineers and Scientists; 2017. p. 295–323.

  66. Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, et al. MEG and EEG data analysis with MNE-Python. Front Neurosci. 2013;7(267):1–13.

    Google Scholar 

  67. Vallat R. Raphaelvallat/Antropy: Antropy: Entropy and complexity of (EEG) time-series in Python. https://github.com/raphaelvallat/antropy.

  68. Zhang R, Xu P, Chen R, Li F, Guo L, Li P, et al. Predicting Inter-session Performance of SMR-Based Brain-Computer Interface Using the Spectral Entropy of Resting-State EEG. Brain Topogr. 2015;28(5):680–90. https://doi.org/10.1007/s10548-015-0429-3.

    Article  Google Scholar 

Download references

Acknowledgements

The authors appreciate the experimental work of Mr Jianli Wang (former Master’s student at Shantou University) for Experiment 1 in April 2021. The authors would like to thank Mr Kaichao Wu of Shantou University for providing experimental assistance. The authors are grateful for all the medical staff at Jiaxing 2nd Hospital, China, and participants who made this work possible.

Funding

Li Ka Shing Foundation Cross-Disciplinary Research Grant (2020LKSFG03C) Australian Government (RTP Stipend Scholarship (RSS-SC)) RMIT University (Engineering Top-up Scholarship (E &B))

Author information

Authors and Affiliations

Authors

Contributions

CSC performed the analysis and drafted the manuscript including all Figures and Tables unless stated otherwise. BD designed and conducted Experiment 2 in July 2021, and produced Fig. 5. BJ, KN and QF contributed concepts and edited the manuscript. QF organised and managed this research, and provided the initial concept. BJ also edited all Tables and Fig. 6. AK and SSM contributed technical expertise. XG and JF are responsible for recruiting, monitoring and diagnosing stroke patients. XG and JF also provided advice as medical experts for this work. All authors revised and approved the manuscript.

Corresponding authors

Correspondence to Qiang Fang or Beth Jelfs.

Ethics declarations

Ethics approval and consent to participate

The Ethics Committee of Jiaxing 2nd Hospital Rehabilitation Center, China approved this study which was compliant to the Helsinki Declaration. All subjects signed or agreed to participant in this study.

Consent for publication

Authors agree for the publication of any figures or data included in this article.

Conmpeting interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Figure S1.

α band power scalp maps of stroke patients 1 to 7 (top row to bottom row) from Experiment 1 showing the intensity variations normalised relative to classes: VOICE, MI after VOICE, MI+VR and MI after VR. Red is 1 = maximum; blue is -1 = minimum. Figure S2. β band power scalp maps of stroke patients 1 to 7 (top row to bottom row) from Experiment 1 showing the intensity variations normalised relative to classes: VOICE, MI after VOICE, MI+VR and MI after VR. Red is 1 = maximum; blue is -1 = minimum. Figure S3. α band power scalp maps of stroke patients 8 to 16 (top row to bottom row) from Experiment 2 showing the intensity variations normalised relative to classes: VOICE, MI after VOICE and MI after VR. Red is 1 = maximum; blue is -1 = minimum. Figure S4. β band power scalp maps of stroke patients 8 to 16 (top row to bottom row) from Experiment 2 showing the intensity variations normalised relative to classes: VOICE, MI after VOICE and MI after VR. Red is 1 = maximum; blue is -1 = minimum. Figure S5. Spectral entropy scalp maps of stroke patients 1 to 7 (top row to bottom row) for frequencies 1-40 Hz showing the intensity variations normalised relative to classes from Experiment 1. Red is 1 = maximum; blue is -1 = minimum. Figure S6. Spectral entropy scalp maps of stroke patients 8 to 16 (top row to bottom row) for frequencies 1-40 Hz showing the intensity variations normalised relative to classes from Experiment 2, respectively. Red is 1 = maximum; blue is -1 = minimum.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choy, C.S., Fang, Q., Neville, K. et al. Virtual reality and motor imagery for early post-stroke rehabilitation. BioMed Eng OnLine 22, 66 (2023). https://doi.org/10.1186/s12938-023-01124-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12938-023-01124-9

Keywords