Image enhancement using MVEF algorithm

Sonograms are usually affected by speckle noises, which hinder the analysis of musculoskeletal geometry. Taking into account the fact that the fiber in sonograms are tubular and include coherent orientation tendencies, we apply MVEF to enhance the sonograms before the line angle detection. The MVEF method is based on the second order local structure, with excellent noise and background suppression performance [31]. The method includes three steps: the Hessian matrix estimation (including the choice of Gaussian kernels), computation of eigenvector for each scale and processing for the maximum vesselness response. More details can be found in [31].

Extraction of object regions

In this study, objects to be extracted from sonograms are regions which may represent long and thin muscle fibers. Since fibers, both aponeuroses and other fascia structures, have higher intensity than the background in a sonogram, a straightforward approach to find the potential regions of fibers is to apply a threshold on the enhanced image, so that the pixels whose value is greater than the threshold are regarded as the pixels located in the potential object regions. In this paper, the Otsu’s method is employed to get the optimal threshold [32]. This will result in a binary map, I _map , where the white components represent the candidate regions for muscle fibers.

Estimation of MFO

According to our observations, shapes of the interested object regions could vary and be divided into three different patterns. RA: regions which are long and thin, where each of them represents one major muscle fiber. RB: regions which are long but have branches because of adherence of two or several muscle fibers. RC: regions which are short and possibly from a 'broken’ line caused by partial imaging one single muscle fiber.

Orientations of RA and RC would be calculated as the angle between the y-axis and major axis of ellipse that has the same normalized second central moments as the region. Angles of RB would be calculated using Hough transform (HT).

Specifically speaking, 3 shape measures for the region, aspect ratio Ar, width ω and length L, will be used for classification of RA, RB and RC. In this study, L and ω are calculated respectively as the length of the major and minor axis of the ellipse that has the same normalized second central moments as the region, and the aspect ratio is defined as L/ω.

The procedures of the proposed framework for MFO estimation on the binary map are shown in Figure 1.

The proposed framework detects lines one by one in I _map , starting with the longest region and detailed steps are:

Step 1.  Setting parameters T ₁ , T ₂ , T ₃ and N; n = 1; $I_{\mathit{map}}^n=I_{\mathit{map}}$. (T ₁ , T ₂ are thresholds for shape measurements Ar and ω respectively. T ₃ is the ratio of the length of the last and first detected lines. N is the upper limit of line number for each image).

Step 2.  Extracting the longest region and calculating its aspect ratio Ar ⁿ , width ω ⁿ and length L ⁿ .

Step 3.  If Ar ⁿ  > T ₁ and ω ⁿ  < T ₂ , the orientation of the region is estimated as the angle between the y-axis and major axis of ellipse that has the same normalized second central moments as the region; Otherwise, applying HT on the region, and the line that with global maximum in the accumulator array is detected.

Step 4.  Removing pixels close to the line detected in step #3 and getting the updated map $I_{\mathit{map}}^{n+1}$. (This step can remove noises near the line and avoid the duplication in the angle measurement of RC).

Step 5.  Check whether n = N or L ⁿ  < L ¹ × T ₃ . If not, n = n + 1 and repeat from step #2.

Experiments

Experiment setup for normal subjects

The proposed framework is first evaluated using the dataset from previous reports [3, 5], which were acquired on biceps and forearm muscles during various typical exercise tasks, from three healthy adult male volunteers, and the detailed experiment setup can be found in [3, 5].

Then to further evaluate the proposed framework on same muscle but from different subjects, we designed an experiment example on the sonograms of gastrocnemius. Eleven healthy male subjects (mean ± SD, age = 29.4 ± 1.8 years; body weight 65.9 ± 9.3 kg; height = 170.3 ± 5.1 cm) volunteered to participate in this experiment. No participant had a history of neuromuscular disorders, and all were aware of experimental purposes and procedures. The human subject ethical approval was obtained from the relevant committee in the Hong Kong Polytechnic University, Hung Hum, Hong Kong and informed consents were obtained from subjects prior to the experiment.

The testing position of the subject was in accordance with the Users Guide of a Norm dynamometer (Humac/Norm Testing and Rehabilitation System, Computer Sports Medicine, Inc., Massachusetts, USA). Each subject was required to put forth his maximal effort of isometric plantar flexion for a period of 3 seconds with verbal encouragement provided. The maximal voluntary contraction (MVC) was defined as the highest value of torque recorded during the entire isometric contraction. A rest of 5 min was allowed before the subject performing another MVC test. The MVC torque was then calculated by averaging the two recorded highest torque values from the two tests. The subject was instructed to generate a torque waveform in rough sinusoid shape, up to 90% of his MVC, using ankle plantar flexion movements in prone position. The torque was measured by the aforementioned dynamometer and the reason for choosing 90% MVC as the highest value was to avoid muscle fatigue.

A real-time B-mode ultrasonic scanner (EUB-8500, Hitachi Medical Corporation, Tokyo, Japan) with a 10 MHz electronic linear array probe (L53L, Hitachi Medical Corporation, Tokyo, Japan) was used to obtain ultrasound images of muscles. The long axis of the ultrasound probe was arranged parallel to the long axis of the gastrocnemius and on its muscle belly. The ultrasound probe was fixed by a custom-designed foam container with fixing straps, and a very generous amount of ultrasound gel was applied to secure acoustic coupling between the probe and skin during muscle contractions, as shown in Figure 2. The probe was adjusted to optimize the contrast of muscle fascicles in ultrasound images. Then the B-mode ultrasound images were digitized by a video card (NI PCI-1411, National Instruments, Austin, USA) at a rate of 25 frame/s for later analysis.

Surface electromyography (EMG) signals were collected from the gastrocnemius muscle using bipolar Ag-AgCl electrodes (Axon System, Inc., NY, USA), amplified by a multiple channel amplifier (RM6280 Multi-Channel Biosignal Collection and Processing System, Chengdu Instrument Company, Chengdu, China), with a gain of 2000, filtered separately by 10–400 Hz, 5–100 Hz band-pass analog filters within the amplifier, and then digitized by a 12-bit data acquisition card (NI-DAQ 6024E, National Instruments Corporation, Austin, TX, USA) with a sampling rate of 1 kHz. Ultrasound image sequences, surface EMG and torque signals were simultaneously collected and stored by a custom-made program for ultrasonic measurement of motion and elasticity (UMME, http://www.tups.org).

Totally eleven sequences of gastrocnemius ultrasound images were acquired and each includes 200 images. For each sequence, 3 frames corresponding to the torque at 0%, 45% and 90% MVC were used to evaluate the proposed framework.

Furthermore, one sequence including all 200 frames from one representative subject is used to evaluate the performance of MFO tracking, following the practice of [4, 7].

Experiment setup for an aged subject with cerebral infarction

We also tried to preliminarily evaluate the performance of the proposed framework on sonograms from other than healthy or young subjects. One male subject with unilateral limb dysfunction caused by cerebral infarction (age = 68 years; body weight = 71 kg; height = 1.72 m; right leg dysfunctional) volunteered to participate in this study. The human subject ethical approval was obtained from the relevant committee in Zhujiang Hospital, Guangzhou, China before carrying out the experiment. The subject was briefed about the procedure of experiment and written consents were collected prior to the experiment. The subject was seated with both right hip and knee angles of 90. During measurement, the subject was asked to perform plantar flextion both in left leg (normal) and in right leg (dysfunctional) with his best efforts, and the rough contraction time is about .4 seconds in one exercise. A laptop ultrasound system (SS-10, Sonostar Technologies Co., Limited, Guangzhou, China), with a 7.5 MHz electronic linear array probe, was used to obtain the ultrasonic image sequences. The long axis of the probe was arranged parallel to the long axis of the gastrocnemius muscle (as shown in Figure 3) and during the muscle contraction the probe was managed to keep on well coupled to the muscle belly with a luxury usage of gel by an experienced operator. 128 consecutive frames from each leg were captured. The caregiver of the patient had the SonoStar scanner available only, and they allowed no third-party scanner used inside their medical facilities.

Results on the dataset from previous reports

Totally 314 lines were detected in the 45 sonograms from previous reports [3, 5], among which 313 were regarded as being valid according to visual verification and 8 lines were detected using HT.

A typical original sonogram of biceps and the corresponding results obtained using the proposed method, are shown in Figure 4. The lines and their angle values were marked with the order in which they got detected. MFO here was computed as the angle between the detected line and the vector pointing from the up-left to bottom-left corner of the image, which was the same as the definition of the NIH Image (National Institutes of Health, USA) software.

In order to verify the validity of the method, two operators who were experienced in ultrasound imaging of muscles, were employed to draw lines manually on the original images at locations where lines were detected using the proposed framework, and read the drawn angles using NIH Image software. As a result, 313 lines were marked out manually, all included in the 314 lines detected by the proposed framework. The angles estimated by our method, operator #1 and operator #2 were defined as ap, aa 1 and aa 2 respectively and displayed in Figure 5.

To investigate how well our results fit the manually drawn values, the mean of aa 1 and aa 2, named as “Op”, was compared with the corresponding value of ap. The correlation analysis of Op and ap is shown in Figure 6. Further Bland-Altman plot [33] illustrated in Figure 7 presents the details of the angle measurements statistics. The angle differences between the manual and automatic measurements is 0.11 ± 1.80 at the 95% confidence level.

Results on sonograms of gastrocnemius from 11 subjects

Totally 231 lines were detected in the 33 sonograms of gastrocnemius from 11 normal subjects, among which 217 were regarded as being valid according to visual verification and only 1 line was detected using HT. Blamd-Altman plot of angle detection results for these data, obtained by the proposed framework and the 2 operators, is displayed in Figure 8. The angle differences between the manual and automatic measurements is 0.03 ± 2.72 at the 95% confidence level.

Results on MFO tracking from 200 frames of one representative subject

We also tracked the orientation changes of a selected dominant fascicle and the deep aponeurosis in a sequence. Comparisons of these two orientations estimated using the proposed and manual methods were shown in Figure 9. It was observed that there were good correlations among the results obtained by the two operators and the proposed framework. Angles of the selected fasicle and the deep aponeurosis were also shown in Figure 10(a-b) together with the corresponding normalized torque signal and the normalized root mean square (RMS, 256-points) values of EMG signal in Figure 10(c-d).

Results on sonograms from the aged subject with cerebral infarction

Two typical frames from left and right legs of the aged subject and their line detection results estimated using the proposed method are shown in Figure 11. Based on the triceps surae architecture, the maximum line per sonogram N was set to 4 for the data from aged subject. 440 and 452 lines were detected for frames for normal and dysfunctional limbs respectively, and among which 3 was invalid according to visual verification. However, totally 67 interested muscle fibers were not detected by the proposed method.

The effects of the parameters on the line detection

Results of the method are affected by at least 4 parameters, i.e., T ₁ , T ₂ , T ₃ and N.

The total number of detected lines in a sonogram could be determined by T ₃ and N. The maximum line per sonogram N could be easily tuned up, if necessary, to accommodate the detection of more localized linear patterns. Taking 45 sonograms of biceps and forearm as examples, if we set N to 9 instead of 7, the number of total detected lines would increase from 314 to 378, while the number of false lines not actually corresponding to any one single muscle fiber increased from 1 to 4.

The last-to-first ratio T ₃ was set to exclude very short regions which may be noises, artefacts or the shorter regions of the broken fiber (whose angles could be the duplications of the longest segment from the same broken fiber). It is clear that the larger T ₃ is, the less possible that a false line would be wrongly detected. While on the other hand, if this ratio is too large, less lines could be detected especially if the longest line in the image are very long. For 45 sonograms of biceps and forearm, when we tune T ₃ from 10% up to 50%, all detected lines were valid but the number of total detected lines decreased from 314 to 118.

As for T ₁ and T ₂ , which are thresholds for aspect ratio Ar and width ω respectively, taking an incoming region as an example, the larger its Ar and the smaller its ω is, the more possible it will falls into RA group. Simply speaking, T ₁ and T ₂ affect the number of lines detected using HT. The larger T ₁ and the smaller T ₂ is, the more lines will be detected using HT, and the slower the algorithm will be. However, smaller T ₁ or larger T ₂ could cause higher risks to mis-classify a RB as a RA, which may lead to a wrong angle estimation. Therefore, one has to compromise between the computation efficiency and correctness and tune the parameters based on the specific image contents.

Evaluation of the validity and accuracy of the proposed method

As for the quantitative performance of our method, taking the result on the dataset used in previous study [3, 5] as an example, a very good linear correlation between the results of the manual and proposed methods was obtained (shown in Figure 6, R² = 0.987). Meanwhile, Bland-Altman plots in Figure 7 and Figure 8 also echoed the good agreement between the results of the manual and proposed methods both in the previous dataset and sonograms of gastrocnemius from normal subjects.

As shown in Table 1, the angle differences between the manual and automatic measurements, 0.11 ± 1.80 at the 95% confidence level, are comparable to the previous results of 0.13 ± 1.79 obtained using RVHT with Gabor filtering [5].

Results of the proposed method for interested MFO tracking agreed well with those of the manual method, as shown in Figure 9. It can be observed in Figure 10 that the interested MFOs detected were not as smooth as the torque signal, but followed the general cyclic changes along with the torque output.

The results from the aged subject with cerebral infarction have given us some preliminary confidence on the proposed framework. However, the proposed method failed to detect all valid major muscle fibers (about 7.5% missed). This could be related to the fact that the images from this subject are quite different from young or healthy subjects in terms of both echo intensity and image texture [34].

Evaluation of the efficiency of the proposed method

Because MVEF has less computation than Gabor Filtering [20], we compared speed of our framework with that of RVHT both with and without MVEF procedure.

It can be seen from Table 2, that in general, the speed of the proposed framework is much faster than that of RVHT, which makes significant difference in terms of massive data.

Two major reasons contributed to the higher efficiency of the proposed method: Firstly, the new framework skipped the voting step as much as possible. Instead, the shape properties of muscle fibers were used to output the MFO in most cases. The time complexity for computing the orientation of the ellipse that has the same normalized second central moments of the region is O(n²), which is smaller than Hough/Radon transform for line detection in an image with the time complexity O(n³). Subsequently, the time complexity of the whole framework is related to the shape of fibers. Taking the result on the dataset used in previous study [3, 5] as an example, only a very few regions are branch-structured and therefore detected using HT in sonograms. Experiments showed that the number of lines detected using HT was only 2.5% of the total number of detected lines for the 45 sonograms. As for the 230 (30 from 10 subjects and a sequence of 200 frames from the representative subject) sonograms of gastrocnemius from normal subjects, only one line was detected using HT. In other words, in most cases our framework detects lines without the voting procedures, and HT remains only a complementary operation for branch-structured fiber.

Secondly, in our framework, candidates in each voting step were pixels of a sub-region, which is different from the previous RVHT where all pixels in the image would be involved in a voting step.

Improving computation efficiency contributes a lot to the real-time application of ultrasonic diagnosis. The benefit of improved computation speed will grow much larger when we are shifting today from dealing with individual scans to several stream of video captured during the scan procedures. The high efficiency of the method is crucial for applications in the evaluation of the muscle dynamic behavior using sonograms where massive data are involved (such as [4, 6–12]).