Tendon-motion tracking in an ultrasound image sequence using optical-flow-based block matching
© The Author(s) 2017
Received: 24 October 2016
Accepted: 30 March 2017
Published: 20 April 2017
Tendon motion, which is commonly observed using ultrasound imaging, is one of the most important features used in tendinopathy diagnosis. However, speckle noise and out-of-plane issues make the tracking process difficult. Manual tracking is usually time consuming and often yields inconsistent results between users.
To automatically track tendon motion in ultrasound images, we developed a new method that combines the advantages of optical flow and multi-kernel block matching. For every pair of adjacent image frames, the optical flow is computed and used to estimate the accumulated displacement. The proposed method selects the frame interval adaptively based on this displacement. Multi-kernel block matching is then computed on the two selected frames, and, to reduce tracking errors, the detailed displacements of the frames in between are interpolated based on the optical flow results.
In the experiments, cadaver data were used to evaluate the tracking results. The mean absolute error was less than 0.05 mm. The proposed method also tracked the motion of tendons in vivo, which provides useful information for clinical diagnosis.
The proposed method provides a new index for adaptively determining the frame interval. Compared with other methods, the proposed method yields tracking results that are significantly more accurate.
KeywordsTendon tracking Ultrasound Optical flow Block matching
A tendon is a band of fibrous tissue that connects muscle to bone. Muscle contraction pulls the tendon, which causes the limbs to move; thus, tendon motion is important for evaluating the status of limb and joint functions. Tendinopathy of the finger, such as trigger finger (a.k.a. stenosing tenosynovitis), has become a frequent occupational disease in recent decades. A patient with trigger finger will need surgical treatment if the symptom is at a serious stage. The first annular pulley (A1 pulley) will be cut off to increase the sliding space of the tendon. The percutaneous release technique developed by Lorthioir  uses a specially designed knife to divide the pulley. Jou et al.  also proposed a new ultrasound-assisted minimally invasive surgical technique to increase the safety of percutaneous release. In these surgical techniques, ultrasound imaging is used to observe the tendon position and appearance. Some studies report that tendinopathy changes tendon behavior. Klauser et al.  found a significant difference in tendon stiffness between patients with diseased Achilles tendons and healthy controls. Sahu et al.  also found that tendinopathy limits the motion of tendons.
In the past decade, optical flow and block matching have been the two most frequently used methods to track tissue using ultrasound [8–17]. Zahnd et al.  and Lai et al.  proposed the Kalman-based block matching method for carotid and tendon motion. Ayvali and Desai  and Tenbrinck et al.  applied the optical flow to track the motion of needle head and left ventricle. Furthermore, Barbosa et al.  developed a tracking method that combines optical flow and block matching for left ventricle motion in 4-D ultrasound sequences. Korstanje et al.  developed a multi-kernel ultrasound speckle tracking method to quantify tendon displacement and reduce the tracking error.
Although block matching and the optical flow method were used to track a moving target in ultrasound images in the above referenced studies, there are still several challenges that need to be resolved. First, the tracking error in the block matching method will increase over time . In the block matching method, the sub-pixel displacement between frames usually generates a small amount of template error during template updating. This error will accumulate over time and will make the tracking results dissimilar to the initial tracking template. Thus, a template updating procedure should be used for appropriate frames to lower the tracking error. Second, the displacement cannot be too large when using the optical flow method . Because of the intensity consistency constraint, the optical flow method will underestimate the displacement if the motion is too large. Finally, the tracking frame intervals in the block matching process can also affect the tracking results significantly and must be set. Dilley et al.  reported that different frame intervals affected the cross-correlation tracking method. In their cases, increasing the frame interval improved the accuracy of the tracking results for slower velocity data. Nevertheless, they did not provide a mechanism to automatically adjust the frame interval during image tracking. Although many methods use adjacent frames or constant frame intervals in block matching, an adaptive frame interval based on the properties of a tracked image sequence will make speckle tracking more flexible and accurate.
In this research, we propose an optical-flow-trend-based multi-kernel block matching (OFTB-MKBM) method that combines block matching with optical flow methods for automatic speckle tracking. The OFTB-MKBM is intended to provide a new index for adaptively determining the frame interval. Optical flow and MKBM methods are used to compare the tracking accuracy with the OFTB-MKBM, and adaptive MKBM using linear interpolation rather than the optical flow trend is evaluated to illustrate the effectiveness of this new method.
Optical flow method
Multi-kernel block matching
Block matching is a detection and tracking method in image processing. It is used to compute the similarity between a reference block and a target block. However, block matching is sensitive to speckle noise. The speckle noise is the small scale brightness variations of speckle which affect the tracking results when the variations are significant.
Korstanje et al.  proposed an MKBM scheme to solve this problem. MKBM is a multi-kernel block matching method that separates the reference block into several sub-blocks. Each sub-block is initially examined using the block matching method to find the block that is the closest match. The matching results of all sub-blocks are then combined to obtain the overall matching result. By utilizing the multiple block matching, MKBM computes the normalized-cross-correlation (NCC) weighted average as the tracking result which is less affected by the speckle variations. However, MKBM still cannot perform well if the motion of tracking target is too small.
Optical-flow-trend-based multi-kernel block matching
In the implementation, we chose the number of flow points (N) conveniently based on V a . For example, if the average displacement of the top 5% flow points is 1.5 pixels (V a = 1.5), the accurate displacement should be computed using the top 20% flow points (N = 20). The region displacement can then be obtained by averaging the displacements from the specific number of flow points.
Results and discussion
Determining window size
Threshold of accumulated displacement
Validating accuracy using standard ultrasound phantom
Validating accuracy using cadaver data
The error metrics of the proposed results for cadaver (1 pixel = 0.0265 mm)
OFTB-MKBM results of in vivo data
Comparisons with other methods
Error metrics of the four methods compared with the ground truth
Average absolute error Ea (pixels)
Elbow (1 pixel = 0.075 mm)
Finger (1 pixel = 0.0265 mm)
We proposed a new ultrasound image tendon-tracking algorithm (Additional file 10). The OFTB-MKBM and MKBM methods were used to track the tendon motions in an elbow and in a finger. The accuracy of the proposed method was validated. Moreover, our proposed method yielded better tracking results than did the traditional optical flow and MKBM methods. The results interpolated based on optical flow were also better than were those of the adaptive MKBM method.
- A1 pulley:
first annular pulley
flexor digitorum superficialis
flexor digitorum profundus
optical-flow-trend-based multi-kernel block matching
multi-kernel block matching
sum of absolute difference
BIC developed the algorithm method, carried out all experiments, and drafted the manuscript. JHH developed the algorithm method and carried out all experiments. LC provided anatomical knowledge and experimental consulting. IMJ provided clinical setup, consultation, and experiments. FCS designed the data acquisition protocol, provided anatomical knowledge, and support clinical experiments. YNS suggested and designed the algorithms, arranged experiments, and helped to draft the manuscript. All authors read and approved the final manuscript.
The authors would like to thank Shyh-Hau Wang, Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan, for providing the CIRS tissue-mimicking phantom data, and Medical Device Innovation Center (MDIC), National Cheng Kung University, for supporting the ultrasound devices. We would also like to thank Shu-Ya Li and Yuh-Ping Tsai, Department of Biomedical Engineering, National Cheng Kung University, for their helps in the ultrasound data acquisition.
The authors declare that they have no competing interests.
Availability of data and materials
Consent for publication
Authorization has been granted to publish the results of the tests.
Ethics approval and consent to participate
All participants provided written informed consent to participate in the study and to allow their data to be used for the study purpose.
This work was supported by MOST grant 104-2221-E-006-097-MY3 from the Taiwan Ministry of Science and Technology.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Lorthioir J. Surgical treatment of trigger-finger by a subcutaneous method. J Bone Joint Surg Am. 1958;40:793–5.View ArticleGoogle Scholar
- Jou IM, Chern TC. Sonographically assisted percutaneous release of the a1 pulley: a new surgical technique for treating trigger digit. J Hand Surg: Br Eur. 2006;31(2):191–9.View ArticleGoogle Scholar
- Klauser AS, Faschingbauer R, Jaschke WR. Is sonoelastography of value in assessing tendons? Seminars in musculoskeletal radiology. Stuttgart: Thieme Medical Publishers; 2010.Google Scholar
- Sahu RL, Gupta P. Experience of percutaneous trigger finger release under local anesthesia in the Medical College of Mullana, Ambala, Haryana. Ann Med Health Sci Res. 2014;4(5):806–9.View ArticleGoogle Scholar
- Zhou Y, et al. Dynamic measurement of pennation angle of gastrocnemius muscles during contractions based on ultrasound imaging. Biomed Eng Online. 2012;11(1):1.MathSciNetView ArticleGoogle Scholar
- Masum MA, et al. Accuracy assessment of Tri-plane B-mode ultrasound for non-invasive 3D kinematic analysis of knee joints. Biomed Eng Online. 2014;13(1):1.View ArticleGoogle Scholar
- Gupta R, et al. Curvelet based automatic segmentation of supraspinatus tendon from ultrasound image: a focused assistive diagnostic method. Biomed Eng Online. 2014;13(1):1.View ArticleGoogle Scholar
- Zahnd G, et al. Evaluation of a Kalman-based block matching method to assess the bi-dimensional motion of the carotid artery wall in B-mode ultrasound sequences. Med Image Anal. 2013;17(5):573–85.View ArticleGoogle Scholar
- Lai TY, et al. Application of a novel Kalman filter based block matching method to ultrasound images for hand tendon displacement estimation. Med Phys. 2016;43(1):148–58.View ArticleGoogle Scholar
- Ayvali E, Desai JP. Optical flow-based tracking of needles and needle-tip localization using circular hough transform in ultrasound images. Ann Biomed Eng. 2015;43(8):1828–40.View ArticleGoogle Scholar
- Tenbrinck D, et al. Histogram-based optical flow for motion estimation in ultrasound imaging. J Math Imag Vision. 2013;47(1-2):138–50.MathSciNetView ArticleMATHGoogle Scholar
- Barbosa D, et al. Fast tracking of the left ventricle using global anatomical affine optical flow and local recursive block matching. Proceedings of the MICCAI Challenge on Endocardial Three-dimensional Ultrasound Segmentation-CETUS (2014): 17–24.
- Korstanje JWH, et al. Development and validation of ultrasound speckle tracking to quantify tendon displacement. J Biomech. 2010;43(7):1373–9.View ArticleGoogle Scholar
- Matsuzawa R, et al. Monitoring of lesion induced by high-intensity focused ultrasound using correlation method based on block matching. Jpn J Appl Phys. 2012;51:0726.View ArticleGoogle Scholar
- Siddig AMA, Yousif RK, Alanwer M. Motion estimation in ultrasound image using dynamic multi-shape search. International Conference on Computing, Electrical and Electronics Engineering (ICCEEE), 2013 IEEE; 2013.
- Li J, et al. Estimation and visualization of longitudinal muscle motion using ultrasonography: a feasibility study. Ultrasonics. 2014;54(3):779–88.View ArticleGoogle Scholar
- Karamanidis K, et al. Use of a Lucas–Kanade-Based template tracking algorithm to examine in vivo tendon excursion during voluntary contraction using ultrasonography. Ultrasound Med Biol. 2016;42(7):1689–700.View ArticleGoogle Scholar
- Alvarez L, Weickert J, Sánchez J. Reliable estimation of dense optical flow fields with large displacements. Int J Comput Vision. 2000;39(1):41–56.View ArticleMATHGoogle Scholar
- Dilley A, et al. The use of cross-correlation analysis between high-frequency ultrasound images to measure longitudinal median nerve movement. Ultrasound Med Biol. 2001;27(9):1211–8.View ArticleGoogle Scholar
- Gibson JJ. The perception of the visual world. 1950.
- Lucas BD, Kanade T. An iterative image registration technique with an application to stereo vision. New York: IJCAI; 1981.Google Scholar