Research  Open  Published:
Reconstruction of freehand 3D ultrasound based on kernel regression
BioMedical Engineering OnLinevolume 13, Article number: 124 (2014)
Abstract
Introduction
Freehand threedimensional (3D) ultrasound has the advantages of flexibility for allowing clinicians to manipulate the ultrasound probe over the examined body surface with less constraint in comparison with other scanning protocols. Thus it is widely used in clinical diagnose and imageguided surgery. However, as the data scanning of freehand–style is subjective, the collected Bscan images are usually irregular and highly sparse. One of the key procedures in freehand ultrasound imaging system is the volume reconstruction, which plays an important role in improving the reconstructed image quality.
System and methods
A novel freehand 3D ultrasound volume reconstruction method based on kernel regression model is proposed in this paper. Our method consists of two steps: binfilling and regression. Firstly, the binfilling step is used to map each pixel in the sampled Bscan images to its corresponding voxel in the reconstructed volume data. Secondly, the regression step is used to make the nonparametric estimation for the whole volume data from the previous sampled sparse data. The kernel penalizes distance away from the current approximation center within a local neighborhood.
Experiments and results
To evaluate the quality and performance of our proposed kernel regression algorithm for freehand 3D ultrasound reconstruction, a phantom and an invivo liver organ of human subject are scanned with our freehand 3D ultrasound imaging system. Root mean square error (RMSE) is used for the quantitative evaluation. Both of the qualitative and quantitative experimental results demonstrate that our method can reconstruct image with less artifacts and higher quality.
Conclusion
The proposed kernel regression based reconstruction method is capable of constructing volume data with improved accuracy from irregularly sampled sparse data for freehand 3D ultrasound imaging system.
Introduction
Imageguided technique has been widely used in minimally invasive surgery which can greatly reduce the patient’s pain, improve the operation success rate and accelerate postoperative recovery. Compared with computed tomography (CT) imaging and magnetic resonance imaging (MRI), ultrasound is more suitable in intraoperative imaging for its realtime imaging advantage. Further more, ultrasound is noninvasive, nonionizing, portable, and lowcost.
As 2D ultrasound cannot provide complete volume data of tissues and organs, 3D ultrasound imaging system is put forward to overcome such limitations by constructing abundant 3D data. A number of approaches for constructing 3D ultrasound volume data have been proposed, which can be grouped into three categories: dedicated 3D probes, mechanical scanning and freehand scanning [1]. 3D probes relies on an sophisticated mechanism for scanning a predefined region of interested (ROI), and can provide 3D volume data in realtime. Nevertheless they are expensive and have limitation on scanning large organs [2]. The mechanical scanning is usually formed of a scanning head as well as a stepping motor, which move and rotate the 2D transducer and synchronously record positions and orientations data [3]. Unfortunately, the scanning range is still limited [2]. In comparison with the above approaches, freehand 3D ultrasound is relatively lowcost and flexible. The irregularly sampled Bscan images with corresponding positions and orientations are scanned and recorded by a 2D probe with transducer, which can be manipulated freely to view the desired anatomical section.
A number of algorithms for reconstructing 3D ultrasound images from these irregularly 2D Bscan images. These algorithms fall into three groups: VoxelBased Methods (VBM), PixelBased Methods (PBM) and FunctionBased Methods (FBM) [4]. VBMs traverse all voxels in a target volume and inserts corresponding pixels from the input images. An usual method of implementation this way is the Voxel Nearest Neighbor (VNN) [5]. VNN traverses each volume voxel and assigns the value of nearest image pixel. VNN method can preserve the most original texture patterns from Bscan images, but it also has a trend to generate large reconstruction artifacts when the distance of the voxel to the Bscan image plane is large. Another algorithm is the DistanceWeighted (DW) [6]. DW traverses each volume voxel and assigns the value of average its local neighborhood pixels weighted by the inverse distances between those pixels and the voxel center. DW method trends to averaging, so it has the advantage of suppress speckle noise, however, it also smooth out the 3D image boundaries. In [7], Huang et al. introduce a medianfilterbased reconstruction method to improve the quality of volume reconstruction by utilizing median filter to reduce speckle noise. In [2, 8–10], Huang et al. propose some algorithms by improving the DW algorithm. Such as the recently present a adaptive Gaussian distance weighted (AGDW) method to improve the quality of reconstructed image. The AGDW evaluates the homogeneity of the neighborhood for each voxel according to the local variance/mean of neighboring pixels. If a voxel is deemed as a homogeneous region, its neighboring pixels are averaged. Otherwise, the voxel value is calculated using an adaptive Gaussian distance weighted method with respect to the local statistics. However, the AGDW method trends to overemphasize the effect of speckle reduction, the patchlike pattern can be observed from their experimental results. The Probe Trajectory (PT) method [11] builds on the DW algorithm. Instead of using the orthogonal projection of points to the nearest Bscans, a probe trajectory is estimated and used for finding the corresponding pixels in the nearest Bscans. PBMs traverse the input pixels and insert them into the corresponding target volume voxels. it consist of two steps: a Distribution Step (DS) and a HoleFilling Step (HFS). An usual method of implementation this way is the Pixel Nearest Neighbor (PNN). In the DS, PNN traverses each pixel in all the 2D Ultrasound images, each pixel value is filled into the nearest voxel (Distribution Step). if multiple pixels contribute to the same voxel, the averaged value [12, 13], the maximum value [12], the most recent value [14] or the first value [15] are usually selected for assigned it. In the HFS, the volume from the DS has some empty voxels, each empty voxel is filled with the nearby already filled voxels. The usual method for this purpose is average or maximum [12, 16] or a median [16] of those voxels. Dewi et al. [17] propose a Holefilling algorithm by improving the Olymplic operator. The algorithm shows a better empty voxel estimation than conventional Olympic, Average, and Maximum. The PNN method causes great blurring as well as relatively large reconstruction error. In [18], a method named direct frame interpolation (DFI) method is presented to creates additional intermediate image frames by directly interpolating between two or more adjacent image frames of the original image series. Then the DFI method fills the the target volume using the original frames in combination with the additionally constructed frames. The DFI method requires the sampled Bscan slices not to be intersected. However, the intersection among the Bscan slices could not be avoided in the freehand scanning configuration. In [3], a method based on Cyclic Regularized SavitzkyGolay (CRSG) filter is introduced to reconstruct freehand 3D ultrasound volume with mechanical linear scanning sweep. However, such parallel scanning sweep is usually not available for the freehand scanning. To preserve the detail of boundaries, An Fast Marching Method (FMM) algorithm has been recently proposed by [1], it consists of DS and HFS. In DS, it similar to DS of PNN, In HFS, it advances the interpolation boundary along its normal direction and fills the area closest to known voxel points in first. it improvement in reconstruction accuracy and efficiency. FBMs utilize estimate functions by taking use of the input data for creating the voxel grid. Rohling et al. [6] propose the Radial Basis Function (RBF) interpolation is an approximation with splines that tries to use the underlying shape of the data in the volume reconstruction. Overfitting phenomenon is usually unavoidable for the Spline method. Sanches and Marques [19] present the Rayleigh reconstruction/interpolation with a Bayesian framework estimates a function for the tissue by statistical methods. The Rayleigh method also trend to suppress spckle noise but smooth out the boundaries.
This paper develops a new freehand 3D ultrasound reconstruction algorithm based on kernel regression (KR) aims to suppress speckle noise and preserve boundaries. its corresponding ideas come from nonparametric estimation [20]. Nonparametric estimation has various applications, such as 2D image denoising, upscaling, interpolation and fusion [21], feature extraction for 3D MR image denoising [22]. In the following, we demonstrate the details of KR applied in freehand 3D ultrasound reconstruction.
System and methods
System
The freehand 3D ultrasound system consists of three modules: a conventional 2D ultrasound scanner (DC7, Mindray Medical International Ltd., Shenzhen, China) used to acquire ultrasound image, an electromagnetic spatial sensing device (Aurora [23], NDI, Ontario, Cannada) acquiring the position and orientation of ultrasound images and surgical instruments, and a workstation with customdesigned software used for data collection, volume reconstruction, and visualization [1]. Figure 1 illustrate this system framework. The spatial information (position (x,y,z) and orientation (R _{ x },R _{ y },R _{ z })) of ultrasonic probe and surgical instrument embeding sensor is recorded by Aurora system connects to the workstation through its USB port so that it can be acquired by a customdesigned software by use of Aurora system API. Besides, the realtime ultrasound video is captured by a video capture card (RGB133, VTimage Inc., Shenzhen, China) installed in the workstation.
Data collection
Before volume reconstruction, data collection is a very important step which influences accuracy and efficiency of reconstruction. Data collection consists of two steps: calibration between spatial data and 2D Bscan images, and selecting ROI (region of interest). In this paper, the set of Bscan image I _{ i } and its position T _{ i } are collected by the data collection method of [1].
Volume reconstruction
A 3D volume data is reconstructed from the collected data, including the 2D Bscan images and its spatial information. In this study, the algorithm for volume reconstruction is composed of two stages: binfilling and regression.
Binfilling stage
The binfilling stage is to map the pixel in 2D Bscans into the voxel in 3D volume data based on its corresponding positional information. In this freehand 3D ultrasound system, the mapping of the coordinate system from the 2D Bscans to 3D volume is named after forward mapping and is defined as
where V _{ p } is the physical position, M is the forward transformation matrix, and V _{ r } is the resulting voxel location in the reconstructed 3D volume data. The forward transformation matrix M must be decomposed and implemented to find an matrix for this transformation, which is discussed in detail in [1].
Regression stage
Since the scanning of freehandstyle is subjective, the collected Bscan images are usually irregular and highly sparse. Therefore, there are some gaps in the reconstructed volume after the binfilling stage, as addressed in Figure 2(a). The goal of the regression stage is to make the nonparametric estimation [20] for the whole volume data from the previous sparse volume data.
The sparse volume data after the binfilling stage are given by
where r(·) is the regression function, X _{ i } = (X _{ i 1},X _{ i 2},X _{ i 3})^{T} is the threedimension coordinate of data, ε _{ i }s are the independent and identically distributed zero mean noise values.
Specifically, if X is near the sample at X _{ i }, we can approximate it with a Nterm Taylor series
A leastsquares formulation capturing this idea is to solve the folowing optimization problem:
where K(·) is the kernel function which penalizes distance away from the local position where the approximation is centered, and the smoothing parameter h (bandwidth) controls the strength of this penalty. In particular, the function K is a symmetric function which attains its maximum at zero, satisfying
where c is a constant value. The choice of the particular form of the function K is usually Gaussian, exponential, or other forms, which comply with the above constraints. Because the choice of the kernel has little impact on the accuracy of estimation. Therefore, the Gaussian kernel, being computational complexity, is selected in this paper.
For the estimation problem based on Least Square Method upon showed in Equation 4, the order N effect the accuaracy and complexity of local approximation of the volume data. Therefore, it must be appropriately chosen. In the nonparametric statistics literature, locally constant, linear, and quadratic approximations (corresponding to N=0,1,2) have been considered most widely [24–27].
The Kernel function K is now a function of 3 variables. Given a nonsingular positive definite 3 × 3 bandwidth matrix H, which is defined
Often, one scales each covariate to have the same mean and variance, then use the kernel
where K is any onedimensional kernel. therefore, there is a single bandwidth parameter h. At a target value X = (X _{1},X _{2},X _{3})^{T}, the local sum of squares is given by
where
The estimator is
where $\hat{\beta}={\left(\hat{{\beta}_{0}},\cdots ,\hat{{\beta}_{3}}\right)}^{T}$ is the value of β = (β _{0},⋯,β _{3})^{T} that minimizes the weighted sums of squares. The solution $\hat{\beta}$ is
where
and W _{ x } is the diagonal matrix whoses (i,i) element is w _{ i }(X). Therefore,
where e _{1} is a column vector with all elements equal to zero but the first element equal to one, $\hat{r}(X)$ is the final values of reconstructed volume data.
Experiments and results
To evaluate the quality and performance of our proposed KR algorithm for freehand 3D ultrasound reconstruction, a phantom and an invivo liver organ of human subject are scanned with our freehand 3D reconstruction system. The parameters of the order N, the size of kernel R and the bandwidth h of KR are assigned with 1, 15 and 0.5 respectively. The proposed method is compared with the classical VNN, PNN, Spline and PT methods and the recent FMM method. The accuracy of the reconstruction results is evaluated via root mean square error (RMSE). The running time of different methods on all scanned datasets are given for their performance and efficiency comparison. All of the methods are implemented in C++, and the computer is equipped with a Intel Core i32120 3.3 GHz and 2 GB RAM.
Qualitative results
The first experiment is conducted on an abdominal phantom (CIRS Model 057), which is made from proprietary materials to accurately mimic human tissues under MRI, CT, and ultrasound. The phantom mainly contains simulated lungs, liver, hepatic vesels, ribs, vertebra, kidneys, and abdominal aorta. The freehand scan of the phantom is performed with a 4.5 MHz probe. 102 Bscan slices are collected. Each Bscan is cropped to 400 × 400 pixels. A depth setting of 200 × 200 mm is used giving a resolution of 0.5 mm/pixel. Figure 3 shows the phantom picture, its inner structure and the reconstructed volume data with our freehand 3D ultrasound reconstruction system.The second experiment object is an invivo liver organ of human, 167 Bscan slices are collected. each Bscan is cropped to 347 × 242 pixels. A depth setting of 177 × 123 mm is used giving a resolution of 0.511945 mm/pixel. Figure 2 shows the binfilled volume without interpolation and the reconstructed volume using our KR algorithm.
The coronal, sagittal and transverse slice of the phantom and liver using the VNN, PNN, Spline, PT, FMM and KR algorithm are showed in Figure 4 and Figure 5. According to Figure 4(b) and Figure 5(b), the reconstructed slices using the VNN algorithm look sharp and present more texture patterns. However, the anatomical structure is actually distorted due to the misalignments of the pixels, as addressed by [6]. Figure 4(c) and Figure 5(c) show the PNN method causes great blurring as well as relatively large reconstruction error. Overfitting phenomenon is usually unavoidable for the Spline method, as demonstrated in Figure 4(d) and Figure 5(d). Figure 4(e) and Figure 5(e) show the PT method preserve more texture details. However, there are some gaps in the sagittal slice. Figure 4(f) and Figure 5(f) show the recent FMM method performs much better than the classical methods. However, it can’t preserve enough details in the big gap. Compared with the classical methods and the recent FMM method, our KR algorithm has advantages both in recovering missing data with more details and suppressing speckle noises, as demonstrated in Figure 4(g) and Figure 5(g).
Quantitative results
For the quantitative evaluation, we deliberatly remove a slice from the collected BScan slices. Then different reconstruction algorithms are applied to the removed data. Such evaluation method is capable of measuring the ability of a reconstruction algorithm in preserving true intensity values at the locations where a part of original data is removed. A good reconstruction algorithm should interpolate the removed data with values very close to the original data. In this study, root mean square error (RMSE) [28] is used for the image quality measure. It can be described as follow:
where I _{ o } is the removed slice from Bscan images, I _{ r } is an image resliced the reconstructed volume with I _{ o }’ position. (x,y) is the coordinate of pixels.
Table 1 and Table 2 summarize the RMSE of the evaluation tests using the VNN, PNN, Spline, PT, FMM and our KR reconstruction algorithms. The RMSE of our KR algorithm is smallest among these reconstruction mehods. Figure 6 and Figure 7 show that the KR based reconsturcted slice is more close to the original slice.
Computation complexity analysis
In order to perform an objective and theoretical estimations to the computation time, the big O notation is used to identify the complexity function. Table 3 lists the computational time complexity of VNN, PNN, Spline, PT, FMM and our KR approaches.
The practical time costs are demonstrated in Table 4 for the phantom test (with a dimension of 424×421×131) and Table 5 for the liver test (with a dimension of 387×338×350). From Table 4 and Table 5, the KR takes more computation time than VNN, PNN, Spline, PT, and FMM algorithms.
Conclusion
In this paper, A novel freehand 3D ultrasound system based on kernel regression is proposed. A phantom and an invivo liver organ of human subject are scanned and reconstructed with this system for experiments. Root mean square error (RMSE) is used for the quantitative evaluation. Both of the qualitative and quantitative experimental results have demonstrated that our method can reconstruct image with less artifacts and higher quality than the classical VNN, PNN, Spline, PT methods and the recent FMM method. Therefore, the proposed kernel regression based reconstruction method is capable of constructing volume data with improved accuracy from irregularly sampled sparse data for freehand 3D ultrasound imaging system. Of course, the practical time cost experiments show it costs more computation time. So the time complexity is required to improve for higher efficiency in freehand 3D ultrasound reconstruction in the future work. More over, in respect to quality, automatic and dynamic bandwidth will be studied for spatial adaptation.
Abbreviations
 2D:

Twodimensional
 3D:

Threedimensional
 VBM:

Voxelbased methods
 PBM:

Pixelbased methods
 FBM:

Functionbased methods
 VNN:

Voxel nearest neighbor
 PNN:

Pixel nearest neighbor
 DW:

Distanceweighted
 FMM:

Fast marching method
 PT:

Probe trajectory
 KR:

Kernel regression
 RBF:

Radial basis function
 ROI:

Region of interest.
References
 1.
Wen T, Zhu Q, Qin W, Li L, Yang F, Xie Y, Gu J: An accurate and effective fmmbased approach for freehand 3d ultrasound reconstruction. Biomed Signal Process Control 2013,8(6):645–656. 10.1016/j.bspc.2013.05.009
 2.
Huang QH, Zheng YP, Lu MH, Chi Z: Development of a portable 3d ultrasound imaging system for musculoskeletal tissues. Ultrasonics 2005,43(3):153–163. 10.1016/j.ultras.2004.05.003
 3.
Toonkum P, Suwanwela NC, Chinrungrueng C: Reconstruction of 3d ultrasound images based on cyclic regularized savitzky–golay filters. Ultrasonics 2011,51(2):136–147. 10.1016/j.ultras.2010.07.003
 4.
Solberg OV, Lindseth F, Torp H, Blake RE, Nagelhus Hernes TA: Freehand 3d ultrasound reconstruction algorithmsa review. Ultrasound Med Biol 2007,33(7):991–1009. 10.1016/j.ultrasmedbio.2007.02.015
 5.
Sherebrin S, Fenster A, Rankin RN, Spence D: Freehand threedimensional ultrasound: implementation and applications. In Medical Imaging 1996. Bellingham: International Society for Optics and Photonics; 1996:296–303.
 6.
Rohling R, Gee A, Berman L: A comparison of freehand threedimensional ultrasound reconstruction techniques. Med Image Anal 1999,3(4):339–359. 10.1016/S13618415(99)800280
 7.
Huang QH, Zheng YP: Volume reconstruction of freehand threedimensional ultrasound using median filters. Ultrasonics 2008,48(3):182–192. 10.1016/j.ultras.2007.11.005
 8.
Huang QH, Zheng YP: An adaptive squareddistanceweighted interpolation for volume reconstruction in 3d freehand ultrasound. Ultrasonics 2006, 44: 73–77. 10.1016/j.ultras.2005.08.004
 9.
Huang Q, Lu M, Zheng Y, Chi Z: Speckle suppression and contrast enhancement in reconstruction of freehand 3d ultrasound images using an adaptive distanceweighted method. Appli Acoustics 2009,70(1):21–30. 10.1016/j.apacoust.2008.02.002
 10.
Huang Q, Zheng Y, Lu M, Wang T, Chen S: A new adaptive interpolation algorithm for 3d ultrasound imaging with speckle reduction and edge preservation. Comput Med Imaging Graph 2009,33(2):100–110. 10.1016/j.compmedimag.2008.10.006
 11.
Coupé P, Hellier P, Morandi X, Barillot C: Probe trajectory interpolation for 3d reconstruction of freehand ultrasound. Med Image Anal 2007,11(6):604–615. 10.1016/j.media.2007.05.002
 12.
Nelson TR, Pretorius DH: Interactive acquisition, analysis, and visualization of sonographic volume data. Int J Imaging Syst Technol 1997,8(1):26–37. 10.1002/(SICI)10981098(1997)8:1<26::AIDIMA4>3.0.CO;2V
 13.
Gobbi D, Peters T: Interactive intraoperative 3d ultrasound reconstruction and visualization. Med Image Comput ComputerAssisted InterventionMICCAI 2002, 2489: 156–163.
 14.
Ohbuchi R, Chen D, Fuchs H: Incremental volume reconstruction and rendering for 3d ultrasound imaging. In Visualization in Biomedical Computing. Bellingham: International Society for Optics and Photonics; 1992:312–323.
 15.
Trobaugh JW, Trobaugh DJ, Richard WD: Threedimensional imaging with stereotactic ultrasonography. Comput Med Imaging Graph 1994,18(5):315–323. 10.1016/08956111(94)900027
 16.
Estépar R, MartínFernández M, AlberolaLópez C, Ellsmere J, Kikinis R, Westin CF: Freehand ultrasound reconstruction based on roi prior modeling and normalized convolution. Medic Image Comput ComputerAssisted InterventionMICCAI 2003,6(Pt 2):382–390.
 17.
Dewi D, Wilkinson M, Mengko T, Purnama I, van Ooijen P, Veldhuizen A, Maurits N, Verkerke G: 3d ultrasound reconstruction of spinal images using an improved olympic holefilling method. In Instrumentation, Communications, Information Technology, and Biomedical Engineering (ICICIBME), 2009 International Conference On. Bandung: IEEE; 2009:1–5.
 18.
Scheipers U, Koptenko S, Remlinger R, Falco T, Lachaine M: 3d ultrasound volume reconstruction using the direct frame interpolation method. Ultrason Ferroelectr Freq Control IEEE Trans 2010,57(11):2460–2470.
 19.
Sanches JM, Marques JS: A rayleigh reconstruction/interpolation algorithm for 3d ultrasound. Pattern Recognit Lett 2000,21(10):917–926. 10.1016/S01678655(00)000532
 20.
Fan J: Local linear regression smoothers and their minimax efficiencies. Annals Stat 1993,21(1):196–216. 10.1214/aos/1176349022
 21.
Takeda H, Farsiu S, Milanfar P: Kernel regression for image processing and reconstruction. Image Process IEEE Trans 2007,16(2):349–366.
 22.
LópezRubio E, FlorentínNúñez MN: Kernel regression based feature extraction for 3d mr image denoising. Med Image Anal 2011,15(4):498–513. 10.1016/j.media.2011.02.006
 23.
Triple Modality 3D Abdominal Phantom Model 057 [ http://www.cirsinc.com/products/all/65/triplemodality3dabdominalphantom/]
 24.
Hardle W: Applied Nonparametric Regression, vol. 5. Cambridge England and New York: Cambridge University Press; 1990.
 25.
Härdle W: Smoothing Techniques: with Implementation in S. New York: Springer; 1991.
 26.
Wand MMP, Jones MC: Kernel Smoothing, vol. 60. UK: Crc Press; 1995.
 27.
Hardle W: Nonparametric and semiparametric models. Springer, Verlag Berlin and Heidelberg GmbH & Co. K; 2012.
 28.
Wang Z, Bovik AC: A universal image quality index. Signal Process Lett IEEE 2002,9(3):81–84.
Acknowledgements
This study was financed partially by the National Natural Science Foundation of China (NSFC: 61401451, 61472411, 61103165, 81171402), the National 863 Program of China (Grant No. 2012AA02A604), the Next generation communication technology Major project of National S&T (Grant No. 2013ZX03005013), the Key Research Program of the Chinese Academy of Sciences, and the Guangdong Innovation Research Team Funds for ImageGuided Therapy and Lowcost Healthcare.
Author information
Additional information
Competing interests
Chen is interested in computer graphics and image processing. His research focuses on medical image processing.
Authors’ contributions
XKC carried out the algorithm design and implementation, and drafted the manuscript. TXW participated in the design of the study and the system framework. WJQ, DL, WZP, XL and JG conceived of the study, and contributed to discussions and suggestions to complete the manuscript. All authors read and approved the final manuscript.
Xiankang Chen, Tiexiang Wen contributed equally to this work.
Rights and permissions
About this article
Received
Accepted
Published
DOI
Keywords
 Freehand ultrasound
 Kernel regression
 Reconstruction
 Interpolation
 Nonparametric statistics