A 3D Freehand Ultrasound System for Multi-view Reconstructions from Sparse 2D Scanning Planes
© Yu et al; licensee BioMed Central Ltd. 2011
Received: 7 October 2010
Accepted: 20 January 2011
Published: 20 January 2011
A significant limitation of existing 3D ultrasound systems comes from the fact that the majority of them work with fixed acquisition geometries. As a result, the users have very limited control over the geometry of the 2D scanning planes.
We present a low-cost and flexible ultrasound imaging system that integrates several image processing components to allow for 3D reconstructions from limited numbers of 2D image planes and multiple acoustic views. Our approach is based on a 3D freehand ultrasound system that allows users to control the 2D acquisition imaging using conventional 2D probes.
For reliable performance, we develop new methods for image segmentation and robust multi-view registration. We first present a new hybrid geometric level-set approach that provides reliable segmentation performance with relatively simple initializations and minimum edge leakage. Optimization of the segmentation model parameters and its effect on performance is carefully discussed. Second, using the segmented images, a new coarse to fine automatic multi-view registration method is introduced. The approach uses a 3D Hotelling transform to initialize an optimization search. Then, the fine scale feature-based registration is performed using a robust, non-linear least squares algorithm. The robustness of the multi-view registration system allows for accurate 3D reconstructions from sparse 2D image planes.
Volume measurements from multi-view 3D reconstructions are found to be consistently and significantly more accurate than measurements from single view reconstructions. The volume error of multi-view reconstruction is measured to be less than 5% of the true volume. We show that volume reconstruction accuracy is a function of the total number of 2D image planes and the number of views for calibrated phantom. In clinical in-vivo cardiac experiments, we show that volume estimates of the left ventricle from multi-view reconstructions are found to be in better agreement with clinical measures than measures from single view reconstructions.
Multi-view 3D reconstruction from sparse 2D freehand B-mode images leads to more accurate volume quantification compared to single view systems. The flexibility and low-cost of the proposed system allow for fine control of the image acquisition planes for optimal 3D reconstructions from multiple views.
There is a strong interest in developing effective 3D ultrasound imaging systems in ultrasonography. The basic advantage of 3D systems is that they enable us to provide quantitative measurements of different organs without assuming simplified geometrical models associated with conventional 2D systems. Here, our focus is on developing a 3D ultrasound system for accurate 3D reconstructions from arbitrary scanning geometries (freehand) that are validated on calibrated 3D targets.
Most research in 3D echocardiography is focused on the use of 3D probes (volume-mode probes) where 3D volume is imaged directly from a single probe position. This approach simplifies the reconstruction and visualization of the 3D data set since the geometry of acquired slices is known and real-time 3D reconstruction is possible. This approach does not allow for fine control of the location of the 2D planes.
Clinically, 3D probes have not been widely adopted. The overwhelming majority of ultrasound exams are still using standard 2D ultrasound probes. Routine clinical diagnosis is still depended on acquiring optimal 2D acoustic views. On the other hand, 3D ultrasound can be used by non-experts to avoid the need for training on how to acquire optimal 2D views . In current clinical practice, 3D datasets are often communicated to expert readers that will then have to extract optimal 2D views that are needed for documenting clinical diagnosis.
Robust system components:
Several required image processing system components (see Figure 2) are often treated separately in the literature. In this paper, we address several system issues for reliable performance (see preliminary results in [8–10]). Successful 3D reconstruction in the system begins with reliable electromagnetic interference detection for accurate 3D position and orientation sensing. It also requires accurate 2D to 3D calibration. A hybrid active contour segmentation and parameter optimization is used to develop a robust segmentation method. It is important to note that we segment each 2D plane independently. Due to sparse sampling, 3D segmentation methods are not applicable here. Following segmentation, a robust, coarse to fine, multi-view registration method is used for registering multiple 3D volumes. Here, the registration is based on the 3D geometric shape, and does not depend on the varied gray-scale intensities of different view acquisitions.
Reconstructions from sparse acquisition geometries:
We are particularly interested in quantifying reconstruction error as a function of the number of acquired ultrasound image planes. The use of a limited number of planes can achieve acceptable 3D accuracy and speed up the clinical examination. It also allows fast screening of normal cases. In our system, 2D images are acquired from different acoustic windows during routine clinical diagnosis. The sparsely sampled 2D images are automatically registered into 3D volumes as opposed to the use of registering densely sampled 3D images in 3D real-time echocardiography.
There is relatively limited research on the use of multiple views with automated registration. Yet, a large number of imaging artefacts are associated with single view ultrasound image acquisition. As an example, the presence of shadows due to ribs and lungs can significantly limit our imaging capability . Furthermore, anatomical structures produce weak reflections or no reflections when they are parallel to the ultrasound beam. In such cases, almost no echo energy is reflected back to the transducer. These limitations can be addressed through the use of multiple acoustic windows and views where the ultrasound beam can propagate behind obstructions while imaging organ interfaces at the directions that are not parallel to the ultrasound beam.
In our 3D freehand ultrasound prototype system, an electromagnetic position and orientation measurement device (EPOM) is attached on a conventional 2D clinic ultrasound probe (see Figure 1). This allows 3D reconstruction from arbitrary sampling geometries and multiple acoustic windows. It provides a simple hardware system that allows for great flexibility in choosing suitable acoustic windows according to clinical practice. Once a region of interest has been identified, a freehand 3D system allows the experts to take very dense samples around the abnormality from an appropriate acoustic window and view, providing very accurate reconstructions in the region of interest.
Interest in the use of multiple views for providing 3D reconstructions has been primarily focused on reconstructions of the left ventricle. Legget et al.  used a 2D freehand scanning protocol and manual registration to combine parasternal and apical windows in-vitro. A similar system was reported by Leotta et al.. Later, Ye et al.  used a 3D rotational probe and an electromagnetic spatial locator to combine apical long-axis view and parasternal short-axis view. The reconstructions are fused by features and weighted by the image acquisition geometry. Reconstruction was limited by the lack of automated registration. Good spatial alignment of the dense rotational sweeps between different views was assumed. Long acquisition time for each view (3-4 minutes in ) may result in unstable motion of the probe.
For ultrasound image registration, Rohling et. al. developed an automatic registration method in gall bladder reconstruction . Six, slightly different sweeps were collected and the first sweep is used as baseline. Spatial compounding was performed by registering the last five sweeps to the first baseline based on the estimation of the cross-correlation of 3D gradient magnitude  or the usage of landmarks . A review of cardiac image registration among multiple imaging modalities is available in . As stated in , due to the varied image quality associated with cardiac ultrasound images, there are few publications focused on echocardiography image registration [16–18]. Mutual information methods also presented difficulties associated with ultrasound image characteristics .
For real-time 3D echocardiography registration, Soler et al.  used manual marked and segmented meshes of the left ventricle to register two different views from apical window by intensity similarity measure. Grauet al.  registered parasternal and apical views using phase and orientation similarity measures. Their method relied on the use of manual landmark initialization. Here, we note that effective 3D ultrasound registration cannot be based on image intensity alone due to large intensity variations within the same tissue structures and between the different views.
Unlike prior research based on the use of dense 3D samples, this paper is based on the use of sparse 2D planes. We develop a fully-automated registration without manual initialization. To the best of our knowledge, no such research has been reported in the literature. To achieve reliable registration performance, we use a coarse to fine volumetric registration method. We initialize searching for the global optimal registration parameters using a 3D Hotelling transform to construct a reference frame to coarsely register 3D volumes from different acoustic windows. Then, feature-based high accuracy registration is performed using a robust, non-linear least squares algorithm.
Automatic feature segmentation is carried before multi-view registration. Automatic segmentation techniques of echocardiographic images face a number of challenges due to poor contrast, high-level speckle noise, weak endocardial boundaries and small boundary gaps. Recently, promising segmentation results have been obtained using methods based on deformable models. There are mainly parametric and geometric deformable models . In parametric models, the evolving segmentation curve is explicitly represented. In geometric models, the evolving segmentation curve is implicitly expressed as level sets of a higher dimensional scalar function. Unlike parametric models, geometric deformable models can handle topological changes automatically and be easily extended to higher dimensional applications.
Parametric deformable models have been used in semi-automatic segmentation for both the epicardial and endocardial borders over the entire cardiac cycle [22–24]. Here, statistical models of the cardiac structure features (e.g. shape, intensity appearance and temporal information) were derived from large training data sets for segmenting endocardial boundaries [25–28]. For these methods, we note that there is significant overhead associated with providing large and appropriate training data sets and also significant efforts in setting up the point correspondences.
In clinical practice and especially in paediatric cardiology, there can be significantly topological variability associated with ventricle wall boundaries. Due to the wide variability in possible abnormal cases, it is difficult to provide significant populations for each abnormal classification. This difficulty further limits the applicability of parametric model-based approaches. More recently, geometric level set models have been developed to address these limitations. These models can handle topological changes automatically without the need for extensive parameter training. Level set methods have been used for echocardiography image segmentation . Other variational level set segmentation strategies also integrate prior knowledge (shape, statistical distribution etc.) [30–32]. However, the use of prior information often requires off-line training. It can be tedious and expert-dependent.
Alternatively, Corsi et al.  developed a semi-automatic level set segmentation method that did not require prior knowledge. The authors applied the method to real-time 3D echocardiography images for reconstructing the left ventricle. In this study, the initial surface had to be chosen close to the boundaries of the LV chamber.
We describe a new image segmentation method that relaxes the need for accurate initialization. Here, we are proposing a two-step approach. After a rough initialization, we first use a gradient vector flow (GVF) geodesic active contour (GAC) model to move the initial contour closer to the true boundary. This is done by driving the initial contour to the true boundary using strong GVF forces , which is integrated in geodesic active contour (GAC) model [35–38]. Then, in the second step, the evolving curve is driven by image gradient for accurate segmentation. It allows for relatively simple and free initialisation of the model, while minimizing edge leaking. We also present a study of the influence of the segmentation parameters on the model. To the best of our knowledge, no similar parameter optimization is reported in publications in ultrasound image segmentation.
The performance of the 3D system is demonstrated in its ability to provide accurate volume estimates using sparse image plane sampling from multiple acoustic views. To quantify accurate measures, the validation is focused on measures taken on calibrated 3D ultrasound phantoms. However, we also provide measures from in-vivo cardiac data set.
Hardware Setup and Software Flow Chart
For acquiring 2D ultrasound images, we use an Acuson Sequoia C256 (Siemens, USA) with a 7 MHz array transducer probe 7V3C (see Figure 1). A six-degree of freedom EPOM device, the Flock of Birds (FOB) (Ascension, Burlington, VT, USA) is used to record 3D location of each 2D image. For accurate 3D reconstructions, a calibration for determining the spatial relation between the sensor and the 2D images is performed [4, 39].
Figure 2 shows the system software flow chart. We start image acquisition with breath-holding and ECG gating at the standard acoustic window. The purpose of doing this is to avoid the cardiac deformation due to respiration and cyclic cardiac motion. Then, the only significant source of misregistration is due to the rigid movement of the patient. The position and orientation of the transducer associated with each acquired 2D image (640 × 480) are saved in the computer. The region of interest (ROI) is quickly outlined to reduce the computational and memory requirements. 2D+T image sequences are segmented automatically to identify the endocardial boundaries. The 3D surface of the LV is reconstructed with automated registration using segmented boundary walls. The whole software is developed in C programming language and MATLAB (MathWorks).
Hybrid Gradient Vector Flow (GVF) Geometric Active Contour (GAC) Model
Level sets segmentation allows the development of new image segmentation methods that can adapt to complex object boundaries. We developed a new hybrid model that can deliver accurate segmentation results from relatively simple initializations.
where d is the signed distance from point (x, y) to the boundary of an region Ω (Γ(t) bounds the region Ω) at t = 0. If d = 0, the point (x, y) is on the boundary. For points that are inside the initial boundary, φ(x, y,t = 0) takes on negative values. For points that are outside the initial boundary, φ(x, y,t = 0) takes on positive values.
where F norm denotes motion in the normal direction; F curv denotes motion in the curvature direction, and F adv denotes motion due to an externally generated velocity field, independent of the front itself.
where α is a constant strength coefficient. g is close to zero in regions where the gradient is high, and is approximately one in homogenous regions.
To minimize the edge leakage, the expansion term in the normal direction is excluded in this model.
To allow the deformable models to be initialized away from the object boundary, GVF vector field (u(x, y), ν (x, y)) is used for external driving force at the beginning. It diffuses the image gradient toward the homogenous region, allowing curve evolution in edge-free regions. It also allows for bi-directional flow that propagates the curve toward the object boundary from inside or outside the boundary. The edge indicator function g is also used for controlling the strength of the advection term.
Segmentation Parameter Optimization
Corsi et al.  set the parameters empirically: α = 0.1, β = 6, ε = 0.5. For the new model, a segmentation parameter optimization is also implemented.
The proposed hybrid model requires pre-setting a single threshold parameter. It is important to optimize this parameter since it can affect the performance of the segmentation method. For example, if the threshold value is too low, the hybrid method may not be able to reach the true boundary because of the relaxed initialization. For very high values, the evolving curve may pass the true boundary given as a result leakage at the edges. For that reason, it is necessary to consider all the possible values for the threshold in the optimization. In order to cover a wider range, a logarithmic sampling of the threshold values is chosen: T res = [5, 20, 50, 125.6, 315.5, 792.4, 1990.5, 5000].
The same logarithmic scale is also considered for the values of the ε, β 1, β 2 parameters. A total of 10 different values per parameter is set in the following way: ε = [0.1, 0.21, 0.46, 1, 2.15, 4.64, 10, 21.54, 46.42, 100] and β 1= β 2 = [0.6, 1.29, 2.79, 6, 12.92, 27.85, 60, 129.27, 278.50, 600].
To evaluate segmentation performance for each parameter combination, representative images are selected. Then, a simple curve which is set to be inside the ROI but far from the true boundary is provided to the algorithm as an initialization.
Two metrics are used to determine the optimal parameters: Hausdorff distance and mean absolute difference (MAD) between the manually and automatically segmented boundaries. The Hausdorff distance measures the distance between the points on two curves that differ the most, while MAD provides the mean difference between the two curves. Finally, the minimum values of both metrics determine the optimal parameters.
Multi-view Reconstruction with Automatic Registration
Automatic registration is a required and important step for combining acquisitions between different views. Clearly, misregistration is a big problem in freehand 3D ultrasound that affects the accuracy of the reconstruction and volume estimation. In general, there are three sources that cause misregistration in freehand 3D ultrasound: (i) spatial location error of 2D image, (ii) target movement during intra-view (deformation by cardiac motion and respiration) and inter-view scans (rigid movement of the subject), and (iii) unstable probe pressure applied on the scanning surface. The first error is largely reduced by the electromagnetic interference detector in the system . The misregistration due to unstable probe movement is reduced by short acquisition time (about 15 seconds) for each view acquisition. Intra-view deformation can be addressed by breath-holding and ECG gating. Rigid movement of the subject in inter-view scans causes the majority of the registration errors.
Our basic assumption for achieving automatic registration is that there is a partial overlap between the image acquisitions from different views. And we do require that the images from different views share some common features or anatomical structures, such as chamber wall surfaces. As we pointed out earlier in this paper, only voxel intensity-based registration can lead to significant errors. This is due to ultrasound-gain variation, speckle noise, and viewing artefacts. Instead of intensity-based registration, we use a feature-based geometric approach.
The basic idea is to reconstruct each view in 3D and then register the views together. The reconstructed 3D surfaces are obtained by 3D reconstruction of the 2D planes. Here, each plane is generated using the difference between two binary images. First, we generate a binary image of the segmented region that contains all of the pixels that fall inside the object of interest. The second binary image is generated by eroding the first one using a circular element of radius of 4 to 8 pixels (based on target size). The difference image captures the boundary wall. A 3D reconstruction of the 2D planes generates a 3D binary surface model.
We note that registration is possible as long as the reconstructed wall surfaces exhibit some overlap. To satisfy the partial overlap criterion, we require that at-least one of the views is a full-sweep, covering the entire object of interest. We expect that the inherent appearances of chamber wall surfaces will guarantee the existence of a unique global minimum for the registration parameters.
To reach the globally optimal value, we first apply a global registration method by using 3D Hotelling transform to construct an object-based reference volume. This is needed to avoid local minima and ensure a significant overlap between different view acquisitions. Then, we perform a higher accuracy registration using a robust, non-linear least squares algorithm to archive the optimal parameters.
We then reconstruct the 3D volume with the largest number of 2D image planes (for example, view V 1 ) over a regular Cartesian grid, and then register the 2D image slices from the rest of the views (for example, view V 2) to it.
where O is the overlapping region between the two volumes, I R refers to the reference 3D reconstruction, I N refers to the "new" 3D reconstruction to register, and n is the number of the voxels within set O. Once the images are registered, the 3D reconstructed volume is achieved by averaging the intensities from the different view volumes to attenuate artefacts and reduce noise.
Data Sets and Acquisition
Quantitative Comparison on Phantom Image Sequence Segmentation
Hausdorff Distance (mm)
1 (40 frames)
0.9828 (σ = 0.4963)
0.9479 (σ = 0.5209)
3.1076 (σ = 2.0168)
2.8946 (σ = 1.739)
2 (40 frames)
0.6540 (σ = 0.3876)
0.6516 (σ = 0.8613)
1.4230 (σ = 0.8147)
1.4156 (σ = 0.8125)
3 (40 frames)
0.7858 (σ = 0.5158)
0.7523 (σ = 0.2932)
2.7191 (σ = 2.0428)
2.500 (σ = 1.4683)
4 (40 frames)
0.4805 (σ = 0.2927)
0.4802 (σ = 0.2919)
1.1981 (σ = 0.7535)
1.1971 (σ = 0.7504)
5 (44 frames)
0.5105 (σ = 0.2299)
0.5464 (σ = 0.2418)
2.1746 (σ = 1.1174)
1.6694 (σ = 0.8680)
6 (41 frames)
0.4687 (σ = 0.2287)
0.4246 (σ = 0.2044)
1.9629 (σ = 0.9913)
1.0510 (σ = 0.5140)
7 (44 frames)
0.4769 (σ = 0.1677)
0.4815 (σ = 0.1069)
1.8469 (σ = 1.1158)
1.8016 (σ = 0.9674)
8 (47 frames)
0.3849 (σ = 0.1655)
0.5153 (σ = 0.2260)
1.5914 (σ = 0.8436)
1.2425 (σ = 0.5320)
While most of the current research has focused on adult cardiology, our primary focus here has been on applications in paediatric cardiology. In paediatric echocardiography, smaller heart size, higher heart rate, and more complicated cardiac anatomy make accurate 3D reconstruction even harder.
Four sequences of in-vivo paediatric cardiac image videos are used in the cardiac experiment. Data sets are acquired from the parasternal short-axis view and the apical long-axis view from a six year old healthy child volunteer. Breath holding (15 seconds) and ECG gating are used to minimize the deformation from respiration and cardiac motion. In each view acquisition, the transducer is moved slowly and evenly to scan the heart. Due to the standard frame rate of the frame grabber (nearly 30 frames per second), 433 images with 640 × 480 resolution are collected in 15 seconds, which is a much shorter scanning time than the acquisition time reported by Ye et al.. The subject does not have to remain still during the time it takes to switch to a different acoustic window. The image acquisition procedure for 3D is done exactly in the same way as the regular routine echocardiography examination at the hospital.
For the phantom image, the minimum Hausdorff distance is achieved for: ε = 0.1, β1 = β2 = 0.6, T res ≥ 50 (see Figure 4). From the plots of Figure 4, it is interesting to note that the optimization level is relatively flat within a certain region of the ε-β plane. In particular, the heuristically derived point given by ε = 0.8, β1 = β2 = 6, T res = 50 (used in our earlier investigations) gives essentially equivalent performance to the optimal line given by ε = 0.1, β1 = β2 = 0.6, T res ≥ 50. To minimize computational complexity, we use the point given T res = 50. It allows for the evolving front to converge quickly under GVF force to the region around the boundary, before it allows for local gradient to slowly fine-tune the final result. Here, the optimal results require that β values be at-least six times larger than the ε value where ε lies in the interval of [0.1, 100].
For the cardiac image, the optimal segmentation parameters are given by: ε = 0.1, β 1 = β 2 = 1.29 and 5 < T res < 125 (see Figure 5). As compared to the phantom image, the actual cardiac images require lower value of T res , and much larger values for the β parameters. For optimal results, we require a threshold value of 20 with β values to be at-least ten times larger than ε where ε lies in the interval of [0.1 100].
We evaluate the performance of the new segmentation method on eight sets of phantom image (2D+T) sequences. To initialize the segmentation, we require that the users provide an initial curve from a frame in the middle of the image sequence. The segmentation procedure then proceeds to automatically segment images before and after the middle image. For each frame, segmentation is initialized by using the segmented curve from the previous frame, allowing for a quick convergence and accurate segmentation.
In order to discuss the effects of the segmentation parameters, we present results for both optimized and non-optimized (empirical) segmentation. For the non-optimized segmentation, we set the parameters empirically, after a few experiments with a couple of training images (ε = 0.8, β 1 = β 2 = 6, T res = 50). For the optimized results, we follow the optimization method that we described in the Methods Section.
Table 1 shows that the optimal parameters gave the lowest Hausdorff distance errors in all cases. For the MAD, the optimized approach gives the best results in the majority of the cases. On average, the MAD stays below 1mm while the maximum segmentation error stays below 3mm (Hausdorff distance). It is interesting to note that there is more consistency in the optimized approach, in the sense that the standard deviations of the Hausdorff distances is found to be remain lower than the empirical approach.
Quantitative results on echocardiography sequence segmentation
Hausdorff Distance (mm)
Parasternal (26 frames)
0.794 (σ = 0.272)
0.775 (σ = 0.276)
3.656 (σ = 1.210)
3.610 (σ = 1.401)
Apical (26 frames)
1.267 (σ = 0.266)
0.911 (σ = 0.171)
4.261 (σ = 0.899)
2.966 (σ = 1.127)
Multi-view Reconstruction with Registration
We provide validation in terms of volume measures for the calibrated phantom and left ventricle in paediatric echocardiography. We observe a relation between the relative volume error and the number of image frames and the number of views in calibrated phantom.
Phantom volume measures and relative error for one-view, two-view, and three-view reconstructions
Small Egg (7.2cc)
Top window, short-axis
Top window, long-axis
Side window, short-axis
Side window, long-axis
Top & Side windows
Top window, short-axis 21
Top window, long-axis 22
Side window, long-axis 22
We note that two-view and three-view reconstructions that combine all available images gave the most accurate reconstructions. In fact, registering image planes from different views had obtained approximately the same error as reconstructions with the same number of image planes from a single view. This shows that automatic registration of multi-view sparse images was successful. Further discussion of this figure is given in Discussion section.
Volume measures using two-view reconstruction with automatic segmentation and manual segmentation
Top window: long-axis and short-axis
Side window: long-axis and short-axis
Top window short-axis and side window long-axis
Top window long-axis and side window short-axis
Left ventricular volume measures and relative error at ED and ES phases
Our experimental results on calibrated phantom and in-vivo paediatric echocardiography show that the new freehand 3D system can be used to provide accurate object volume measures using sparse 2D images. The new deformable model uses GVF to provide quick convergence to the true boundaries that are relatively insensitive to the initial curves. Also, it is more robust to speckle noise, poor edges and small boundary gaps that are often found in ultrasound images.
One of the most important factors that affect the performance of the algorithm is the threshold and parameters values in the deformable model. We investigated optimal parameter regions for the new approach. Volume measures from multi-view reconstructions are found to be consistently and significantly more accurate than those from the commonly used, single view reconstructions. More importantly, we have presented the relation between percentage error and the number of views and the number of planes for 3D phantom (see Figure 10).
There are a number of important observations that can be made from Figure 10. First, we note that reconstruction error is always reduced with the increasing number of image planes. Second, we note that when the number of images is 20, single view reconstruction volume has the smallest error. The number of image planes collected from each view in two-view acquisition is 10, while in three-view acquisition it is approximately 6 or 7. The image planes in each view are too sparse to register accurately in multi-view reconstruction in this case. When the number of images is 30 and 40 (40 is the maximal number of images from single view in the experiment), two-view and three-view reconstructions consistently give more accurate results than single view reconstruction. This shows that multi-view reconstruction with volumetric registration is successful. The new registration method allows us to register different views with sparse image planes and no manual initialization. Two-view and three-view reconstructions that combine images give the most accurate reconstructions when the total number of image frames keeps the same as that in single view acquisition. Since this phantom example had no viewing obstructions, we would clearly expect that multi-view systems will provide even more dramatic improvements when obstruction is an issue. Last, we note that the graph of Figure 10 may be used as a general guide for the required number of image planes for achieving desired reconstruction accuracy in our system. As an example, for multi-view reconstructions, we would require about 40 registered image planes for a relative accuracy of 5% or less. For clinical imaging, this information can be used to determine the minimum amount of acquisition time. This is clearly a topic of special concern in clinical imaging applications.
We also discuss our results for in vivo cardiac data. Here, using the new registration method, we have reduced the required image acquisition time to 15 seconds per view. We have also found that small children can hold their breath during the required 15-second period. No requirement of stay still for the small children between different view acquisitions. Multi-view reconstructions only require partial overlaps from the different views and we have achieved good registration results despite the use of very small numbers of 2D planes. Qualitative and quantitative results in Figure 11 and Table 5 show significant improvements in two-view reconstructions versus any of the single-view reconstructions.
On the other hand, registration performance can clearly be affected by the segmentation results. We found that in our experiments, the significant overlaps may be required for complex endocardial boundaries in echocardiography.
In this paper, we combine 3D reconstructions from different views using simple averaging. Alternative fusion strategies have been reported [13, 20]. A weighted averaging approach may be used to reduce view-dependent artifacts, further reduce noise, and emphasize anatomical structures of interest.
We have presented a new freehand ultrasound system that allows 3D reconstructions from sparse sampling geometries and multiple views. The system allows the imaging specialist to optimize view selection as done in routine 2D echocardiography. The utility of the system has been carefully measured on a 3D calibrated ultrasound phantom and also in cardiology clinical settings.
More specifically, 3D performance of the proposed system has greatly benefited from a new hybrid method for image segmentation, and a new coarse-to-fine registration method. The proposed hybrid segmentation method gives optimal results over a wide range of parameters. Unlike previous methods, the proposed registration method does not require good spatial alignment between different views, and it also does not require manual initialization. Registration is performed in a fully automatic mode. On the calibrated phantom example, our approach shows that reconstruction accuracy always increases as a function of the number of views and the number of acquired 2D planes.
Extensive clinical validation is still required before the system can be employed in standard practice. We recognize that the lack of large clinical datasets is a limitation of the current study. Having said this though, it is also clear that 3D reconstruction accuracy is very hard to establish on real, clinical datasets. To address this, it is common practice to report results on calibrated 3D ultrasound phantoms, as we did in this paper.
The authors would like to acknowledge financial support for Dr. Yu and Professor Pattichis by the Department of Electrical and Computer Engineering at the University of New Mexico.
- Vieyres P, Poisson G, Courrèges F, Smith-Guerin N, Novales C, Arbeille P: A Tele-operated Robotic System for Mobile Tele-echography: The OTELO Project. In M-Health: Emerging Mobile Health Systems Edited by: Istepanian RH, Laxminarayan S, Pattichis CS. 2006.Google Scholar
- Gooding MJ, Kennedy S, Noble JA: Volume segmentation and reconstruction from freehand three-dimensional ultrasound data with application to ovarian follicle measurement. Ultrasound in Medicine and Biology 2008, 34: 183–195. 10.1016/j.ultrasmedbio.2007.07.023View ArticleGoogle Scholar
- Kawai J, Tanabe K, Morioka S, Shiotani H: Rapid freehand scanning three-dimensional echocardiography: Accurate measurement of left ventricular volumes and ejection fraction compared with quantitative gated scintigraphy. Journal of the American Society of Echocardiography 2003, 16: 110–115. 10.1067/mje.2003.4View ArticleGoogle Scholar
- Detmer PR, Bashein G, Hodges T, Beach KW, Filer EP, Burns DH, SDE Jr: 3D ultrasonic image feature localization based on magnetic scanhead tracking: In vitro calibration and validation. Ultrasound in Medicine & Biology 1994, 20: 923–936.View ArticleGoogle Scholar
- Legget ME, Leotta DF, Bolson EL, McDonald JA, Martin RW, Li XN, Otto CM, Sheehan FH: System for quantitative three-dimensional echocardiography of the left ventricle based on a magnetic-field position and orientation sensing system. IEEE transactions on Biomedical Engineering 1998, 45: 494–504. 10.1109/10.664205View ArticleGoogle Scholar
- Leotta DF, Munt B, Bolson EL, Kraft C, Martin RW, Otto CM, Sheehan FH: Three-Dimensional Echocardiography by Rapid Imaging from Multiple Transthoracic Windows: In Vitro Validation and Initial In Vivo Studies. Journal of the American Society of Echocardiography 1997, 10: 830–840. 10.1016/S0894-7317(97)70043-9View ArticleGoogle Scholar
- Ye X, X JA, Noble JA, Atkinson D: 3-D freehand echocardiography for automatic left ventricle reconstruction and analysis based on multiple acoustic windows. IEEE Transactions on Medical Imaging 2002, 21: 1051–1058. 10.1109/TMI.2002.804436View ArticleGoogle Scholar
- Yu H, Pattichis MS, Goens MB: A Robust Multi-view Freehand Three-dimensional Ultrasound Imaging System Using Volumetric Registration. Proc. of the IEEE International Conference on System, Man, and Cybernetics 2005, 4: 3106–3111. full_textGoogle Scholar
- Yu H, Pattichis MS, Goens MB: Robust Segmentation and Volumetric Registration in a Multi-view 3D Freehand Ultrasound Reconstruction System. Proc. of the Fortieth Annual Asilomar Conference on Signals, Systems, and Computers 2006, 1978–1982.Google Scholar
- Yu H, Pattichis MS, Goens MB: Multi-view 3D Reconstruction with Volumetric Registration in a Freehand Ultrasound Imaging System. Proc. of the SPIE International Symposium on Medical Imaging 2006, 6147–6. 45–56Google Scholar
- Snider AR, Serwer GA, Ritter SB, Gersony RA: Echocardiography in pediatric heart disease. 2nd edition. Louis: Mosby; 1997.Google Scholar
- Rohling RN, Gee AH, Berman L: Automatic registration of 3-D ultrasound images. Ultrasound in Medicine & Biology 1998, 24: 841–854.View ArticleGoogle Scholar
- Rohling RN, Gee AH, Berman L: 3-D spatial compounding of ultrasound images. Medical Image Analysis 1997, 1: 177–193. 10.1016/S1361-8415(97)85009-8View ArticleGoogle Scholar
- Makela T, Clarysse P, Sipila O, Pauna N, Quoc Cuong P, Katila T, Magnin IE: A review of cardiac image registration methods. IEEE Transactions on Medical Imaging 2002, 21: 1011–1021. 10.1109/TMI.2002.804441View ArticleGoogle Scholar
- Grau V, Becher H, Noble JA: Registration of Multiview Real-Time 3-D Echocardiographic Sequences. IEEE Transactions on Medical Imaging 2007, 26: 1154–1165. 10.1109/TMI.2007.903568View ArticleGoogle Scholar
- Shekhar R, Zagrodsky V, Garcia MJ, Thomas JD: Registration of real-time 3-D ultrasound images of the heart for novel 3-D stress echocardiography. IEEE Transactions on Medical Imaging 2004, 23: 1141–1149. 10.1109/TMI.2004.830527View ArticleGoogle Scholar
- Zagrodsky V, Walimbe V, Castro-Pareja CR, Jian Xin Q, Jong-Min S, Shekhar R: Registration-assisted segmentation of real-time 3-D echocardiographic data using deformable models. IEEE Transactions on Medical Imaging 2005, 24: 1089–1099. 10.1109/TMI.2005.852057View ArticleGoogle Scholar
- Ledesma-Carbayo MJ, Kybic J, Desco M, Santos A, Suhling M, Hunziker P, Unser M: Spatio-temporal nonrigid registration for ultrasound cardiac motion estimation. IEEE Transactions onMedical Imaging 2005, 24: 1113–1126. 10.1109/TMI.2005.852050View ArticleGoogle Scholar
- Mellor M, Brady M: Phase mutual information as a similarity measure for registration. Medical Image Analysis 2005, 9: 330–343. 10.1016/j.media.2005.01.002View ArticleGoogle Scholar
- Soler P, Gerard O, Allain P, Saloux E, Angelini E, Bloch I: Comparison of Fusion Technique for 3D+T Echocardiography acquisitions from Different Acoustic Windows, presented at Computers in Cardiology. 2005, 25–28.Google Scholar
- Sonka M, Fitzpatrick JM: Medical Image Processing and Analysis. In Handbook of Medical Imaging. Volume 2. Bellingham, Washington: SPIE Press; 2000.Google Scholar
- Chalana V, Haynor DR, Kim Y: Left-ventricle boundary detection from short-axis echocardiograms: the use of active contour models. SPIE Image Processing 1994, 2167: 786–798.Google Scholar
- Chalana V, Linker DT, H DR, Kim Y: A multiple active contour model for cardiac boundary detection on echocardiographic sequences. IEEE Transactions on Medical Imaging 1996, 15: 290–298. 10.1109/42.500138View ArticleGoogle Scholar
- Mikic I, Krucinski S, Thomas JD: Segmentation and tracking in echocardiographic sequences: active contours guided by optical flow estimates. IEEE Transactions on Medical Imaging 1998, 17: 274–284. 10.1109/42.700739View ArticleGoogle Scholar
- Cootes TF, Hill A, Taylor CJ, Haslam J: The use of active shape models for locating structures in medical images. Image and vision computing 1994, 12: 355–366. 10.1016/0262-8856(94)90060-4View ArticleGoogle Scholar
- Cootes TF, Beeston C, Edwards GJ, Taylor CJ: A unified framework for atlas matching using active appearance models. In Information Processing in Medical Imaging. Edited by: AS Kuda AS, M, Berlin M. Germany: Springer-Verlag; 1999:322–333. full_textView ArticleGoogle Scholar
- Mitchell SC, Bosch JG, Lelieveldt BPF, van der Geest RJ, Reiber JH, Sonka M: 3-D active appearance models: segmentation of cardiac MR and ultrasound images. IEEE Transactions on Medical Imaging 2002, 21: 1167–1178. 10.1109/TMI.2002.804425View ArticleGoogle Scholar
- Bosch JG, Mitchell SC, Lelieveldt BPF, Nijland F, Kamp O, Sonka M: Automatic segmentation of echocardiographic sequences by active appearance motion models. IEEE Transactions on Medical Imaging 2002, 21: 1374–1383. 10.1109/TMI.2002.806427View ArticleGoogle Scholar
- Noble JA, Boukerroui D: Ultrasound Image Segmentation: A Survey. IEEE Transaction on Medical Imaging 2006, 25: 987–1010. 10.1109/TMI.2006.877092View ArticleGoogle Scholar
- Lin N, Yu W, Duncan JS: Combinative Multi-Scale Level Set Framework for Echocardiographic Image Segmentation. Medical Image Analysis 2003, 7: 529–537. 10.1016/S1361-8415(03)00035-5View ArticleGoogle Scholar
- Paragios N: A Level Set Approach for Shape-Driven Segmentation and Tracking of the Left Ventricle. IEEE Transactions on Medical Imaging 2003, 22: 773–776. 10.1109/TMI.2003.814785View ArticleGoogle Scholar
- Sarti A, Corsi C, Mazzini E, Lamberti C: Maximum likelihood segmentation of ultrasound images with Rayleigh distribution. IEEE Transactions on Ultrasonics, Ferroelectricity and Frequency Control 2005, 52: 947–960. 10.1109/TUFFC.2005.1504017View ArticleGoogle Scholar
- Corsi C, Saracino G, Sarti A, Lamberti C: Left ventricular volume estimation for real-time three-dimensional echocardiography. IEEE Transactions on Medical Imaging 2002, 21: 1202–1208. 10.1109/TMI.2002.804418View ArticleGoogle Scholar
- Xu C, Prince JL: Snakes, shapes, and gradient vector flow. IEEE Transactions on Image Processing 1998, 7: 359–369. 10.1109/83.661186MathSciNetView ArticleGoogle Scholar
- Caselles V, Kimmel R, Sapiro G: Geodesic Active Contours. International journal of Computer Vision 1997, 22: 61–79. 10.1023/A:1007979827043View ArticleGoogle Scholar
- Xu C, Yezzi A Jr, Prince JL: On the relationship between parametric and geometric active contours. In Proc. of 34th Asilomar Conference on Signals, Systems, and Computers 2000, 1: 483–489.Google Scholar
- Hang X, Greenberg NL, Thomas JD: A geometric deformable model for echocardiographic image segmentation. Computers in Cardiology 2002, 77–80.View ArticleGoogle Scholar
- Paragios N, Mellina-Gottardo O, Ramesh V: Gradient vector flow fast geometric active contours. IEEE Transactions on Pattern Analysis and Machine Intelligence 2004, 26: 402–407. 10.1109/TPAMI.2004.1262337View ArticleGoogle Scholar
- Leotta DF, Detmer PR, Martin RW: Performance of a miniature magnetic position sensor for three-dimensional ultrasound imaging. Ultrasound in Medicine & Biology 1997, 23: 597–609.View ArticleGoogle Scholar
- Osher S, Sethian JA: Pronts propagating with curvature-dependent speed: algorithm based on Hamilton-Jacobi formulations. Journal of Computational Physics 1988, 79: 12–49. 10.1016/0021-9991(88)90002-2MathSciNetView ArticleGoogle Scholar
- Sethian JA: Level Set Methods and Fast Marching Methods Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. 2nd edition. New York: Cambridge University Press; 1999.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.