 Research
 Open Access
 Published:
Personalized heterogeneous deformable model for fast volumetric registration
BioMedical Engineering OnLine volume 16, Article number: 30 (2017)
Abstract
Background
Biomechanical deformable volumetric registration can help improve safety of surgical interventions by ensuring the operations are extremely precise. However, this technique has been limited by the accuracy and the computational efficiency of patientspecific modeling.
Methods
This study presents a tissue–tissue coupling strategy based on penalty method to model the heterogeneous behavior of deformable body, and estimate the personalized tissue–tissue coupling parameters in a datadriven way. Moreover, considering that the computational efficiency of biomechanical model is highly dependent on the mechanical resolution, a practical coarsetofine scheme is proposed to increase runtime efficiency. Particularly, a detail enrichment database is established in an offline fashion to represent the mapping relationship between the deformation results of highresolution hexahedral mesh extracted from the raw medical data and a newly constructed lowresolution hexahedral mesh. At runtime, the mechanical behavior of human organ under interactions is simulated with this lowresolution hexahedral mesh, then the microstructures are synthesized in virtue of the detail enrichment database.
Results
The proposed method is validated by volumetric registration in an abdominal phantom compression experiments. Our personalized heterogeneous deformable model can well describe the coupling effects between different tissues of the phantom. Compared with highresolution heterogeneous deformable model, the lowresolution deformable model with our detail enrichment database can achieve 9.4× faster, and the average target registration error is 3.42 mm, which demonstrates that the proposed method shows better volumetric registration performance than stateoftheart.
Conclusions
Our framework can well balance the precision and efficiency, and has great potential to be adopted in the practical augmented reality imageguided robotic systems.
Background
Surgical procedures are traditionally supported with preoperative images, such as the computed tomography (CT) images and magnetic resonance (MR) images. The image quality can be very good, while when it comes to the surgical procedures, the link between images and patient is lost. In this regard, it can be intuitively displayed as an overlay of preoperative images onto the patient’s body, which creates an augmented reality environment that enables surgeons to visualize the structures of interest [1]. Imaging looks inside the patient’s body, exposing the patient’s anatomy beyond what is visible on the surface.
A variety of methods have been developed to provide intraoperative image registration, which can be mainly classified into rigid and nonrigid registration. Rigid registration is generally applied when the target anatomy fulfills the criterion of rigidity and spatial distortions are not introduced in the image acquisition process [2]. It is a relatively quick and straightforward process that uses a rigid motion model with rotation and translation parameters of the target objects where tissue deformation can be ignored. Unfortunately, purely rigid transformation is not sufficient to describe the mechanical behaviors of human organ for most of the surgeries. To this end, this technique cannot produce an optimal alignment when human organ undergoes deformations caused by external forces (such as surgical tools) or natural motions (such as respiration). In such cases, nonrigid registration is required when the imaged anatomy nonrigidly deforms between acquisitions, which can provide a relatively accurate alignment for cases of nonrigid deformations. Readers can refer to [3] for a thorough and comprehensive introduction of nonrigid registration.
Moreover, nonrigid registration can be broadly classified as either surface registration or volumetric registration. Both of these two approaches have advantages and weaknesses. Surface registration methods [4–10] have been shown to accurately align the highly complex morphological details on the surface of the human organ. Although these methods can offer the possibility to achieve visually coherent surface registration, they are limited to surface overlay without considering heterogeneous internal structures, such as vessels and tumors. In this regard, it is crucial to consider volumetric registration [11, 12]. Generally, volumetric registration methods can provide a correspondence field across the entire human organ, including common target regions (such as tumors and vessels) that are not in the domain of the surfacebased alignment procedures.
Biomechanical model has proven to be a promising way for nonrigid volumetric registration [11, 13–20], which can accurately estimate the motion of indepth volumetric structures. The finite element method (FEM) is the most widely used physicsbased approach for developing deformable model, which can accurately describe the mechanical behaviors of human organ as continuous medium. Many researchers have reported certain success in achieving accurate volumetric registration based on FEM model. However, conventional FEM is so complicated that makes solution procedure timeconsuming, which limits its application in clinical practice [21]. In this paper, we employ the total Lagrangian explicit dynamics (TLED) FEM to model the mechanical response of human organ under interactions. This is because TLED FEM is an efficient numerical algorithm that is based on the FEM while using the total Lagrangian formulation, where stresses and strains are measured with respect to the original configuration which allows for precomputing of most spatial derivatives before the commencement of the timestepping procedure [22]. Besides, TLED FEM is capable of handling both geometric and material nonlinearities, which is beneficial to perform the large deformation analysis induced by tool–tissue interactions. In addition, the accuracy of the FEM relies heavily on the quality of geometric models meshes, while the geometric models of human organs are often complicated and irregular for representing the morphological details of the organs. Here we directly extract the uniform hexahedral mesh from the segmented medical images, which can greatly reduce the complexity of volumetric geometric mesh reconstruction and provide the highquality mesh for biomechanical modeling.
In this paper, we are motivated to achieve fast volumetric registration for both surface and indepth anatomical structures, where we have to determine the location of target region with great accuracy and avoid damaging the vessels which are needed for the postoperative rehabilitation of human organs. The contributions of this paper are as follows:

Personalized heterogeneous deformable model. Our deformable model is based on a uniform and highresolution hexahedral mesh directly extracted from the MR images, which is beneficial for the accuracy of TLED FEM analysis. A novel and effective tissue–tissue coupling strategy based on penalty method is proposed to model the indepth anatomical structure of deformable body, and the personalized tissue–tissue coupling parameters are estimated in a datadriven way.

Coarsetofine scheme. We propose a coarsetofine scheme to reduce the computational complexity of heterogeneous deformable model for fast volumetric registration. In more detail, we perform the TLED FEM on lowresolution hexahedral mesh first and then synthesize the microstructures using a detail enrichment database constructed by the highresolution heterogeneous deformable model.
Methods
Materials
In this paper, we employ the triple modality 3D abdominal phantom (Model 057A, Computerized Imaging Reference Systems, Inc.) as the experimental object. It is a plastic model for medical education usage including artificial liver, vessels and tumors. In addition, the anatomical structures of the phantom can be identified by 3D MR images, which are acquired using Siemens 3T MAGNETOM Trio system in Paul C. Lauterbur Research Center For Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. The scanned images of the phantom being compressed is used as the ground truth to validate the proposed volumetric registration method. The phantom has the advantage of providing controllable boundary conditions, and has been widely used as the experimental material to verify the registration method such as some other similar work of Lavely et al. [23] and Serban et al. [24]. To acquire 3D MR images of abdominal phantom under different compressions, we build a containing device by 3D printing technique to facilitate applying displacements during the MR imaging. The compression positions are on top of phantom and the MR scanning scene are shown in Fig. 1. We totally obtained 6 image datasets which consist of a dataset of initial status and 5 datasets of compressions (marked as A, B, C, D, E), and each dataset consists of 111 MR images. In addition, we label 25 landmarks (11 landmarks on liver surface and 14 internal landmarks inside the liver) in the initial status dataset and 5 compression datasets as groundtruth. Those landmarks are selected according to the structure of the tissues or organs, such as the internal surface or corners.
Hexahedral mesh construction
In this paper, we focus on the cases when lesions (such as tumors) are present inside the parenchyma, these can be taken into account also from the mechanical point of view as they usually introduce significant heterogeneity. This heterogeneity can be straightforwardly included in the model through volumetric mesh. In this regard, hexahedral mesh or tetrahedral mesh [11–13] are often employed rather than triangular mesh. Note that traditional volumetric registration methods are based on reconstructed tetrahedral mesh, while the reconstruction process may cause loss of precision. Considering that the segmented MR images are pixel level representation of the scanned phantom, we can avoid this limitation by directly constructing the hexahedral mesh. More importantly, we concentrate on improving the efficiency of volumetric registration, however, the tetrahedral mesh is usually irregular and contains distorted elements when deforms [25, 26], which requires remeshing and results in huge computational cost and makes the simulation timeconsuming [27, 28]. While the hexahedral mesh is known to be efficient in terms of stability and computational cost [25, 26], and hexahedral mesh presents better accuracy and efficiency than tetrahedral mesh in solid mechanics and structural engineering problems [29].
Thus, we directly construct a uniform highresolution hexahedral mesh from the 3D MR images of vascularized liver. First, we manually segment the 3D MR images of vascularized liver, which can be classified as parenchyma, vessels and tumors. By setting the resolution (\(74\times 18\times 54\)), we divide each MR images into voxels and construct the uniform hexahedral mesh by connecting voxels in each MR image. The hexahedral mesh construction process can be observed in Fig. 2. Note that each hexahedron can only serve as one kind of tissue element, and the relationship between landmarks and the hexahedral mesh are established using the landmarks’ pixel coordinates.
It is worth noting that there are truly jaggy structures in our method around the hexahedral boundary no matter what kind of resolution it uses, and this will degrades the performance of our method in a certain degree. However, this is a kind of inevitable precision loss in the process of geometric mesh generation, even for the commonly adopted triangular mesh or tetrahedral mesh, since converting the raw medical images to surface/volumetric mesh leads to the loss of precision. Theoretically, the higher resolution the hexahedral mesh is, the more accurate registration method is. In this regard, we adopt a relative high resolution \(74 \times 18 \times 54\) to reduce the precision loss induced by jaggy structures as much as possible. Meanwhile, the coarsetofine scheme proposed in this paper guarantees our registration efficiency is not influenced by the highresolution hexahedral mesh.
Personalized heterogeneous deformable model
In this work, we propose a personalized heterogeneous deformable model to accurately and efficiently model the mechanical behavior of human organ under interaction, aiming at achieving fast volumetric registration. The overall process is illustrated in Fig. 3.
Highresolution heterogeneous deformable model
Taking liver tissue containing a tumor as an example, the hexahedron containing both part of tumor and soft tissue is called boundary hexahedron. Here the shared vertices of liver hexahedral and tumor hexahedral meshes forms the internal surfaces and these vertices are called internal surface vertices. By representing the internal surface vertices using their neighboring hexahedral vertices, the coupling forces can be transmitted to the neighboring hexahedra vertices whose displacements can also be reflected on the internal surface vertices.
As shown in Fig. 4, purple vertices are parenchyma (such as liver soft tissue), red vertices are lesion (such as tumors), which are separated by the internal surface. Here we define \(\mathbf {P}=\{p_1,p_2,...\}\) as the parenchyma vertices, \(\mathbf {Q}=\{q_1,q_2,...\}\) as the lesion vertices. As the parenchyma and the lesion are coupled all the time on the boundary, here we treat boundary vertex i as two corresponding splitted vertices \(bp_i\) and \(bq_i\), called boundary vertices, which are on the same position and obviously they should keep in contact at every time step. We define \(\mathbf {BP}=\{bp_1,bp_2,...\}\) as boundary vertices of parenchyma and \(\mathbf {BQ}=\{bq_1,bq_2,...\}\) as boundary vertices of lesion. The highresolution heterogeneous deformation is described as follows: first, we exert a displacement by the interactive tool on the parenchyma vertices and the new position of each parenchyma vertex is solved with TLED FEM. The boundary vertices are not involved in the TLED FEM modeling and they are represented by its support domain using moving leastsquares (MLS) [30], as shown in Fig. 4.
Suppose two corresponding boundary vertices \(bp_i\) and \(bq_i\), \(\mathbf {P}_i^\Omega\) and \(\mathbf {Q}_i^\Omega\) are their support domain, respectively. At time step t, after applying boundary condition on the parenchyma vertices, the parenchyma boundary vertices and lesion boundary vertices are calculated by the following equation.
where \(\Phi _{p,i}, \Phi _{q,i}\) are shape functions according to the MLS method.
During the mechanical coupling procedure, \(bp_i\) and \(bq_i\) suffer a coupling force and induce lesion to deform. Here we propose a tissue–tissue coupling strategy based on penaltybased method [31], which is a computationally simple and effective solution for collision response between parenchyma, tumor and vessels. The coupling force is
where \(\delta ^t=bp_i^tbq_i^t\) is the interpenetration of parenchyma and lesion, \(k_{c}\) is a personalized tissue–tissue coupling coefficient to calculate coupling force in heterogeneous deformable model. For computing the coupling forces between parenchyma and tumor, parenchyma and vessels, \(k_{c}\) is represented by \(k_{c,st}\) and \(k_{c,sv}\), respectively.
After obtaining the coupling force between lesion boundary vertex \(bq_i\) and parenchyma boundary vertex \(bp_i\), we transmit the coupling force to their support domain \(\mathbf {Q}_i^{\Omega ,t}\) and \(\mathbf {P}_i^{\Omega ,t}\), respectively.
Then the parenchyma vertices and lesion vertices are solved with the TLED FEM and the support domains of boundary vertices \(bp_i\) and \(bq_i\)’s are updated as \(\mathbf {P}_i^{\Omega ,t+1},\) \(\mathbf {Q}_i^{\Omega ,t+1},\) respectively. The positions of \(bp_i\) and \(bq_i\) in the time step \(t+1\) are
Actually, on the boundary, the parenchyma and lesion are coupled all the time, and they satisfy the Signorini’s law [32]:
where \(\perp\) indicates there is a complementarity relation between \(\delta ^{t+1}\) and \(f_{i,t+1}^{coupling}.\) The workflow of the highresolution heterogenous deformable model is illustrated in Fig. 5.
Data driven parameters estimation
To precisely model volumetric deformation, we have to obtain the tissue–tissue coupling parameters for our heterogeneous deformable model. It is unrealistic to find universal coefficients \(k_{c,st}\) and \(k_{c,sv}\) which fit all the patients and circumstances [33]. There is an important variation of values when it comes to the parameters estimation of our deformable model. The parameters should be selected according to the personalized application.
From the experimental data, we adopt 3 datasets of compression for parameters estimation and the other 2 datasets of compression for validation through calculating the landmarks’ registration errors. For each dataset for parameter estimation, deformation can be described as:
where \(\mathbf {X}^{t+1}\) is the deformed position set of vertices on highresolution heterogeneous deformable model. \(\mathbf {X}^t\) are the vertices on undeformed highresolution heterogeneous deformable model. \(\mathbf {K}(k_{c,st},k_{c,sv})\) are the tissue–tissue coupling parameters of the highresolution heterogeneous deformable model, and they are obtained by the following equation,
where \(\mathbf {X}_{g}^{t+1}\) is the groundtruth set of 25 landmarks. The parameters are trained by pairs of the landmarks positions on the highresolution heterogeneous deformable model.
Dynamics
In this paper, we model the dynamics of human organ using the following mathematical formulation:
where \(\mathbf {M}\) is the constant mass, \(\mathbf {X}\) is the current nodes’ displacements. \(\mathbf {D}(\dot{\mathbf {X}})\) is the damping force, \(\mathbf {K}(\mathbf {X})\mathbf {X}\) represents elastic force, and \(\mathbf {R}\) is the current applied force field equivalently distributed to each node of the object’s mesh. In addition, we model the deformable body as NeoHookean material.
Coarsetofine scheme
The highresolution heterogeneous deformable model can provide a promising way for volumetric registration with high fidelity. However, it needs too much computational cost. To achieve fast volumetric registration, we propose a coarsetofine scheme which numerically coarsens the highresolution hexahedral mesh into a lowresolution hexahedral mesh of \(24\times 18\times 18\). Each vertex on lowresolution hexahedral mesh can find a corresponding vertex on the highresolution hexahedral mesh. As shown in Fig. 6, in a 2D illustration, \(h_1, h_2, ... , h_9\) are vertices of the highresolution hexahedral mesh. After constructing the lowresolution hexahedral mesh, \(\mathbf {P}=\{h_1, h_2, h_3, h_4\}\) are served as the vertices of lowresolution hexahedron \(H_j\), \(\mathbf {X}=\{h_5, h_6, h_7, h_8, h_9\}\) are the microstructures which are synthesized by MLS method, \(\Phi _i\) are shape functions.
However, there is a difference between highresolution and lowresolution hexahedral meshes. For the highresolution hexahedral mesh, the parenchyma and lesion are separated from each other by the internal boundary as shown as the two splitted vertices and imaginary line, which have no direct connection with the parenchyma or lesion hexahedral mesh. The force transmission between parenchyma and lesion is realized by the internal boundary vertices using MLS method. For the lowresolution hexahedral mesh, the parenchyma hexahedral and lesion hexahedral meshes are connected directly and force transmission between them is also direct. For each material’s hexahedral mesh, we assign different Young’s modulus for them, each hexahedron’s tissue type is determined by the largest number of pixel inside that hexahedron. To achieve accurate volumetric registration with lowresolution hexahedral mesh, we store the mapping relationship between vertices on highresolution hexahedral mesh and corresponding lowresolution hexahedral mesh using MLS method on different compression conditions (respectively 50 different displacements and 20 different orientations at each of the 5 compression positions) to build a detail enrichment database.
At runtime, we can employ the lowresolution hexahedral mesh to achieve fast firststep deformation, and then synthesize the microstructures according to the detail enrichment database. During the construction of detail enrichment database, each hexahedron on lowresolution hexahedral mesh is regarded as an element. Besides, we can use the 8vertices on lowresolution hexahedral mesh to describe all the vertices in corresponding highresolution hexahedral mesh by MLS method. For the tth displacement, the ith 8vertices element \(H_i\)’s all shape functions can be calculated using MLS method, donated as,
After constructing the detail enrichment database, for a specific displacement exerted on the phantom we need to find out the corresponding deformation information under the similar compression conditions in the detail enrichment database to synthesize the microstructures. Suppose the ith 8vertices element \(H_i\)’s 8 vertices are \(\mathbf {P}_i=(P_{i,1},P_{i,2},...,P_{i,8})\), for lowresolution hexahedral mesh element \(H_c\), the microstructures can be calculated as,
Figure 7 explains the mechanism of how to construct the mapping between two levels of hexahedral meshes. First, we construct the lowresolution hexahedral mesh according to the highresolution hexahedral mesh and calculate the mapping relationship by MLS method [30]. Then, we build the detail enrichment database by deforming the highresolution heterogeneous model and recording the mapping relationship between highresolution and lowresolution hexahedral meshes. Finally, we compute the deformation of the lowresolution hexahedral mesh, and synthesize the microstructures to achieve the final deformation of the highresolution hexahedral mesh.
Experiments
We conducts several experiments to validate our method mainly from two perspectives: accuracy and efficiency. Young’s modulus for parenchyma, vessel and tumor used in the phantom are respectively \(2\times 10^5\), \(10^6\) pa and \(5\times 10^6 pa\) pa, and the Poisson’s ratio for them are 0.49. All experiments are conducted on a PC equipped with Intel Xeon CPU E31230 V2 (3.30GHz) CPU, 4G RAM and NVIDIA GeForce GTX 650 Ti.
Datadriven parameters estimation We divide the five compressions datasets into two groups (three training datasets and two validation datasets). Figure 8 illustrates the volumetric liver registration results on training datasets (A, B and C) and validation datasets (D and E). Each row represents the data from one compression position. For the phantom, we have labeled 25 landmarks on five cross sections, which are corresponding to the five column in Fig. 8. Each column represents a MR image section with landmarks. To estimate the personalized parameters accurately, we iteratively conduct the deformation experiments for highresolution heterogeneous deformable model until obtaining the minimum average target registration error (TRE) for training datasets. It is observed from the training datasets A, B and C in Fig. 8 that the calculated results (red crosses) are consistent with the groundtruth (green crosses). The minimum average TRE of the training datasets is 2.00 mm. The estimated tissue–tissue coupling parameters \(k_{c,st}\) and \(k_{c,sv}\) are 600 and 500 N/m respectively. The average computation time per frame for highresolution heterogeneous deformable model is 344.83 ms and the average frame rate is 2.9 fps. By applying the estimated parameters on validation datasets, we obtain the volumetric registration results with average TRE 2.55 mm, which demonstrates the effectiveness of the estimated personalized parameters.
Coarsetofine scheme We build a lowresolution deformable model and construct a details enrichment dataset by deforming highresolution heterogeneous deformable model and recording the mapping relationship using the MLS method. In our method, we direct perform deformation on lowresolution deformable model and synthesize the microstructures according to the detail enrichment dataset. The deformation results of our method (lowresolution deformable model with detail enrichment database) are illustrated in Fig. 9, which demonstrates that the calculated results are consistent with the groundtruth. Figure 10 shows the TRE of highresolution heterogeneous deformable model and our method, respectively. We demonstrate the deformation comparisons in volumetric registration for highresolution heterogeneous deformable model and our method. At equilibrium we obtained the displacements of both solutions for dataset E (see Fig. 11a, b). Then we measured the difference of the displacements computed by the two approaches. Results are presented in Fig. 11c (using the same scale as the initial displacements). The average TRE is 3.27 mm, which indicates the coarsetofine strategy can achieve registration results with good accuracy.
In addition, we visualize the intermediate deformation poses of our hexahedral mesh and the corresponding positions of the 25 landmarks in this progress. As shown in Fig. 12, we divided the 25 landmarks into three types: surface landmarks, tumor landmarks and vessel landmarks, since these three kinds of landmarks are doctor’s structures of interest. We compare the positions of the 25 landmarks in these deformed model with the ground truth, which indicates that the calculated landmark positions are approaching the ground truth gradually during the registration process.
Comparison We validate our method (lowresolution deformable model with detail enrichment database) from the following perspectives:

Compare the overall registration accuracy and performance of our method with the registrations using highresolution heterogeneous deformable model and Han et al. [13], which is also a nonrigid registration method based on the TLED FEM while without considering the tissue heterogeneity.

Compare the surface and internal registration results of our method with the above two registration methods respectively, demonstrating the accuracy of heterogeneity representation of our method.
Figure 13a illustrates the average TRE of volumetric registration on our method, highresolution heterogeneous deformable model and Han’s work [13] for datasets A, B, C, D and E. It can be observed that the average TRE of Han’s method (7.74 mm) for datasets A, B, C, D and E are all larger than those of highresolution heterogeneous deformable model (2.22 mm) and our method (3.42 mm). Because the highresolution heterogeneous deformable model includes the tissue–tissue coupling process in the volumetric mesh, which results in less registration error than Han’s method, which did not include the tissue–tissue coupling process. Besides, our method first deforms the lowresolution hexahedral mesh and then synthesizes the microstructures of the highresolution hexahedral mesh according to the detail enrichment database, which is constructed by the mapping relationship of the deformed highresolution heterogeneous deformable model and lowresolution deformable model. The detail enrichment database itself already includes the effects of the tissue–tissue coupling process.
Meanwhile, we evaluate the registration accuracy of 11 surface landmarks and 14 internal landmarks on the phantom model respectively, as shown in Fig. 13b. For highresolution heterogeneous deformable model, the average TRE of surface landmarks and internal landmarks are respectively 2.45 and 2.04 mm. While for our method, the average TRE of surface landmarks and internal landmarks are 3.46 and 3.38 mm, respectively. Using Han’s method, the average TRE of surface landmarks and internal landmarks are 6.39 and 8.81 mm, respectively. Experimental results indicate our method can well describe the heterogeneity of human organ, and achieve better registration accuracy than Han’s method [13].
In addition, we achieve 27.2 fps of volumetric registration by our method with the average computation time 36.76 ms, this frame rate can fulfill the requirement of realtime tracking system. It is also worth noting that at the expense of accuracy for about 1 mm, our method can speed up \(9.4\times\) than the highresolution heterogeneous deformable model, as well as about \(9.4\times\) faster than Han’s method which is applied on the highresolution deformable model. The experimental results demonstrate that our method can well balance the computational efficiency and accuracy.
Discussion and conclusion
In this paper, an efficient personalized heterogeneous deformable model is presented for volumetric registration. Our method includes three core components: a heterogeneous deformable model, a personalized tissue–tissue coupling strategy and a coarsetofine scheme. In more detail, we propose the highresolution heterogeneous deformable model for a uniform and highresolution hexahedral mesh and model the mechanical behavior of heterogeneous anatomical structure with TLED FEM and penalty method. Besides, we present a data driven parameters estimation method for the highresolution heterogeneous deformable model to obtain the tissue–tissue coupling parameters of our method in vivo for FE analysis. In addition, we put forward a coarsetofine scheme to achieve fast volumetric registration, which is to first perform the volumetric deformation on the lowresolution hexahedral mesh and then synthesize the microstructures according to the detail enrichment database. We have tested the highresolution heterogeneous deformable model and our method with the real compression of phantom in five experiments. The experimental results indicate our method can achieve fast and accurate registration results, which are essential for clinical applications.
Biomechanical deformable model is an effective way to reliably predict deformation for volumetric registration and many researchers have demonstrated good nonrigid registration that meets the accuracy requirements of specific surgery. AlMayah et al. [17] proposed a 3D FEM based biomechanical model for image registration of head and neck cancer treatment, they applied the linear elastic material properties to their method and adopted linear geometry. Oktay et al. [16] proposed a linear FEM deformation based registration method for preoperative and intraoperative 3D image fusion for laparoscopy surgery. Though the linear FE analysis is an approximation that makes the analysis of the structure more tractable, the assumptions of linearity are often not adequate for real tissues which often undergoes nonlinear behaviour. Compared with the work of [16, 17], our TLED FEM analysis adopts nonlinear elastic material properties which can provide more accurate biomechanical analysis. Hopp et al. [15] presented a nonlinear biomechanical FEM analysis based registration method for Xray mammograms with DCEMRI volumes. The mean TRE was 13.2 mm that was within the clinically relevant range. However, the deformable body was modelled as homogeneous soft tissue which could not describe the deformation distribution inside the soft tissue. Thus their method is not suitable for the registration of organs with internal heterogeneous structures such as tumors or vessels. To address the heterogeneous issue, Haouchine et al. [11] used a deformable volumetric biomechanical model accounting for heterogeneity and anisotropy in hepatic surgery guidance with the best tumor registration accuracy of 2.5 mm. Samavati et al. [18] proposed a biomechanical model with heterogenous material property for deformable prostate image registration with average registration accuracy of 4.8 mm, also they have presented a hybrid biomechanical intensity based deformable image registration method for lung 4DCT [19] and achieved average registration accuracy of 2.9 mm. Han et al. [13] proposed a patientspecific biomechanical modeling framework for heterogeneous breasts based on nonlinear FEM solver, which achieved relative accurate volumetric breasts registration with the best registration accuracy of \(3.18\pm 1.69\) mm by anisotropic heterogeneous model. They assigned different material properties for different tissues to construct the heterogeneous structures. Different from the above heterogenous models, our method constructs the heterogeneous deformable on a uniform and highresolution hexahedral mesh directly extracted from the MR images for investigating the motion of liver’s vessels and tumors, and the different types of tissues are coupled by a penalty method. The proposed highresolution heterogeneous deformable model achieves average registration accuracy of 2.22 mm on a fine resolution grid of \(74\times 18\times 54\). Our method achieves average registration accuracy of 3.42 mm on a lowresolution grid of \(24\times 18\times 18\) with detail enrichment database, while the method of Han et al. [13] achieves TRE of 7.74 mm.
In spite of high registration accuracy achieved by our heterogeneous model, the efficiency due to the tremendous computation has limited the applications of many work [11, 13, 18, 19], whose registration process is timeconsuming. However, the efficiency is an essential issue in the imageguided surgery for the reason that even in a few seconds the registration target would deform or shift a lot which could lead to the failure of the registration. It is crucial to develop a fast biomechanical model based registration methods which incorporate the advantages of high accuracy and efficient computation. To achieve fast volumetric registration, we propose a practical coarsetofine scheme and establish a detail enrichment database at the preprocessing stage. At runtime, we simulate the mechanical behavior of the lowresolution hexahedral mesh and synthesize the microstructures in virtue of detail enrichment database. With the expense of accuracy for about 1mm, our method can speed up \(9.4\times\) than the highresolution heterogeneous deformable model.
There are certain limitations of our method. In clinical practice, the personalized Young’s modulus is unknown while it is very essential for the construction of personalized deformation model. We plan to obtain the patientspecific Young’s modulus by ultrasound elastography in vivo [34]. Besides, the accuracy of the lowresolution deformable model is not as accurate as the highresolution one, so that we plan to incorporate the generalized moving least squares (GMLS) [35] into our coarsetofine scheme to improve its accuracy. In addition, we intend to improve the setup of boundary conditions for the temporal registration phase, which is an important and challenging problem. Besides, we will also adopt our method to align the preoperative volumetric liver MR images to intraoperative ultrasound image and obtain quantitative errors on real data.
Abbreviations
 CT:

computed tomography
 MR:

magnetic resonance
 MRI:

magnetic resonance imaging
 FEM:

finite element method
 TLED:

total Lagrangian explicit dynamics
 MLS:

moving least squares
 GMLS:

generalized moving least squares
 TRE:

target registration error
 DCEMRI:

dynamic contrastenhanced magnetic resonance imaging
 4DCT:

4dimension computed tomography
References
 1.
Sridhar AN, HughesHallett A, Mayer EK, Pratt PJ, Edwards PJ, Yang GZ, Darzi AW, Vale JA. Imageguided robotic interventions for prostate cancer. Nat Rev Urol. 2013;10(8):452–62.
 2.
Markelj P, Tomaževič D, Likar B, Pernuš F. A review of 3d/2d registration methods for imageguided interventions. Med Image Anal. 2012;16(3):642–61.
 3.
Sotiras A, Davatzikos C, Paragios N. Deformable medical image registration: a survey. Medical Imaging IEEE Trans. 2013;32(7):1153–90.
 4.
PuertoSouza GA, Cadeddu JA, Mariottini GL. Toward longterm and accurate augmentedreality for monocular endoscopic videos. Biomed Eng IEEE Trans. 2014;61(10):2609–20.
 5.
van de Ven WJ, Hu Y, Barentsz JO, Karssemeijer N, Barratt D, Huisman HJ. Biomechanical modeling constrained surfacebased image registration for prostate mr guided trus biopsy. Med Phys. 2015;42(5):2470–81.
 6.
Dumpuri P, Clements LW, Dawant BM, Miga MI. Modelupdated imageguided liver surgery: preliminary results using surface characterization. Prog Biophys Mol Biol. 2010;103(2):197–207.
 7.
Rucker DC, Wu Y, Clements LW, Ondrake JE, Pheiffer TS, Simpson AL, Jarnagin WR, Miga MI. A mechanicsbased nonrigid registration method for liver surgery using sparse intraoperative data. Med Imaging IEEE Trans. 2014;33(1):147–58.
 8.
Mostayed A, Garlapati RR, Joldes GR, Wittek A, Roy A, Kikinis R, Warfield SK, Miller K. Biomechanical model as a registration tool for imageguided neurosurgery: evaluation against bspline registration. Ann Biomed Eng. 2013;41(11):2409–25.
 9.
Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD. Augmented reality during robotassisted laparoscopic partial nephrectomy: toward realtime 3dct to stereoscopic video registration. Urology. 2009;73(4):896–900.
 10.
Plantefève R, Peterlik I, Haouchine N, Cotin S. Patientspecific biomechanical modeling for guidance during minimallyinvasive hepatic surgery. Ann Biomed Eng. 2016;44(1):139–53.
 11.
Haouchine N, Dequidt J, Peterlik I, Kerrien E, Berger MO, Cotin S. Imageguided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery. In: Mixed and augmented reality (ISMAR), 2013 IEEE international symposium; 2013. pp. 199–208. (IEEE)
 12.
Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, dos Santos T, MaierHein L, Wagner M, Wünscher J, Kenngott H, et al. Physicsbased shape matching for intraoperative image guidance. Med Phys. 2014;41(11):111901.
 13.
Han L, Hipwell JH, Tanner C, Taylor Z, Mertzanidou T, Cardoso J, Ourselin S, Hawkes DJ. Development of patientspecific biomechanical models for predicting large breast deformation. Phys Med Biol. 2012;57(2):455.
 14.
AlMayah A, Moseley J, Velec M, Brock K. Toward efficient biomechanicalbased deformable image registration of lungs for imageguided radiotherapy. Phys Med Biol. 2011;56(15):4701.
 15.
Hopp T, Dietzel M, Baltzer PA, Kreisel P, Kaiser WA, Gemmeke H, Ruiter NV. Automatic multimodal 2d/3d breast image registration using biomechanical fem models and intensitybased optimization. Med Image Anal. 2013;17(2):209–18.
 16.
Oktay O, Zhang L, Mansi T, Mountney P, Mewes P, Nicolau S, Soler L, Chefdhotel C. Biomechanically driven registration of preto intraoperative 3d images for laparoscopic surgery. In: International conference on medical image computing and computerassisted intervention. Berlin: Springer; 2013. pp. 1–9.
 17.
AlMayah A, Moseley J, Hunter S, Brock K. Radiation dose response simulation for biomechanicalbased deformable image registration of head and neck cancer treatment. Phys Med Biol. 2015;60(21):8481.
 18.
Samavati N, McGrath DM, Jewett MA, van der Kwast T, Ménard C, Brock KK. Effect of material property heterogeneity on biomechanical modeling of prostate under deformation. Phys Med Biol. 2014;60(1):195.
 19.
Samavati N, Velec M, Brock K. A hybrid biomechanical intensity based deformable image registration of lung 4dct. Phys Med Biol. 2015;60(8):3359.
 20.
Sahillioğlu Y, Kavan L. Skuller: a volumetric shape registration algorithm for modeling skull deformities. Med Image Anal. 2015;23(1):15–27.
 21.
SanVicente G, Aguinaga I, Celigueta JT. Cubical massspring model design based on a tensile deformation test and nonlinear material model. Vis Computer Gr IEEE Trans. 2012;18(2):228–41.
 22.
Miller K, Joldes G, Lance D, Wittek A. Total lagrangian explicit dynamics finite element algorithm for computing soft tissue deformation. Commun Numer Methods Eng. 2007;23(2):121–34.
 23.
Lavely WC, Scarfone C, Cevikalp H, Li R, Byrne DW, Cmelak AJ, Dawant B, Price RR, Hallahan DE, Fitzpatrick JM. Phantom validation of coregistration of pet and ct for imageguided radiotherapy. Med Phys. 2004;31(5):1083–92.
 24.
Serban M, Heath E, Stroian G, Collins DL, Seuntjens J. A deformable phantom for 4d radiotherapy verification: Design and image registration evaluation. Med Phys. 2008;35(35):1094–102.
 25.
Ramos A, Simoes J. Tetrahedral versus hexahedral finite elements in numerical modelling of the proximal femur. Med Eng Phys. 2006;28(9):916–24.
 26.
Bourdin X, Trosseille X, Petit P, Beillas P. Comparison of tetrahedral and hexahedral meshes for organ finite element modeling: an application to kidney impact. In: 20th International technical conference on the enhanced safety of vehicle, Lyon;2007.
 27.
Chen Y, Lee J, Eskandarian A. Meshless methods in solid mechanics. Springer: Berlin; 2006.
 28.
Tournois J, Wormser C, Alliez P, Desbrun M. Interleaving delaunay refinement and optimization for practical isotropic tetrahedron mesh generation. ACM Trans Gr. 2009;28(3):1–9.
 29
Brauer JR. What every engineer should know about finite element analysis. Dry Technol. 1995;13(4):1041–2.
 30
Horton A, Wittek A, Joldes GR, Miller K. A meshless total lagrangian explicit dynamics algorithm for surgical simulation. Int J Numer Methods Biomed Eng. 2010;26(8):977–98.
 31
Tang M, Manocha D, Otaduy MA, Tong R. Continuous penalty forces. ACM Trans Gr. 2012;31:107–11079.
 32
Duriez C, Dubois F, Kheddar A, Andriot C. Realistic haptic rendering of interacting deformable objects in virtual environments. Vis Comput Gr IEEE Trans. 2006;12(1):36–47.
 33
Bilston LE. Brain tissue mechanical properties. Stud Mechanobiol Tissue Eng Biomat. 2011;3:69–89.
 34
Wang Y, Cheng JZ, Ni D, Lin M, Qin J, Luo X, Xu M, Xie X, Heng PA. Towards personalized statistical deformable model and hybrid point matching for robust mrtrus registration. IEEE Trans Med Imag. 2016;35(2):589–604.
 35
Martin S, Kaufmann P, Botsch M, Grinspun E, Gross M. Unified simulation of elastic rods, shells, and solids. ACM Trans Gr (TOG). 2010;29(4):39.
Authors’ contributions
WS, XYL, QW and PAH participated in literature search, data analysis, manuscript writing and editing. WXS and XYL contributed equally to this work. All authors read and approved the final manuscript.
Acknowledgments
This paper is supported by Hong Kong Research Grants Council General Research Fund (Project Nos. CUHK 14202514 and CUHK 14203115), National Natural Science Foundation of China (Nos. 61233012, 61305097 and 81601576) and Guangdong province science and technology plan project (No. 2016A020220013).
Competing interests
The authors declare that they have no competing interests.
Declarations
Please contact author for data requests.
Author information
Affiliations
Corresponding author
Additional information
Weixin Si, Xiangyun Liao and Pheng Ann Heng are coauthors
Weixin Si and Xiangyun Liao contributed equally to this work
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Si, W., Liao, X., Wang, Q. et al. Personalized heterogeneous deformable model for fast volumetric registration. BioMed Eng OnLine 16, 30 (2017). https://doi.org/10.1186/s1293801703213
Received:
Accepted:
Published:
Keywords
 Biomechanical deformable volumetric registration
 Tissue–tissue coupling
 Datadriven parameters estimation
 Coarsetofine scheme