Augmenting intraoperative ultrasound with preoperative magnetic resonance planning models for percutaneous renal access

(i). US calibration

The calculation of T _C and T _T will be given later in Section 3. After the calibration, each pixel in all acquired US slices can be positioned in the intraoperative space as Z _US = {z _US,i }.

(ii). Respiratory gating

Because of the organ motion due to respiration, inconsistency will occur between the positions of the anatomical features such as vascular structures in different US slices. This will lead to significant registration error. Therefore, we expect to acquire US slices at the same stages of the respiration cycles. It has been shown that for free respiration the kidney assumes the same positions at equivalent lung volumes [15]. Moreover, the end-exhale represents the longest natural pause in a cycle [16]. Thus, we expect to use only US slices at the maximum exhalation positions.

This can be achieved by an optical tracking based respiratory gating technique. First, the US probe is placed on the caudal end of the patient’s sternum, where the US slice plane is approximately parallel to the transverse section of the human body. At this position, we define a cranio-caudal line l ₀ that is perpendicular to the transverse section of the human body and passes through the central point of the US transducer face. The line l ₀ is considered as a reference axis. In order to acquire one US slice of interest at the maximum exhalation, the probe is placed at the specified location, and a certain number of slices are acquired and stored at a stable acquiring rate. Meanwhile, the instantaneous distance from the centroid of the optically-tracked markers mounted on the probe to the reference axis l ₀ is calculated. The US slice with the minimum distance is considered as the required maximum exhalation slice, as shown in Figure 3. In such a case, the maximum exhalation serves as a trigger for slice selection, accounting for the breathing motion and ensuring consistency of all used US slices.

(iii). MR to US affine registration

We aim to transfer the preoperative planning onto the patient in situ by means of MR to US registration. With respect to clinical demand, surgeons prefer an effective image guidance that optimizes the surgeon's hand-eye coordination, but are often reluctant to accept over complicated workflow. To this end, we do not use a MR to densely-sampled US volume registration studied by several groups, which requires overlong intraoperative processing time [17]. Meanwhile, we decide not to use the VNav method after careful consideration, which often suffer from less accuracy [8]. Instead, we propose an efficient approach on top of an affine registration based on two pairs of orthogonal US images.

The affine registration is equivalent to find a transformation T _R that best aligns the target point set X _MR with the source point set Z _US. The iterate closest point (ICP) algorithm [18] is often used to estimate a transformation that minimizes the mean square distance between two point sets. However, the original ICP is sensitive to the initial pose and could be prone to fall into local minima, especially for noisy feature sets, e. g., a point-cloud selected in US images. Moreover, the free landmark extraction is often inefficient and leads to inaccurate registration, mainly because it utilizes insufficient “geometrical information”. In order to overcome these limitations, we propose an orthogonal-slice ICP (OICP) algorithm detailed as follows.

First, the US probe is swept near the 11th intercostal space. A few US images are acquired at the maximum exhalation positions based on the respiratory gating described above. Two pairs of almost orthogonal images, U ₁, U ₂, U ₃ and U ₄ are selected, where U ₁, U ₂ are approximately parallel to the transverse section of the body and U ₃, U ₄ are acquired with the probe along the midaxillary line. U ₁, U ₂ contain clearly visible hilum vein, inferior vena cava (IVC), and a transverse kidney contour while U ₃, U ₄ contain a clearly visible longitudinal kidney contour. Note that both the transverse and longitudinal contours should better be complete, or at least represent most of the kidney.

The estimate starts with calculating an initiate transformation T ₀ by aligning three pairs of landmark points selected from the cranial end, caudal end and kidney hilum on both the US slices and the MR segmentation images, as shown in Figure 4. The initial alignment can be calculated using the Procrustes analysis [19]. This preprocessing allows quick algorithm convergence without falling into local minima. Assuming a fine initial registration, the following optimization can be constrained within a small translation range ± A and a small rotation angle range ± α, thus reducing the computing time while improving the reliability. The translation and rotation are then further optimized to minimize the mean square error (MSE). The entire optimization is summarized below.

Input: K _US and V _US manually picked from orthogonal US slices U ₁, U ₂, U ₃ and U ₄, K _MR, V _MR preoperatively segmented from the MR volume;

Output: T _R ;

Initiation: The starting registration T ₀ is obtained by aligning the three pairs of landmark points.

Iteration: for n = 1 to n _max or until convergence do

1. 1.

   Compute the closest point $c{k}_i=C\left(T_{n-1}{k}_{\text{US},i},K_{\text{CT}}\right)$for i = 1,…, K ₂ and $c{v}_i=C\left(T_{n-1}{v}_{\text{US},i},V_{\text{CT}}\right)$ for i = 1,…, V ₂;

    
2. 2.

   Compute an update T _n that minimizes the MSE between T _n-1 Z _US and {ck _i }∪{cv _i } with translation within ± A and rotation within ± α.

    
3. 3.

   End the iteration when n = n _max or the decrease of the MSE is below a threshold h.

    

With the final estimate of T _R , the transformation from the US plane to the MR image space S _MR can be given by $T=T_R^i{T}_T{T}_C$. Figure 5 shows the corresponding US and MR slices from a healthy volunteer, where the longitudinal kidney contour is well aligned.

(iv). Augmented US-based guidance

Based on T _R , the planning and MR anatomical models can be transferred from the preoperative space S _MR into intraoperative space S _tra. A puncture trestle is mounted to the US probe to restrict the needle trajectory to several given angles within the US plane. The needle position can be precisely measured by the tracker in real time and fused in the guidance image as a virtual model. The visual guidance is provided by augmenting the US images with 3D anatomical models, the planning and the virtual needle, as shown in Figure 6.

One should notice that the transformation T _R is calculated using only US slices at the maximum exhalation positions. Therefore, at the other stages of the respiratory circle, the accuracy of registered needle trajectory cannot be guaranteed because of the organ shift and soft-tissue deformation. In such a case, we expect to perform the puncture at maximum exhalation positions. To this end, we make use of the respiratory gating technique proposed above to plot the respiratory curve (as shown in Figure 3) in real time. The maximum exhalation positions can then be detected by visual inspection of the surgeon. Meanwhile, the needle is inserted rapidly into the intrarenal target. Under the presented image guidance, the puncture trajectory can be guided and guaranteed to be coincident with the planning.

(i). Registration evaluation

Here, we aim to measure the registration accuracy in terms of the root mean square (RMS) target registration error (TRE) on MR and US data provided by volunteers. Because no gold standard is available, the MR to densely-sampled US volume registration proposed in [17] was used as a bronze standard [20].

The true-FISP MR data was acquired by scanning four healthy volunteers (distinguished by A, B, C, and D) on a Siemens MAGNETOM Trio Tim 3.0 T machine. Written informed consent from all volunteers was obtained. The voxel resolution was set to 1.12 × 1.12 × 1.00 mm³ to approximate the isotropy. The US images were acquired using a Mindray DC-7 machine with a 3.5 MHz abdominal probe and then captured using an EDIUX NX video grabber from CANOPUS. A passive Polaris system from Northern Digital Incorporation (NDI) was used for position tracking. The US probes was mounted with optically-tracked reflective markers such that the position of the probe can be tracked. The Mindray US machine provides a built-in calibration application to output the distance from the left bottom of the US slice to the center of the arc-shaped probe surface, such that the transformation T _C can be calculated directly. Note that for each imaging depth, the built-in US calibration algorithm only need to run once before the surgery. Then, the output can be stored in a mapping table between transformation T _C and imaging depth for future use. The transformation T _T was calculated automatically by means of the tracker’s real time output. Given T _C and T _T , all US slices can be calibrated and located in the tracker space S _tra, as described in section 2.

For acquiring one desired US slice at the maximum exhalation via the proposed respiratory gating, 50 to 100 slices were acquired at a stable rate of 20 slices per second using the video grabber. For calculating the bronze standard registration, 125 US images were selected at the maximum exhalation positions from each volunteer, covering from transverse to longitudinal views of the kidney. Four urologists with expertise in US-guided renal intervention were asked to individually conduct the proposed registrations and the bronze standard registration for each volunteer (denoted as Test 1–4). For calculating the OICP registration, the translation range and the rotation range are set to A = 28 mm and α = 30°, respectively. The threshold h for terminating the iterations was set to 0.1.

The accuracy can then be measured as follows. All voxel positions within the kidney model were transformed from S _MR into S _tra, using both the proposed registration method and the bronze standard. The RMS error between the two corresponding position sets was then calculated. Thus, the accuracy in terms of RMS TRE was obtained. To evaluate the precision, or repeatability of the proposed method, the RMS distance from the transformed positions to their average, i.e., the standard deviation, was calculated. The processing time for the proposed registration was also recorded, including the Procrustes initial registration and OICP optimization.

(ii). Phantom trials

A triple-modality (CT, MR, US) abdominal phantom model 057 from Computerized Imaging Reference Systems (CIRS) was used for the phantom trials. The internal structure of the model 057 includes partial abdominal aorta, partial vena cava, spine and two partial kidneys each with a lesion. The lesions are high contrast relative to the background in MR and can be barely identified in US. Each lesion can be punctured 3–5 times. These features make it a useful tool for evaluating the targeting accuracy of a multimodality image-guided PRA.

First, the phantom was scanned with the same Simons MR machine. The MR volume data was then pulled onto the augmented-US based guidance system for surgery planning. With the 3D reconstruction of the segmented kidney, lesions, spine and skin, two planning trajectories were defined, each including an entry point on the skin and a target point within a lesion.

The same four interventionists were asked to perform the RFA on the phantom (Test 1–4). Two pairs of orthogonal US slices containing longitudinal and transverse contours of the kidney were selected from real time US data for MR-US registration. Because the phantom is approximately rigid, the breathing gate technique was not used here. Three points from the cranial end, the middle, and the caudal end of the kidney were used for the initial alignment. 8 to 15 points from the kidney surface and the vena cava were used for the OICP registration. A rigid plastic needle equipped with four fixed optically-tracked markers was used. A puncture trestle was mounted to the US probe such that the needle trajectory was restricted, as shown in Figure 7. This needle can be located in real time in space S _tra and was intended to reach the target lesion. By observing the proposed visualized guidance, each interventionist performed two needle punctures. The environment of the phantom test is shown in Figure 8. After each PRA trial, MR scanning was performed to assess the puncture accuracy. The distance between the needle tip and the lesion center, denoted as needle-target distance (NTD), was measured based on the multiplanar reconstructed images.

The perceptual quality of the image guidance system for surgeons is very important, as their satisfaction relates to the therapeutic impact in selecting the system in clinical practice. The perceptual quality for the proposed PRA was rated in terms of three criteria according to a custom scoring system, as follows:

Intervention Improvement from 5 to 1 respectively denotes significant, meaningful, moderate, fair, and little localization improvement when the proposed guidance is employed. Workflow Impact indicates the acceptability of the proposed workflow, where 5 to 1 respectively denote positive, acceptable, acceptable after training, acceptable with reluctance, and unacceptable. Clinical Relevance denotes the clinical value of the proposed framework, where scores 5 to 1 correspond to values of high, moderately high, medium, moderately low to low. The evaluators were allowed to rate x.5 that represents an assessment between x and x + 1.