- Open Access
An optical tracker based robot registration and servoing method for ultrasound guided percutaneous renal access
- Dongwen Zhang†1, 2Email author,
- Zhicheng Li†1,
- Ken Chen1,
- Jing Xiong1,
- Xuping Zhang1 and
- Lei Wang1Email author
© Zhang et al.; licensee BioMed Central Ltd. 2013
- Received: 29 December 2012
- Accepted: 15 May 2013
- Published: 24 May 2013
Robot-assisted needle steering facilitates the percutaneous renal access (PRA) for their accuracy and consistency over manual operation. However, inaccurate image-robot correspondence and uncertainties in robot parameters make the needle track deviate from the intrarenal target. This paper aims to simplify the image-tracker-robot registration procedure and improves the accuracy of needle alignment for robot assisted ultrasound-guided PRA.
First, a semi-automatic rigid registration is used for the alignment of the preoperative MR volume and the intraoperative orthogonal US slices. Passive markers are mounted both on US probe and robot end-effector, the planned puncture path is transferred from the MR volume frame into optical tracker frame. Tracker-robot correspondence and robot calibration are performed iteratively using a simplified scheme, both position and orientation information are incorporated to estimate the transformation matrix, only several key structural robot parameters and joint zero-positions are calibrated for simplicity in solving the inverse kinematic. Furthermore, an optical tracker feedback control is designed for compensating inaccuracies in robot parameters and tracker-robot correspondence, and improving the accuracy of needle alignment. The intervention procedure was implemented by a telemanipulated 5R1P robot, two experiments were conducted to validate the efficiency of robot-tracker registration method and the optical tracker feedback control, robot assisted needle insertion experiment was conducted on kidney phantom to evaluate the system performance.
The relative positioning accuracy of needle alignment is 0.24 ± 0.08 mm, the directional accuracy is 6.78 ± 1.65 × 10-4rad; the needle-target distance of needle insertion is 2.15 ± 0. 17 mm. The optical tracker feedback control method performs stable against wide range of angular disturbance over (0 ~ 0.4) radians, and the length disturbance over (0 ~ 100) mm.
The proposed optical tracker based robot registration and servoing method is capable of accurate three dimension needle operation for PRA procedure with improved precision and shortened time.
- Needle Insertion
- Haptic Device
- Optical Tracker
- Optical Marker
- Robot Calibration
Robot-assisted needle insertion facilitates many minimally invasive percutaneous procedures such as biopsy, electrolytic ablation and renal intervention, where a similar establishment of reliable and consistent access track from skin to the inside anatomical feature is required. In percutaneous renal intervention, it is important to locate the needle tip as well as the track of the needle shaft under intra-operative guidance of x-ray or ultrasound images [1–3]. Accurate steering and placement of needle is challenging due to uncertainties in image-robot correspondence, which makes the needle track deviate from the target.
Numbers of robotic systems have been proposed for eliminating radiation exposure and simultaneously increasing accuracy in radiologic interventions. Bzostek et al.  used a stereo-pair of two x-ray views registered to a common fiducial system with an active robot to assist needle placement. Yu Zhou et al. introduced a CT-guided robotic needle biopsy technique for lung nodules. Based on the nodule respiratory motion model, needle placement is planned to follow an optimal timing and path, and is triggered based on the respiratory phase tracking . The PAKY-RCM incorporates a passive robotic arm and a friction transmission with axial loading system, which allows a urologist to remotely align the needle along a selected trajectory path under fluoroscopic guidance using the superimposed registration principle . These methods all require time consuming pre-operative registration procedures between robot, imaging system and the patient's anatomy. Patriciu et al. uses the laser markers readily available on any CT scanner for robot registration in computer tomography imaging systems. This approach does not require additional hardware, laser alignment being performed on the instrument used in the clinical application . An automatic image-guided control based on visual servoing and principles of projective geometry is presented for automatic and uncalibrated needle placement under CT-fluoroscopy. The approach demonstrated good targeting accuracy by using the procedure needle as a marker, without additional registration hardware .
Robotic percutaneous interventions guided by ultrasound are developed in recent decades for that ultrasound (US) is radiation-free, real-time and easy-to-use . J Hong et al. proposed an ultrasound-driven needle insertion robot for percutaneous cholecystostomy, which is capable of modifying the needle path by real-time motion compensation through visual servo control before needle insertion . Robot assisted and ultrasound guided ablative therapy and biopsy operation are also studied [9, 10], an optical/electromagnetic marker mounted on ultrasound probe are used to measure the transducer’s position and orientation, once the puncture path is defined, the robotic arm moved automatically to the planned insertion path. Ultrasound image-based visual servoing techniques have not been used in percutaneous interventions for that the abdominal US is often related to limited anatomy identification and targeting abilities, providing only two-dimensional(2D) anatomical information with poor quality [3, 11, 12].
In our previous works, an augmenting intraoperative ultrasound with preoperative magnetic resonance planning models for PRA was proposed and evaluated by urologists on a kidney phantom. With careful setup it can be efficient for overcoming the limitation of current US-guided PRA [13, 14]. In this paper, a telemanipulated 5R1P robot is employed for needle operation. We present an optical tracker based robot registration and servoing method for ultrasound-guided PRA, optical tracker serves as intermediate coupling tool for image-robot registration and error feedback control for needle alignment. The rest of the paper is organized as follows, introduction of experiment setup and navigation systems, image-robot registration and robot control scheme are illustrated in Sec. II. The last two sections describe the experiment and discussions.
Procedures of robot assisted percutaneous renal intervention
The robot assisted percutaneous renal intervention surgery workflow consists of preoperative surgical planning, intraoperative surgical navigation and semi-autonomous telemanipulated needle operation. First, the patient is scanned by magnetic resonance (MR), kidney, vessels and tumor are then segmented from the MR volume as 3D model, such that a surgeon can make a optimal surgical plan preoperatively. During the surgery, a semi-automatic rigid registration is performed for the alignment of the US slices and the MR volume, the preoperative planning can be transferred onto the patient in situ. With an image-guidance interface, the surgeon guide the robot to the insertion point, needle alignment and interventional puncture can be performed autonomously in accordance with the surgical planning. Verification that the needle has successfully gained access to the collecting system will be provided by the return of urine through the trocar needle. The needle will be detached from the robot, and subsequent surgical procedure continues.
Registration of image-robot-tracker
Registration of the robot to the image space provides us with the essential relationship between the needle location and the targets in image coordinate. Indeed, inaccurate robot-image calibration has a direct impact on the accuracy of the needle steering.
(i). Image-Tracker registration
In this section, we propose a simplified registration method for both robot-tracker correspondence and robot calibration.
A dual number quaternion based algorithm was employed to estimate the transformation matrix , which incorporates both orientation and translation information. However, inaccuracy in robot forward kinematics seriously affects the validity of registration result. Robot calibration is required to reduce the registration error as well as inaccuracies in robot parameters of links and joint angles.
(iii). Calibration of robot parameters
then the robot parameters can be compensated by , . The least square method tends to change the mechanical structure of robot completely when the estimated parameters deviate a lot from the actual ones. Only 5 rotational joint zero-positions, 4 link lengths and 3 marker positions are chosen to calibrate for consistency and simplicity in solving the inverse kinematic.
(iv). Simplified two-step scheme for robot-track registration
In this section, we introduce a simplified two-step registration scheme for both robot-tracker correspondence and robot calibration. The entire registration is summarized as follow.
Input: corresponded frame pairs, k = 1 ⋯ K of the optical makers measured via optical tracker and robot forward kinematics respectively;
Output: transformation and robot parameters (X, Θ);
Initialization: robot parameters (X0, Θ0) are initialized by the nominal settings;
Compute the transformation matrix by minimizing the object function (7);
Update the marker positions and deviation matrix using the newer robot kinematics F(X n , Θ n );
- 3.Calibrate and compensate robot parameters(16)
End the iteration when n = n max or the decrease of the MSE below a threshold h.
With the image-guidance interface, the surgeon telemanipulated the robot to approach the insertion point manually in free space, needle alignment and interventional puncture are performed autonomously in accordance with the surgical planning. In this study, the haptic device acted as the master controller, while the 5R1P robot performed as the slave needle operator. The master and the slave were connected through a communication network.
(i). Master–slave control for manually needle approaching
where Λ is a scaling diagonal matrix, different scaling ratio was assigned to each joint pair according to their contributions to the translation and rotation of the end-effector. Small ratio helps reduce disturbances in manual input. The calculated joint velocities were then sent to the Mitsubishi alternate current servo-unit, all five joints were controlled simultaneously to approach the puncture point, the linear motor were controlled by safe button on the joystick of master separately for needle insertion.
(ii). Optical tracker feedback control for needle alignment
Inaccuracy of robot-tracker correspondence and robot parameters impacts the absolute precision severely when using the internal control system of the robot itself. But since the relative accuracy is better than the allowed tolerances the robot can be adjusted until the absolute accuracy is good enough [20, 21]. This section presents an optical tracker feedback control method to improve the accuracy of needle alignment for manual or robotic needle steering operations in soft tissue.
Initialize the pose of end-effector by the estimated ;
Solve the inverse kinematics of robot , command the joints move to Θ k ;
- 3.Measure the actual pose of end-effector using optical tracker, and compute the error,
Modify a new command by error compensation , go to step 2;
Stop until iteration time k = k max or the error below threshold h.
Three experiments were conducted to validate the efficiency of robot-tracker registration method and the optical tracker feedback control for needle alignment task.
The nominal parameters of robot
Position of maker (mm): (40.00, 0.00, 160.00)
Calibrated robot parameter
Position of maker (mm): (40.32, 0.71, 158.02)
Optical tracker based error compensation experiment
the approximation holds only for small directional deviations. To compensate robot parameter disturbances, the robot was driven to the modified poses iteratively, and the minimum error was selected in 10 iterations with position threshold 0.2. In this case, the same target pose was used in both stages.
Combined disturbance levels in robot parameters
Link length error (mm)
Joint angle error (rad)
Marker position (mm)
Robot-tracker orientation (rad)
Robot-tracker displacement (mm)
Error magnitudes of optical tracker feedback control
Uncorrected translational error (mm)
Uncorrected rotational error (rad)
Corrected translational error (mm)
Corrected rotational error (E-4 rad)
Robot assisted needle insertion experiment
A triple-modality (CT, MR, US) abdominal phantom model 057 from Computerized Imaging Reference Systems (CIRS) was used for the in vivo data Test. The internal structure of the model 057 includes partial abdominal aorta, partial vena cava, spine and two partial kidneys each with a lesion. The lesions are high contrast relative to the background in MR and can be barely identified in US.
First, the phantom was scanned with Siemens MAGNETOM Trio Tim 3.0 T machine, meanwhile the robot was calibrated to the optical tracker frame following the process in section 2. To avoid the accumulated US-MR registration error in robot assisted needle insertion experiment, 7 silicon square makers were attached to the surface of phantom, a rigid registration was employed to transform the MR image to optical tracker frame directly by the corresponded position pairs of the silicon markers both in MR image frame and optical tracker frame.
Results of robot assisted needle insertion experiment on kidney
Position error of alignment (mm)
Direction error of alignment (E-4 rad)
Position error of insertion (mm)
In previous study , the accuracy of the volume navigation was evaluated via puncture tests on a customized phantom. The mean needle-target distance was 2.7 mm for the trials performed by an experienced radiologist, while 3.1 mm for a medical resident without experience. With the help of the optical tracker based feedback control, precise needle alignment could facilitate the follow-up manual needle insertion or robotic needle steering. When the positioning accuracy of tracking system increases, the absolute positioning accuracy of needle alignment will increase. However, in needle steering stage, the positioning information from the tracker was incorrect due to bending of needle shaft in soft tissue. Further work will use magnetic sensor to track the precise needle tip or intra-operation visual servoing technique, more dexterous needle steering inside tissue will be studied.
This paper presents an integrated needle operation robot system for percutaneous renal intervention. A simplified image-tracker-robot registration procedure was introduced. Variations in robot geometric parameters and tracker-robot correspondence account for the needle positioning accuracy of robot. An optical tracker feedback control was proposed and validated to compensate these disturbance for needle alignment. The accuracy is inherited from the optical positioning system. Experiments show that the control scheme is capable of providing accurate 3D needle alignment, and compensating wide range of disturbance in robot parameters and tracker-robot correspondence. Robot assisted needle insertion experiments were performed on kidney phantom, precise needle alignment could improve the precision of needle insertion. Robot-assisted needle steering has the potential to improve the accuracy through more dexterous control of the needle-tip trajectory, further work will involve tip/base needle manipulation for needle steering in soft tissue .
This study was financed partially by the Projects of National Natural Science Foundation of China (Grant Nos. 60932001 and 61072031), the National 863 Program of China (Grant No. 2012AA02A604), the National 973 Program of China (Grant No. 2010CB732606), the Next generation communication technology Major project of National S&T (Grant No. 2013ZX03005013), the Key Research Program of the Chinese Academy of Sciences (Grant No.), and the Guangdong Innovation Research Team Funds for Low-cost Healthcare and Image-Guided Therapy.
- Bzostek , Schreiner S, Barnes AC, Cadeddu JA, Roberts WW, Anderson JH, Taylor RH, Kavoussi L: An automated system for precise percutaneous access of the renal collecting system. Lecture Notes in Computer Science 1997, 1205: 299–308. 10.1007/BFb0029249View ArticleGoogle Scholar
- Su LM, Stoianovici D, Jarrett TW, Patriciu A, Roberts WW, Cadeddu JA, Ramakumar S, Solomon SB, Kavoussi LR: Robotic percutaneous access to the kidney: comparison with standard manual access. J Endourol 2002, 16(7):471–5. 10.1089/089277902760367421View ArticleGoogle Scholar
- Lee JY, Choi BI, Chung YE, Kim MW, Kim SH, Han JK: Clinical value of CT/MR-US fusion imaging for radiofrequency ablation of hepatic nodules. Eur J Radiol 2012, 81(9):2281–9. 10.1016/j.ejrad.2011.08.013View ArticleGoogle Scholar
- Zhou Y, Thiruvalluvan K, Krzeminski L, Moor WH, Xu Z: Liang Z. CT-guided robotic needle biopsy of lung nodules with respiratory motion–experimental system and preliminary test: International Journal of Medical Robotics and Comput Assist Surgery; 2012 Jun 13. http://onlinelibrary.wiley.com/doi/10.1002/rcs.1441/pdfGoogle Scholar
- Patriciu A, Solomon SB, Kavoussi LR, Stoianovici D: Robotic kidney and spine percutaneous procedures using a new laser-based CT registration method. MICCAI 2001 Lecture Notes in Computer Science 2001, 2208: 249–257.Google Scholar
- Loser MH, Navab N: A new robotic system for visually controlled percutaneous interventions under CT fluoroscopy. MICCAI 1999, Lecture Notes in Computer Science, Springer-Verlag 2000, 1935: 887–896.Google Scholar
- Hosseini MM, Hassanpour A, Farzan R, Yousefi A, Afrasiabi MA: Ultrasonography-guided percutaneous Nephrolithotomy. J Endourol 2009, 23(4):603–60. 10.1089/end.2007.0213View ArticleGoogle Scholar
- Hong J, Dohi T, Hashizume M, Konishi K, Hata N: An Ultrasound-driven needle insertion robot for percutaneous cholecystostomy. Phys Med Biol 2004, 49(3):441–55. 10.1088/0031-9155/49/3/007View ArticleGoogle Scholar
- Deng S, Jiang L, Cao Y, Liang P, Ren H, Tong L, Wang Y: A needle-holding robot for ultrasound guided percutaneous hepatic microwave ablation and initial experiments. Intelligent Robotics and Applications Lecture Notes in Computer Science 2008, 5315: 1173–1182. 10.1007/978-3-540-88518-4_125View ArticleGoogle Scholar
- Kettenbach J, Kronreif G, Figl M, Fürst M, Birkfellner W, Hanel R, Bergmann H: Robot-assisted biopsy using ultrasound guidance: initial results from in vitro tests. Eur Radiol 2005, 15(4):765–71. 10.1007/s00330-004-2487-xView ArticleGoogle Scholar
- Bachta W, Krupa A: Towards ultrasound image-based visual servoing. IEEE Int. Conf. on Robotics and Automation, ICRA'2006 2006, 4112–4117.Google Scholar
- Novotny P, Howe R, Dupont P: Real-time 3D ultrasound-based servoing of a surgical instrument. IEEE Int. Conf. on Robotics and Automation, ICRA'2006 2006, 613–618.Google Scholar
- Zhicheng L, Kai L, Hailun Z, Ken C, Jia G, Lei W: Augmenting intraoperative ultrasound with preoperative magnetic resonance planning models for percutaneous renal access. Biomed Eng Online 2012, 11–60.Google Scholar
- Zhicheng L, Jacob C, Jia G: Ultrasound-based surgical navigation for percutaneous renal intervention: in vivo measurements and in vitro assessment. IEEE International Conference on Image Processing (ICIP) 2011, 11–14.Google Scholar
- Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G: User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 2006, 31(3):1116–1128. 10.1016/j.neuroimage.2006.01.015View ArticleGoogle Scholar
- Besl PJ, McKay ND: A method for registration of 3-D shapes. IEEE Trans. on Patt. and Mach. Intell. 1992, 14(2):239–256. 10.1109/34.121791View ArticleGoogle Scholar
- Walker MW, Shao L, Volz RA: Estimation 3D location parameters using dual number quaternions. Image Understanding 1991, 54(3):358–367. 10.1016/1049-9660(91)90036-OView ArticleGoogle Scholar
- Haytai SA: Robot arm geometric link parameters estimation. Proceedings of the 22nd Conference on Decision and Control 1983, 1477–1483.View ArticleGoogle Scholar
- Wu C-H: Robot accuracy analysis based on kinematics. IEEE J Robot Autom 1986, RA-2(3):171–179.Google Scholar
- Sunnanbo A: Laser feedback control for robotics in aircraft assembly. Sweden: Department of Production Systems, Linköping University; 2003.Google Scholar
- Bajracharya M, Dicicco M, Backes P, Nickels K: Visual end-effector position error compensation for planetary robotics. Journal of Field Robotics 2007, 24(5):399–420. 10.1002/rob.20186View ArticleGoogle Scholar
- Wiles AD, Thompson DG, Frantz DD, Accuracy assessment and interpretation for optical tracking systems: Medical imaging: visualization, image-guided procedures, and display. 2004, 5367: 421–432.View ArticleGoogle Scholar
- Reed KB, Majewicz A, Kallem V, Alterovitz R, Goldberg K, Cowan NJ, Okamura AM: Robot-assisted needle steering. IEEE Robot Autom Mag 2011, 18(4):35–46.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.