- Open Access
Interlaboratory comparison of femur surface reconstruction from CT data compared to reference optical 3D scan
BioMedical Engineering OnLine volume 17, Article number: 29 (2018)
The present study contrasts the accuracy of different reconstructed models with distinctive segmentation methods performed by various experts. Seven research groups reconstructed nine 3D models of one human femur based on an acquired CT image using their own computational methods. As a reference model for accuracy assessment, a 3D surface scan of the human femur was created using an optical measuring system. Prior to comparison, the femur was divided into four areas; “neck and greater trochanter”, “proximal metaphysis”, “diaphysis”, and “distal metaphysis”. The deviation analysis was carried out in GEOMAGIC studio v.2013 software.
The results revealed that the highest deviation errors occurred in “neck and greater trochanter” area and “proximal metaphysis” area with RMSE of 0.84 and 0.83 mm respectively.
In conclusion, this study shows that the average deviation of reconstructed models prepared by experts with various methods, skills and software from the surface 3D scan is lower than 0.79 mm, which is not a significant discrepancy.
Over the past decade, the application of medical imaging technology in clinical diagnostics and therapy has expanded to diverse approaches such as evaluation of joint biomechanics, patient specific implant development and evaluation, statistical modelling, three-dimensional (3D) printing and rapid prototyping, computer-assisted surgery, and preoperative planning [1,2,3,4]. In order to examine the biomechanical behavior of human joints and tissues for all above-mentioned applications, subject specific image based finite element (FE) analysis has been used as the most commonly used method . Extracting the bone geometry from medical images, generating an optimum FE mesh, assigning proper material properties, and defining actual boundary conditions are the main inputs for FE analysis , and therefore their accuracy affects the precision of the FE analysis result . The file format for performing 3D analysis is different from medical image formats for computed topography (CT) to magnetic resonance imaging (MRI). Digital imaging and communication in medicine (DICOM) file formats must be converted to a format readable by CAD or finite element software for further 3D analyses, e.g. the stereolithography (STL) format. The first milestone of construction workflow of image-based biomechanical analysis is accurate segmentation (extracting desired bone geometry from medical image data) from source data [8,9,10]. Therefore, FE analysis for the aforementioned purposes requires accurate 3D shape representation of bones. This is fundamental to obtain segmented bone data as authentic as possible to the patient’s morphology. Since there are many commercial segmentation software packages and algorithms, The STL models segmented with commercially available software packages may have discrepancies compared to the actual bone, and the accuracy of the segmented bone could then vary based on the segmentation method and operator’s skills. Hence, the accuracy assessment of 3D reconstructed bone based on medical images has been recently investigated extensively [8, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]. Yet, it is concerned that such segmented models do not represent the accuracy of the original bone in a FE model. Therefore, the purpose of this study is to investigate the effect of using different segmentation methodologies conducted by experts with different experiences and skills through a round robin test including seven biomechanics laboratories. The deviation of segmented bone models from the cadaver bone geometry obtained by optical 3D scanning was investigated and the discrepancies between reconstructed models from CT images and the model obtained from optical 3D scanning was quantified.
Seven different research groups were invited to participate in this interlaboratory comparison. The results of this research were anonymized and only the principal investigator had access to non-anonymized data. The laboratories were randomly numbered from 1 to 7. In case that more than one model was created by a laboratory, the first and second model of each laboratory was named by additional letters A and B, respectively. Figure 1 presents a flowchart showing the research methodology applied in this study. It briefly shows how the reference STL file and also different laboratories STL file were created.
CT scan acquiring
The CT image of the right femur of a 58 year-old male cadaver was acquired from Trauma Center Murnau with a SOMATOM Definition AS + CT scanner (Siemens AG, Erlangen, Germany). This unique CT image was used by all seven laboratories for segmentation. Soft tissues were removed from the bone prior to the scan. The CT image was saved as DICOM with resolution of 0.29 × 0.29 mm and slice thickness of 0.6 mm for deviation analysis. To reduce partial volume effects, the femur was scanned in a water bath. In order to achieve a calibrated scan, a bone mineral density phantom was scanned in the same setup.
Optical 3D digitization
The outer geometry of the femur was scanned using an optical measuring system at the Fraunhofer Application Centre of Large Structures in Production Engineering (AGP) in Rostock (Fig. 2). The ATOS series of industrial optical 3D scanners provide accurate scans with detailed resolution at high speeds (GOM-Gesellschaft für Optische Messtechnik mbH, Braunschweig, Germany). Instead of measuring single points, ATOS captures an object’s full surface geometry and primitives precisely in a dense point cloud or polygon mesh. The 3D scanner consists of two cameras and a projection system in which the projector projects special stripe light patterns on a surface of an object, being recognized by the cameras. Every scan produces a point cloud with up to 4 Billion points. Several scans from different points of view can be registered by special reference points with a defined diameter, which will be automatically detected by the software. Table 1 presents the specifications of the ATOS scanner for scanning the femur.
Reconstruction of 3D models
DICOM files obtained from the CT scan were used to segment the surface of femurs by four different segmentation software packages: AMIRA® (FEI Visualization Sciences Group, Oregon, USA), Mimics® (Materialise N.V., Leuven, Belgium), YaDiv (Welfenlab, Leibniz Universität Hannover, Hannover, Germany) , and Fiji Life-Line . The bony regions were labeled in all slices of the CT images based on the certain values of Hounsfield units (HU) for bones. The process of labelling of bony structures is a visual and subjective procedure in which a primary HU value for bones was selected from the literature which is around 200–250 up to 3000 [28,29,30,31]. The automated segmentation of the bone started by thresholds of HU and was followed by manually editing the slices to obtain more accurate surfaces [28,29,30,31]. A triangulated surface of the femurs was created with the segmentation software using a semi-automatic method. Removal of the holes and sharp edges which were formed due to semi-automated segmentation was implemented in the above mentioned software [31,32,33,34]. Software information, methods and duration of the segmentation process for each model are tabulated in Table 2. The reconstruction process of CT image was performed by researchers with 3–5 years of experience in segmentation.
Stereolithography (STL) files were collected from all project partners and imported into GEOMAGIC studio v.2013 (Raindrop Geomagic, NC, USA) for deviation analysis. Thereby, the researcher conducting the analysis was blinded towards the participant’s identity in order to avoid bias. Prior to comparison, the femur was divided into 4 areas: “neck and greater trochanter” area, “diaphysis”, “proximal metaphysis” and “distal metaphysis”. Five different planes were defined in global coordinates to divide all models into above-mentioned areas. Proximal end, upper proximal metaphysis, lower proximal metaphysis, distal metaphysis and distal end were the predefined planes for splitting the models into four aforementioned parts . Figure 3 illustrates the predefined cutting planes of the femur. The Neck area includes “neck and greater trochanter” and the “proximal metaphysis” contains the area of lesser trochanter. The “diaphysis” defined as long bone known as the femur shaft and the last part excludes the epiphysis named “distal metaphysis”.
Seven laboratories prepared nine reconstructed models out of one single human femur CT scan. The outer surface of the femur was scanned with high point resolution using an optical 3D scanner. The results of 3D deviation analysis for the four femoral parts were processed in GEOMAGIC studio and compared to the other models. Table 3 presents the average deviation values of the 9 different segmented models for all four predefined areas of the femur. The highest deviation was observed in “neck and greater trochanter” area with RMSE of 0.84. The negative values for the estimated percentage error of the surface areas represent the deviation of the underestimated areas and the positive values show the overestimated areas (see Table 3). Root means square error (RMSE) was used as a standard statistical metric for comparison and evaluation of simulation models performance [36, 37]. Figure 4 also illustrates the visual deviation using color-coded map to show the differences of each model compared to the bone optical 3D scan. Figure 5 illustrates the estimated surface areas of the nine segmented models and the bone optical 3D scan. “Diaphysis” and “neck and greater trochanter” areas have the largest percentage errors of outer surface area with 2.92 and 2.57% respectively. This figure indicates that the outer surface areas of the reconstructed models are not exceedingly different from the reference model.
Finite element models are commonly used based on specific geometrical characteristics extracted from medical imaging data. This study presents a deviation analysis to evaluate different segmentation methods based on CT scan compared to optical 3D surface scan of the same bone. Thereby, the reconstruction results of seven different biomechanics laboratories were compared to evaluate how human skills, methods of segmentation, and different software packages can cause imprecision in image-based reconstructed models.
This study investigated a variety of conditions, which may have influenced the accuracy of the segmentation process. The segmentations of “neck and greater trochanter” area and “proximal metaphysis” showed the greatest deviations with RMSE of 0.84 and 0.83 mm respectively (Table 3). Thevenot et al.  reported the accuracy of a novel method for automatically reconstructed 3D model from 2D hip radiograph and Verim et al.  evaluated the reconstructed proximal femur from different images of different devices. They both found that the greatest error happened in the trochanter area which is in good agreement with our results. Vaananen et al.  assessed the 3D shape of proximal femur using two different methods; shape template and bone mineral density image. They also found out that the maximum discrepancies are in trochanter area, ranging from 0.7 to 2.6 mm. Our results also showed the similar range of discrepancies. Schumann et al.  used clinically relevant morphometric parameters measurement of the proximal femur to examine the accuracy of their reconstructed method. In their study, the highest average deviations were also observed in trochanter area. Rathnayaka et al.  conducted a study to compare the accuracy of MRI and CT reconstructed 3D models where they also estimated the highest deviations was observed in the “neck and greater trochanter” region. The highest deviations, usually observed in the “neck and greater trochanter” area, are probably due to geometrical complications exist in this area. In the current study, the highest estimated discrepancy from the reconstructed models is 0.79 mm. The previous studies of Glaude et al. [11, 43] on accuracy assessment of reconstructed models based on medical images, suggest that the mean 3D deviation of reconstructed models should be in the range of 1 mm. Since clinical hip fractures commonly occur in the neck area , more accuracy in reconstruction of this area is required to have more precise FE analysis results. Furthermore, as observed in Fig. 5, there was no outlier in the accuracy assessment comparison, and all the reconstructed 3D models have similar range of deviations. However, if peak discrepancies are observed, they can be simply disregarded because they are local. The outer surface areas of the reconstructed models providing the surface meshes for FE analysis were also estimated in this study. Highest errors of the outer surface area were observed in “diaphysis” and “neck and greater trochanter” regions. This is also illustrated in Fig. 4 using color-coded map to show that “diaphysis” and “neck and greater trochanter” regions have the highest surface discrepancies compare to the real bone 3D optical scan. Therefore, these two regions are the most critical regions for reconstruction of 3D models based on medical images and should be processed carefully. The results also suggest, that the quality of the image segmentation is rather independent of reconstruction processing software. The differences observed in segmentation times can be associated with either individual investigator speed of segmentation or the usability of the software. For future works, the effect of the negligible discrepancies on the FE analysis results will be examined using a controlled load case in an experimental setup.
This study shows that the average deviation of CT based models, prepared by experts with different skills using various software packages, from a bone surface scan is very low. This reveals that the effect of human expertise and use of different software packages and corresponding methodologies have a negligible effect on the accuracy of the reconstruction procedure from medical images. Therefore, image-based reconstructed models are reliable to use in FE models for clinical applications.
Ellis RE, Tso CY, Rudan JF, Harrison MM. A surgical planning and guidance system for high tibial osteotomy. Comput Aided Surg. 1999;4(5):264–74.
Blemker SS, Asakawa DS, Gold GE, Delp SL. Image-based musculoskeletal modeling: applications, advances, and future opportunities. J Magn Reson Imaging. 2007;25(2):441–51. https://doi.org/10.1002/jmri.20805.
Kanlić EM, DeLaRosa F, Pirela-Cruz M. Computer assisted orthopaedic surgery-CAOS. Bosn J Basic Med Sci. 2006;6(1):7–13.
Rengier F, Mehndiratta A, von Tengg-Kobligk H, Zechmann CM, Unterhinninghofen R, Kauczor H, Giesel FL. 3D printing based on imaging data: review of medical applications. Int J Comput Assist Radiol Surg. 2010;5(4):335–41.
Pauchard Y, Fitze T, Browarnik D, Eskandari A, Pauchard I, Enns-Bray W, Pálsson H, Sigurdsson S, Ferguson SJ, Harris TB, Gudnason V, Helgason B. Interactive graph-cut segmentation for fast creation of finite element models from clinical ct data for hip fracture prediction. Comput Methods Biomech Biomed Eng. 2016;19(16):1693–703.
Pahr DH, Zysset PK. From high-resolution CT data to finite element models: development of an integrated modular framework. Comput Methods Biomech Biomed Eng. 2009;12(1):45–57.
Cattaneo PM, Dalstra M, Frich LH. A three-dimensional finite element model from computed tomography data: a semi-automated method. Proc Inst Mech Eng. 2001;215(2):203–12.
Kang S, Kim M, Kim H, Zhengguo P, Lee S. Accuracy assessment of image-based surface meshing for volumetric computed tomography images in the craniofacial region. J Craniofac Surg. 2014;25(6):2051–5.
Pinheiro M, Alves JL. A new level-set-based protocol for accurate bone segmentation from CT imaging. IEEE Access. 2015;3:1894–906.
van Den Broeck J, Vereecke E, Wirix-Speetjens R, Vander Sloten J. Segmentation accuracy of long bones. Med Eng Phys. 2014;36(7):949–53. https://doi.org/10.1016/j.medengphy.2014.03.016.
Gelaude F, Vander Sloten J, Lauwers B. Accuracy assessment of CT-based outer surface femur meshes. Comput Aided Surg. 2008;13(4):188–99.
Wang LI, Greenspan M, Ellis R. Validation of bone segmentation and improved 3-D registration using contour coherency in CT data. IEEE Trans Med Imaging. 2006;25(3):324–34.
Eckstein F, Charles HC, Buck RJ, Kraus VB, Remmers AE, Hudelmaier M, Wirth W, Evelhoch JL. Accuracy and precision of quantitative assessment of cartilage morphology by magnetic resonance imaging at 3.0 T. Arthritis Rheum. 2005;52(10):3132–6.
Fitzwater KL, Marcellin-Little DJ, La Harrysson O, Osborne JA, Poindexter EC. Evaluation of the effect of computed tomography scan protocols and freeform fabrication methods on bone biomodel accuracy. Am J Vet Res. 2011;72(9):1178–85.
Lalone EA, Willing RT, Shannon HL, King GJ, Johnson JA. Accuracy assessment of 3D bone reconstructions using CT: an intro comparison. Med Eng Phys. 2015;37(8):729–38. https://doi.org/10.1016/j.medengphy.2015.04.010.
Loubele M, Maes F, Schutyser F, Marchal G, Jacobs R, Suetens P. Assessment of bone segmentation quality of cone-beam CT versus multislice spiral CT: a pilot study. Oral Surg Oral Med Oral Pathol Oral Radiol Endodontol. 2006;102(2):225–34. https://doi.org/10.1016/j.tripleo.2005.10.039.
Choi J, Choi J, Kim N, Kim Y, Lee J, Kim M, Lee J, Kim M. Analysis of errors in medical rapid prototyping models. Int J Oral Maxillofac Surg. 2002;31(1):23–32.
Pan Y, Zheng R, Liu F, Jing W, Yong C, Liang X, Bing W. The use of CT scan and stereo lithography apparatus technologies in a canine individualized rib prosthesis. Int J Surg (London, England). 2014;12(5):71–5. https://doi.org/10.1016/j.ijsu.2013.10.006.
Pinsky HM, Dyda S, Pinsky RW, Misch KA, Sarment DP. Accuracy of three-dimensional measurements using cone-beam CT. Dentomaxillofac Radiol. 2006;35(6):410–6.
Shu D, Liu X, Guo B, Ran W, Liao X, Zhang Y. Accuracy of using computer-aided rapid prototyping templates for mandible reconstruction with an iliac crest graft. World J Surg Oncol. 2014;12(1):1.
Taddei F, Martelli S, Reggiani B, Cristofolini L, Viceconti M. Finite-element modeling of bones from CT data: sensitivity to geometry and material uncertainties. IEEE Trans Biomed Eng. 2006;53(11):2194–200.
Wang J, Ye M, Liu Z, Wang C. Precision of cortical bone reconstruction based on 3D CT scans. Comput Med Imaging Graph. 2009;33(3):235–41.
White D, Chelule KL, Seedhom BB. Accuracy of MRI vs CT imaging with particular reference to patient specific templates for total knee replacement surgery. Int J Med Robot Comput Assist Surg. 2008;4(3):224–31.
Oka K, Murase T, Moritomo H, Goto A, Sugamoto K, Yoshikawa H. Accuracy analysis of three-dimensional bone surface models of the forearm constructed from multidetector computed tomography data. Int J Med Robot Comput Assist Surg. 2009;5(4):452–7.
Piller G, McCoy S, Collins C, Sokn S, Ploeg H. Geometric accuracy of physical and surface models created from computed tomography data. Master’s thesis, University of Wisconsin-Madison, Madison, WI. 2012.
Friese K, Blanke P, Wolter F. YaDiV—an open platform for 3D visualization and 3D segmentation of medical data. Vis Comput. 2011;27(2):129–39.
Schindelin J, Arganda-Carreras I, Frise E, Kaynig V, Longair M, Pietzsch T, Preibisch S, Rueden C, Saalfeld S, Schmid B. Fiji: an open-source platform for biological-image analysis. Nat Methods. 2012;9(7):676–82.
Ahmady A, Soodmand E, Soodmand I, Milani TL. The effect of various heights of high-heeled shoes on foot arch deformation: finite element analysis. J Foot Ankle Res. 2014;7(Suppl 1):A78.
Soodmand E, Natsakis T, Jonkers I, Vander Sloten J, editors. Intra-articular pressure based stress analysis of the distal tibia following insertion of a total ankle replacement. London: Taylor & Francis; 2015.
Treece GM, Prager RW, Gee AH. Regularised marching tetrahedra: improved iso-surface extraction. Comput Graph. 1999;23(4):583–98.
Kluess D, Wieding J, Souffrant R, Mittelmeier W, Bader R. Finite element analysis in orthopaedic biomechanics. Rijeka: INTECH Open Access Publisher; 2010.
Kluess D, Souffrant R, Mittelmeier W, Wree A, Schmitz K, Bader R. A convenient approach for finite-element-analyses of orthopaedic implants in bone contact: modeling and experimental validation. Comput Methods Programs Biomed. 2009;95(1):23–30.
Kluess D, Schultze C, Lubomierski A, Mittelmeier W, Schmitz K, Bader R. Finite-element-analysis of a cemented ceramic knee arthroplasty under worst case scenarios: abstracts of the 16th Congress, European Society of Biomechanics. J Biomech. 2008;41(Supplement 1):S237.
Chu C, Chen C, Liu L, Zheng G. FACTS: fully Automatic CT segmentation of a hip joint. Ann Biomed Eng. 2015;43(5):1247–59.
BruceBlaus.com staff. Blausen gallery 2014. Wikiversity J Med. 1(2). https://doi.org/10.15347/wjm/2014.010. ISSN 20018762.
Willmott CJ, Ackleson SG, Davis RE, Feddema JJ, Klink KM, Legates DR, O’donnell J, Rowe CM. Statistics for the evaluation and comparison of models. J Geophys Res. 1985;90(C5):8995–9005.
Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014;7(3):1247–50.
Thevenot J, Koivumaki J, Kuhn V, Eckstein F, Jamsa T. A novel methodology for generating 3D finite element models of the hip from 2D radiographs. J Biomech. 2014;47(2):438–44. https://doi.org/10.1016/j.jbiomech.2013.11.004.
Verim O, Tasgetiren S, Er MS, Timur M, Yuran AF. Anatomical comparison and evaluation of human proximal femurs modeling via different devices and FEM analysis. Int J Med Robot Comput Assist Surg MRCAS. 2013;9(2):e19–24. https://doi.org/10.1002/rcs.1442.
Vaananen SP, Isaksson H, Julkunen P, Sirola J, Kroger H, Jurvelin JS. Assessment of the 3-D shape and mechanics of the proximal femur using a shape template and a bone mineral density image. Biomech Model Mechanobiol. 2011;10(4):529–38. https://doi.org/10.1007/s10237-010-0253-3.
Schumann S, Tannast M, Nolte L, Zheng G. Validation of statistical shape model based reconstruction of the proximal femur—a morphology study. Med Eng Phys. 2010;32(6):638–44. https://doi.org/10.1016/j.medengphy.2010.03.010.
Rathnayaka K, Momot KI, Noser H, Volp A, Schuetz MA, Sahama T, Schmutz B. Quantification of the accuracy of MRI generated 3D models of long bones compared to CT generated 3D models. Med Eng Phys. 2012;34(3):357–63. https://doi.org/10.1016/j.medengphy.2011.07.027.
Gelaude F, Vander Sloten J, Lauwers B. Semi-automated segmentation and visualisation of outer bone cortex from medical images. Comput Methods Biomech Biomed Eng. 2006;9(1):65–77.
ES: CT image segmentation, major contributor in writing the manuscript, performing the data analysis. DK: designing the study and organizing participants. PAV: CT image segmentation. RC: CT image segmentation. MS: CT image segmentation. DG: CT image segmentation. FN: Designing 3D optical scanning. DP: CT image segmentation. MW: CT image segmentation and design the study. All authors read and approved the final manuscript.
We thank Prof. Dr.-Ing. H. Seitz for support using Materialise Mimics® software. This study was organized within the structures of the Musculoskeletal Biomechanics Network (MSB-Net) of the Basic Research section within the German Society for Orthopaedics and Trauma. The human bone specimen was obtained through an anatomic gifts program (Science Care, Phoenix, AZ, USA) and its use was authorized by the responsible ethical committee (BLAEK 2011-058, Munich, Germany).
The authors declare that they have no competing interests.
Availability of data and materials
The datasets generated and/or analyzed during the current study are available in the Biomechanics and Implant Technology Research Laboratory, Department of Orthopedics, University Medicine Rostock repository. The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.
Consent for publication
Ethics approval and consent to participate
The human bone specimen was obtained through an anatomic gifts program (Science Care, Phoenix, AZ, USA) and its use was authorized by the responsible ethical committee (BLAEK 2011-058, Munich, Germany).
We acknowledge the internal funding program of University Medicine Rostock (‘Förderung zur Vorbereitung von Anträgen der Verbundforschung’) to financially support this project.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Soodmand, E., Kluess, D., Varady, P.A. et al. Interlaboratory comparison of femur surface reconstruction from CT data compared to reference optical 3D scan. BioMed Eng OnLine 17, 29 (2018). https://doi.org/10.1186/s12938-018-0461-0
- Accuracy assessment
- Deviation analysis
- Image-based model
- Bone segmentation
- Shape reconstruction
- Medical imaging
- Round robin test