Open Access

Reproducibility of image-based computational models of intracranial aneurysm; methodological issue

BioMedical Engineering OnLine201615:109

Received: 11 August 2016

Accepted: 11 August 2016

Published: 16 September 2016


Aneurysm Angiography CFD CTA MRA Reproducibility Methodology Misdiagnosis Mismanagement

I was interested to read the paper by Ren and colleagues published in May 2016 issue of Biomed Eng Online [1]. The authors aimed to evaluate the reproducibility of computational models of intracranial aneurysm reconstructed from computed tomography angiography (CTA), magnetic resonance angiography (MRA) and 3D rotational angiography (3DRA) [1, 2].

As the authors mentioned, model geometry and hemodynamic parameters were compared between the three models. They reported that in respect of hemodynamic parameters, all three models showed a similar distribution: low average WSS at the sack, high OSI at the body and high average WSSG at the neck. However, there was a large variation in the average WSS (Δ = 34 ± 5.13 % for CM, Δ = 40.6 ± 9.21 % for MM) [1, 2].

Regarding reliability, it is crucial to know that an individual based approach instead of group based (average) should be considered [37]. The reason is in reliability assessment; we should consider individual results and not global average. Therefore, intra class correlation coefficient (ICCC) single measure instead of average measure should be reported to correctly assess the reliability. In other words, possibility of getting exactly the same average of a variable between three models with no reliability at all is high [37].

Moreover, reporting significant difference between two methods is completely different methodological issue from clinically importance of the mentioned difference. In reliability analysis, depending on sample size, a negligible clinical difference can be statistically significant [37].

As the authors pointed out in their conclusion, CTA and MRA have no significant differences in reproducing intracranial aneurysm geometry. There might be some significant differences in hemodynamic parameters between the three imaging-based models and this need to be considered when interpreting the CFD results of different imaging-based models. Such a conclusion can be a misleading message and should be avoided by researcher otherwise; misdiagnosis and mismanagement of the patients cannot be avoided.

Reply to Letter by Dr. Sabour

Yuan Ren, Guo-Zhong Chen, Zhen Liu, Yan Cai, Guang-Ming Lu, Zhi-Yong Li

We thank Dr. Sabour for his interest in our recent study [1, 2], which gives our opportunity to discuss and clarify our results further.

We would like firstly to point out that our results should in no way be used as guidance for clinical decision-making or patient management for three different imaging modalities. In our study, no significant differences in both morphological and hemodynamic parameters were detected. Only one suggestion was made in terms of the interpretation of the computational fluid dynamics (CFD) results because some variation of average wall shear stress (WSS) was found between the three patient-specific models based on different angiographies.

In this study, we found a similar distribution in hemodynamic parameters in three types of CFD models developed from three different types of angiographies. The difference of each parameter was calculated as: (|CM (or MM) − DM|)/DM × 100 %. The means and standard errors of all morphological and hemodynamic parameters were calculated. The differences between DM and CM or DM and MM were analyzed by a paired nonparametric Wilcoxon test. This study was designed to detect any difference in the simulation results rather than to test the reliability of different imaging methods or measurements. Therefore we have not used intra-class correlation coefficient (ICC). We have only taken DSA as a golden standard and compared the difference in geometrical and CFD parameters from models reconstructed from CTA, MRA and DSA.

In the current literature, there are many different types of patient-specific CFD models that are developed from different types of clinical imaging. Therefore, this study was focused to investigate whether these CFD results are comparable or not. There were no significant differences between the commonly used morphological and CFD parameters between the three different models. Some variation in WSS was found although it was not significant and therefore we suggest that we need to be careful when interpreting the simulation results in the future. This study was not aimed to compare any clinically used parameters and it should not be taken to influence any clinical-decision making.




This study was partially supported by the National 973 Basic Research Program of China [No. 2013CB733800], the National Natural Science Foundation of China (NSFC) (No. 11272091, 11422222, 31470043), and ARC (FT140101152).

Competing interests

The author declare that he has no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

Safety Promotion and Injury Prevention Research Center, Shahid Beheshti University of Medical Science
Department of Clinical Epidemiology, School of Health, Shahid Beheshti University of Medical Sciences
Biomechanics Laboratory, School of Biological Science and Medical Engineering, Southeast University
School of Chemistry, Physics and Mechanical Engineering, Queensland University of Technology


  1. Ren Y, Chen GZ, Liu Z, Cai Y, Lu GM, Li ZY. Reproducibility of image-based computational models of intracranial aneurysm: a comparison between 3D rotational angiography, CT angiography and MR angiography. Biomed Eng Online. 2016;15(1):50. doi: ArticleGoogle Scholar
  2. Rothman KJ, Greenland S, Lash TL. Modern Epidemiology, 4th edition, Baltimore. United States: Lippincott Williams & Wilkins; 2010.Google Scholar
  3. Szklo M, Nieto. FJ. Epidemiology beyond the basics. 2nd ed. Manhattan: Jones and Bartlett Publisher; 2007.Google Scholar
  4. Sabour S. Reliability of a new modified tear breakup time method: methodological and statistical issues. Graefes Arch Clin Exp Ophthalmol. 2016;254(3):595–6. doi: (Epub 2015 Aug 28).View ArticleGoogle Scholar
  5. Sabour S. Spinal instability neoplastic scale: methodologic issues to avoid misinterpretation. AJR Am J Roentgenol. 2015;204(4):W493. doi: ArticleGoogle Scholar
  6. Sabour S. Reliability of automatic vibratory equipment for ultrasonic strain measurement of the median nerve: common mistake. Ultrasound Med Biol. 2015;41(4):1119–20. doi: (Epub 2015 Jan 16).View ArticleGoogle Scholar
  7. Sabour S. Does the experience level of the radiologist, assessment in consensus, or the addition of the abduction and external rotation view improve the diagnostic reproducibility and accuracy of MRA of the shoulder? Clin Radiol. 2015;70(3):333–4. doi: (Epub 2015 Jan 1).View ArticleGoogle Scholar


© The Author(s) 2016