- Research
- Open access
- Published:
Assessment of the functional severity of coronary lesions from optical coherence tomography based on ensembled learning
BioMedical Engineering OnLine volume 22, Article number: 127 (2023)
Abstract
Background
Atherosclerosis is one of the most frequent cardiovascular diseases. The dilemma faced by physicians is whether to treat or postpone the revascularization of lesions that fall within the intermediate range given by an invasive fractional flow reserve (FFR) measurement. The paper presents a monocentric study for lesions significance assessment that can potentially cause ischemia on the large coronary arteries.
Methods
A new dataset is acquired, comprising the optical coherence tomography (OCT) images, clinical parameters, echocardiography and FFR measurements collected from 80 patients with 102 lesions, with stable multivessel coronary artery disease. Having the ground truth given by the invasive FFR measurement, the dataset is challenging because almost 40% of the lesions are in the gray zone, having an FFR value between 0.75 and 0.85. Twenty-six features are extracted from OCT images, clinical characteristics, and echocardiography and the most relevant are identified by examining the models’ accuracy. An ensembled learning is performed for solving the binary classification problem of lesion significance considering the leave-one-out cross-validation approach.
Results
Ensemble models are designed from the multi-features voting from 5 features models by prediction aggregation with a maximum accuracy of 81.37% and a maximum area under the curve score (AUC) of 0.856.
Conclusions
The proposed explainable supervised learning-based lesion classification is a new method that can be improved by training with a larger multicenter dataset for further designing a tool for guiding the decision making of the clinician for the cases outside the gray zone and for the other situation extra clinical information about the lesion is needed.
Background
Cardiovascular diseases, especially coronary artery disease (CAD), currently affect a major part of the population, along with the presence of arterial hypertension (AHT) [1]. The diagnosis and treatment of these patients is costly in terms of human and material resources. Many efforts are directed toward improving clinical management by reducing the diagnosis time interval, creating multidisciplinary teams, and optimizing the material costs.
In this context, medical imaging is helpful, and because of that, it is a continually growing domain that unifies the efforts of physicians, scientists, and engineers. Its impact on daily clinical management is high [2], especially in cardiovascular diseases and cancer, the two leading causes of death worldwide. This was empowered by the growth of computational power and data storage, which enable more accurate image classification, object detection, and image enhancement by means of image processing [3].
One of the most important cardiovascular diseases is atherosclerosis, which consists of a narrowing of the blood vessel lumen, mainly due to fat deposits. It may affect the major coronary arteries, causing myocardial ischemia at different degrees of exertion [4].
Optical coherence tomography (OCT) is acquired during invasive coronary X-ray angiography (XA), and it provides high-resolution images of the proximal coronary arteries with accurate quantification of the lumen and the structure of the vessel walls [5]. It may be used in the planning of interventional clinical procedures by selecting the diameter and length of the stents, the need for additional lesion preparation for highly calcified lesions and in the evaluation of stent expansion, apposition, and the presence of dissections. Direct measures of coronary stenoses, such as minimal luminal area (MLA) or minimal luminal diameter (MLD), are generally used for this aim [6, 7].
At the time of the invasive clinical procedure, the functional significance of a stenosis is optimally classified based on the evaluation of the fractional flow reserve (FFR), which is measured as the ratio between the pressure distal to the stenosis and the aortic pressure. In the medical literature and practice, the optimal cutoff value is 0.8 [8, 9]. A value less than 0.8 indicates functionally significant stenosis, requiring a revascularization procedure (either surgical or interventional) in addition to medical treatment.
Disadvantages of the invasive FFR are:
-
a.
Invasive procedure which involves inserting a catheter into the arteries. This procedure carries inherent risks such as bleeding, infection, and artery damage.
-
b.
Patient discomfort because it can cause pain and anxiety for patients. It often involves the use of a contrast agent, which may have side effects, and typically requires patients to remain still for an extended period.
-
c.
Additional costs ranging from 500 to 2000 euros are needed when compared with non-invasive methods due to the equipment and expertise required for the catheterization procedure.
-
d.
Additional time both for the patient in terms of preparation and recovery.
The gray zone with high uncertainty regarding the lesion significance is generally considered in the interval of 0.75 ≤ FFR ≤ 0.85 for which the physician will need extra information including the invasive FFR measurement.
Generally, the lesions situated in the gray zone have the medical recommendation for revascularization. Still, the physician must consider the clinical procedure risks and the overall clinical state of the patient (age, obesity, etc.) for deciding the treatment.
Virtual FFR (vFFR) can be estimated via machine learning (ML) algorithms from XA [10], OCT [11], intravascular ultrasound (IVUS) [12] and combinations of OCT and IVUS imaging [13] based on the computational fluid dynamics (CFD). All these approaches rely on features that characterize the vascular geometry, specifically the arterial lumen, and on clinical parameters.
Virtual FFR-based XA has the potential to alter decision making and it can increase the operators’ confidence in their decision [14].
Intracoronary OCT and FFR were acquired for the left anterior descending artery lesions in [11] for 125 patients with an accuracy of the OCT-based machine learning algorithm of 95.2%.
A total of 41 coronary stenoses in 30 patients were assessed consecutively in the paper [15] by quantitative coronary angiography (QCA), FFR, and intracoronary OCT. The study revealed that the diagnostic capability of MLA and MLD in identifying significant stenoses was moderate, with an area under the curve (AUC) of 0.80 for MLA and 0.76 for MLD. The optimal cutoff of OCT-measured MLA to identify stenoses with FFR ≤ 0.80 was 1.62 mm2.
FFR estimation from intracoronary OCT imaging based on CFD modeling, also known as OCT-based optical flow ratio (OFR), was addressed in [16], and a prototype software package (OctPlus) was built. Bifurcation fractal laws were applied to correct the step-down phenomenon lumen size for 125 vessels from 118 patients, with an average FFR of 0.80 ± 0.09. The overall vessel-level diagnostic accuracy was 90%, with a sensitivity and specificity of 87% and 92%, respectively.
Another study aimed to evaluate the diagnostic performance of the OFR [17] to compare it with the angiography-based quantitative flow ratio (QFR), using wire-based FFR as the gold standard in 212 vessels from 181 patients. The average FFR was 0.82 ± 0.10, and 40.1% of vessels had an FFR ≤ 0.80. The diagnostic accuracy, sensitivity, and specificity of OFR to identify FFR ≤ 0.80 were 92%, 86% and 95%, respectively.
An angiography-based machine learning (ML) algorithm was developed in [18] to classify lesions based on FFR cutoff value with an overall accuracy of 82% and AUC of 0.87.
The goal of the present study is to solve the binary classification problem for predicting the intermediate coronary lesions significance based on the patient’s medical characteristics and the features extracted from the lumen radii from intracoronary OCT.
Results
Population characteristics
Baseline patient and lesion characteristics are summarized in Tables 1 and 2: 80 patients with 102 intermediate coronary lesions are included in this study, where 57 are located on the left anterior descending coronary artery (LAD) with a mean FFR of 0.76, 20 on the left circumflex artery (LCX) with a mean FFR of 0.86, and 25 on the right coronary artery (RCA) with a mean FFR of 0.83.
The histogram of the corrected FFR values is presented in Fig. 1 to reveal the data concentration around the gray zone. This is justified by the fact that the physicians generally perform invasive FFR measurements when there is an uncertainty in the decision-making of revascularization.
Lesion severity classification performance
The tenfold cross-validation strategy was applied for each univariate model with the same random seed variation. Each one of the features was analyzed with the following algorithms: SVM with different kernels, decision tree, k-nearest neighbors, random forest with different numbers of trees [19], AdaBoost [20] and naive Bayes [21]. From a total of 12 algorithms, only the one with the highest accuracy was selected in Table 3.
The models with a maximum accuracy of 60% (the top 14 feature models) were further analyzed through a box plot [22], as shown in Fig. 2. Two features have strong correlation, and radius rapport was eliminated from the study.
For feature analysis, the correlation matrix was computed and shown in Fig. 3, from where it can be observed that two features, radius rapport and percentage diameter reduction have perfect correlation, hence, the last one was eliminated from the feature map.
Some OCT features, for example, percentage diameter reduction and proximal radius rapport, mean radius, and mean lumen radius for stenosis region, have good positive or negative correlations due to their linear computation formulas (see Table 9). Still, in practice, if one of the multicollinear feature is removed, the ensemble model performance is decreasing, suggesting that their combination in voting is valuable.
In the simple and weighted voting approach, the accuracy is improved after running the ROC analysis and the models’ aggregation for the top 13 models. The performances of the ensembled model are displayed in Table 4 along with their corresponding AUC scores in Table 5.
The model aggregation of the crisp labels of all features using the mean ROC cutoff value (M3 method) for the voting combination of 7, 8, 9, 10, 11, 13 features models led to the same evaluation metrics with an accuracy of 80.39%.
The best accuracy was obtained for 5 features (minimal lumen diameter, proximal radius rapport, percentage diameter reduction, mean radius, minimum lumen radius) ensemble model with 81.37% (Table 6). The ROC analysis for both crisp labels and class probabilities are further displayed in Fig. 4.
For a narrower gray zone of 0.75 ≤ FFR ≤ 0.83 [23, 24] the model has an accuracy of only 71.43%. Outside this interval, the model performed significantly better (Table 7).
The best ensembled model is further analyzed regarding its dependencies with other clinical parameters, such as sex, vessel type, if the vessel is proximal LAD, if the patient is hypertensive or if he has diabetes (Table 8).
The LAD vessels are harder to diagnose due to an accuracy of 78.95%, as well as, if they are not proximal. Additionally, the patient’s sex is a factor of discordances, quantified by a difference in accuracy of 6% between women and man. The presence of diabetes or hypertension give similar performances and they are not discordance factors.
A multivariate model was tested for the top 5 to 13 features and the best accuracy was obtained for random forest classifier on the top 13 features with an accuracy of 71.56% which is lower than in the aggregation approach.
Discussion
Machine learning in stenosis classification
The paper proposed a new ML model as a potential tool for deciding the lesion significance based on the patient clinical characteristics and features extracted from the radii profile achieved from the intracoronary OCT. In the study, it was assumed that this medical imaging technique will provide better quality data in the assessment of the radii profiles.
The results sustain the hypothesis that the most important features in the ensemble model are extracted from OCT imaging; hence, the best model uses the top 5 features, all related to the radii profile: the minimal lumen diameter, proximal radius rapport, percentage diameter reduction, mean radius, and minimum lumen radius.
The criterion for performance evaluation was considered the models’ accuracy due to balanced classes of the dataset. Nevertheless, the confidence intervals of the other ensemble models’ performances overlap, and the differences are not statistically significant.
There are no similar methods reported in the scientific literature, but a comparison is made with other papers related to the virtual FFR.
ML-based techniques were employed in [11] for 125 patients considering intracoronary OCT and FFR. The partition ratio was 5:1 for splitting the training and testing datasets. A random forest model was used, with the six most important features as inputs: MLA, percentage of the stenotic area, lesion length, proximal lumen area, preprocedural platelet count, and hypertension. The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the OCT-based machine learning-FFR for the testing group were 100%, 92.9%, 87.5%, 100%, and 95.2%, respectively.
Another attempt considered in [14] by the same authors identified 36 features, including 16 clinical lesion characteristics and 20 OCT extracted features. The classification performances were assessed using sensitivity, specificity, positive predictive value, negative predictive value, and accuracy as 85.7%, 100%, 100%, 77.8%, and 90.5%, respectively.
Even if the accuracies of these models are higher, it must be noted that the splitting ratio of 5:1 for training versus testing inherit a risk of generalization lack. In the present study, with almost the same number of cases, the leave group out approach was chosen for preventing this issue.
All these papers don’t mention if their private dataset have FFR values inside the gray zone.
A study aimed to evaluate the OFR diagnostic performance [17] in comparison with the angiography-based QFR, having the wire-based FFR as a gold standard for 212 vessels from 181 patients with an average FFR of 0.82 ± 0.10 from which 40.1% of vessels had an FFR ≤ 0.80. The diagnostic accuracy, sensitivity, and specificity of OFR to identify FFR ≤ 0.80 were 92%, 86% and 95%, respectively. The computational burden of this algorithm is one of the most important issues.
The study [18] tends to be more relevant in our comparison, used an angiography-based ML algorithm for classifying the lesions significance using the FFR cutoff value with an overall accuracy of 82% and AUC of 0.87 for 1501 patients with 1501 intermediate lesions.
In the scientific paper [25], a deep learning (DL) approach was used for estimating the FFR based on the CFD simulations applied to synthetically generated coronary anatomy with an accuracy of 83.2%.
The best comparison with the current study that used the same OCT dataset, is reported in [26], which used DL algorithms applied on raw radii profiles extracted from the images. Few-shot learning reached the highest accuracy of 77.5%. The present model outperformed the DL-based model due to an enhanced dataset that included the clinical characteristics of the patient.
An independent comparison study [27] was done for 5 software/methods of angio-FFR estimation on a large dataset. The ROC analysis was performed for each method for detecting the FFR ≤ 0.8 and the AUC values were between 0.73 and 0.75, which are lower than the previously diagnostic performances reported in their validation studies.
Although the model accuracy of 81.37% is slightly lower than ones from the other studies which reported accuracies ranging from 82% to 95.2%, it is essential to consider that our cohort consisted of nearly 50% stenoses in the gray zone, with an FFR of approximately 0.8, unlike the others with a smaller representation in this region. Still, for cases with FFR outside the gray zone, the model reached an accuracy of 85.14%.
Our method claims a better computational time, due to the fact it does not need the 3D modeling reconstruction, CFD simulations or deep learning implementation, making it suitable for a further development in the clinical practice.
Ensemble learning reduces the generalization error and improves the overall performance of the individual model accuracy from an average of 67.35% to 81.37%. Its inputs were the crisp labels (significant lesion or not significant) for all features using the mean ROC cutoff value approach.
The ensembled model boosted the model performances in comparison to the multivariate model from 71.56% to 81.37%.
From the current study, the following interpretations can be concluded:
-
measurements related to the stenosis region are the dominant features due to their strong correlation with FFR,
-
weight is an important physiological characteristic of the patient.
-
features related to the lesion region have an important impact on the classification output,
-
AUC score for the crisp labels outperforms the class probability approach.
Clinical implications
To evaluate the clinical impact, the study focused only on non-culprit lesions of patients with ACS and multivessel disease. There were no other restrictions or regulations regarding the data acquisition which offer a broader perspective of the clinical applications.
The explainability goal was targeted for a better understanding and acceptance as decision support in daily clinical practice.
The accuracy for a narrower gray zone of 0.75 ≤ FFR ≤ 0.83 for clinical decision making of percutaneous coronary intervention (PCI) reached 71.43% and it can suggest that the invasive FFR measurement is mandatory for the critical region, whereas, for the cases of outside this interval, the model performance improves substantially to an accuracy of more than 85%.
Therefore, the model offers good performances outside the gray zone that can guide the decision making for choosing the proper treatment that may include the immediate revascularization or taking medications.
For the cases inside the interval, the algorithm suggests the need for getting extra information about the lesion type by performing the invasive FFR measurement. Nevertheless, additional medical information may also include the evaluation of the coronary flow reserve which imply the coronary blood flow velocity measuring. This can be done both invasively using an ultrasound transducer-based catheter or non-invasively, using positron emission tomography (PET).
The importance of the study is given by the fact that it is not possible to rely only on 2D invasive coronary angiography and FFR measurement for treating non-culprit lesions. FFR measurement is subject to artefacts which will yield inaccurate results and it’s producing a hemodynamic disturbance due to the hyperemia inducing state [28]. Moreover, having a model that could simplify the amount of information coming from all invasive assessments (OCT or FFR) could be a real help for interventionists who must take important and instant decisions on treating patients in cardiac catheterization labs.
Clinical implementation
To implement the model in a clinical setting, we envisage the followings: patients are included only after signing an informed consent form; a set of initial inclusion and exclusion criteria are also checked, and if they are met, the XA and OCT exams are performed; data is then extracted and annotated using the dedicated tool.
The XA and OCT data are processed using a cloud-based or on-premise application and the prediction model will run using an artificial intelligence service. Finally, the prediction outputs are interpreted by the clinical expert, who then takes the final diagnosis and treatment decision.
Limitations
The study was limited by a single-center acquisition and there was no other dataset similar to be found in public databases, which constrained the current study to only 80 patients. Finally, to generalize the proposed model, further validation is necessary in a large multicenter cohort of subjects of different races.
The ground truth that is built based on a fixed FFR cutoff value, forced the inclusion of confidence intervals to make them more clinically relevant.
The lumen geometry extracted from OCT imaging may be influenced by the errors in the acquisition process, by the contour detection and its corrections performed in the built-in software.
Contributions and further perspectives
The study contributions include new features extraction from intracoronary OCT (Table 9) capable of quantifying stenosis lesion severity and building the ensemble model obtained using the ML approach for classifying the significance of coronary lesions.
A further perspective includes the extension of the dataset at least with 100 new patients for testing other ML strategies for preserving the explainability goal, which creates a classification mechanism that is interpretable and can be reproductible by humans.
The inclusion of other clinical biomarkers and angiography results can contribute to designing a holistic model for clinical decision making of vascularization.
Conclusions
OCT-based machine learning lesion classification can be used to acquire morphological and functional information into a single procedure, suggesting that it may enhance the treatments of coronary artery stenoses.
The paper introduces a single-center study for evaluating the importance of intermediate coronary lesions that may lead to ischemia in the major coronary arteries. The assessment primarily involves extracting features from both OCT images and patient characteristics.
Methods
Data acquisition
The dataset was collected from the Clinical Emergency Hospital, Bucharest, Romania, and it was conducted in compliance with the Declaration of Helsinki for investigation in human beings.
The study protocol was approved by the local ethics committee of the hospital, after all patients gave their written informed consent before enrollment.
The dataset comprises 80 patients with 102 lesions with stable coronary artery disease or acute coronary syndrome (ACS) and multivessel disease. Only non-culprit lesions were considered in ACS patients. Culprit lesions were examined and treated during hospitalization, whereas non-culprit lesions were evaluated based on XA, OCT and invasive FFR during a second hospitalization, generally after two weeks.
OCT imaging was performed using Optis (St. Jude Medical/Abbott, St. Paul, MN, USA) and Dragon Fly catheters. The fiber probe was pulled back at a constant speed, and cross-sectional images were acquired with 5 frames/mm during manual contrast injection. The maximum vessel length that can be evaluated during one pullback is 75 mm.
FFR measurement was performed using a Quantum system (St. Jude Medical/Abbott, Minneapolis, MN, USA). The measurement was performed after the administration of adenosine, either intravenously at a constant rate of 140 μg/kg/min or as an intracoronary bolus (50–100 μg for the right and 100–400 μg for the left coronary artery) [29].
The clinical protocol states that after recording FFR, the pressure wire is pulled back with the sensor at the tip of the guiding catheter to measure the pressure drift. If these values differed by more than ± 3 mmHg, pressures must be re-equalized, and the measurements are repeated.
Echocardiography was performed for all patients, and the relevant clinical measurements were given by the physician: ejection fraction (echo EF), diastolic pattern, and interventricular septum size. From the blood tests, hemoglobin (Hb) and hematocrit (Ht) levels are used in the study.
Data processing
The OCT images were anonymized and exported in Digital Imaging and Communications in Medicine (DICOM) format [30] with a spatial resolution of 704 × 704 pixels.
The images were analyzed by the clinical team, and the inner vessel contour was automatically traced in the OCT console. Another verification was done on acquired data and some slices with improper contours were eliminated or they had been corrected in a built-in software.
The interventional cardiologist annotated the dataset as follows:
-
OCT frames related to the proximal and distal region of the vessel segment,
-
OCT frames related to the proximal and distal region of the lesion,
-
tracing the lumen border from the medical imaging device under medical imaging expert guidance for computing the vessel’s radii and the estimation of MLD.
Figure 5a is a sample of OCT slice with the coronary artery lumen tracing after contour correction, which is used to build the unidimensional signal of the diameters’ evolution along the vessel length (Fig. 5b).
The 12 features mentioned in Table 9 are computed directly from that unidimensional signal, considering the proximal and distal radii of the lesion (r_proximal and r_distal respectively) given by the medical expert.
The difference between MLD and minimum lumen radius is that the first one is estimated by the physician and the last one is computed directly from the evolution of the diameters along the vessel length (Fig. 5b). This signal is subject to changes due to the OCT imaging preprocessing done in the built-in software, where the contours are manually adjusted when it was needed.
Thirteen clinical characteristics of the patients were used as features in the algorithm: sex, age, height (cm), weight (kg), smoking (0 no; 1 past; 2 present), AHT, dyslipidemia, diabetes, ejection fraction, diastolic pattern, interventricular septum size, hemoglobin, and hematocrit levels.
The ground truth is related to FFR measurement, which has been corrected previously considering the pressure drift (D), which represents the arithmetic difference between aortic pressure (drift_Pa) and distal pressure (drift_Pd). The drift is done at the end of the measurements to check the pressure curves if they are superimposed [31].
Even if drift values of 1–2 mmHg are generally accepted as insignificant in medical practice [32], a drift of 2–3 mmHg was considered relevant in the study, improving the models’ accuracy. Hence, a correction of the measured distal pressure (corrected_Pd) was introduced considering the drift measurement:
where Pd is the distal pressure measured at the beginning of the FFR investigation.
The updated value of FFR (\({\text{corrected}}\_{\text{FFR}}\)), which is measured at maximal hyperemia, will be computed as in the medical literature [21]:
where Pa is the aortic pressure measured at the beginning of the invasive FFR investigation.
Only 15 lesions out of 102 were adjusted according to the above-mentioned formula (2). Twelve lesions had a drift variation of 2 mmHg, and only 3 lesions had a drift variation of 3 mmHg. The correction value influenced the ground truth for two lesions by underestimating the initial value of 0.8 and setting the case into the positive class.
The medical data were extracted both from the clinical records and from the medical images as following:
-
OCT imaging and invasive FFR for each one of three coronary arteries (RCA, LCX or LAD) were performed in PCI during the angiography procedure,
-
The FFR value was corrected using Eq. (2),
-
A database was constructed in which the cardiologist recorded the start and end of the OCT frames to delimitate the vessel segment, the start and end of the OCT slices corresponding to the lesion,
-
Patient characteristics were extracted from the clinical records: physiological, anatomical, and behavioral features correlated to coronary artery disease.
The algorithm overview
The ground truth is computed based on the corrected value of FFR into two classes, if the coronary lesions are hemodynamically significant or not:
The feature matrix was first normalized using L1 norm or Manhattan distance which computes the sum of the magnitudes of the features.
Twelve base learners were tested for 10 different seeds, such as support vector machine (SVM) with different kernels, decision tree, random forest with different number of trees, k-nearest neighbors, AdaBoost, logistic regression or naive Bayes [33,34,35]. The usage of the same random seed per model computation for each type of classifiers, can guarantee a precise, reproducible, and uniform evaluation of each algorithm.
The leave-group out method [36] was applied at the patient level, hence 80 training/testing folds for each model computation, as follows: for each fold, a threshold value was set to balance sensitivity and specificity on the respective training set. Finally, the chosen threshold is applied to classify the test sample(s).
To evaluate the results, diagnostic statistics [37] were computed for all approaches: accuracy, sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV).
The accuracy is computed as the ratio between the correct predictions and all samples’ predictions, as revealed in relation (4):
where TP is the true positive, TN is the true negative, FP is the false positive and FN is the false negative.
The area under the curve by receiver-operating characteristic (ROC) analysis [38] was used on mean probabilities of the positive or negative class (significant or nonsignificant lesion) and on mean prediction output for all models. Instead of the popular Youden index, the closest to (0,1) criteria [39] gave better results in identifying the best cutoff value for the given inputs. Each optimal threshold and its average are further used in constructing the final model.
Ensemble learning groups different models for solving difficult problems to improve the overall performance accuracy and reduce the variance at the cost of increasing the bias [40, 41].
For feature analysis and dependencies, the correlation matrix was computed based on Pearson correlation [42]. Percentage diameter reduction and radius rapport revealed a perfect correlation, hence, the last one was eliminated from the feature map.
The following experiments were conducted considering an ensemble model made of 13 univariate models obtained for each feature individually illustrated in Fig. 6.
Based on models’ performance, eleven features were extracted from the intracoronary OCT frames: MLD, proximal radius rapport, percentage diameter reduction, mean radius, minimum lumen radius, mean lumen radius for stenosis region, mean radius per length, maximum radius rapport, stenosis lesion length, and maximum lumen radius. In addition to that, the weight, the hematocrit level, and the interventricular septum are added into the final feature map.
Considering the univariate models of the top 13 most relevant features obtained after tenfold cross validation, a multi-feature voting is used for building the ensemble model. ROC analysis was applied, and the ensemble predictions were computed by averaging the output prediction of all 10 models.
The output of the class label (Oi) can take different types of values according to the information provided by the classifiers Ci:
In the scientific literature [41] it is suggested that the class probabilities estimated by most classifiers have, in general, lower performance, except for the situation of a careful calibration.
Considering the crisp label and the calibrated class probability as inputs for the ROC analysis, the final outputs of each model are used in a voting system to obtain the three different aggregation strategies:
-
M1—the mean ROC cutoff value of the class probabilities of all univariate models is the threshold of the final predictions that are further used as inputs into the voting system.
-
M2—the cutoff values after the ROC analysis of each corresponding feature model are the thresholds for the resulting models that are further used as inputs into the voting system.
-
M3—the optimal cutoff value obtained after the ROC analysis of the crisp labels of each univariate model is the threshold for the final models that are further used as inputs into the ensemble model.
For comparison, a simple majority vote and a weighted voting [43] are implemented in the classification system having as input the mean of the models’ outputs after performing the ROC analysis.
In the simple majority vote approach, every classification model votes for one class label, and the final output class label is the one that receives more than half of the votes.
As implementation, the ensemble model’s output is given by the combination of each one of the 13 univariate models that can vote. A 0.5 threshold is applied to their mean for computing the final prediction output.
Weighted voting assumes that the individual classifiers have unequal performance and it will give more voting power to the stronger classifiers. The weights should be proportional to the performance of the individual learners.
The optimal weights (wi) assigned to the classifier (Ci) are computed from [43] with the following formula (6):
where acci denotes the accuracy of the classifier Ci and \(i=\stackrel{-}{\mathrm{1,13}}\) features.
Finally, the mean of each output (Oi) is multiplied by the corresponding weight and compared to threshold value of 0.5 to compute the ensembled model output.
The implementation of the algorithms was performed using the scikit-learn Python library [44].
Availability of data and materials
The data that support the findings of this study are available from the local ethics committee of the Clinical Emergency Hospital, Bucharest, but restrictions apply to the availability of these data, which were used under license for the current study and are not publicly available. Data are, however, available from the corresponding author I.-A.T. upon reasonable request and with permission of the local ethics committee of the Clinical Emergency Hospital, Bucharest.
Abbreviations
- ACS:
-
Acute coronary syndrome
- AHT:
-
Arterial hypertension
- AUC:
-
Area under the curve
- CAD:
-
Coronary artery disease
- CFD:
-
Computational fluid dynamics
- DICOM:
-
Digital Imaging and Communications in Medicine
- DL:
-
Deep learning
- EF:
-
Ejection fraction
- FFR:
-
Fractional flow reserve
- FN:
-
False negative
- FP:
-
False positive
- Hb:
-
Hemoglobin level
- Ht:
-
Hematocrit level
- IVS:
-
Interventricular septum
- IVUS:
-
Intravascular ultrasound
- LAD:
-
Left Anterior Descending artery
- LCx:
-
Left circumflex artery
- ML:
-
Machine Learning
- MLA:
-
Minimal lumen area
- MLD:
-
Minimal lumen diameter
- NPV:
-
Negative predictive value
- OCT:
-
Optical coherence tomography
- OFR:
-
OCT-based optical flow ratio
- PCI:
-
Percutaneous coronary intervention
- PET:
-
Positron emission tomography
- Poly SVM:
-
Support vector machine with polynomial kernel
- PPV:
-
Positive predictive value
- QCA:
-
Quantitative coronary angiography
- QFR:
-
Angiography-based quantitative flow ratio
- M1:
-
Class probabilities for all features using the mean ROC cutoff values
- M2:
-
Class probabilities using the corresponding ROC cutoff for each feature
- M3:
-
Predicted output for all features using the mean optimal ROC cutoff value
- RCA:
-
Right coronary artery
- RF:
-
Random forest
- ROC:
-
Receiver-operating characteristic
- SVM:
-
Support vector machine
- TN:
-
True negative
- TP:
-
True positive
- XA:
-
X-ray Coronary Angiography
References
Stanciu S, Rusu E, Miricescu D, Radu AC, Axinia B, Vrabie AM, Ionescu R, Jinga M, Sirbu CA. Links between metabolic syndrome and hypertension: the relationship with the current antidiabetic drugs. Metabolites. 2023;13:87. https://doi.org/10.3390/metabo13010087.
Ostensen H. Diagnostic imaging: what is it? When and how to use it where resources are limited?. World Health Organization; 2001, report.
Bankman IN. Handbook of medical image processing and analysis. 2nd ed. Elsevier Inc; 2009. p. 1–69. https://doi.org/10.1016/b978-0-12-373904-9.x0001-4.
https://www.nhlbi.nih.gov/health/coronary-heart-disease, Accessed 21 July 2022.
Gutiérrez-Chico JL, Alegría-Barrero E, Teijeiro-Mestre R, et al. Optical coherence tomography: from research to practice. Eur Heart J Cardiovasc Imaging. 2012;13(5):370–84. https://doi.org/10.1093/ehjci/jes025.
Ali ZA, Maehara A, Généreux P, et al. ILUMIEN III: OPTIMIZE PCI Investigators Optical coherence tomography compared with intravascular ultrasound and with angiography to guide coronary stent implantation (ILUMIEN III: OPTIMIZE PCI): a randomized controlled trial. Lancet. 2016;388(10060):2618–28. https://doi.org/10.1016/S0140-6736(16)31922-5.
Gonzalo N, Escaned J, Alfonso F, et al. Morphometric assessment of coronary stenosis relevance with optical coherence tomography: a comparison with fractional flow reserve and intravascular ultrasound. J Am Coll Cardiol. 2012;59(12):1080–9. https://doi.org/10.1016/j.jacc.2011.09.0781.
Matsumura-Nakano Y, Kawaji T, Shiomi H, Kawai-Miyake K, Kataoka M, Koizumi K, Matsuda A, Kitano K, Yoshida M, Watanabe H, Tazaki J, Kato T, Saito N, Shizuta S, Ono K, Togashi K, Morimoto T, Kimura T. Optimal cutoff value of fractional flow reserve derived from coronary computed tomography angiography for predicting hemodynamically significant coronary artery disease. Circ Cardiovasc Imaging. 2019. https://doi.org/10.1161/circimaging.119.008905.
Tomaniak M, Ochijewicz D, Kołtowski Ł, Rdzanek A, Pietrasik A, Jąkała J, Kochman J. OCT-derived plaque morphology and FFR-determined hemodynamic relevance in intermediate coronary stenoses. J Clin Med. 2021;10(11):2379. https://doi.org/10.3390/jcm10112379.
Cho H, et al. Angiography-based machine learning for predicting fractional flow reserve in intermediate coronary arterylesions. J Am Heart Assoc. 2019;8: e011685. https://doi.org/10.1161/jaha.118.011685.
Cha JJ, Son TD, Ha J, et al. Optical coherence tomography-based machine learning for predicting fractional flow reserve in intermediate coronary stenosis: a feasibility study. Sci Rep. 2020;10:20421. https://doi.org/10.1038/s41598-020-77507-y.
Lee JG, et al. Intravascular ultrasound-based machine learning for predicting fractional flow reserve in intermediate coronary artery lesions. Atherosclerosis. 2020;292:171–7. https://doi.org/10.1016/j.atherosclerosis.2019.10.022.
Guo X, Maehara A, Matsumura M, et al. Predicting plaque vulnerability change using intravascular ultrasound + optical coherence tomography image-based fluid–structure interaction models and machine learning methods with patient follow-up data: a feasibility study. BioMed Eng OnLine. 2021;20:34. https://doi.org/10.1186/s12938-021-00868-6.
Gosling RC, Adam Z, Barmby DS, Iqbal J, Morgan KP, Richardson JD, Rothman AMK, Lawford PV, Rodney Hose D, Gunn JP, Morris PD. The impact of virtual fractional flow reserve and virtual coronary intervention on treatment decisions in the cardiac catheter laboratory. Can J Cardiol. 2021;37(10):1530–8. https://doi.org/10.1016/j.cjca.2021.06.004. (ISSN 0828-282X).
Zafar H, Ullah I, Dinneen K, Matiullah S, Hanley A, Leahy MJ, Sharif F. Evaluation of hemodynamically severe coronary stenosis as determined by fractional flow reserve with frequency domain optical coherence tomography measured anatomical parameters. J Cardiol. 2014;64(1):19–24. https://doi.org/10.1016/j.jjcc.2013.11.009.
Yu W, Huang J, Jia D, Chen S, Raffel OC, Ding D, Tian F, Kan J, Zhang S, Yan F, Chen Y, Bezerra HG, Wijns W, Tu S. Diagnostic accuracy of intracoronary optical coherence tomography-derived fractional flow reserve for assessment of coronary stenosis severity. EuroIntervention. 2019;15(2):189–97. https://doi.org/10.4244/EIJ-D-19-00182.
Huang J, Emori H, Ding D, Kubo T, Yu W, Huang P, Zhang S, Gutiérrez-Chico JL, Akasaka T, Wijns W, Tu S. Diagnostic performance of intracoronary optical coherence tomography-based versus angiography-based fractional flow reserve for the evaluation of coronary lesions. EuroIntervention. 2020;16(7):568–76. https://doi.org/10.4244/EIJ-D-19-01034.
Cho H, Lee J, Kang S, Kim W, Choi S, Ko J, et al. Angiography-based machine learning for predicting fractional flow reserve in intermediate coronary artery lesions. J Am Heart Assoc. 2019. https://doi.org/10.1161/jaha.118.011685.
Louppe G. Understanding random forests: from theory to practice, PhD thesis; 2014. arXiv:1407.7502v3. https://doi.org/10.48550/arXiv.1407.7502
Wang R. AdaBoost for feature selection, classification and its relation with SVM, a review. Phys Procedia. 2012;25:800–7. https://doi.org/10.1016/j.phpro.2012.03.160. (ISSN 1875-3892).
Schonlau M. The Naive Bayes classifier. In: Applied statistical learning. Statistics and computing. Springer: Cham; 2023. p. 143–60. https://doi.org/10.1007/978-3-031-33390-3_8.
Williamson D, Parker RA, Kendrick J. The box plot: a simple visual method to interpret data. Ann Intern Med. 1989;110:916–21. https://doi.org/10.1059/0003-4819-110-11-916.
Strisciuglio T, Barbato E. The fractional flow reserve gray zone has never been so narrow. J Thorac Dis. 2016;8(11):E1537–9. https://doi.org/10.21037/jtd.2016.11.52.
Petraco R, Sen S, Nijjer S, Echavarria-Pinto M, Escaned J, Francis DP, Davies JE. Fractional flow reserve-guided revascularization: practical implications of a diagnostic gray zone and measurement variability on clinical decisions. JACC Cardiovasc Interv. 2013;6(3):222–5. https://doi.org/10.1016/j.jcin.2012.10.014.
Itu L, Rapaka S, Passerini T, Georgescu B, Schwemmer C, Schoebinger M, Flohr T, Sharma P, Comaniciu D. A machine-learning approach for computation of fractional flow reserve from coronary computed tomography. J Appl Physiol. 2016;121(1):42–52. https://doi.org/10.1152/japplphysiol.00752.2015.
Hatfaludi C-A, Tache I-A, Ciușdel CF, Puiu A, Stoian D, Itu LM, Calmac L, Popa-Fotea N-M, Bataila V, Scafa-Udriste A. Toward a deep-learning approach for prediction of fractional flow reserve from optical coherence tomography. Appl Sci Basel. 2022;12(14):6964. https://doi.org/10.3390/app12146964. (ISSN 2076-3417).
Ninomiya K, Serruys PW, Kotoku N, Zhou J, Kageyama S, Masuda S, Revaiah PC, Wang B, He X, Tsai T-Y, Kageyama M, Sevestre E, Sharif F, Garg S, Akasaka T, Escaned J, Patel MR, Onuma Y. Anonymous comparison of various angiography-derived fractional flow reserve software with pressure-derived physiological assessment. JACC Cardiovasc Interventions. 2023;16(14):1778–90. https://doi.org/10.1016/j.jcin.2023.04.026.
Cantoni V, Green R, Zampella E, D’Antonio A, Cuocolo A. Revascularization of non-culprit lesions: a common dilemma. J Nucl Cardiol. 2023;30(4):1745–8. https://doi.org/10.1007/s12350-022-03176-1.
Kern MJ, Lim MJ. Intravascular lesion assessment: physiology and imaging. In: Kern MJ, Sorajja P, Lim MJ, editors. The interventional cardiac catheterization handbook. 4th ed. Elsevier; 2018. p. 107–58 (ISBN 9780323476713).
Bidgood WD Jr, Horii SC, Prior FW, Van Syckle DE. Understanding and using DICOM, the data interchange standard for biomedical imaging. J Am Med Inform Assoc. 1997;4(3):199–212. https://doi.org/10.1136/jamia.1997.0040199.
Cottens D, Ferdinande B, Polad J, Vrolix M, Ameloot K, Hendrickx I, Poels E, Maeremans J, Dens J. FFR pressure wire comparative study for drift: piezo resistive versus optical sensor. Am J Cardiovasc Dis. 2022;12(1):42–52. https://doi.org/10.1080/00015385.2021.1939510.
Cook CM, Ahmad Y, Shun-Shin MJ, Nijjer S, Petraco R, Al-Lamee R, Mayet J, Francis DP, Sen S, Davies JE. Quantification of the effect of pressure wire drift on the diagnostic performance of fractional flow reserve, instantaneous wave-free ratio, and whole-cycle Pd/Pa. Circ Cardiovasc Interv. 2016;9(4): e002988. https://doi.org/10.1161/circinterventions.115.002988.
Jayatilake SMDAC, Ganegoda GU. Involvement of machine learning tools in healthcare decision making. J Healthc Eng. 2021;2021:6679512. https://doi.org/10.1155/2021/6679512.
Caballé-Cervigón N, Castillo-Sequera JL, Gómez-Pulido JA, Gómez-Pulido JM, Polo-Luque ML. Machine learning applied to diagnosis of human diseases: a systematic review. Appl Sci. 2020;10:5135. https://doi.org/10.3390/app10155135.
Belyadi H, Haghighat A. Chapter 5—Supervised learning. In: Belyadi H, Haghighat A, editors. Machine learning guide for oil and gas using python. Gulf Professional Publishing; 2021. p. 169–295 (ISBN 9780128219294).
Sammut C, Webb GI. Leave-one-out cross-validation, encyclopedia of machine learning. Boston: Springer; 2011. p. 600–1 (ISBN 978-0-387-30768-8).
Trevethan R. Sensitivity, specificity, and predictive values: foundations, pliabilities, and pitfalls in research and practice. Front Public Health. 2017;5:307. https://doi.org/10.3389/fpubh.2017.00307.
Melo F. Area under the ROC curve. In: Dubitzky W, Wolkenhauer O, Cho KH, Yokota H, editors. Encyclopedia of systems biology. Springer: New York; 2013. p. 38–9 (ISBN 978-1-4419-9862-0).
Unal I. Defining an optimal cut-point value in ROC analysis: an alternative approach. Comput Math Methods Med. 2017. https://doi.org/10.1155/2017/3762651.
Ganaie MA, Hu M, Malik AK, Tanveer M, Suganthan PN. Ensemble deep learning: a review, engineering applications of artificial intelligence; 2022. arXiv:2104.02395v3. https://doi.org/10.1016/j.engappai.2022.105151
Zhang C, Yunqian M. Ensemble machine learning. New York: Springer; 2012. p. 1–34 (ISBN 978-1-4419-9325-0).
Qu G, Hariri S, Yousif M. A new dependency and correlation analysis for features. IEEE Trans Knowl Data Eng. 2005;17(9):1199–207. https://doi.org/10.1109/TKDE.2005.136.
Zhou Z. Ensemble methods: foundations and algorithms. Chapman & Hall/CRC; 2012. p. 67–95 (ISBN:978-1-4398-3003-1).
Kumar A, Mayank J. Ensemble Learning for AI Developers. New York: Springer; 2020. p. 1–48 (ISBN-13 (pbk): 978-1-4842-5939-9).
Acknowledgements
The authors would like to thank all participants in the study for helping us find a better diagnosis and treatment of atherosclerosis.
The authors would like to thank for the support of the Academy of Romanian Scientists in the framework of Scientific session of young researchers in the competition AOSR-TEAMS, edition 2022–2023.
Funding
The research leading to these results has received funding from EEA Grant 592 2014-2021 under Project contract no. 33/2021. This work was partially supported by a grant from the Romanian National Authority for Scientific Research and Innovation, CCCDI—UEFISCDI, project number ERANET-PERMED-HeartMed, within PNCDI III. This work has been partly funded under the European Union’s Horizon 2020 research and innovation programme under grant agreement N° 101017578 (SIMCor: In-Silico testing and validation of Cardiovascular IMplantable devices).
Author information
Authors and Affiliations
Contributions
Conceptualization I-AT, C-AH, and LMI; methodology, I-AT, C-AH, and AP; software, I-AT, C-AH, and AP; validation, LC, N-MP-F, and AS-U; formal analysis, LC; investigation, LC, and N-MP-F; resources, I-AT and LMI; data curation, I-AT and LC; writing—original draft preparation, I-AT, C-AH and LMI; writing—review and editing, I-AT; visualization, I-AT, C-AH, and LMI; supervision, LMI, and AS-U; project administration, LMI, and AS-U; funding acquisition, LMI, and AS-U All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The study was conducted according to the guidelines of the Declaration of Helsinki and did not require Institutional Review Board approval, as this work was performed using anonymized images collected previously. This work was performed using an anonymized dataset collected from the hospital. Informed consent was obtained prior to medical procedures.
Consent for publication
Not applicable.
Competing interests
Irina-Andra Tache, Andrei Puiu, and Lucian Mihai Itu are employees of Siemens SRL, Advanta, Brasov, Romania. Cosmin-Andrei Hatfaludi received a scholarship from Siemens SRL, Advanta, Brasov, Romania. The other authors declare no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Tache, IA., Hatfaludi, CA., Puiu, A. et al. Assessment of the functional severity of coronary lesions from optical coherence tomography based on ensembled learning. BioMed Eng OnLine 22, 127 (2023). https://doi.org/10.1186/s12938-023-01192-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12938-023-01192-x