Skip to main content

Machine learning associated with respiratory oscillometry: a computer-aided diagnosis system for the detection of respiratory abnormalities in systemic sclerosis

Abstract

Introduction

The use of machine learning (ML) methods would improve the diagnosis of respiratory changes in systemic sclerosis (SSc). This paper evaluates the performance of several ML algorithms associated with the respiratory oscillometry analysis to aid in the diagnostic of respiratory changes in SSc. We also find out the best configuration for this task.

Methods

Oscillometric and spirometric exams were performed in 82 individuals, including controls (n = 30) and patients with systemic sclerosis with normal (n = 22) and abnormal (n = 30) spirometry. Multiple instance classifiers and different supervised machine learning techniques were investigated, including k-Nearest Neighbors (KNN), Random Forests (RF), AdaBoost with decision trees (ADAB), and Extreme Gradient Boosting (XGB).

Results and discussion

The first experiment of this study showed that the best oscillometric parameter (BOP) was dynamic compliance, which provided moderate accuracy (AUC = 0.77) in the scenario control group versus patients with sclerosis and normal spirometry (CGvsPSNS). In the scenario control group versus patients with sclerosis and altered spirometry (CGvsPSAS), the BOP obtained high accuracy (AUC = 0.94). In the second experiment, the ML techniques were used. In CGvsPSNS, KNN achieved the best result (AUC = 0.90), significantly improving the accuracy in comparison with the BOP (p < 0.01), while in CGvsPSAS, RF obtained the best results (AUC = 0.97), also significantly improving the diagnostic accuracy (p < 0.05). In the third, fourth, fifth, and sixth experiments, different feature selection techniques allowed us to spot the best oscillometric parameters. They resulted in a small increase in diagnostic accuracy in CGvsPSNS (respectively, 0.87, 0.86, 0.82, and 0.84), while in the CGvsPSAS, the best classifier's performance remained the same (AUC = 0.97).

Conclusions

Oscillometric principles combined with machine learning algorithms provide a new method for diagnosing respiratory changes in patients with systemic sclerosis. The present study's findings provide evidence that this combination may help in the early diagnosis of respiratory changes in these patients.

Introduction

Systemic sclerosis (SSc) is a chronic connective tissue disease characterized by thickening and fibrosis of the skin and internal organs such as the heart, lungs, kidneys, and gastrointestinal tract [1, 2]. Pulmonary complications are the most common causes of death in SSc, and pulmonary arterial hypertension has become the most crucial life-threatening complication. The most common pulmonary manifestation is interstitial lung disease, associated with pulmonary fibrosis, where the lungs lose their compliance. This abnormality occurs in approximately 80% of cases and is associated with reduced survival [2].

The forced oscillation technique (FOT), also known as respiratory oscillometry, is a system identification method used to evaluate respiratory system resistance and reactance. This method provides a detailed analysis of the respiratory system's mechanical properties, addressing different properties from that evaluated by spirometry, the most traditional method of analyzing respiratory diseases. Indeed, oscillometry is likely complementary to spirometry [3]. The measurement is based on applying low-pressure oscillations to the airway opening to stimulate the respiratory system and measure the associated flow response. Therefore, this technique requires minimal cooperation and no forced expiratory maneuvers and can be used in situations when standard measurements of lung function by spirometry are difficult or not feasible, including children, the elderly, and patients in advanced stages of the disease [4].

Our laboratory successfully applied FOT to obtain a detailed description of the respiratory changes in sarcoidosis [5] and silicosis [6, 7]. Several other research groups also used FOT to diagnose respiratory mechanics changes associated with interstitial lung disease [8,9,10,11]. In general, these patients presented increased resistance and frequency-dependency of resistance, as well as more negative reactance. Studies in rheumatoid arthritis showed reactance values and frequency-dependent behavior in resistance significantly different from those of the healthy subjects [12]. Recent studies showed that respiratory reactance reflects fibrosis and restrictive ventilatory deficiency in idiopathic pulmonary fibrosis [13]. FOT allows a simple, not invasive, and detailed analysis of the respiratory system [4]. Taken together, these previous results and features indicate that FOT is a promising tool to facilitate the diagnosis of respiratory abnormalities in patients with SSc.

However, oscillometry is not currently widely used in pulmonary function tests, even with these important clinical advantages. This limitation arises because this method is based on concepts derived from the electrical engineering area, which are not easily interpreted in the clinical environment. Thus, although oscillometry exams are simple, the interpretation of resistance and reactance curves and the derived parameters is difficult for the busy untrained pulmonologist, requiring training and experience.

Machine learning (ML) algorithms have been offered an important contribution to improving lung function tests [4]. In the particular case of oscillometry, previous studies provided clear evidence that these algorithms simplify the interpretation of the results, and therefore, the clinical use of oscillometry [14, 15]. There is also evidence that these algorithms’ use may help increase diagnostic accuracy [14, 16]. Despite the high potential of combining these two methods in lung diseases, there are no previous studies using oscillometry combined with ML methods to diagnose respiratory changes in SSc.

In this context, we hypothesized that using ML methods associated with respiratory oscillometry analysis would improve the diagnosis of respiratory changes in systemic sclerosis. This paper has two key aims. First, to evaluate several ML algorithms to aid in the diagnostic of respiratory changes in SSc. Second, to find out the best configuration for this task.

The next four sections of this paper initially provided a description of the patient groups and the measurement protocol in “Materials and methods” section. We also describe the investigated classifiers, the indexes used for performance evaluation, and the experimental design. The findings of the research are presented in the third section. These findings are discussed and criticized in the fourth section. Finally, the “Conclusion” section summarizes this research’s primary outcomes, focusing on the two key proposed objectives.

Results

The biometric and spirometric features of the studied subjects are exhibited in Table 1. The three studied groups’ biometric features were similar, and there were no significant differences between the groups. As shown in Table 1, patients with SSc presented significant reductions in the spirometric parameters (p < 0.05).

Table 1 Anthropometric spirometric characteristics of the studied subjects [mean ± SD and (minimal–maximal values)]

The bar charts in Fig. 1 show the characteristics of individuals from the control group (CG), patients with sclerosis and normal spirometry (PSNS), and patients with sclerosis and altered spirometry (PSAS). The mean values of each oscillometric parameter were calculated at a 95% confidence interval. Using the analysis of variance (ANOVA), all oscillometric parameters showed a significant difference in their respective mean values (p < 0.001). An increase was observed in the mean values of R0, Rm, and Zrs in patients with Systemic Sclerosis. Thus, disease carriers have higher resistance values (R0, Rm) and higher impedance values (Zrs). On the other hand, resonance frequency (fr) and the slope of the resistance curve (S) have close CG and PSNS group values. However, fr has higher values for PSAS, and more negative values for S. Cdyn has higher values for the CG and similar values in the PSNS and PSAS groups.

Fig. 1
figure 1

Mean values ± 95% confidence interval of each FOT parameter. Control group (CG), patients with sclerosis and normal spirometry (PSNS), and patients with sclerosis and altered spirometry (PSAS). The analysis of variance (ANOVA) showed that all parameters presented a significant difference in their respective mean values (p < 0.001)

Figure 2 resumes the results of experiment 1. One can see that Cdyn is the best oscillometric parameter (BOP) to discriminate SSc, presenting moderate diagnostic accuracy (AUC = 0.77) for the situation CGvsPSNS and presenting high diagnostic accuracy (AUC = 0.94) in the scenario CGvsPSAS. Tables and figures with more detail about these results can be found in the supplement (Additional file 1: Tables S1, S2, Additional file 2: Figures S1, and S2).

Fig. 2
figure 2

Results of experiment 1, describing the diagnostic accuracy of oscillometry in sclerosis. fr: resonance frequency; Xm: mean respiratory reactance; R0: respiratory resistance extrapolated at 0 Hz; S: slope of the linear relationship of resistance versus frequency; Rm: mean respiratory resistance; Zrs: absolute value of respiratory impedance in 4 Hz; Cdyn: respiratory system dynamic compliance

Figure 3 presents the AUCs of the BOP, the ML algorithms, and the MIL classifier obtained in experiment 2. One can see that the ML algorithms improved the AUC in the situation CGvsPSNS. KNN achieved the best result with AUC = 0.90. This result indicates that the algorithm provides a highly accurate diagnosis (0.9 ≤ AUC ≤ 1.0). The second-best performance was realized by the ADAB, with AUC = 0.88. The AUCs’ comparison with the methodology proposed by Delong et al. [17] has shown that KNN, ADAB, RF, and XGB presented a statistically significant difference concerning the BOP.

Fig. 3
figure 3

Comparative analysis of the diagnostic accuracy in experiment 2, considering the best oscillometric parameter (BOP) obtained without the use of classifiers), machine learning algorithms, and the MIL classifier. K-NN K-Nearest Neighbor, ADAB Adaboost with decision tree classifiers, RF Random Forests, MIL Multiple Instance Learning, XGB Extreme Boosting Gradient Classifiers, AUC area under the ROC curve. Also, “*” indicates that there a statistically significant difference comparing to BOP (p < 0.05) and “**” (p < 0.01)

In the scenario, CGvsPSAS, ADAB, RF, and XGB could provide a small improvement in the AUC. RF and XGB exhibited a statistically significant difference regarding the BOP, while the RF classifier has achieved the best performance with AUC = 0.97. Table 2 presents the five oscillometric parameters selected in these experiments.

Table 2 Five oscillometric parameters selected by MIL and RFE

Figure 4 shows the results of experiment 3, presenting the AUCs obtained by the following strategies: BOP, best ML algorithms with all seven oscillometric parameters (ML7), best ML algorithm with five oscillometric parameters selected by MIL (MIL5 + ML), and the best ML algorithm with five oscillometric parameters selected by RFE (RFE5 + ML).

Fig. 4
figure 4

Summary of Experiment 3 (MIL5 + ML: MIL as five feature selector) and Experiment 4 (RFE5 + ML: RFE as a five feature selector)—AUCs for the best oscillometric parameter (BOP), for the best ML algorithms in experiments 3 and 4, and the best ML algorithm with oscillometric parameters (ML7). The figure indicates the best oscillometric parameter and the best ML algorithm in each case. Also, “*” indicates that there a statistically significant difference comparing to BOP (p < 0.05) and “**” (p < 0.01)

Regarding MIL’s selection, when one compares obtained AUCs with those obtained in the second experiment, it is worth noting that there is only a small decrease in the scenario CGvsPSNS. In the CGvsPSAS scenario, the obtained AUCs stay the same. In both situations, the AUCs’ comparison with the methodology proposed by Delong et al. has shown a statistically significant difference concerning the BOP. KNN achieves the best AUC in the scenario CGvsPSNS with feature selection done by MIL (AUC = 0.87), while in the CGvsPSAS, the best AUC was obtained by RF (AUC = 0.97).

The fifth and sixth experiments were designed to train ML algorithms with the selection of the three best features. Table 3 presents the selected features, and Fig. 5 resumes the results.

Table 3 Three oscillometric parameters selected by MIL and RFE
Fig. 5
figure 5

Summary of Experiment 5 (MIL3 + ML: MIL as three feature selector) and Experiment 6 (RFE3 + ML: RFE as a three feature selector)—AUCs for the best oscillometric parameter (BOP), for the best ML algorithms in experiments 5 and 6, and the best ML algorithm with oscillometric parameters (ML7). The figure indicates the best oscillometric parameter and the best ML algorithm in each case. Also, “*” indicates that there a statistically significant difference comparing to BOP (p < 0.05) and “**” (p < 0.01)

Even with only three features, AUCs’ comparison has shown a statistically significant difference concerning the BOP in both cases. In CGvsPSNS, once again, there is a small decrease in the performance (AUC = 0.84). In the CGvsPSAS, the performance was the same (AUC = 0.97). Figure 6 presents a 3D picture of the CG and PSNS.

Fig. 6
figure 6

Representation of the dataset CGvsPSNS using three features: R0, Zrs, and Cdyn

For additional analysis of the ROC curves, Figs. 7 and 8 show, respectively, the Se observed at an Sp of 90% and at an Sp of 75% (representing bearable specificity). We included the 90% specificity level since it allows only 10% false positives, introducing the most difficult cases into the correct group. It is also noticeable that the sensitivities at 90% Sp of the best ML classifiers were higher than those observed using the BOP in all performed experiments. Best ML classifiers invariably presented better results than BOP at 75% Sp.

Fig. 7
figure 7

Summary of the experiments describing comparisons of the sensitivity at 90% Sp obtained using the best oscillometric parameter (BOP) and ML methods in all experiments. The sensitivity at 90% Sp presented is that of the best classifier

Fig. 8
figure 8

Summary of the experiments describing comparisons of the sensitivity at 75% Sp obtained using the best oscillometric parameter (BOP) and ML methods in all experiments. The sensitivity at 75% Sp presented is that of the best classifier

The interested reader may find a detailed description of the results obtained in experiments 2 to 6 in the supplement (Additional file 1: Tables S3 to S22, Additional file 2: Figures S3 to S22).

Discussion

This is the first study on designing an automatic classifier to assist in diagnosing respiratory abnormalities in patients with SSc using respiratory oscillometry. It was shown that it could simplify lung function’s clinical evaluation and improve these exams’ diagnostic accuracy.

In the first experiment, the dynamic compliance (Cdyn) was the oscillometric parameter that obtained the best individual performance in both scenarios: CGvsPSNS and CGvsPSAS. These findings are in close agreement with the diffuse fibrosis that affects the interstitium and alveolar septa in SSc [1, 2]. They are also in line with the reduction in compliance observed by Greenwald et al. [18] using the esophageal balloon technique and the increase in reactance area obtained by Aronsson et al. [19] using impulse oscillometry.

In the first scenario, as expected, due to the small differences in the measured parameters (Fig. 1), it was challenging to separate the control group from the patients with sclerosis and normal spirometry, which yields an AUC = 0.77, indicating moderate diagnostic accuracy (Fig. 2). In the second scenario, the increase in physiological abnormalities resulted in increased differences in the measured parameters (Fig. 1). This allowed Cdyn to easily separate the two groups and present an AUC = 0.94, which stands for high diagnostic accuracy (Fig. 2). These results are consistent with previous studies showing increased Cdyn diagnostic accuracy with physiological abnormalities in sarcoidosis [5], adults with sickle cell anemia [20], and COPD [21].

In the second experiment (Fig. 3), it is possible to note that the best result for the scenario CGvsPSNS was achieved by the KNN (AUC = 0.90). The use of the KNN, ADAB, RF, and XGB algorithms resulted in a significant improvement in diagnostic accuracy. KNN was followed by ADAB, RF, and XGB using all oscillometric parameters, with ADAB remarkably adjacent to high diagnostic accuracy (AUC = 0.88). In accordance with the present results, previous studies have demonstrated similar increase in accuracy in sickle cell anemia [22], the differential diagnosis of asthma and restrictive respiratory diseases [23], and the early diagnosis of smoking-induced respiratory changes [16].

Similar to previous studies [23, 24], feature selection allowed the reduction of the used features without a significant reduction in performance (Fig. 4). In CGvsPSNS, feature selection helped spot the most relevant features. Although the methods selected a different set of features, there is a significant intersection (R0, Rm, and Cdyn), which agrees with what can be seen in Fig. 1. In the other scenario, CGvsPSAS, RF’s best results were achieved, followed by XGB and ADAB (Fig. 4). For this scenario, the feature selection has shown that the same results could be achieved using fewer oscillometric parameters. As mentioned in the Introduction, pulmonary manifestation in SSc is characterized by interstitial lung disease associated with pulmonary fibrosis [2]. In this sense, one interesting finding was obtaining Cdyn between the most relevant features in the two studied scenarios (Table 2). This is in close accordance with pathophysiological fundamentals involved in this disease, in which the lungs lose their compliance [2].

Figure 5 summarizes all the results obtained in the fifth and sixth experiments and compares the results in experiments 1 and 2. In CGvsPSNS, the feature selection did not increase diagnostic accuracy, but it indicated important features R0, Zrs, and Cdyn, which agrees with Fig. 1. The use of the reduction of attributes was intended to reduce the complexity of the analysis. The current study found selected features consistent with the presence of lung fibrosis [1, 2]. This rather interesting result is consistent with clinically relevant abnormalities that are known to be associated with reduced survival in these patients [2]. The three selected main features allowed us to visually inspect the division between groups. These results further support the idea of a simple visual analysis to help the clinical use of FOT [23]. This optimized interpretation allowed us to observe that the SSc presents smaller values for Cdyn and higher values for R0 and Zrs. Due to its direct physiological translation, this simple spatial description may help interpret the proposed medical decision support system’s results, contributing to its use in the clinical scenario.

Concerning the use of the MIL algorithm, it was efficient selecting attributes (Tables 2 and 3), where it was able to achieve a better result for the control versus normal spirometric analysis than the one obtained by the specialist selection. However, the MIL algorithm was not as efficient as the traditional classifiers (Figs. 3, 4, 5, 7, and 8).

Recent studies have shown the importance of improving our respiratory system knowledge [25] and the non-invasive lung function tests [26,27,28]. Respiratory oscillometry has been widely perceived as the state-of-the-art lung function analysis [29], and one of the most promising emerging technologies in this area [3, 30]. However, although its advantages associated with a detailed and straightforward examination are particularly important, this method is not yet widely used. One of the main aspects limiting its wide routine application is that the obtained indexes' interpretation is based on electric models, requiring training and practice. Previous research has established that diagnostic easiness is a fundamental attribute for occupied non-specialist clinicians [31]. The present study supports previous evidence [14, 16] and contributes to this direction showing that ML algorithms can improve SSc patients’ medical services, simplifying the use of respiratory oscillometry and improving the diagnosis of the cited disease.

Early diagnosis of the abnormal respiratory changes in SSc could support early intervention, thus possibly restricting the disease’s progression, mitigating adverse symptoms, improving general well-being, restraining complications and comorbidities, and early mortality. Artificial intelligence/machine learning methods have improved pulmonary function analysis since the 1980s [4]. The current study extends these findings providing evidence that a combination of respiratory oscillometry and a clinical decision support system based on ML techniques might indicate early abnormal respiratory changes in SSc.

Finally, some important limitations need to be considered and clarified to the reader. First, the study was limited to the Brazilian population at a specific practice site. Thus, it is not possible to ensure its generalizability to a different population. It is suggested that multicenter data be investigated in future studies to expand the generalizability of the findings. It is worth mentioning that by examining the adopted inclusion and exclusion criteria and the present study’s biometric features, readers can easily evaluate whether they are likely to achieve similar findings in their patient population. It is also pertinent to mention that the experimental design of the present work enhances its generalizability. Globally recognized inclusion and exclusion criteria were used, and the work was conducted under usual clinical procedures in a typical setting.

Second, SSc is a disease of low incidence, making it hard to obtain a high number of patients. As a result, the datasets available are relatively small, which requires care to control the complexity of the ML models. In addition to all the care taken in this study to avoid overfitting, such as controlling the hyperparameters, feature selection can also help control overfitting by diminishing the inputs. Another reason to employ feature selection is that a smaller number of features can help simplify the analysis. Moreover, if one uses only three features, it is possible to visualize the separation between groups, which can aid the diagnostic explanation.

Conclusion

We designed and tested various classifier methods to achieve a clinical decision support system to assist in detecting respiratory abnormalities in patients with systemic sclerosis. The respiratory oscillometry parameters alone can only reach moderate diagnostic accuracy (AUC = 0.77) in scenario CGvsPSNS. The ML classifiers' use allowed us to enhance accuracy, reaching high accuracy (AUC ≥ 0.9) in this situation, representing the disease’s initial stages. In the CGvsPSAS, the oscillometric parameter alone could reach high diagnostic accuracy (AUC = 0.94); nevertheless, ML algorithms could provide a small enhancement (AUC = 0.97). The developed system may also help simplify oscillometry use in detecting respiratory changes in patients with systemic sclerosis. Notably, the adoption of feature selection has spotted the most crucial oscillometric parameters, which simplify the analysis. Taken together, the results of the present study and these practical considerations provide clear evidence that respiratory oscillometry combined with machine learning classifiers’ may help to improve lung function tests in systemic sclerosis.

This study’s next steps include improving the understanding and management of systemic sclerosis by integrating ML algorithms and telemedicine systems based on respiratory oscillometry.

Materials and methods

Eighty-two volunteers were included in the study. Fifty-two presented SSc, and 30 were healthy, composing the control group. The patients with SSc were divided into two groups: (1) the normal spirometry group (n = 22), which included patients diagnosed with SSc and showing normal spirometry, and (2) the altered spirometry group (n = 30) that was composed of patients diagnosed with SSc and presenting altered spirometry, associated with restrictive ventilatory disorder [2].

The exams were conducted at the Pulmonary Function Testing Laboratory of the Pedro Ernesto University Hospital and the Biomedical Instrumentation Laboratory of the State University of Rio de Janeiro. The Hospital Ethical Committee approved the study (approval number 456 CEP/HUPE), and all subjects gave informed written consent. This study is in agreement with The Declaration of Helsinki. The inclusion criteria in the present study were a confirmed diagnosis of SSc according to the American College of Rheumatology [32], including volunteers from both genders. The exclusion criteria were a history of exacerbation of disease in the previous 90 days, smoking, and presence of tuberculosis or pneumonia, chronic lung diseases, presence of respiratory infections in the previous 30 days, chest trauma or surgery, inability to perform the tests, and chemotherapy and radiotherapy for cancer.

The control group was composed of healthy volunteers from both genders without a history of cardiovascular or lung disease or smoking. These individuals did not present respiratory infections and showed normal spirometry [33].

The main elements in this study are the respiratory oscillometry measurements, impedance estimation, and clinical decision support system development and performance evaluation. The complete process is shown in Fig. 9. Each operation will be described in the next sections.

Fig. 9
figure 9

Simplified block diagram describing the main steps in this study. K-NN K-Nearest Neighbor, ADAB Adaboost with decision tree classifiers, RF Random Forests, MIL Multiple Instance Learning, XGB Extreme Boosting Gradient Classifiers

Respiratory oscillometry measurements and parameters

These analyses were performed using as input excitation small amplitude pressure oscillations (≤ 2 cmH2O), which were produced by a loudspeaker and applied during tidal breathing at the entrance of the individual’s airway through the oral cavity. The result of the exams was generated as the mean of three tests, each 16 s long. These tests were considered adequate if they were free of pauses and presented stable rate and tidal volumes. A pseudo-random noise signal between 4 and 32 Hz was used, and the exams are repeated until all analyzed frequencies presented the minimal coherence function of 0.9. We used a coefficient of variability ≤ 10% in the lowest frequency (4 Hz) in the three used tests to avoid outlying values. The experiments were conducted using an impedance analyzer described previously [34].

Linear regression in the respiratory resistance values in the 4–16 Hz range was used to interpret the obtained results. This yielded resistance at 0 Hz (R0), the mean resistance in this frequency range (Rm), and the slope of the relationship between the resistive values and frequency (S). R0 describes the low-frequency range. This parameter integrates the Newtonian effects related to the airways, lung, and chest wall resistance and the effect of gas redistribution [35]. The mid-frequency range is described by Rm, which reflects the resistance in the central airways [36]. S is associated with ventilation non-homogeneities [37].

The reactive results were interpreted using four indexes: the mean reactance (Xm), resonance frequency (fr), the impedance module (Zrs), and the dynamic compliance (Cdyn). Xm was calculated using the 4- to 32-Hz frequency range and describes ventilation inhomogeneity. The fr occurs when the elastic and inertive properties cancel out, and the respiratory reactance becomes zero [38]. Cdyn was calculated based on the reactance at 4 Hz (Cdyn = 1/2πfX4) and reflects the respiratory compliance, comprising pulmonary, chest wall, and airway compliances. This parameter is also associated with ventilation homogeneity [36]. Zrs includes the effects of resistance and elastic loads in 4 Hz, representing the respiratory system’s total mechanical load [39].

Data sets

In the present work, experiments were executed in a dataset that consisted of 246 measurements acquired from the volunteers. Healthy volunteers contributed with 90 measurements of the oscillometric parameters, patients with sclerosis and normal spirometry with 66, and patients with sclerosis and altered spirometry supplied 90 measurements.

Machine learning algorithms

Machine learning algorithms can discover crucial relationships among the features in a data set [4, 40]. These models’ inference can be carried out with minimal user intervention through several techniques such as linear models, graphic models, ensemble strategies, hybrid approaches, and artificial neural networks, among others. In our previous research [14, 15, 24], we have experimented with a wide diversity of models and concluded that ensemble strategies had outstanding performance. In this study, we want to investigate the Extreme Gradient Boosting (XGB) algorithm, a type of ensemble derived from gradient boosting. The final inference model is an assemblage of weak inference models, routinely decision trees. It builds the model in a stepwise mode, where its step is designed to model the error of the previous ones. XGB implements Gradient Boosting, focusing on regularization to control overfitting, which gives it better performance. Besides, we also want to explore Multiple Instance Learning (MIL) to the early examination of respiratory changes in patients with systemic sclerosis. Therefore, in this study, the following ML algorithms were appraised:

  • K-Nearest Neighbor (KNN) [41];

  • Adaboost with decision trees [42];

  • Random Forest (RF) [43];

  • Extreme Gradient Boosting (XGB) [44];

  • Multiple Instance Learning (MIL) [45];

The first three algorithms have already been briefly described in the previous studies [14, 15, 24]; therefore, we will provide a condensed description of the two algorithms that have not been used in our studies before. A complete description of them can be found in the references.

The Extreme Gradient Boosting is a more efficient, regularized version of Gradient Boosting. In Gradient Boosting, one fits an additive model (ensemble) in a forward manner. There is an introduction of a weak learner to cope with the previous weak learners’ shortcomings in each stage. These shortcomings can be described by the residuals (errors) left by the previous weak learners. Hence, the weak learner to be added must fit the residuals to the ensemble to produce better results. The relation of this algorithm with gradient descent (GD) is since the residuals can be seen as negative gradients, and the GD can employ them to locate the minimum value of the loss function. Common choices for the loss function are root mean squared error (regression) and log-loss (classification).

The multi-instance learning (MIL) paradigm was introduced by [45] focused on an application in biochemistry. MIL is considered an extension of supervised learning, where the labels are assigned to a set of instances, known as bags, and not to each instance individually. MIL’s central idea is related to the notion of bags: it is labeled as a negative bag (Bi−) if the total instances contained in it are negative and labeled as positive (Bi+) if, at best, one of the instances is positive. In this way, a bag can be defined as a collection of instances or regions. The Diverse Density (DD) algorithm was originally introduced by [46], where the algorithm is described as an assessment of the intersection of positive bags minus the union of negative bags. The algorithm’s central idea is to find a concept point in the feature space close to at least one instance of each positive bag and far from the negative bag instances.

Experimental design

This study executed a total of six experiments. The purpose of the first experiment was to investigate the proficiency of a single oscillometric parameter alone to correctly spot the airway obstruction level in patients with systemic sclerosis. We considered two different situations: the control group versus patients with sclerosis and normal spirometry (CGvsPSNS) and the control group versus patients with sclerosis and altered spirometry (CGvsPSAS). The remaining experiments also evaluate the two situations described.

The second experiment exploited ML algorithms and compared them with the results obtained by a single oscillometric parameter to reveal if the ML algorithms could achieve superior performance. The area under the ROC curve (AUC) was then chosen as the measurement of the performance since it is regularly employed in medicine [47,48,49,50] and yields a superior way to confront classifiers than accuracy [51]. We did not implement feature selection; thus, all of the oscillometric indexes were used. The classifiers described previously were realized with Scikit-learn [52], a machine learning library written in python. On the other hand, Multiple Instance Learning was implemented by the library described in [53]. Since the dataset contains only 246 oscillometric measurements, the k-fold validation procedure [54] is indicated to allow the valuation of the generalization proficiency in the whole dataset. Hyperparameter tuning is a crucial step in model selection. Scikit-learn possesses several strategies to allow hyperparameter fine-tuning, such as grid search, which experiments with all possible combinations of the hyperparameters. Table J0 presents the classifiers and their respective chosen hyperparameters for tuning.

The third experiment evaluates the capability of MIL as a feature selector with the purpose of complexity reduction and to gain knowledge about the importance of different oscillometric parameters [55]. Its role is to select five oscillometric parameters in a previous step before the classifier training. The fourth experiment employs the recursive feature selection (RFE) to select five oscillometric parameters before the classifier training. RFE is a wrapper strategy that can use several ML algorithms to assess the performance. In this paper, the ML algorithm’s choice was the linear support vector machine classifier with L1 regularization. The fifth experiment uses MIL to select three oscillometric parameters, and the sixth employs RFE to choose three oscillometric parameters.

The hypothesis test is a requisite for contrasting ML algorithms. There are a wide variety of parametric tests available, which are commonly based on the t-test [40, 56, 57]. Some of the nonparametric tests most used are McNemar’s and Wilcoxon’s [56, 58, 59]. In this work, the hypothesis test was carried out with AUCs by applying the methodology specified in Delong et al. [17].

Availability of data and materials

The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

%:

Percentile of the predicted values

ADAB:

AdaBoost with decision trees

ANOVA:

Analysis of variance

AUC:

Area under the receiver operating characteristic curve

BOP:

Best oscillometric parameter

Cdyn:

Dynamic compliance

CG:

Control group

DD:

Diverse density algorithm

FOT:

Forced oscillation technique

FEV1 :

Forced expiratory volume in the first second

fr:

Resonant frequency

FVC:

Forced vital capacity

FEF25–75% :

Forced expiratory flow between 25 and 75%

KNN:

K-Nearest Neighbors

MIL:

Multiple instance learning

MIL5 + ML:

Best ML algorithm with 5 oscillometric parameters selected by MIL

ML:

Machine learning

ML7:

Best ML algorithms with all 7 oscillometric parameters

Ns:

Not significant

PSNS:

Patients with sclerosis and normal spirometry

PSAS:

Patients with sclerosis and altered spirometry

RF:

Random Forest

RFE:

Recursive feature selection

RFE5 + ML:

Best ML algorithm with 5 oscillometric parameters selected by RFE

R 0 :

Intercept resistance

R m :

Mean resistance in the 4–16 Hz range

ROC:

Receiver operating characteristic curve

S :

Angular coefficient of resistance in the 4–16 Hz range

Se:

Sensitivity

Sp:

Specificity

SSc:

Systemic sclerosis

XGB:

Extreme Gradient Boosting algorithm

X 4 :

Respiratory reactance at 4 Hz

Zrs:

Absolute value of respiratory impedance in 4 Hz

X m :

Mean reactance evaluated considering the 4 to 32 Hz frequency range

References

  1. Ostojic P, Cerinic MM, Silver R, Highland K, Damjanov N. Interstitial lung disease in systemic sclerosis. Lung. 2007;185(4):211–20.

    Article  Google Scholar 

  2. Miranda IA, Dias Faria AC, Lopes AJ, Jansen JM, de Melo PL. On the respiratory mechanics measured by forced oscillation technique in patients with systemic sclerosis. PLoS ONE. 2013;8(4):e61657.

    Article  Google Scholar 

  3. MacIntyre NR. The future of pulmonary function testing. Respir Care. 2012;57(1):154–64.

    Article  Google Scholar 

  4. King GG, Bates J, Berger KI, Calverley P, de Melo PL, Dellaca RL, Farre R, Hall GL, Ioan I, Irvin CG, et al. Technical standards for respiratory oscillometry. Eur Respir J. 2020;55(2):1900753.

    Article  Google Scholar 

  5. Faria AC, Lopes AJ, Jansen JM, Melo PL. Assessment of respiratory mechanics in patients with sarcoidosis using forced oscillation: correlations with spirometric and volumetric measurements and diagnostic accuracy. Respir Int Rev Thorac Dis. 2009;78(1):93–104.

    Google Scholar 

  6. Lopes AJ, Mogami R, Camilo GB, Machado DC, Melo PL, Carvalho AR. Relationships between the pulmonary densitometry values obtained by CT and the forced oscillation technique parameters in patients with silicosis. Br J Radiol. 2015;88(1049):20150028.

    Article  Google Scholar 

  7. de Sa PM, Lopes AJ, Jansen JM, de Melo PL. Oscillation mechanics of the respiratory system in never-smoking patients with silicosis: pathophysiological study and evaluation of diagnostic accuracy. Clinics. 2013;68(5):644–51.

    Article  Google Scholar 

  8. van Noord JA, Clement J, Cauberghs M, Mertens I, Van de Woestijne KP, Demedts M. Total respiratory resistance and reactance in patients with diffuse interstitial lung disease. Eur Respir J. 1989;2(9):846–52.

    Google Scholar 

  9. Sugiyama A, Hattori N, Haruta Y, Nakamura I, Nakagawa M, Miyamoto S, Onari Y, Iwamoto H, Ishikawa N, Fujitaka K, et al. Characteristics of inspiratory and expiratory reactance in interstitial lung disease. Respir Med. 2013;107(6):875–82.

    Article  Google Scholar 

  10. Fujii M, Shirai T, Mori K, Mikamo M, Shishido Y, Akita T, Morita S, Asada K, Suda T. Inspiratory resonant frequency of forced oscillation technique as a predictor of the composite physiologic index in interstitial lung disease. Respir Physiol Neurobiol. 2015;207:22–7.

    Article  Google Scholar 

  11. Takeichi N, Yamazaki H, Fujimoto K. Comparison of impedance measured by the forced oscillation technique and pulmonary functions, including static lung compliance, in obstructive and interstitial lung disease. Int J Chron Obstruct Pulmon Dis. 2019;14:1109–18.

    Article  Google Scholar 

  12. Sokai R, Ito S, Iwano S, Uchida A, Aso H, Kondo M, Ishiguro N, Kojima T, Hasegawa Y. Respiratory mechanics measured by forced oscillation technique in rheumatoid arthritis-related pulmonary abnormalities: frequency-dependence, heterogeneity and effects of smoking. SpringerPlus. 2016;5:335.

    Article  Google Scholar 

  13. Yamamoto Y, Miki K, Tsujino K, Kuge T, Okabe F, Kawasaki T, Matsuki T, Kagawa H, Miki M, Kida H. Oscillometry and computed tomography findings in patients with idiopathic pulmonary fibrosis. ERJ Open Res. 2020. https://doi.org/10.1183/23120541.00391-2020.

    Article  Google Scholar 

  14. Amaral JLM, Lopes AJ, Veiga J, Faria ACD, Melo PL. High-accuracy detection of airway obstruction in asthma using machine learning algorithms and forced oscillation measurements. Comput Methods Programs Biomed. 2017;144:113–25.

    Article  Google Scholar 

  15. Amaral JLM, Lopes AJ, Faria ACD, Melo PL. Machine learning algorithms and forced oscillation measurements to categorise the airway obstruction severity in chronic obstructive pulmonary disease. Comput Methods Programs Biomed. 2015;118(2):186–97.

    Article  Google Scholar 

  16. Amaral JL, Lopes AJ, Jansen JM, Faria AC, Melo PL. An improved method of early diagnosis of smoking-induced respiratory changes using machine learning algorithms. Comput Methods Programs Biomed. 2013;112(3):441–54.

    Article  Google Scholar 

  17. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45.

    Article  MATH  Google Scholar 

  18. Greenwald GI, Tashkin DP, Gong H, Simmons M, Duann S, Furst DE, Clements P. Longitudinal changes in lung function and respiratory symptoms in progressive systemic sclerosis. Prospective study. Am J Med. 1987;83(1):83–92.

    Article  Google Scholar 

  19. Aronsson D, Hesselstrand R, Bozovic G, Wuttge DM, Tufvesson E. Airway resistance and reactance are affected in systemic sclerosis. Eur Clin Respir J. 2015;2:28667.

    Article  Google Scholar 

  20. Marinho CL, Maioli MCP, do Amaral JLM, Lopes AJ, Melo PL. Respiratory resistance and reactance in adults with sickle cell anemia: correlation with functional exercise capacity and diagnostic use. PLoS ONE. 2017;12(12):e0187833.

    Article  Google Scholar 

  21. Ribeiro CO, Lopes AJ, de Melo PL. Oscillation mechanics, integer and fractional respiratory modeling in COPD: effect of obstruction severity. Int J Chron Obstruct Pulmon Dis. 2020;15:3273–89.

    Article  Google Scholar 

  22. Marinho CL, Maioli MCP, Amaral J, Lopes AJ, Melo PL. Respiratory resistance and reactance in adults with sickle cell anemia: Part 2—fractional-order modeling and a clinical decision support system for the diagnosis of respiratory disorders. PLoS ONE. 2019;14(3):e0213257.

    Article  Google Scholar 

  23. Amaral JLM, Sancho AG, Faria ACD, Lopes AJ, Melo PL. Differential diagnosis of asthma and restrictive respiratory diseases by combining forced oscillation measurements, machine learning and neuro-fuzzy classifiers. Med Biol Eng Comput. 2020;58(10):2455–73.

    Article  Google Scholar 

  24. Amaral JLM, Lopes AJ, Jansen JM, Faria ACD, Melo PL. Machine learning algorithms and forced oscillation measurements applied to the automatic identification of chronic obstructive pulmonary disease. Comput Methods Programs Biomed. 2012;105(3):183–93.

    Article  Google Scholar 

  25. Bit A, Chattyopadhay H, Nag D. Study of airflow in the trachea of a bronchopulmonary patient using CT data. Indian J Biomech. 2009:31–36.

  26. Croxton TL, Weinmann GG, Senior RM, Hoidal JR. Future research directions in chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2002;165(6):838–44.

    Article  Google Scholar 

  27. Drummond MB, Buist AS, Crapo JD, Wise RA, Rennard SI. Chronic obstructive pulmonary disease: NHLBI workshop on the primary prevention of chronic lung diseases. Ann Am Thorac Soc. 2014;11(Suppl 3):S154-160.

    Article  Google Scholar 

  28. Busse WW, Erzurum SC, Blaisdell CJ, Noel P. Executive summary: NHLBI workshop on the primary prevention of chronic lung diseases. Ann Am Thorac Soc. 2014;11(Suppl 3):S123-124.

    Article  Google Scholar 

  29. Bates JHT, Irvin CG, Farré R, Hantos Z. Oscillation mechanics of the respiratory system. In: Terjung R, editor. Comprehensive physiology. Hoboken: Wiley; 2011.

    Google Scholar 

  30. Brusasco V, Barisione G, Crimi E. Pulmonary physiology: future directions for lung function testing in COPD. Respirology. 2015;20(2):209–18.

    Article  Google Scholar 

  31. Global Initiative For Chronic Obstructive Lung Disease—UPDATE. Global strategy for the diagnosis, management, and prevention of chronic obstrutive pulmonary disease. 2013.

  32. Masi AT, Diagnostic SFSCotARA, Committee TC. Preliminary criteria for the classification of systemic sclerosis (scleroderma). Arthritis Rheum. 1980;23(5):581–90.

    Article  Google Scholar 

  33. Knudson RJ, Lebowitz MD, Holberg CJ, Burrows B. Changes in the normal maximal expiratory flow-volume curve with growth and aging. Am Rev Respir Dis. 1983;127(6):725–34.

    Google Scholar 

  34. de Melo PL, Werneck MM, Giannella-Neto A. New impedance spectrometer for scientific and clinical studies of the respiratory system. Rev Sci Instrum. 2000;71(7):2867–72.

    Article  Google Scholar 

  35. Lorino AM, Zerah F, Mariette C, Harf A, Lorino H. Respiratory resistive impedance in obstructive patients: linear regression analysis vs viscoelastic modelling. Eur Respir J. 1997;10(1):150–5.

    Article  Google Scholar 

  36. MacLeod D, Birch M. Respiratory input impedance measurement: forced oscillation methods. Med Biol Eng Comput. 2001;39(5):505–16.

    Article  Google Scholar 

  37. Brochard L, Pelle G, de Palmas J, Brochard P, Carre A, Lorino H, Harf A. Density and frequency dependence of resistance in early airway obstruction. Am Rev Respir Dis. 1987;135(3):579–84.

    Google Scholar 

  38. Cavalcanti JV, Lopes AJ, Jansen JM, Melo PL. Detection of changes in respiratory mechanics due to increasing degrees of airway obstruction in asthma by the forced oscillation technique. Respir Med. 2006;100(12):2207–19.

    Article  Google Scholar 

  39. Nagels J, Landser FJ, van der Linden L, Clement J, Van de Woestijne KP. Mechanical properties of lungs and chest wall during spontaneous breathing. J Appl Physiol Respir Environ Exercise Physiol. 1980;49(3):408–16.

    Google Scholar 

  40. Witten IH, Frank E, Hall MA, Pal CJ. Data mining: practical machine learning tools and techniques. Morgan Kaufmann; 2016.

    Google Scholar 

  41. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. New York: Springer; 2009.

    Book  MATH  Google Scholar 

  42. Schapire RE. Explaining adaboost. In: Empirical inference. New York: Springer; 2013. p. 37–52.

    Chapter  Google Scholar 

  43. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.

    Article  MATH  Google Scholar 

  44. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: The 22nd ACM SIGKDD international conference, 2016. ACM Press; 2016. p. 785–794.

  45. Dietterich TG, Lathrop RH, Lozano-Pérez T. Solving the multiple instance problem with axis-parallel rectangles. Artif Intell. 1997;89(1–2):31–71.

    Article  MATH  Google Scholar 

  46. Maron O, Lozano-Pérez T. A framework for multiple-instance learning. In: Jordan MI, Kearns MJ, editors. Advances in neural information processing systems 10. Solla: MIT Press; 1998. p. 570–6.

    Google Scholar 

  47. Hajian-Tilaki K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med. 2013;4(2):627–35.

    Google Scholar 

  48. Kumar R, Indrayan A. Receiver operating characteristic (ROC) curve for medical researchers. Indian Pediatr. 2011;48(4):277–87.

    Article  Google Scholar 

  49. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.

    Article  Google Scholar 

  50. Metz CE. Basic principles of ROC analysis. WB Saunders; 1978. p. 283–98.

    Google Scholar 

  51. Jin H, Ling CX. Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng. 2005;17(3):299–310.

    Article  Google Scholar 

  52. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825–30.

    MathSciNet  MATH  Google Scholar 

  53. Tax VDMJC. MIL, A Matlab toolbox for multiple instance learning. 2016.

  54. Abu-Mostafa YS, Magdon-Ismail M, Lin H-T. Learning from data: a short course. S.l.: AMLbook.com; 2012.

    Google Scholar 

  55. Guyon I, Elisseeff A. An introduction to variable and feature selection. J Mach Learn Res. 2003;3(Mar):1157–82.

    MATH  Google Scholar 

  56. Dietterich TG. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 1998;10(7):1895–923.

    Article  Google Scholar 

  57. Alpaydm E. Combined 5 × 2 cv F test for comparing supervised classification learning algorithms. Neural Comput. 1999;11(8):1885–92.

    Article  Google Scholar 

  58. Demšar J. Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res. 2006;7(Jan):1–30.

    MathSciNet  MATH  Google Scholar 

  59. Japkowicz N, Shah M. Evaluating learning algorithms: a classification perspective. Cambridge University Press; 2011.

    Book  MATH  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study was supported by the Brazilian Council for Scientific and Technological Development (CNPq), the Rio de Janeiro State Research Supporting Foundation (FAPERJ) and in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Author information

Authors and Affiliations

Authors

Contributions

DSMA and LMR performed software development, implemented the computer code and supporting algorithms, analyzed the data, and drafted the manuscript. AJL collected data regarding pulmonary function and provided subject identification, and helped to draft the manuscript. JLMA and PLM mentored DSMA, provided funding, participated in the data analysis process, and helped to draft the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Pedro L. Melo.

Ethics declarations

Ethics approval and consent to participate

The Research Ethics Committee of the Pedro Ernesto University Hospital granted approval for this study and the collection and use of the data analyzed in this study.

Consent for publication

All authors consent for the publication of this manuscript.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Table S1 Results of the experiment 1 (Control group versus Patients with sclerosis and normal spirometry). The Area Under the ROC Curve (AUC), the Standard Error (SE) and the 95% confidence interval (95% CI) of each FOT parameter. Table S2 Results of the experiment 1 (Control group x Patients with sclerosis and altered spirometry). The Area Under the ROC Curve (AUC), the Standard Error (SE) and the 95% confidence interval (95% CI) of each FOT parameter.

Additional file 2.

Figure S1 ROC curves for Experiment 1. ROC curves for each one of the FOT parameters - Control Group versus Patients with Sclerosis and Normal Spirometry (CGvsPSNS). Figure S2 ROC curves for Experiment 1. ROC curves for each one of the FOT parameters - Control Group versus Patients with Sclerosis and Altered Spirometry (CGvsPSAS).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Andrade, D.S.M., Ribeiro, L.M., Lopes, A.J. et al. Machine learning associated with respiratory oscillometry: a computer-aided diagnosis system for the detection of respiratory abnormalities in systemic sclerosis. BioMed Eng OnLine 20, 31 (2021). https://doi.org/10.1186/s12938-021-00865-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12938-021-00865-9

Keywords