Valvular heart disease (VHD) is one of the most prevalent cardiac diseases affecting about 2.5% of the population in the United States (1).
The survival advantage of heart valve replacement makes this method the treatment of choice for patients with severe valve stenosis, regurgitation, or high-degree valve calcification who develop symptoms (2). Such invasive procedures have several complications, such as various comorbidities and higher risk of mortality; therefore, proper risk assessment must be carried out. On the other hand, outcomes and mortality rates vary depending on surgical techniques, valve location, and patient characteristics (3,4).Previous studies have investigated independent risk factors in association with mortality after valve replacement, but they have included a limited number of cases due to fewer surgical treatments for valve replacement than other procedures, e.g., coronary artery bypass graft (CABG) surgery (5,6).
Artificial intelligence and machine learning approaches are new methods consisting of a collection of technologies for analyzing huge amounts of data, which allow the health care system to find algorithms and models for diagnosing diseases and predicting risks and complications during various medical procedures (7). These novel methods of analysis may reveal new prediction models (8). Identifying more robust prediction models helps cardiac surgeons and hospital staff to choose the best management approach and monitor patients with a higher risk of mortality more closely.
In the present study,we collected a significant amount of clinical data related to patients undergoing heart valve replacement surgery at a large referral tertiary hospital center and assessed a variety of machine learning models in order to determine which models are the most effective at predicting in-hospital mortality in patients who underwent prosthetic valve replacement surgery.

 

Table 2. The results of predictors of in-hospital mortality: The univariate analysis

CPB time: Cardio-pulmonary bypass time; K: Potassium; Cr: Creatinine; Mg: Magnesium; UA: Uric Acid; INR: International normalized ratio; Na: Sodium; BUN: Blood urea nitrogen; FBS: Fasting blood sugar, PTT: Partial thromboplastin time; Na: Sodium; ESR: Erythrocyte sedimentation rate; LDH: Lactate dehydrogenase; LDL: Low-density lipoprotein cholesterol; HDL: High-density lipoprotein cholesterol; Plt: Platelet; Hb: Hemoglobin; Hct: Hematocrit; Ca: Calcium; CABG: Coronary artery bypass surgery.

 
Figure 1. Correlations among variables. ALT: Alanine transaminase; AST: Aspartate transferase; CRP: C-reactive protein, the rest of abbreviations are similar to Table 2.

According to the results of final multiple Bayesian logistic regression model, the positive statistically significant association was observed between in-hospital mortality rate and Cardiopulmonary bypass (Pump, CPB) time per minute (OR=1.01), serum BUN (OR=1.102), PTT (OR=1.025), and serum Na (OR=1.173) variables. While, the inverse statistically significant association was observed between mortality rate and Hemoglobin (OR=0.606), PLT (OR=0.973), and serum creatinine (OR=0.618) variables.
Compared to negative proteinuria, individuals with proteinuria 2+ have 9.92 times higher risk of mortality, while we did not find this association in proteinuria 1+. The serum potassium significantly increased in-hospital mortality rate by 7.91 (95% CI: 4.43, 15.3). A positive association was observed between in-hospital mortality and valve replacement site (OR=2.12; 95% CI: 3.22, 1088.6). In the regression analysis, the pulmonary valve replacement had a significantly lower risk of in-hospital mortality risk compared to Aorta, Mitral, and tricuspid valves replacement. The risk of mortality was decreased in both continues and individual suture types, compared to suture-less type, the risk of mortality was decreased by 0.052 (95% CI: 0.015, 0. 16), and 0.082 (95% CI: 0.023, 0.28) for Continues and Individual suture types, respectively (Table 3).

Table 3. The predictors of in-hospital mortality: Results of final multiple Bayesian logistic regression model in multivariate statistical analysis.

The abbreviations are similar to Table 2.
 

The results of ROC curve showed that the last BUN, Cr, Hb, potassium, Plt count, and CPB time had the highest predictive values (Opt cut-off: 27, AUC: 0.862, Se: 67.91%, Sp: 89.83%, P<0.001; Opt cut-off: 1.1, AUC: 0.841, Se: 74.14%, Sp: 81.81%, P<0.001; Opt cut-off: 8.8, AUC: 0.830, Se: 68.32%, Sp: 86.50%, P<0.001; Opt cut-off: 4.6, AUC: 0.840, Se: 71.74%, Sp: 89.78%, P<0.001; Opt cut-off: 140, AUC: 0.862, Se: 69.78%, Sp: 90.67%, P<0.001; Opt cut-off: 139, AUC: 0.703, Se: 55.28%, Sp: 75.67%, P<0.001, Respectively) to predict in-hospital mortality, respectively.

Prediction models

Feature importance for pre-operative, intra-operative, post-operative and all features datasets are presented in Figure 2, Panel A-D, respectively.

 
Figure 2. Feature importance for pre-operative, intra-operative, post-operative and all features datasets. Abbreviations are similar to Table 2 and Figure 1.
 

Figure 3. AUC box plots of different machine learning classifiers among the different number of features in 4 sub-datasets [Pre-operative (Panel A), intra operative (Panel B), post-operative (Panel C), and all features (Panel D)]. Abbreviations are similar to Table 1.

According to Table 4 and Figure 3, AB classifier had better performance than other classifier in all datasets (AUC: 0.82 ± 0.033, 0.68 ± 0.014, 0.90 ± 0.100, and 0.93 ± 0.049 for pre-operative, intra-operative, post-operative, and all features’ datasets, respectively). On the other hand, BNB classifier had the lowest performance in almost all datasets (AUC: 0.500 ± 0.000, 0.52 ± 0.015, 0.50 ± 0.000, and 0.52 ± 0.020 for pre-operative, intra-operative, post-operative, and all features’ datasets, respectively).

Table 4. AUC of different machine learning classifiers among the different number of features in 4 sub-datasets

 
Figure 4. Maximum AUC for the different classifiers in different numbers of features for 4 sub-datasets along with the sensitivity and specificity based one the maximum AUC. Abbreviations are similar to Table 1.

Figure 4 indicates maximum AUC for different classifier in different number of features for 4 sub datasets. In pre-operative dataset, MLP, AB and DT with AUC: 0.88, 0.86 and 0.83 had highest AUC max, respectively. In intra-operative, DT, AB and QDA with AUC: 0.72, 0.71, 0.71 had highest AUC max, respectively. In post-operative, AB had AUC: 0.95 and followed by LR, RF, GNB and QDA had AUC: 0.94. The last dataset with all features, AB and LR had AUC: 0.97, followed by SVM with AUC: 0.96 and RF, QDA and GNB with AUC: 0.94.
In multivariate analysis performance of models was assessed by AUC, Sensitivity (SEN), Specificity (SPE), Positive predictive value (PPV), and Negative predictive value (NPV). Table 5 shows top models based on the above metric in pre-operative, intra-operative, post-operative, and all features’ datasets. Each row indicates the top model based on that metric is bold.

 
Table 5. Top prediction model of 4 datasets

NOF: Number of features; PPV: Positive predictive value; NPV: Negative predictive value; MLP: Multi-layer Perceptron; RF: Random forest; DT: Decision tree; AB: Adaptive boost.
 

In pre-operative dataset MLP classifier with 26 features had the highest AUC (0.878) and the highest SEN (0.969) and NPV (0.994), but RF with 26 features had the highest SPE (0.998) and PPV (0.973). In intra-operative dataset, DT classifier with 7 features had the highest AUC (0.716) and the highest SEN (0.484) and NPV (0.925) but AB with 7 features had the highest SPE (0.998) and PPV (0.955). In post-operative dataset, AB classifier with 12 features had the highest AUC (0.954) and the highest SEN (0.922) and NPV (0.998) but AB with 15 features had the highest SPE (0.998) and PPV (0.982). In all features dataset, AB with 56 features had the highest AUC (0.968), SPE (0.998) and PPV (0.984) and MLP with 72 features had the highest SEN (0.953) and NPV (0.992) (Figure 5).

 
Figure 5. The impact of the number of features and performance of different model classifiers with AUC metric. Abbreviations are similar to Table 1.

Discussion

According to the results, the overall in-hospital mortality rate was 6.36%, which was consistent with the results reported by the Society of Thoracic Surgeons National Cardiac Surgery Database and National Inpatients Sample (NIS), which estimated the in-hospital mortality rate to be about 6.0% to 8.0% based on the target study population (12,13).
The prediction model results indicated that the AB classifier had the best performance in predicting in-hospital mortality after valve replacement surgery. AB is a well-known ensemble machine learning method for classification problems, which can effectively improve multiple weak predictive models by building an ensemble of models. AdaBoost focuses on the cases that were defectively predicted in the previous model, thereby ensuring that the more complex patterns are detected. In other words, AB yields its final output through the combination of the predictions of the individual models. AB offers the advantage of good generalization and can reduce both bias and variance (14). Generally, the AB model is reported to have an acceptable performance in the prediction of in-hospital mortality in patients admitted to ICU regardless of the cause of hospitalization (15). However, the decision tree-based AB algorithms require a quality dataset and are sensitive to outlier and noisy data; thus, they have the potential to overfit the training set. AB had the worst performance in predicting survival after heart transplantation compared to RF and neural network ensembles (16). Therefore, for complex and high dimension data, gradient boosting (GB) outperforms AB due to GB's system optimizations. Similarly, GB had the best performance (AUC: 0.78) for the prediction of acute kidney injury after cardiac surgery (17). Furthermore, in a study by Kilic et al., GB was reported to be a promising model for prediction analytics in cardiac surgery mortality (18). Hajianfar et al. also confirmed the previous results for the GB prediction model in methyl-guanine-DNA methyltransferase (MGMT) methylation status (19). Compared to the other models, the BNB model had the worst performance overall in all four datasets. The obtained results may have been expected because the NB model unrealistically assumes that all of the features are independent and equally important (20). Nilsson et al. used artificial neural network for the prediction of mortality after cardiac surgery, which exhibited an AUC of 0.81, while the MLP performance in our study had an AUC of 0.92 and was the better performance (21). However, the study population in the investigation was comprised of all cardiac surgery procedures whereas our target population was limited to only those who underwent valvular replacement surgeries. In this study, preoperative LDH had the highest feature importance in the model design, which was consistent with the results of the study by Zhong et al. in which LDH and platelet had a higher rate of appearance among other predictors in patients undergoing open heart surgery (22).
The performance of all models was relatively poor in the intra-operative dataset; however, machine learning in our study favored the inclusion of a wide range of variables that had acceptable performance in the pre- and post-operative datasets.

 

Limitations

This study had several limitations: 1) Only in-hospital mortality was considered and not the 30-day mortality, with all consequences correlated. The in-hospital mortality could be affected by a bad management of the discharge among other factors. 2)  This was a single-center study, and the results may lack enough power to be generalized.

 

Conclusion

The results of machine learning algorithms in the prediction of in-hospital mortality were promising, and different algorithms performed better than the univariate results. Therefore, machine learning-based predictive models, such as AB, can be used to enhance significant data in prosthetic valve replacement surgery and to propose prediction and risk stratification models. AB is an advanced machine learning algorithm, which exhibited the highest discriminatory performance for the prediction of in-hospital mortality following valve replacement surgery.

 

Acknowledgements

We would like to express our thanks to Mr. Peyman Tabatabaei; Mrs. Mehrnoosh Ebrahimi; Mrs. Nasim Jafari for their fruitful consultations and cooperation through the study.

 

Conflicts of Interest

   None Declared.

 

Funding

   None Declared.