A Calibrated Multi-Backbone Ensemble Learning for Multi-Label Chest X-Ray Pathology Detection with Automated Structured Reporting

Abstract

Chest radiography is one of the most important and diagnostic methods that are widely used in the world, but the interpretation of the chest X-rays (CXRs) to identify various pathologies is difficult. This paper presents a new ensemble deep learning model to do the multi-label classification of the NIH ChestX-ray14 dataset. The model combines three convolutional neural network (CNN) architectures - DenseNet, ResNet, and EfficientNet - and adds a class-wise AUC weighting scheme that fuses the predictions of each pathology by the architectures. Besides this, we use test-time augmentation (TTA) as a means of augmenting robustness. The ensemble was also optimized on the pre-trained models and tested on ChestX-ray14, giving a mean area under the ROC (AUC) of 0.891 across 14 chest pathologies, which is better than the individual constituent models. Importantly, our approach achieves state-of-the-art results when applied to multiple disease classes, and large average precision (AP) values on important abnormalities. We also produce Gradient-weighted Class Activation Mapping (Grad-CAM) visualizations to understand the decision made by the model, which are also clinically relevant when highlighted on the X-rays. The presented weighted-ensemble approach has shown that the multi-disease detection on chest X-rays by taking advantage of the complementary capabilities of various CNN architectures combined with class-specific weighting and TTA can significantly increase the performance. The work could be beneficial to radiologists in that they can make proper predictions with visual explanations of every condition identified in a particular situation.