Breast cancer remains one of the leading causes of cancer-related mortality among women. Accurate classification of breast cancer subtypes, based on molecular biomarkers such as oestrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2), is critical for guiding treatment decisions and predicting outcomes. Noninvasive imaging technologies have the potential to identify these biomarkers without requiring biopsy procedures. Recent advances in deep learning have demonstrated promise in interpreting medical images, particularly in predicting cancer-related biomarkers. A recent review published in Insights Into Imaging explores the effectiveness of a multi-task deep learning model trained on 3D whole breast ultrasound (3DWBUS) images for accurately predicting ER, PR, and HER2 expression and enhancing interpretability.
Deep Learning Models for Biomarker Prediction
Traditional single-task deep learning models analyse ultrasound data to predict the expression of individual biomarkers. However, these models often need more explicit constraints to interpretability issues. In contrast, the multi-task learning approach combines two tasks: tumour segmentation and biomarker prediction. By integrating segmentation, the model is encouraged to focus on regions of interest within the breast, thus improving prediction accuracy and model interpretability.
The study analysed 388 breast cancer patients who underwent 3DWBUS imaging. The dataset was divided into training, validation, and test sets. A 3D ResNet architecture was used to build both single-task and multi-task models, with the former focusing solely on biomarker expression prediction and the latter combining segmentation and classification tasks. Model performance was evaluated using metrics such as the Area Under the Receiver Operating Characteristic Curve (AUC), sensitivity, specificity, and interpretability through visualisation techniques like Grad-CAM++.
Performance of Single-Task vs. Multi-Task Models
Both single-task and multi-task models demonstrated strong predictive capabilities for ER, PR, and HER2. In individual evaluations, the single-task model showed higher performance for ER prediction with an AUC of 0.809. In contrast, the multi-task model performed better for PR and HER2 predictions with AUCs of 0.767 and 0.697, respectively. However, when considering the overall performance, the multi-task model consistently outperformed the single-task model across multiple metrics. Specifically, the multi-task model achieved a higher macro AUC (0.733) than the single-task model (0.708).
The multi-task model's improved performance can be attributed to its ability to focus on the lesion area more effectively through the segmentation task. By leveraging the additional segmentation constraint, the model was better able to distinguish relevant tumour features associated with biomarker expression, leading to more accurate predictions. This was evident in the enhanced attention to diseased tissue areas, as visualised by Grad-CAM++ heatmaps, which concentrated on the lesion regions.
Explainability and Visualisation with Grad-CAM++
The multi-task model was further validated for its ability to improve interpretability. Grad-CAM++ was employed to visualise the attention regions within the ultrasound images. Heatmaps generated by this technique revealed that the multi-task model exhibited a stronger focus on diseased tissue areas compared to the single-task model. The attention was directed towards blue or red areas, representing regions of high predictive value, while purple areas, with lower predictive values, were less emphasised.
The t-distributed stochastic neighbour embedding (t-SNE) analysis further highlighted the improved discriminatory ability of the multi-task model, particularly for PR and HER2. The visualisation showed clear differentiation between positive and negative samples for these biomarkers, while the single-task model performed better in distinguishing ER expression. This enhanced visualisation not only improves model transparency but also aids clinicians in understanding the underlying decision-making process of the deep learning model.
This study demonstrates that both single-task and multi-task deep learning models can effectively predict breast cancer biomarker expression from 3DWBUS images. The multi-task model, which integrates tumour segmentation with biomarker prediction, exhibits superior overall performance and enhanced interpretability. This model’s ability to focus on lesion regions while generating heatmaps through Grad-CAM++ provides a clear visualisation of the predictive process, making it a promising tool for noninvasive breast cancer diagnosis and targeted therapy selection. While further validation is necessary to generalise the model to diverse clinical settings, the study establishes a foundational framework for explainable AI in breast cancer imaging, paving the way for improved diagnostic and treatment workflows.
Source: Insights Into Imaging
Image Credit: iStock
