Where, why, and how is bias learned in medical image analysis models? A study of bias encoding within convolutional networks using synthetic data

Emma A M Stanley; Raissa Souza; Matthias Wilms; Nils D Forkert

doi:10.1016/j.ebiom.2024.105501

Where, why, and how is bias learned in medical image analysis models? A study of bias encoding within convolutional networks using synthetic data

EBioMedicine. 2024 Dec 12:111:105501. doi: 10.1016/j.ebiom.2024.105501. Online ahead of print.

Authors

Emma A M Stanley¹, Raissa Souza², Matthias Wilms³, Nils D Forkert⁴

Affiliations

¹ Biomedical Engineering Graduate Program, University of Calgary, Calgary, Canada; Department of Radiology, University of Calgary, Calgary, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Canada. Electronic address: emma.stanley@ucalgary.ca.
² Biomedical Engineering Graduate Program, University of Calgary, Calgary, Canada; Department of Radiology, University of Calgary, Calgary, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Canada.
³ Department of Radiology, University of Calgary, Calgary, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Canada; Department of Pediatrics, University of Calgary, Calgary, Canada; Department of Community Health Sciences, University of Calgary, Calgary, Canada.
⁴ Department of Radiology, University of Calgary, Calgary, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Canada; Department of Clinical Neuroscience, University of Calgary, Calgary, Canada.

Abstract

Background: Understanding the mechanisms of algorithmic bias is highly challenging due to the complexity and uncertainty of how various unknown sources of bias impact deep learning models trained with medical images. This study aims to bridge this knowledge gap by studying where, why, and how biases from medical images are encoded in these models.

Methods: We systematically studied layer-wise bias encoding in a convolutional neural network for disease classification using synthetic brain magnetic resonance imaging data with known disease and bias effects. We quantified the degree to which disease-related information, as well as morphology-based and intensity-based biases were represented within the learned features of the model.

Findings: Although biases were encoded throughout the model, a stronger encoding did not necessarily lead to the model using these biases as a shortcut for disease classification. We also observed that intensity-based effects had a greater influence on shortcut learning compared to morphology-based effects when multiple biases were present.

Interpretation: We believe that these results constitute an important first step towards a deeper understanding of algorithmic bias in deep learning models trained using medical imaging data. This study also showcases the benefits of utilising controlled, synthetic bias scenarios for objectively studying the mechanisms of shortcut learning.

Funding: Alberta Innovates, Natural Sciences and Engineering Research Council of Canada, Killam Trusts, Parkinson Association of Alberta, River Fund at Calgary Foundation, Canada Research Chairs Program.

Keywords: Algorithmic bias; Artificial intelligence; Synthetic data.