Research Highlights Racial Biases in AI's Performance with Retinal Images

According to a study, AI algorithms have demonstrated the capability to determine race not just from color retinal fundus images but also from grayscale retinal vessel maps that remove any visual cues allowing humans to infer race. The authors of the study have expressed concerns regarding potential racial biases in AI that could impact diagnosis and treatment decisions.

The research involved a selected sample from the Imaging and Informatics in Retinopathy of Prematurity cohort study, comprising 94 infants reported as Black and 151 infants reported as White by their parents or guardians. A total of 4,095 color retinal fundus images were collected to train, validate, and test 40 ResNet-18 convolutional neural network (CNN) models.

Subsequently, all color retinal fundus images were segmented into grayscale retinal vessel maps, which underwent iterative transformations such as thresholding, binarizing, and skeletonizing. These transformations aimed to eliminate any potential information related to vessel pigmentation, size, and caliber that could be used to differentiate between individuals of Black and White race.

The models exhibited a remarkably accurate ability to predict self-reported race (SRR) based on color retinal fundus images.

"Although race itself is a social construct, it is associated with variations in skin and retinal pigmentation," the authors wrote, highlighting the connection between race and physical attributes.

Surprisingly, the CNN demonstrated comparable precision in inferring Black versus White race even from grayscale retinal vessel maps, a task that humans are unable to perform. "Even images that appeared devoid of information to the naked eye retained predictive information," the authors emphasized, indicating the AI's ability to extract meaningful race-related features from the retinal vessel maps.

The authors expressed concerns regarding the implications of their diagnostic study, stating, "Results of this diagnostic study suggest that it can be very challenging to remove information relevant to self-reported race (SRR) from fundus photographs. As a result, AI algorithms trained on fundus photographs have the potential for biased performance in practice." This highlights the potential for AI algorithms to exhibit racial biases in their performance, which can have significant consequences in real-world diagnosis and treatment scenarios.