Joint conditional generative adversarial networks for eyelash artifact removal in ultra-wide-field fundus images

Jiong Zhang; Dengfeng Sha; Yuhui Ma; Dan Zhang; Tao Tan; Xiayu Xu; Quanyong Yi; Yitian Zhao

doi:10.3389/fcell.2023.1181305

Joint conditional generative adversarial networks for eyelash artifact removal in ultra-wide-field fundus images

Front Cell Dev Biol. 2023 May 5:11:1181305. doi: 10.3389/fcell.2023.1181305. eCollection 2023.

Authors

Jiong Zhang^{1

2}, Dengfeng Sha^{1

3}, Yuhui Ma¹, Dan Zhang⁴, Tao Tan⁵, Xiayu Xu^{6

7}, Quanyong Yi², Yitian Zhao^{1

2}

Affiliations

¹ Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, China.
² The Affiliated Ningbo Eye Hospital of Wenzhou Medical University, Ningbo, China.
³ Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, China.
⁴ School of Cyber Science and Engineering, Ningbo University of Technology, Ningbo, China.
⁵ Faulty of Applied Sciences, Macao Polytechnic University, Macao, Macao SAR, China.
⁶ The Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an, China.
⁷ Zhejiang Research Institute of Xi'an Jiaotong University, Hangzhou, China.

Abstract

Background: Ultra-Wide-Field (UWF) fundus imaging is an essential diagnostic tool for identifying ophthalmologic diseases, as it captures detailed retinal structures within a wider field of view (FOV). However, the presence of eyelashes along the edge of the eyelids can cast shadows and obscure the view of fundus imaging, which hinders reliable interpretation and subsequent screening of fundus diseases. Despite its limitations, there are currently no effective methods or datasets available for removing eyelash artifacts from UWF fundus images. This research aims to develop an effective approach for eyelash artifact removal and thus improve the visual quality of UWF fundus images for accurate analysis and diagnosis. Methods: To address this issue, we first constructed two UWF fundus datasets: the paired synthetic eyelashes (PSE) dataset and the unpaired real eyelashes (uPRE) dataset. Then we proposed a deep learning architecture called Joint Conditional Generative Adversarial Networks (JcGAN) to remove eyelash artifacts from UWF fundus images. JcGAN employs a shared generator with two discriminators for joint learning of both real and synthetic eyelash artifacts. Furthermore, we designed a background refinement module that refines background information and is trained with the generator in an end-to-end manner. Results: Experimental results on both PSE and uPRE datasets demonstrate the superiority of the proposed JcGAN over several state-of-the-art deep learning approaches. Compared with the best existing method, JcGAN improves PSNR and SSIM by 4.82% and 0.23%, respectively. In addition, we also verified that eyelash artifact removal via JcGAN could significantly improve vessel segmentation performance in UWF fundus images. Assessment via vessel segmentation illustrates that the sensitivity, Dice coefficient and area under curve (AUC) of ResU-Net have respectively increased by 3.64%, 1.54%, and 1.43% after eyelash artifact removal using JcGAN. Conclusion: The proposed JcGAN effectively removes eyelash artifacts in UWF images, resulting in improved visibility of retinal vessels. Our method can facilitate better processing and analysis of retinal vessels and has the potential to improve diagnostic outcomes.

Keywords: artifact removal; conditional GAN; retina; ultra-wide-field fundus images; vessel segmentation.

Grants and funding

This work was supported in part by the National Natural Science Foundation Program of China under Grants (62103398 and 61906181), in part by the Zhejiang Provincial Natural Science Foundation (LQ23F010002, LZ23F010002, LR22F020008, LZ19F010001, and LQ21F010007), in part by the Ningbo Natural Science Foundation (2022J143, 20221JCGY010608), in part by the Youth Innovation Promotion Association CAS (2021298), the Research Startup Fund of Ningbo University of Technology (2022KQ29), and in part by 2025 S&T Megaprojects (2019B10033, 2019B10061, and 2021Z054).