Learning from crowds in digital pathology using scalable variational Gaussian processes
Metadatos
Mostrar el registro completo del ítemEditorial
Nature
Fecha
2021-06-02Referencia bibliográfica
López-Pérez, M... [et al.]. Learning from crowds in digital pathology using scalable variational Gaussian processes. Sci Rep 11, 11612 (2021). [https://doi.org/10.1038/s41598-021-90821-3]
Patrocinador
Agencia Estatal de Investigacion of the Spanish Ministerio de Ciencia e Innovacion PID2019-105142RB-C22/AEI/10.13039/501100011033; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Cancer Institute (NCI) U01CA220401 U24CA19436201; La Caixa Banking Foundation (Barcelona, Spain) Barcelona, Spain) through La Caixa Fellowship 100010434 LCF/BQ/ES17/11600011Resumen
The volume of labeled data is often the primary determinant of success in developing machine
learning algorithms. This has increased interest in methods for leveraging crowds to scale data
labeling efforts, and methods to learn from noisy crowd-sourced labels. The need to scale labeling is
acute but particularly challenging in medical applications like pathology, due to the expertise required
to generate quality labels and the limited availability of qualified experts. In this paper we investigate
the application of Scalable Variational Gaussian Processes for Crowdsourcing (SVGPCR) in digital
pathology. We compare SVGPCR with other crowdsourcing methods using a large multi-rater dataset
where pathologists, pathology residents, and medical students annotated tissue regions breast
cancer. Our study shows that SVGPCR is competitive with equivalent methods trained using goldstandard
pathologist generated labels, and that SVGPCR meets or exceeds the performance of other
crowdsourcing methods based on deep learning. We also show how SVGPCR can effectively learn
the class-conditional reliabilities of individual annotators and demonstrate that Gaussian-process
classifiers have comparable performance to similar deep learning methods. These results suggest
that SVGPCR can meaningfully engage non-experts in pathology labeling tasks, and that the classconditional
reliabilities estimated by SVGPCR may assist in matching annotators to tasks where they
perform well.