A tutorial on the segmentation of metallographic images: Taxonomy, new MetalDAM dataset, deep learning-based ensemble model, experimental analysis and challenges Luengo Martín, Julián Sevillano García, Iván Charte Luque, Francisco David Peláez Vegas, Adrián Mesejo Santiago, Pablo Herrera Triguero, Francisco Image segmentation Metallography Machine learning Deep learning Microscopy images Computer vision This publication is supported by ArcelorMittal, Luxembourg Global R&D, specifically the project granted by ArcelorMittal Global R&D Digital Portfolio in collaboration with the Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) , University of Granada. This publication is supported by the Andalusian Excel-lence, Spain project P18-FR-4961 and SOMM17/6110/UGR. D. Charte is supported by the Spanish Ministry of Universities, Spain under the FPU program (Ref. FPU17/04069) . Funding for open access charge: Universidad de Granada/CBUA. Image segmentation is an important issue in many industrial processes, with high potential to enhance the manufacturing process derived from raw material imaging. For example, metal phases contained in microstructures yield information on the physical properties of the steel. Existing prior literature has been devoted to develop specific computer vision techniques able to tackle a single problem involving a particular type of metallographic image. However, the field lacks a comprehensive tutorial on the different types of techniques, methodologies, their generalizations and the algorithms that can be applied in each scenario. This paper aims to fill this gap. First, the typologies of computer vision techniques to perform the segmentation of metallographic images are reviewed and categorized in a taxonomy. Second, the potential utilization of pixel similarity is discussed by introducing novel deep learning-based ensemble techniques that exploit this information. Third, a thorough comparison of the reviewed techniques is carried out in two openly available real-world datasets, one of them being a newly published dataset directly provided by ArcelorMittal, which opens up the discussion on the strengths and weaknesses of each technique and the appropriate application framework for each one. Finally, the open challenges in the topic are discussed, aiming to provide guidance in future research to cover the existing gaps. 2021-11-15T10:24:51Z 2021-11-15T10:24:51Z 2021-09-30 journal article Julián Luengo... [et al.]. A tutorial on the segmentation of metallographic images: Taxonomy, new MetalDAM dataset, deep learning-based ensemble model, experimental analysis and challenges, Information Fusion, Volume 78, 2022, Pages 232-253, ISSN 1566-2535, [https://doi.org/10.1016/j.inffus.2021.09.018] http://hdl.handle.net/10481/71515 10.1016/j.inffus.2021.09.018 eng http://creativecommons.org/licenses/by-nc-nd/3.0/es/ open access Atribución-NoComercial-SinDerivadas 3.0 España Elsevier