Multispectral High Dynamic Range Polarimetric Imaging applied to scene segmentation and object classification

Abstract

Different advanced techniques of digital imaging such as multispectral imaging, high dynamic range (HDR) imaging, polarimetric imaging or Near-Infra-Red imaging, have been developed and applied separately for years. Researchers are trying to merge some of these techniques together into a single integrated system. However this integration is rather challenging, specially if we are dealing with general purpose applications, such as capturing outdoor urban or natural scenes. This dissertation proposes capturing system designs, as well as algorithms and processing techniques for improving and simplifying the systems currently present in the state of the art of these different imaging techniques. This way, high dynamic range multispectral polarimetric images in the visible and near infrared can be captured and processed for many applications such as image segmentation, objects or materials classification, vegetation monitoring, food inspection, remote sensing, surveillance, etc. A new multispectral image capturing system is proposed, based on a novel generation of sensors which are still under development. Based on simulations, this work takes advantage of the spectral tunability of these sensors, and combines it with color filter arrays, to propose an imaging system with 36 spectral channels, achieving very good colorimetric and spectral performance for spectral reflectance estimation. Besides, a new algorithm for the automatic capture of HDR images is proposed, called Adaptive Exposure Estimation (AEE). It can be implemented in any digital imaging system, and it works online, as the capturing is ongoing. It is adaptive to scene content without the need of any prior knowledge about the scene being captured. The proposed method allows the user to tune the performance of the algorithm, keeping the balance between exposure time and signal-to-noise ratio, by just adjusting two free parameters. It can also capture the full dynamic range of the scene (or region of interest), or just a part of it. The proposed AEE algorithm is also adapted to multispectral polarimetric image capture. Based on a previous work which uses a Liquid Crystal Tunable Filter, a new full framework for capturing and processing 31-channels MultiSpectral HDR Polarimetric (MSHDRPol) images is proposed. New techniques for segmentation and classification of objects present in indoors scenes are proposed and tested. The results show that the algorithm outperforms other methods proposed in previous studies. As an additional contribution, the whole capturing workflow is adapted to an 8- channels filter-wheel-based imaging system covering the visible and NIR ranges up to 1000 nm. Therefore a system and a framework able to automatically capture MultiSpectral HDR Polarimetric Visible and Near Infra- Red (MSHDRPolVISNIR) images of outdoor scenes are proposed. A set of 8 outdoors scenes have been captured using the proposed system and methods and they will be made publicly available after the defense of this doctoral thesis.