PTDC/EIA-EIA/122454/2010

Vision, Imaging and 3D graphics systems can be classified into two types according to the geometry of image formation: central and non-central. Central systems, also known as single viewpoint systems (SVP) are systems where all the projecting rays intersect at a single point, the effective viewpoint, which is the case, for example, of the common perspective cameras. In this case the effective viewpoint is the center of projection of the camera. One particular class of non-central imaging systems are omnidirectional vision systems. These systems have the advantage of providing images with wide fields of view which are advantageous for many applications ranging from surveillance, computer graphics, rendering, robotics and also in some medical applications. One of the common approaches to obtain omnidirectional images is to use catadioptric imaging systems (CIS). Catadioptric imaging systems combine mirrors and lenses. However there are CIS that are central, as shown in [Bak99]. These are obtained only with very specific configurations namely: -a perspective camera whose center of projection is placed on the focus of hyperbolic or elliptical mirror; -- an orthographic camera aligned with the axis of a parabolic mirror. These configurations require the careful placement of the camera and/or a special camera (orthographic). All the other configurations using mirrors lead to non-central CIS which implies that the rays do not intersect at a single point. As mentioned non-central imaging systems occur not only in CIS that but also in many other cases and applications. Cameras using fish eye lenses are also non-central as are cameras operating immersed in a fluid or interfacing other refractive environments. That is the case of medical applications such as endoscopy, colonoscopy or minimally invasive surgery. It is also the case in underwater robotics. 3D reconstruction from non-central images has been studied namely in [Mic04], [Gon04], [Kan06] and [Din09]. In all these cases multiple non-central images acquired with multiple mirrors (or with a moving mirror) used. 3D reconstruction has also been performed using a single image and using distortion to estimate depth. This project aims at studying calibration and 3D reconstruction in the following non-central cases: --Using 3D cameras (for example time-of-flight cameras) looking at a curved mirror or fish-eye lens to perform 3D reconstruction; --Having multiple perspective cameras looking at the same curved mirror to perform 3D reconstruction; In the case of 3D cameras we will also study the sampling of the plenoptic function performed by these imaging systems namely in what concerns the radiometric issues. To the best of the PI knowledge there are no results in the literature modeling the combination of 3D cameras with curved mirrors or modeling the use of 3D cameras in other non-central configurations namely when immersed in non-homogeneous environments. Also 3D reconstruction in these cases has not yet been modeled. The case of 3D reconstruction using multiple perspective cameras looking at a single curved mirror has not been fully addressed in the literature. In this case there are issues related to radiometric aspects namely with the resolution and focus that have not been studied. The PI has, in the last few years, addressed several problems related to central and non-central catadioptric systems ([Gon04], [Gon07a], [Gon09], [Gon07b] ) including modeling, calibration and 3D reconstruction. The use of a single mirror or lens has the significant advantage for applications of allowing more compact imaging systems. On the other hand 3D data is useful in many applications. In medical applications, namely in endoscopy/colonoscopy 3D images would significantly facilitate not only the navigation of the tool but also the recognition of the pathologies. Colonoscopes/Endoscopes are non-central systems. Furthermore they operate usually immersed in fluid which further accentuates the non-centrality. Surveillance applications also benefit from 3D data since recognition of human behavior can be significantly improved. The results of the project will allow better modeling of image formation in non-central imaging systems which will also benefit 3D Computer Graphics. The topics of this project correspond, partially, to the problems addressed by the Ph.D. students Pedro Miraldo and Luís Perdigoto on their ongoing dissertations.