Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
1998-08-11
2001-01-16
Boudreau, Leo H. (Department: 2721)
Image analysis
Applications
3-d or stereo imaging analysis
C382S284000, C382S294000, C348S042000, C348S048000
Reexamination Certificate
active
06175648
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to processes for producing cartographic data in three dimensions from n two-dimensional images of a scene, provided by n sensors having different points of view of the scene.
It has for a very long time been known to define the position in space of structures which are present in a scene and can be seen in two images taken under different viewing angles, using stereoscopic techniques. This process has been generalized to the case of n images, n being an integer greater than 2, these n images constituting a stereoscopic system having a plurality of baselines.
Processes which include the following steps are, in particular, known:
the n sensors are calibrated (by using 3D knowledge of their relative position with respect to the scene which is observed and/or pattern recognition processes, so as to provide parameters of n models F
i
(x,y,z), each defining the relationship between a point in the scene, with coordinates x,y,z, and the coordinates (p,q)
i
of its projection into each of the n images, for i ranging from 1 to n;
the n images are set in correspondence, so as to locate the coordinates of the projection in the images of the same point in three-dimensional space;
3D reconstruction is performed, consisting in obtaining the coordinates x, y and z of the 3D point corresponding to each match between images, on the basis of knowledge of the models F
i
, and the matched image points.
A process of this type is described in the article by Sing Bing Kang et al. “A Multibaseline Stereo System with Active Illumination and Real-time Image Acquisition”, Proceedings IEEE Int. Conf. on Computer Vision, pages 88-93, June 1995. The process proposed in this article employs four cameras whose optical axes converge approximately at the same point. The image provided by one of the cameras is chosen as a reference. Given that the axes of the cameras are not parallel, the associated epipolar lines are not parallel to the image lines. In order to simplify recovery of the altitude from the stereoscopic images, that is to say 3D reconstruction, the images are subjected to rectification which converts each original pair of images into another pair such that the epipolar lines resulting therefrom are parallel, equal and coincident with image scanning lines. The correspondence method uses a variable &lgr;, defined as the distance from the optical centre along the viewing axis passing through the optical centre of the reference camera and the point in question, in order to calculate the search zone for potential homologues in the images to be matched with the reference image. Use of this variable &lgr; inevitably leads to a model with non-linear transition between the images, which makes the calculations more complicated. The strategy taught by the article, consisting in assigning equal significance to each pair, is a source of error whenever points are masked in one or more of the images.
A detailed study of algorithms for merging a plurality of representations in order to recover 3D cartographic data from a plurality of 2D images of a scene is given in the thesis at the Universit{acute over (e)} de Paris Sud, Centre d'Orsay, May 1988 “Construction et Fusion de Representations Visuelles 3D: Applications {grave over (a)} la Robotique Mobile” [Constructing and Merging 3D Visual Representations: Applications in Mobile Robotics] by N. Ayache.
SUMMARY OF THE INVENTION
An object of the present invention is to provide a process for producing cartographic data which is improved, in particular in that it makes it easier to establish multi-pair correspondence of n images (n being an integer greater than 2) and is flexible enough to adapt readily to a number of fields, such as:
detection of obstacles and autonomous guidance of a mobile robot in a fixed scene,
3D modelling of real sites,
mapping,
aerial reconnaissance, for obtaining a terrain profile,
the modelling of optimum trajectories during the preparation of an air mission.
To this end, the process uses in particular the observation that, irrespective of the nature of the sensors (pinhole, linear array of photodetector sites, scanning sensor) and their relative position when the images are being acquired (on condition that they are stereoscopic and that there is overlap between a plurality of images) it is always possible to find n−1 resemblance curves, each corresponding to a pair of images including a same reference image, as a function of the disparity, defined in an arbitrary one of the pairs. To do this, a change of reference frame of the resemblance curves is performed, which can always be done with an affine model for changing from the disparity of one pair to that of another pair.
It will in general be possible to liken the disparity to the curvilinear abscissa of one point relative to another along the epipolar line; it can be measured as a number of pixels in the frequent case of an image represented by elementary points which are each assigned at least one radiometric value (luminance and/or chrominance).
The invention provides, in particular, a process for producing cartographic data in three dimensions from n two-dimensional images of the scene, which are provided by n sensors with different points of view, n being an integer greater than 2, comprising the steps of:
(a) calibrating each sensor of order i in order to estimate the parameters of n models F
i
(x,y,z) defining the relationship between the coordinates x,y,z of a point in the scene and the coordinates (p,q)
i
of its projection into the image i among the n images;
(b) matching each of the n−1 pairs of images, all including the same reference image chosen from among the n images, by looking for the homologue of each pixel or zone in the reference image along the corresponding epipolar of the other image in the pair;
(c) in each of the n−1 pairs of two images, each comprising one reference image, and for each pixel or zone in the reference image, establishing a resemblance curve (curve of variation of a similarity index) as a function of the disparity along the epipolar of the other image;
(d) bringing all curves into a common reference frame using a model, for example an affine model, in order to match the largest possible number of images;
(e) summing the curves, while possibly removing each peak located at a singular disparity relative to those of all the other curves, and adopting the highest peak of the resultant curve; and
(f) computing the coordinates x,y,z from the disparity of the adopted peak and the parameters of the n models F
i
(x,y,z).
It is also possible to calculate a plurality of disparity images while taking different images as a reference. The merging may be carried out after having calculated the coordinates in three dimensions; it may also be carried out at the disparity image level, this being an advantageous solution when the sensors are calibrated in a projective space.
The results of multipair correspondence or matching may be merged using a majority vote, by assigning a higher weighting coefficient to the peaks corresponding to the points of view most distanced from the reference image.
The process which has just been described makes it possible, in most cases, to alleviate the difficulties encountered in order to be certain to find the homologue of a point of the image in one or more other images. The use of a number n greater than 2 makes it possible to avoid the consequences of occultation in one or more images. The geometrical coherence of the multiple matches reduces the errors and removes ambiguities. The presence of images having similar points of view makes it possible to process images which resemble one another and have little occultation between them; the use of images having distant points of view makes it possible to obtain accurate 3D information.
The abovementioned characteristics, as well as others, will become more clearly apparent on reading the following description of particular embodiments of the invention, which are given by way of non-limiting exam
Ayache Nicolas
Canu David
Sirat Jacques Ariel
Boudreau Leo H.
Larson & Taylor PLC
Matra Systems et Information
Sherali Ishrat
LandOfFree
Process for producing cartographic data by stereo vision does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Process for producing cartographic data by stereo vision, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Process for producing cartographic data by stereo vision will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2446557