Image analysis – Applications – 3-d or stereo imaging analysis
Reexamination Certificate
1999-08-25
2003-01-14
Chang, Jon (Department: 2623)
Image analysis
Applications
3-d or stereo imaging analysis
C382S284000, C382S285000, C382S294000
Reexamination Certificate
active
06507665
ABSTRACT:
FIELD OF THE INVENTION
The invention relates generally to the field of panoramic image formation, and in particular to generating environment maps containing information extracted from pairs of stereo images.
BACKGROUND OF THE INVENTION
There is a growing number of imaging applications where the viewer has the perception of being able to move about a virtual environment. One method of developing such a virtual environment is to capture a plurality of images, which can be combined into a 360° panoramic view. Panoramic images can be considered as capturing an image as if the capture medium, e.g., film, were wrapped in a cylinder. For applications such as virtual reality, a portion of the image is transformed to appear as if the image were captured with a standard photographic system. However, since optical systems are not easily built which capture images on cylindrical photoreceptors, a variety of methods have been developed to provide the functionality. Panoramic stitching is a method that can be used to generate a 360° (or less) panoramic image from a series of overlapping images acquired from an ordinary camera. S. E. Chen describes the process of panoramic stitching in
Quick Time VR—An Image
-
based Approach to Virtual Environment Navigation, Proc. SIGGRAPH
'95, 1995, pp. 29-38.
Because conventional panoramic images do not have range associated with the objects in the scene, there are many potential applications of virtual-reality which are not accomplished easily. One such application is the ability to introduce objects synthetically into a panoramic image and interact with the image as one might like. For instance, if the objects are a distance d from the camera and a synthetic object is de be placed midway and have the zooming property of virtual images to operate in a manner appearing normal, the range must be known. Also, if synthetic objects are to interact with real objects in the image, the range information is critical. However, since the panoramic image capture systems do not acquire images from different vantage points, it is unreasonable to expect the system to estimate the range of the objects to the image capture point. Estimating range information can be accomplished in many ways, though a common and well-known method involves the use of stereo image pairs.
Range estimation by stereo correspondence is a topic that has been heavily researched. By capturing images from two different locations that are a known distance apart, orthographic range to any point in the scene can be estimated if that point can be found in both images. (Orthographic range is the orthogonal distance from a real world point to a plane parallel to an image plane passing through the rear nodal point of the capture device.) The difficulty in this method of range estimation is finding correspondence points. A variety of stereo correspondence algorithms exists; see for example. Y. Ohta and T. Kanade,
Stereo by Intra
-
and Inter
-
Scanline Search Using Dynamic Programming, IEEE Transactions on Pattern Analysis and Machine Intelligence
, Vol. PAMI-7, No. 2, March, 1985, pp. 139-154, and I. J. Cox, S. O. Hingorani, S. B. Rao, and B. M. Maggs,
A Maximum Likelihood Stereo Algorithm, Computer Vision and Image Understanding
, Vol. 63, No. 3, May 1996, pp. 542-567.
Conventionally, stereo images are captured using two optical systems having parallel optical axes, with the plane formed by the optical axis and vector between image centers being parallel to the ground. The reason for this arrangement is that stereo images are typically used to give the viewer a perception of three-dimensionality, and the human visual system has this arrangement. There have been systems producing stereo panoramic images, (see Huang and Hung,
Panoramic Stereo Imaging System with Automatic Disparity Warping and Seaming, Graphical Models and Image Processing
, Vol. 60, No. 3, May, 1998, pp. 196-208); however, these systems use a classical side-by-side stereo system as their intent is to utilize the stereo images for a human viewer, and not to estimate the depth of objects. One problem of the side-by-side approach is that panoramic images are best captured when the axis of rotation is at the rear-nodal point of the optical system. In a conventional side-by-side configuration this is geometrically impossible. As a result, at least one of the panoramic images is sub-optimal.
In the conventional camera arrangement for stereo imaging, the horizontal distance between the two cameras is commonly referred to as the baseline distance. With this arrangement the corresponding points in the two images can be found on the same horizontal line in the image. For a digital system this implies that a corresponding point exists in the same scan line for each image, though the position within the scan line of each image will differ depending upon the distance of the associated object from the cameras. According to the aforementioned Ser. No. 09/162,310, which is incorporated herein by reference, stereo imaging is achieved by capturing stereo image pairs that are displaced from each other along a vertical axis through the rear nodal point of a taking lens, rather than being displaced along a horizontal axis as in the conventional stereo image capture systems. This configuration retains the advantage of the conventional stereo capture configuration, in that points in one image of the stereo pair have their corresponding point in the other image of the pair along a line; however, the line is now a vertical line, which simplifies the formation of depth data for panoramic images.
Referring now to
FIG. 1
, the apparatus described in copending Ser. No. 09/162,310 for capturing a collection of stereo images to produce a 360° panoramic image with range data includes a pair of 360° stereoscopic cameras
110
and
110
′ that are mounted on a rotating camera support for rotation about the respective rear nodal points of the cameras. The rotating camera support includes a support rod
118
that is mounted for rotation in a base
119
. The operator rotates the camera assembly after each photograph in the series. The rotation angle is sufficiently small to permit adequate overlap between successive images for the subsequent “image stitching.” An angle of 30° is generally adequate, but the amount of rotation is dependent upon the field of view of the camera system. The cameras
110
and
110
′ are vertically displaced from each other along the rotating camera support rod
118
to provide a baseline distance d.
Each camera
110
,
110
′ has an optical axis, and alignment between the two cameras must assure that the two optical axes are as parallel as possible. The cameras are mounted such that they share the same rotational angle. In order to keep the cameras in vertical alignment a set of level indicators
128
are mounted on the support mechanism. The camera mounts
123
lock a camera to eliminate any side-to-side rotation creating misalignment between the two cameras. The base
127
is like a common camera tripod and the lengths of the three legs are adjustable, each with a locking mechanism
129
. By using the locking mechanism
129
and the level indicators
128
the operator can align the cameras to be displaced solely in the vertical direction. In order to activate the cameras, a remote shutter
125
is used which triggers both cameras simultaneously. The cameras are connected to the shutter control by a pair of wires
122
or by an RF control signal.
The cameras
110
,
110
′ are assumed to be identical and share in the same optical specifications, e.g., focal length and field of view. The baseline distance d of the two cameras
110
,
110
′ directly influences the resolution of depth estimates for each point in the 360° panoramic image. The amount of baseline is a function of the expected distance from the camera to the objects of interest. To permit this adjustment, the camera system has an adjustable vertical distance mechanism
124
, such as an adjustable rack and pinion on the rotating camera support rod
Cahill Nathan D.
Chen Shoupu
Chang Jon
Eastman Kodak Company
Woods David M.
LandOfFree
Method for creating environment map containing information... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for creating environment map containing information..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for creating environment map containing information... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3004911