Image analysis – Applications – Motion or velocity measuring
Reexamination Certificate
2001-04-18
2002-08-06
Johns, Andrew W. (Department: 2621)
Image analysis
Applications
Motion or velocity measuring
C382S284000
Reexamination Certificate
active
06430304
ABSTRACT:
This application contains related subject matter to that of U.S. patent application Ser. No. 09/384,396, filed simultaneously herewith (Attorney Docket Number SAR 13165), and incorporated herein by reference in its entirety.
The invention relates to an image processing method and apparatus and, more particularly, the invention relates to a method and apparatus for enhancing the quality of an image.
BACKGROUND OF THE DISCLOSURE
For entertainment and other applications, it is useful to obtain high-resolution stereo imagery of a scene so that viewers can visualize the scene in three dimensions. To obtain such high-resolution imagery, the common practice of the prior art is to use two or more high-resolution devices or cameras, displaced from each other. The first high-resolution camera captures an image or image sequence, that can be merged with other high-resolution images taken from a viewpoint different than the first high-resolution camera, creating a stereo image of the scene.
However, creating stereo imagery with multiple high-resolution cameras can be difficult and very expensive. The number of high-resolution cameras used to record a scene can contribute significantly to the cost of producing the stereo image scene. Additionally, high-resolution cameras are large and unwieldy. As such, the high-resolution cameras are not easy to move about when filming a scene. Consequently, some viewpoints may not be able to be accommodated because of the size of the high-resolution cameras, thus limiting the viewpoints available for creating the stereo image.
Similarly, in other applications given a collection of captured digital imagery, the need is to generate enhanced imagery for monocular or binocular viewing Examples of such application are resolution enhancement of video and other digital imagery, quality enhancement in terms of enhanced focus, depth of field, color and brightness/contrast enhancement, and creation of synthetic imagery from novel viewpoints based on captured digital imagery and videos.
All the above applications involve combining multiple co-temporal digital sensors (camera for example) and/or temporally separated sensors for the purpose of creation of synthetic digital imagery. The various applications can be broadly divided along the following lines (but are not limited to these):
1. Creation of an enhanced digital image by processing one or more frames of imagery from cameras and or other sensors which have captured the imagery at the same time instant. The synthesized frame represents the view of an enhanced synthetic camera located at the position of one of the real sensors.
2. Creation of enhanced digital imagery by processing frames that have been captured over time and space (multiple cameras/sensors capturing video imagery over time). The synthesized frames represent enhanced synthetic cameras located at the position of one or more of the real sensors.
3. Creation of enhanced digital imagery by processing frames that have been captured over time and space (multiple cameras/sensors capturing video imagery over time). The synthesized frames represent enhanced synthetic cameras that are located at positions other than those of the real sensors.
Therefore, a need exists in the art for a method and apparatus for creating a synthetic high-resolution image and/enhancing images using only one high-resolution camera.
SUMMARY OF THE INVENTION
The disadvantages associated with the prior art are overcome by the present invention for a method and apparatus for accurately computing image flow information as captured by imagery of a scene. The invention computes the image flow information of each point in an image by computing the image flow within windows that are offset with respect to the point for which the image flow is being computed. Additionally, image flow computations are performed over multiple frames of imagery to ensure accuracy of the image flow computation and to facilitate correction of occluded imagery.
In one illustrative embodiment of the invention, the image flow computation is constrained to compute parallax information. The imagery and parallax (or flow) information can be used to enhance various image processing techniques such as image resolution enhancement, enhancement of focus, depth of field, color, and brightness. The parallax (or flow) information can also be used to generate a synthetic high-resolution image that can be used in combination with the original image to form a stereo image. Specifically, the apparatus comprises an imaging device for producing images (e.g., video frame sequences) and a scene sensing device for producing information regarding the imaged scene. An image processor uses the information from the scene sensing device to process the images produced by the imaging device. This processing produces parallax information regarding the imaged scene. The imagery from the imaging device and the parallax information can be used to enhance any one of the above-mentioned image processing applications.
The invention includes a method that is embodied in a software routine, or a combination of software and hardware. The inventive method comprises the steps of supplying image data having a first resolution and supplying image information regarding the scene represented by the image data. The image data and information are processed by, for example, warping the first image data to form a synthetic image having a synthetic view, where the viewpoint of the synthetic image is different from the viewpoint represented in the image data. The synthetic image and the original image can be used to compute parallax information regarding the scene. By using multiple frames from the original imagery and the synthetic view imagery, the inventive process improves the accuracy of the parallax computation.
Alternate embodiments of the invention include but are not limited to, utilizing multiple sensors in addition to the scene sensing device to provide greater amounts of scene data for use in enhancing the synthetic image, using a displacement device in conjunction with the second imaging device to create a viewpoint for the warped image that is at the location of the displacement device, and using a range finding device as the second imaging device to provide image depth information.
REFERENCES:
patent: 5257209 (1993-10-01), Markandey
patent: 5265172 (1993-11-01), Markandey et al.
patent: 5627905 (1997-05-01), Sebok et al.
patent: 5680487 (1997-10-01), Markandey
patent: 5768404 (1998-06-01), Morimura et al.
patent: 5930378 (1999-07-01), Kubota et al.
patent: 5991428 (1999-11-01), Taniguchi
patent: 6081606 (2000-06-01), Hansen et al.
patent: 6298144 (2000-10-01), Pucker, II et al.
patent: 6192156 (2001-02-01), Moorby
K. Kim, M. Siegel, J. Son, “Synthesis of a High-Resolution 3D-Stereoscopic Image Pair From a High-Resolution Monoscopic Image and a Low-Resolution Depth Map”, Proceedings of the 1998 SPIE/IS&T Conference, vol. 3295A, Jan., 1998, pp. 76-86.
Bergen James Russell
Hanna Keith James
Kumar Rakesh
Lubin Jeffrey
Sawhney Harpreet Singh
Burke William J.
Johns Andrew W.
Sarnoff Corporation
LandOfFree
Method and apparatus for processing images to compute image... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for processing images to compute image..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for processing images to compute image... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2966433