Computer graphics processing and selective visual display system – Computer graphics processing – Graph generating
Reexamination Certificate
1999-12-13
2001-11-20
Nguyen, Phu K. (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Graph generating
Reexamination Certificate
active
06320584
ABSTRACT:
BACKGROUND OF THE INVENTION
A. Field of the Invention
This invention relates generally to digital image processing and computer graphics. More particularly, it is concerned with generating full-sphere panorama views using subhemispherical images and simulating free movement within a multidimensional environment, which can be either computer-generated or real.
B. Description of the Related Art
Computer-generated (CG) environments are typically created by representing objects with polygons and associated computer-generated or photographic surfaces or texture maps. Rendering, the construction of an image from a CG model, can be done from any point of view. See Foley [J. D. Foley, A. van Dam, S. K. Feiner, J. F. Hughes,
Computer Graphics: principles and practice
, 2nd ed., Addison-Wesley, 1987]. As such, it provides unrestricted simulated movement within the environment. However, the temporal resolution of unrestricted movement within a realistic CG environment that one can achieve on today's personal computers is severely limited by the computational requirements and by the labor of constructing realistic imagery.
U.S. Pat. No. 4,807,158 to Blanton, et. al discloses a method for reducing the computational requirements of rendering CG images, which could also be applied to natural images. First they build a database of images at selected positions, or “keypoints”, within the environment by rendering them in an off-line process. They store these panoramic images as conic projections. Then in real time, the application approximates the image at any position from that at the nearest keypoint. This approach works well when all objects are about the same distance from the viewer. This is a good assumption in their application, a flight simulator, but the loss of parallax would be a severe limitation in many environments. Objects at different distances move as a unit within the domain of a keypoint, and parallax is only evident when the keypoint changes.
U.S. Pat. No. 5,396,583 to Chen, el. al captures panoramas and project them onto cylindrical surfaces for storage. They are able to rapidly project images from a cylinder to a plane using “scanline coherence”. Unfortunately, like Blanton, their method does not support parallax.
McMillan, et. al. [L. McMillan and G. Bishop,
Plenoptic Modeling. An Image
-
Based Rendering System
, Siggraph '95 Proceedings, 1995], report a method that supports parallax, and apply it to natural images. They also produce a series of reference images off-line, which are captured with a video camera and re-projected to cylinders for storage. To support parallax, they calculate the image flow field between adjacent reference images. Now, when an image is approximated from a nearby reference image, different parts will move differently. Unfortunately, artifacts are quite apparent unless the image flow field is extremely accurate. Occluded regions cause additional artifacts.
The cylindrical surface (Chen and McMillan) is very inefficient for storing panoramic imagery near the vertical. Other panorama projections do not suffer from the limitations of the cylindrical projection. These include spherical, fisheye and cubic representations.
U.S. Pat. No. 5,185,667 to Zimmerman discloses a system for translating a selected portion of a hemispherical fisheye image into a planar projection for display on a CRT. The Zimmerman reference does not disclose how such hemispherical images can be seemed together by edge-to-edge abutment to form a 360 degree panoramic image. Additionally, such hemispherical fisheye images have been found to include several drawbacks that degrade the quality of the resulting translated image portion.
Accordingly, the need remains for providing improved panoramic imagry for simulating free movement within a multidimensional environment.
SUMMARY OF THE INVENTION
It is the object of this invention to simulate movement in a multidimensional space by approximating views at any viewpoint and orientation, with correct perspective and parallax. An additional object of this invention is to support stereography. It is a further object of this invention to provide an efficient method for storing panoramic imagery, especially for orientations near the vertical.
This invention captures panoramic views at many keypoints in the environment, preferably using fisheye photography. It stores these views as projections, from which one can produce views at any position and orientation. The fisheye photography can come from a variety of different lenses including those which have a maximum view angle that is less than, or greater than, 180 degrees. Most preferably, however, a fisheye lens having a view angle of less than 180 is used to avoid the drawbacks of 180 degree or greater fisheye lenses. For instance, several 150 degree images can be taken, overlapped, and seemed together using such tools as the Smoothmove™ seamer from Infinite Pictures, Inc. to form a full panoramic image.
An example of such seaming technique is disclosed herein. Digitized fisheye images are taken in several predefined directions, depending upon the configuration of the lens, to capture the entire image sphere. For instance, in a “bipyramidal” configuration consisting of five fisheye images, a fisheye lens having a view angle of at least approximately 127 degrees is used at 0°, 120° and 240° around the horizontal, straight up and straight down to define the image sphere.
These images are then mapped to a panoramic image. The subhemispherical imagry form overlapping views defining identical imagery in real space. The overlapping pixels defining these portions of real space come from more than one fisheye image. The color and brightness values of the pixel at a particular real space location are a weighted average of such values from overlapping pixels based upon the area of overlap. These overlapped images can then be converted to equirectangular format in an offline intermediate step before ultimately perspectively correcting the view in real time.
In the preferred embodiment, the projections are planar, and consist of polygons that are projections of areas in the environment that are approximately planar. The locations of these areas are stored, giving the playback system the three-dimensional information necessary to infer how the individual polygons move with the viewpoint, and thus simulate parallax.
Because it simulates parallax, the invention can produce stereographic images.
The preferred embodiment solves the occlusion problem in a novel way. Imagery that is occluded at a keypoint but visible at a nearby viewpoint is added to that keypoint, either by extending existing polygons or by creating new ones.
REFERENCES:
patent: 6031540 (2000-02-01), Golin et al.
Smith “Planar 2-Pass Texture Mapping and Warping” Computer Graphics, Jul. 1987, pp. 263-272.
Baker Thomas
Gilbert Scott
Golin Stuart J.
Galbi Elmer
iMove Inc.
Nguyen Phu K.
LandOfFree
Method and apparatus for simulating movement in... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for simulating movement in..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for simulating movement in... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2581604