Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
2001-01-02
2004-07-06
Vo, Cliff N. (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
Reexamination Certificate
active
06760026
ABSTRACT:
BACKGROUND
1. Technical Field:
The invention is related to the rendering of a virtual reality environment for display to a viewer, and more particularly to a system and process for rendering such an environment having an image-based background and geometry-based foreground objects.
1Background Art:
Virtual reality is an attractive technology, which provides users an environment to navigate and play. This technology is often used as a platform for games, virtual tours, and virtual real estate, as well as for many other applications. Traditional virtual reality techniques are usually based on geometrical approaches where the generated scenes may consist of millions or billions of polygons. However, the geometrical modeling and rendering approach has several drawbacks. First, it is very labor-intensive to construct a synthetic scene. Second, in order to achieve a real-time performance, the complexity and rendering quality are usually limited by the rendering engine. And third, the requirement of certain accelerating hardware limits the wide application of the method.
Recently developed image-based modeling and rendering techniques [
1
,
4
] have made is possible to simulate photo-realistic environments. The advantages of image-based rendering methods are that the cost of rendering a scene is independent of the scene complexity and truly compelling photo-realism can be achieved since the images can be directly taken from the real world. One of the most popular image-based rendering software programs is Apple Computer, Incorporated's QuickTime VR [
1
]. QuickTime VR uses cylindrical panoramic images to compose a virtual environment, therefore providing viewers with an immersive experience that can be viewed from a fixed viewpoint. However, an ideal virtual reality environment should provide a virtual 3D space about which a viewer can “move”, and some virtual objects to interact with in the space. QuickTime VR makes an attempt at providing virtual objects by presenting pop-up objects to the viewer. Essentially, pop-up objects are 3D graphic objects presented to the viewer against the photo-realistic panoramic images representing the virtual environment. Unfortunately, because the panoramas only allow a viewer to view the scene from a fixed location, the viewer has no chance to experience 3D motion through the virtual environment.
Other approaches of image-based rendering [
2
,
3
,
5
,
6
] that incorporate 3D graphic objects into scenes have also been studied. However, in these methods, the image-based scenes have to be reconstructed and represented by geometrical models again. Efforts are made on recovering reflection properties of the captured objects and lighting conditions of the scene. These methods inevitably involve much more interactions and manipulation since structure from motion is always a hard problem in vision technology. Briefly, in the rendering aspect, the problem in pure geometry-based virtual reality method is that the 3D rendering engine has to take into account all the geometrical objects in the scene even if a great part of them just acts as a background. Even worse, when a viewer only wants to interact with a small number of the rendered foreground objects, the selection engine has to process all the geometrical objects in the scene anyway—therefore greatly increasing the response time.
Thus, with current image-based virtual reality methods, either the viewer's motions is restricted or the scenes have to be reconstructed via geometrical methods.
It is noted that in the preceding paragraphs, as well as in the remainder of this specification, the description refers to various individual publications identified by an alphanumeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [
1
]” or simply “[
1
]”. Multiple references will be identified by a pair of brackets containing more than one designator, for example, [
2
,
3
,
5
,
6
]. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.
SUMMARY
To present invention is directed toward a novel rendering system and process which allows viewers to freely move about and interact with 3D graphic objects in a virtual environment having an image-based background, without the need to reconstruct the background using geometric models. Essentially, this is accomplished by first rendering an image-based background, and next rendering geometry-based foreground objects. The foreground objects are then integrated into the background environment.
The image-based background scene can be captured and rendered via any appropriate image-based method. However, it is preferred that the system and process described in a co-pending U.S. patent application entitled “Scene Capturing and View Rendering Based On A Longitudinally Aligned Camera Array” having Ser. No. 09/606,857 and a filing date of Jun. 28, 2000, be employed. Essentially, this system and process entails using a collection of images captured by a longitudinally aligned camera array, which rotates along a circle. The captured images are indexed and compressed for efficient rendering of the environment, without the need for explicit geometry.
The aforementioned geometry-based foreground objects can be rendered using any appropriate geometry-based 3D rendering engine. The selected 3D rendering engine is used to integrate 3D graphic object into the background scene. The 3D graphic objects are preferably integrated into the image-based background scene via overlapping approach that involves superimposing the objects onto the background scene. Viewers can move about in a virtual space defined by the image-based background rendering program and interact with the 3D graphic objects. In the case of the preferred rendering of the background environment, the viewer can move about and interact with 3D graphic objects in a 3D cylindrical space. It is also noted that the 3D graphic objects are introduced only if they are needed to interact with the viewers. This keeps the viewer's virtual interaction space uncluttered with unnecessary objects and so significantly reduces the 3D graphic processing time.
The foregoing system and process for integrating geometry-based 3D graphic objects into an image-based background to produce a virtual reality environment has many advantages over the existing systems. For example, unlike traditional geometry-based virtual reality systems that must take into account all the geometrical objects in the scene, the 3D graphic rendering engine employed in the present invention need only deal with a small number of foreground 3D graphic objects. As indicted previously, this significantly reduces the 3D graphic processing time. In addition, there is no need for any geometrical reconstruction in rendering the background scene. This allows for the use of complex image-based background scenes. In fact, since the background scenes can be captured in any real scene, it is possible to tour or play games in very complicated scenes such as forests that are impractical to model via geometric methods. The image-based background scenes can also be made accessible through an Internet website so that others can download them and use them as backgrounds in their games or virtual tours. Furthermore, unlike existing geometry-based virtual reality systems, a viewer using the present system and process can move about in the virtual interaction space and observe the lateral and longitudinal parallaxes and lighting changes typical in a real scene. In addition, occlusions among the 3D graphic objects, as well as occlusions between 3D graphic objects and the background, can be readily portrayed.
In addition to the just described benefits, other advantages of the present invention will become apparent from the detailed description which follows hereinafter when taken in conjunction with the drawing figures which accompany it.
REFERENCES:
S. E. Chen, QuickTime VR—An Image
Li Jiang
Shum Heung-Yeung
Lyon Richard T.
Lyon & Harr LLP
Microsoft Corporation
Vo Cliff N.
LandOfFree
Image-based virtual reality player with integrated 3D... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image-based virtual reality player with integrated 3D..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image-based virtual reality player with integrated 3D... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3214937