Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
1998-07-23
2001-04-10
Nguyen, Phu K. (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
Reexamination Certificate
active
06215496
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to rendering images or sprites that include depth components in their representation and using the depth components in mapping the sprites to render images more efficiently and with greater realism.
BACKGROUND AND SUMMARY OF THE INVENTION
Image based rendering (IBR) techniques are an efficient way of generating novel views of real (e.g., photographic) and synthetic (i.e., computer-generated) objects. With traditional rendering techniques, the time required to render an image increases with the geometric complexity of the scene. The rendering time also increases as the requested shading computations (such as those requiring global illumination solutions) become more ambitious.
A simple example of image based rendering is the use of sprites. Once a complex scene has been rendered from a particular point of view, pieces of the image that correspond to spatially coherent objects will likely look similar from nearby points of view, although each piece may move at a different rate or direction. In this case, the pieces of the original 2D image (sprites) can be slightly altered by an affine or projective transformation to approximate the view from the new camera position. The sprite approximation's fidelity to the correct new view is dependent on the geometry being represented and, in particular, the amount of out-of-plane depth variation in the real scene piece being re-rendered. Another example of image based rendering is to add depth information to an image to produce a depth image. The optical flow that would be induced by a camera shift is then used to warp the scene into an approximation of the new view.
Each of these two methods has its limitations. Simple sprite warping cannot produce the parallax that is produced when parts of the scenes have sizable differences in distance from an approximating plane. Flowing a depth image pixel by pixel can provide proper parallax, but may result in gaps in the image either due to visibility changes when some previously occluded portion of the scene becomes unoccluded, or when a surface is magnified in the new view.
Some solutions have been proposed to the latter problem. One suggested solution is to perform a backward mapping from the output sample location to the input image. This is an expensive operation that requires some amount of searching in the input image. Another possible solution is to treat the input image as a mesh of micro-polygons, and to scan-convert these polygons in the output image. This too is an expensive operation, as it requires a polygon scan-convert setup for each input pixel, an operation that would be difficult without specialized rendering hardware. A third possibility is to simplify the depth component of the depth image, say into larger triangles, and to then use regular texture-mapping hardware or software. However, such a simplified description may fail to capture fine details in the depth map, and also complicates the representation required for the depth image.
This invention provides sprite rendering processes that overcome these limitations and a sprite image data structure that supports these processes. A conventional sprite data structure includes multiple pixels that each have image information (e.g., color and transparency) relating to the appearance of the sprite. The data structure of this invention adds to each pixel a depth component or value representing a displacement of the pixel from a selected plane. The depth components may be included as a separate channel in the conventional sprite data structure or as a separate image.
In one implementation of a sprite rendering process, the depth values of an input or source sprite are forward mapped (i.e., warped) to the depth values of an output or a destination sprite. For many applications of sprites with depth, the depth map will typically vary more slowly or smoothly than the color/intensity distribution. Moreover, motion due to depth parallax will typically be smaller than motion due to the plane perspective transform. These two effects combined make small errors in forward depth mapping much less visible than errors in forward color mapping.
Based upon the forward mapping of the depth values in the destination sprite, a backward mapping is performed on the image information from the source sprite to the destination sprite so that it can be rendered. This provides the destination sprite with parallax-based offsets that can create a very strong impression of three-dimensionality between and within the sprites.
The simplicity and regularity of the process make it a natural candidate for hardware implementation. Thus, the sprite-with-depth data structure could become a new rendering primitive in standard 3D graphics system. Other implementations include factoring the forward mapping into a pure parallax warp followed by a backward global perspective transform, adding a per-pixel parallax correction to the backward mapping process, and forward warping a displacement (flow) map. All of these implementations can be used to generate more realistic renderings of a sprite when it is viewed from different angles.
Additional objects and advantages of the present invention will be apparent from the detailed description of the preferred embodiment thereof, which proceeds with reference to the accompanying drawings.
REFERENCES:
patent: 5977980 (1999-11-01), Aleksicy
Cohen Michael F.
Szeliski Richard S.
Ipsolon LLP
Microsoft Corporation
Nguyen Phu K.
LandOfFree
Sprites with depth does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Sprites with depth, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Sprites with depth will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2489397