Displaying immersive videos using tiled decompression

Details Displaying immersive videos using tiled decompression Displaying immersive videos using tiled decompression

2000-02-16

2003-02-04

Razavi, Micahel (Department: 2672)

Computer graphics processing and selective visual display system

Computer graphics processing

Attributes

C345S423000, C345S555000

Reexamination Certificate

active

06515673

ABSTRACT:

CROSS-REFERENCE TO COMPUTER PROGRAM LISTING APPENDIX
A computer program listing appendix, incorporated herein by reference, is submitted as part of this disclosure. The computer program listing appendix is stored under the file name: “APPENDIX.TXT” residing on one compact disk.
A portion of the disclosure of this patent document contains material, which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but other wise reserves all copyright rights whatsoever.
FIELD OF THE INVENTION
The present invention relates digital imaging. More specifically, the present invention relates to using texture mapping to create environmental projections for immersive video applications.
BACKGROUND OF THE INVENTION
Texture mapping is typically used to add realism to graphic images. Generally, texture mapping involves mapping a two dimensional image, typically referred to as the texture map, onto an object. The texture map contains color information for the object. The texture map is divided into a plurality of texture elements or texels. Texels typically provide color information for the object. The object is divided into a plurality of facets. Each facet is typically a polygon having one or more picture elements (“pixels”). The vertex of each facet is assigned a pair of texture coordinates which index the texture map to choose a texel (i.e., a color) from the texture map. The color of the facet is derived by interpolating between the colors and the vertices of the facet. Thus, the image of the texture map is reproduced onto the object.
At one time, the processing requirements of texture mapping limited texture mapping to professional graphic systems. However, as the processing power of microprocessors has increased, texture mapping software has become useable on consumer level computer systems. Furthermore, special graphics processing hardware capable of texture mapping has also become available for consumer level computer systems. Because texture mapping techniques have become feasible on consumer level computer systems, texture mapping techniques have been adapted for many different applications.
One use of texture mapping is environment mapping. Environment mapping uses computer graphics to display the surroundings or environment of a theoretical viewer. Ideally, a user of the environment mapping system can view the environment at any angle or elevation.
FIG. 1
illustrates the construct used in conventional environment mapping systems. A viewer
105
(represented by an angle with a curve across the angle) is centered at the origin of a three dimensional space having x, y, and z coordinates. The environment of viewer
105
(i.e., what the viewer can see) is ideally represented by a sphere
110
, which surrounds viewer
105
. Generally, for ease of calculation, sphere
110
is defined with a radius of 1 and is centered at the origin of the three dimensional space. More specifically, the environment of viewer
105
is projected onto the inner surface of sphere
110
. Viewer
105
has a view window
130
which defines the amount of sphere
110
viewer
105
can see at any given moment. View window
130
is typically displayed on a display unit for the user of the environment mapping system.
Conventional environment mapping systems include an environment capture system and an environment display system. The environment capture system creates an environment map which contains the necessary data to recreate the environment of viewer
105
. The environment display system uses the environment map to display view window
130
(
FIG. 1
) to the user of the environment mapping system. Typically, the environment capture system and the environment display system are located in different places and used at different times. Thus, the environment map must be transported to the environment display system typically using a computer network, or stored in on a computer readable medium, such as a CD-ROM or DVD.
Computer graphic systems are generally not designed to process and display spherical surfaces. Thus, as illustrated in
FIG. 2
, texture mapping is used to create a texture projection of the inner surface of sphere
110
onto polygonal surfaces of a regular solid (i.e., a platonic solid) having sides that are tangent to sphere
110
. Typically, as illustrated in
FIG. 2
, a texture projection in the shape of a cube
220
surrounds sphere
110
. Specifically, the environment image on the inner surface of sphere
110
serves as a texture map which is texture mapped onto the inner surfaces of cube
220
. A cube is typically used because most graphics systems are optimized to use rectangular displays and a cube provides six rectangular faces. Other regular solids (i.e., tetrahedrons, octahedrons, dodecahedrons, and icosahedrons) have non-rectangular faces. The faces of the cube can be concatenated together to form the environment map. During viewing, the portions of the environment map that correspond to view window
130
(FIG.
1
and
FIG. 2
) are displayed for viewer
105
. Because, the environment map is linear, texture coordinates can be interpolated across the face of each cube based on the vertex coordinates of the faces during display.
An extension to environment mapping is generating and displaying immersive videos. Immersive video involves creating multiple environment maps, ideally at a rate of 30 frames a second, and displaying appropriate sections of the multiple environment maps for viewer
105
, also ideally at a rate of 30 frames a second. Immersive videos are used to provide a dynamic environment rather than a single static environment as provided by a single environment map. Alternatively, immersive video techniques allow the location of viewer
105
to be moved. For example, an immersive video can be made to capture a flight in the Grand Canyon. The user of an immersive video display system would be able to take the flight and look out at the Grand Canyon at any angle.
Difficulties with immersive video are typically caused by the vast amount of data required to create a high resolution environment map and the large number of environment maps required for immersive video. Specifically, transmission and storage of the environment maps for high resolution flicker-free display may be beyond the processing capabilities of most computer systems.
Conventional data compression techniques have been used to compress the environment maps and reduce the amount of data transmitted or stored for immersive video. However, the additional processing time required to decompress a compressed environment map may impair the ability of the environment display system to process an adequate number of environment maps to provide a flicker-free display. Thus, there is a need for a compression and decompression method for immersive videos that minimizes the processing time required for decompressing the environment map.
The excessive data problem for immersive video is compounded by the inefficiencies of the conventional texture projections used to form environment maps. Specifically, although a cubic texture projection can provide realistic environment views, the cubic texture projection is not very efficient, i.e., the average amount of environment information per area is relatively low. The inefficiency of the cubic projection is caused by the lack of symmetry between the amount of spherical area on sphere
110
mapped onto cube
220
. For example, if each surface of cube
220
is subdivided into equal square areas as illustrated in
FIG. 3.
, the square areas do not map to equal areas of sphere
110
. For conciseness and clarity, only cube face
220
_
1
of cube
220
is discussed in detail because each cube face of cube
220
is typically processed in the same manner. Specifically, in
FIG. 3
, cube face
220
_
1
is divided into N
2
squares of equal area. More spherical area is mapped onto the squares near the center of a cube

Affiliated with

Also associated with

Rating

Displaying immersive videos using tiled decompression does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Displaying immersive videos using tiled decompression, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Displaying immersive videos using tiled decompression will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3144202