Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
2000-07-25
2003-05-20
Vo, Cliff N. (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C345S473000
Reexamination Certificate
active
06567086
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates environment display systems. More specifically, the present invention relates to using multiple video streams to increase the resolution of an immersive video systems.
BACKGROUND OF THE INVENTION
As the processing power of microprocessors and the quality of graphics systems have increased, environment mapping systems have become feasible on consumer electronic systems. Environment mapping systems use computer graphics to display the surroundings or environment of a theoretical viewer. Ideally, a user of the environment mapping system can view the environment at any horizontal or vertical angle.
FIG. 1
illustrates the construct used in conventional environment mapping systems. A viewer
105
(represented by an angle with a curve across the angle) is centered at the origin of a three dimensional space having X, Y, and Z coordinates. The environment of viewer
105
(i.e., what the viewer can see) is ideally represented by a sphere
110
, which surrounds viewer
105
. Generally, for ease of calculation, sphere
110
is defined with a radius of
1
and is centered at the origin of the three dimensional space. More specifically, the environment of viewer
105
is captured and then re-projected onto the inner surface of sphere
110
. Viewer
105
has a view window
130
which defines the amount of sphere
110
viewer
105
can see at any given moment. View window
130
is typically displayed on a display unit for the user of the environment mapping system.
Conventional environment mapping systems include an environment capture system and an environment display system. The environment capture system creates an environment map which contains the necessary data to recreate the environment of viewer
105
. The environment display system displays portions of the environment in view window
130
based on the field of view of the user of the environment display system. An environment display system is described in detail by Hashimoto et al., in co-pending U.S. patent application Ser. No. 09/505.442, entitled “ENVIRONMENT DISPLAY USING TEXTURE PROJECTIONS WITH POLYGONAL CURVED SURFACES.” Typically, the environment capture system includes a camera system to capture the entire environment of viewer
105
.
Computer graphic systems are generally not designed to process and display spherical surfaces. Thus, as illustrated in
FIG. 2
, texture mapping techniques are used to create a texture projection of the inner surface of sphere
110
onto polygonal surfaces of a regular solid (i.e., a platonic solid) having sides that are tangent to sphere
110
. As illustrated in
FIG. 2
, a common texture projection is to use a cube
220
surrounding sphere
110
. Specifically, the environment image on the inner surface of sphere
110
serves as a texture map which is texture mapped onto the inner surfaces of cube
220
. A cube is typically used because most graphics systems are optimized to use rectangular displays and a cube provides six rectangular faces. Other regular solids (i.e., tetrahedrons, octahedrons, dodecahedrons, and icosahedrons) have non-rectangular faces. The faces of the cube can be concatenated together to form the environment map. During viewing, the portions of the environment map that correspond to view window
130
(FIG.
1
and
FIG. 2
) are displayed for viewer
105
. Because, the environment map is linear, texture coordinates can be interpolated across the face of each cube based on the vertex coordinates of the faces during display.
Other texture projections can also be used. For example, cylindrical mapping, as illustrated in
FIG. 3
, can be used if view window
130
is limited to a visible range around the equator. Specifically, in
FIG. 3
, a texture projection in the shape of a cylinder
320
surrounds sphere
110
. Portions of the environment image on the inner surface of sphere
110
serves as a texture map which is texture mapped onto the inner surfaces of cylinder
320
. Often, cylinder
320
is approximated using a plurality of rectangular sides to simplify the texture mapping.
FIG. 4
illustrates a texture projection ideally suited for environment mapping. Specifically,
FIG. 4
shows a texture projection comprising a plurality of polygonal curved surfaces, such as polygonal curved surfaces
410
,
420
, and
430
. The polygonal curved surfaces form a sphere
400
having the same radius as sphere
110
(FIG.
1
). The environment image on the inner surface of sphere
110
serves as a texture map for the polygonal curved surfaces. Creation and use of polygonal curved surfaces in environment projection is described in co-pending U.S. patent application Ser. No. 09/505.442, entitled “ENVIRONMENT DISPLAY USING TEXTURE PROJECTIONS WITH POLYGONAL CURVED SURFACES.”
An extension to environment mapping is generating and displaying immersive videos. Immersive video involves creating multiple environment maps, ideally at a rate of 30 frames a second, and displaying appropriate sections of the multiple environment maps for viewer
105
, also ideally at a rate of 30 frames a second. Immersive videos are used to provide a dynamic environment rather than a single static environment as provided by a single environment map. Alternatively, immersive video techniques allow the location of viewer
105
to be moved. For example, an immersive video can be made to capture a flight in the Grand Canyon. The user of an immersive video display system would be able to take the flight and look out at the Grand Canyon at any angle.
Difficulties with immersive video are typically caused by the vast amount of data required to create a high resolution environment map and the large number of environment maps required for immersive video. Thus, most environment mapping systems use very large environment maps, i.e. 1024×1024 or 2048×2048. Conventional high-quality immersive video systems would also require such high resolution environment maps to create high-quality immersive videos. However, conventional video equipment are designed and built to fixed standards regarding image resolution, compression, and other features, which may not provide enough bandwidth for high-quality immersive videos. For example, standard NTSC video streams provides an equivalent resolution of 640 by 480 pixels. While this resolution is adequate for conventional videos, it is inadequate for high-quality immersive videos, which must include the entire environment of a viewer not just the portions being viewed. Hence, there is a need for a method and system of displaying immersive videos while satisfying the constraints of conventional video equipment.
SUMMARY OF THE INVENTION
Accordingly, the present invention provides an immersive video system which utilizes multiple video streams to display high resolution immersive videos using conventional video equipment. In one embodiment of the present invention, an immersive video system for displaying a view window includes a video source, a video decoder coupled to the video source, and an immersive video decoder coupled to the video decoder. The video source is configured to produce a plurality of video streams. The video decoder is configured to decode an active video stream selected by the immersive video decoder from the plurality of video streams. The video streams contain environment data for the environment. Each video stream overlaps at least one other video stream so that the active video stream can be switched between the plurality of video streams without discontinuities. In some embodiments of the present invention, each video stream overlaps at least two other video streams. Generally the amount of overlap is approximately the size of the view window. Additionally, some embodiments of the present invention include view control interfaces to position the view window.
The video source in many embodiments of the present invention includes a video reader and a video storage medium. For example, a specific embodiment of the present invention uses DVD as the video storage medium. However, other embodiments may u
Bever Hoffman & Harms
Enroute, Inc.
Mao Edward S.
Vo Cliff N.
LandOfFree
Immersive video system using multiple video streams does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Immersive video system using multiple video streams, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Immersive video system using multiple video streams will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3081416