Television – Panoramic – With observer selected field of view
Reexamination Certificate
1998-05-27
2002-11-26
Diep, Nhon (Department: 2713)
Television
Panoramic
With observer selected field of view
C348S039000, C345S441000
Reexamination Certificate
active
06486908
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to building photographic panoramas of virtual reality and in particular to building spherical environmental maps utilizing some photographs.
In virtual reality systems, synthesis and navigation of virtual environments are usually accomplished using one of the following approaches: 1) three-dimensional modeling and rendering; 2) branching movies; and 3) image-based approach.
Although three-dimensional modeling and rendering is the traditional approach, creating the three-dimensional geometrical entities is a laborious manual process, and special hardware (e.g., a graphic accelerator) may be necessary for rendering, in real time, a highly complex scene.
The branching movies approach is used extensively in the video game industry. Multiple movie segments are connected together to depict spatial paths of selected branch points. This approach is limited in navigability and interaction, and also requires a large amount of storage space to store movies.
For rendering complex scenes, such as real-world scenery, the image-based approach is often considered to be the most reasonable and feasible approach. The image-based approach renders scenes within a fixed time as well as independently of rendering quality and scene complexity.
The image based approach uses an environment map, which is established from a collection of images and characterizes the appearance of a scene when viewed from a particular position. The environment map contains the pixel values used to display the scene. The image environment map is first wrapped onto an object surface having a certain geometry, such as a cube, a sphere, or a cylinder. Afterwards, by locating the viewing position at the geometrical center of the wrapped object, perspective-corrected views of scenes can be reconstructed from the image on the object surface during playback.
There are many types of environment maps for storing data including cubic maps, spherical maps, cylindrical maps, fish-eye or hemispherical maps, and planar maps. Some environments are capable of storing data of omni-directional scenes, while others cannot. Others have used two additional kinds of environment maps in experimental systems: 1) the textured cylinder-like prism, and 2) the texture sphere-like polyhedron (see e.g., W. K. Tsao, “Rendering scenes in the real world for virtual environments using scanned images”, Master Thesis of National Taiwan University, Advisor: Ming Ouhyoung, 1996). Rendering methods for different types of environment maps are described in U.S. Pat. Nos. 5,396,583; 5,446,833; 5,561,756; and 5,185,667, all of which are incorporated herein by reference.
As for building image-based panoramas, commercially available software (e.g., the QuickTime VR authoring tool suite, from Apple Computer Inc.) can be used to create a seamless panoramic cylindrical image from a set of overlapping pictures. An improved method of a stitching method is described in Hsieh et al., U.S. Ser. No. 08/933,758, filed Sep. 23, 1997, which is incorporated herein by reference. Stitching together photographic images taken from a fish-eye lens is also described in the developer manual of InfinitePictures™, Inc. (“SmoothMove Panorama Web Builder”, Developer Manual (Version 2.0) of InfinitePictures™, Inc., 1996).
Spherical environment maps are used to store data relating to a surrounding scene. There exists many types of spherical environment maps. Spherical mapping mechanisms are described in U.S. Pat. Nos. 5,359,363, 5,384,588, 5,313,306, and 5,185,667, all of which are incorporated herein by reference. The major advantage of spherical mapping systems is that the environment map is able to provide users with 360 degrees of both horizontal and vertical pannings of views within the mapped sphere. However, a common problem with spherical mapping systems relates to image acquisition. Generally, to acquire the image for a spherical environment map, users usually require special and relatively expensive cameras having fish-eye lenses. Two types of particular interest are the “spherical reflection” map and the “parametric spherical environment” map.
A “spherical reflection” map stores the image of an environment as an orthographic projection on a sphere representing a perfect reflection of the surrounding scene (Zimmermann U.S. Pat. No. 5,185,667). These images with circular shapes are all stored in respective square arrays of pixels. The major disadvantage of spherical reflection maps is that the orientations near the silhouette of the sphere (i.e., the ring-like shape when the sphere is orthographically projected on a sphere) are sampled very sparsely. This makes it more difficult to render the spherical reflection maps for omni-directional and interactive viewers.
SUMMARY OF THE INVENTION
The invention relates to building spherical panoramas or spherical environment maps for image-based virtual reality systems. The image-based spherical panoramas can be navigated in any desired view direction (360-degrees) by suitable three-dimensional image browsers or viewers.
In one aspect of the invention, a method of photographing pictures for building the spherical panoramas with a camera system includes determining a number of circles of latitude to build the spherical panorama on the basis of the focal length of the camera system, the height of photographic film used in the camera system, and an overlapping ratio between contiguous ones of the pictures, each circle of latitude including a series of contiguous pictures; and photographing the determined number of pictures.
The approach permits the use of a camera having a standard or wide-angled lenses rather than fish-eye lenses which are more expensive and less popular.
Embodiments of this aspect of the invention may include one or more of the following features.
Determining the number of circles of latitude includes calculating the following equation:
NumCirclesLandscape
=
[
180
°
(
100
-
k
)
%
*
[
2
tan
-
1
(
a
2
f
)
]
-
1
]
.
where:
a is the height of the photographic film;
f is the focal length of the camera system; and
k is the overlapping ratio between contiguous pictures.
Alternatively, the number of circles of latitude can be determined by calculating the following equation:
NumCirclesPortrait
=
[
180
°
-
2
tan
-
1
(
b
2
f
)
+
[
2
k
%
*
tan
-
1
(
a
2
f
)
]
(
100
-
k
)
%
*
[
2
tan
-
1
(
a
2
f
)
]
]
.
where:
a is the height of the photographic film;
b is the width of the photographic film;
f is the focal length of the camera system; and
k is the overlapping ratio between contiguous pictures.
In another aspect of the invention, a method of transforming photographic images of a panoramic scene into a spherical environment map includes warping each of the plurality of photographic images into a parametric spherical environment map; and image cutting each of the warped photographic images into a rectangular image based on an elevation angle of a predetermined sight direction.
This aspect of the invention provides a reliable and efficient approach for transforming photographic images into a spherical environment map. A spherical environment map can be linearly mapped using the spherical coordinate system, in which each pixel is specified in terms of the pixel's azimuthal angle around the horizon and its elevation angle below the north pole, denoted by (&thgr;, &phgr;). Based on angles (&thgr;
v
, &phgr;
v
) which define a picture's sight (or view) direction, a two-dimensional image of a picture can be transformed onto an image region of a spherical environment map using an image warping formula. This is regarded as one kind of coordinate transformation. Although each picture is taken based on the predefined sight direction, direction errors in taking pictures may exist. For computing the actual azimuth angles of camera sight direction, an additional coordinate transformation using the azimuthal coordinate system is performed. Using these computed azimuth angles, the picture images are sea
Chen Ju-Wei
Cheng Tse
Huang Shu-Cheng
Diep Nhon
Fish & Richardson P.C.
Industrial Technology Research Institute
LandOfFree
Image-based method and system for building spherical panoramas does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image-based method and system for building spherical panoramas, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image-based method and system for building spherical panoramas will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2964707