Television – Panoramic
Reexamination Certificate
1998-01-15
2002-03-12
Britton, Howard (Department: 2613)
Television
Panoramic
C348S043000
Reexamination Certificate
active
06356297
ABSTRACT:
FIELD OF THE INVENTION
This invention relates generally to image processing systems and, more particularly, to image-processing systems that allow a user to create and view panoramic images from data representing multiple views of a scene of interest.
This invention also relates to image-processing systems that display video.
BACKGROUND OF THE INVENTION
Conventional three-dimensional graphics applications and associated hardware render scenes of interest composed of one or more three-dimensional objects. The three-dimensional objects are typically represented by geometric primitives, for example triangles. The three-dimensional graphics applications store graphics data, which represents the position and color of the geometric primitives in a model coordinate system. The graphics applications allow a user to manipulate the graphics data to render the scene, i.e. to display those object(s) of the scene that are visible in a viewing window based upon a viewpoint. The user can navigate through the scene of interest by changing the position and orientation of a viewpoint reference (e.g., by altering a camera viewpoint reference). Also, animations can be performed by navigating through a series of pre-selected positions and orientations of the viewpoint reference.
The rendering operation is a computationally intensive process and, thus, is typically performed by specialized graphics hardware. Such systems are powerful but costly due to the need for specialized hardware dedicated to this purpose.
Moreover, conventional three-dimensional graphics systems require that the user provide a three-dimensional model of the scene (i.e., define the graphics data that represent the position and color of the geometric primitives of the scene in the model coordinate system). Such a three-dimensional model may be created with software in conjunction with peripheral devices (e.g., pen tablets, scanners, cameras, etc.). For example, a three-dimensional graphics software package is sold under the name CATIA by Dassault Systems of France. The CATIA system allows a user to build a three-dimensional model of a scene of interest. However, modeling software like CATIA is expensive, and a significant investment in labor may be required to define the graphics data even for a relatively uncomplicated scene.
Because of the high costs associated with conventional three-dimensional graphics systems, alternative solutions have emerged that provide effective yet limited capabilities in creating and viewing a three-dimensional scene. The alternative solutions are typically suitable for use on standard personal computers without the need for dedicated graphics hardware. One such solution is the software developed and distributed by the International Business Machines Corporation under the name of PANORAMIX.
The PANORAMIX system uses a cylindrical or polyhedral environment map to represent a panoramic view of a scene of interest (or collection of images). Different perspective views are rendered by mapping the cylindrical environment map to a desired viewing window.
Another such panoramic viewer is a software system developed and sold by Apple Computer, Inc. of Cupertino, Calif. under the name of QUICKTIMEVR. The QUICKTIMEVR software is segmented into two packages. The first package, which is sold to content providers, is an authoring tool that allows content providers to develop a panoramic image from multiple views of a scene. The second package is a viewer, which is distributed to consumers and which allows the consumer to view the panoramic images created by the authoring tool. A more detailed description of the operation of the QUICKTIMEVR system may be found in Chen, “QuicktimeVR—An Image-based Approach to Virtual Environment Navigation”, ACM SIGGRAPH 1995, Los Angeles, Calif., pp. 29-38, and in U.S. Pat. No. 5,396,583, issued Mar. 7, 1995, entitled “Cylindrical to Planar Image Mapping Using Scanline Coherence”, by Chen et al., which is assigned to Apple Computer, Inc.
The QUICKTIMEVR system uses a cylindrical environment map to represent the panoramic view of a scene of interest (or a collection of images). Different perspective views are rendered by mapping the cylindrical environment map to a desired viewing window.
Other commercial panoramic viewing systems include REALSPACE VR, PHOTOBUBBLES, and JUTVISION. These commercial systems, however, do not enable the incorporation of streaming video within the panoramic view of the scene of interest.
The concept of overlaying streaming video onto a panoramic scene is described in commonly assigned, copending U.S. patent application Ser. No. 08/745,524, filed Nov. 12, 1996, entitled “Video Conferencing using Camera Environment Panoramas” by Chandra Narayanaswami. Therein, video data is overlaid onto a panoramic image as a way to reduce the bandwidth for video conferencing. In Narayanaswami, a panoramic environment map is considered static data that is downloaded once and then browsed interactively. Alternatively, video data is considered dynamic, or time-varying, data. Similarly, U.S. patent application Ser. No. 08/723,970, filed Sep. 27, 1996, entitled “Method and Apparatus for overlaying a Bit Map Image on an Environment Map”, by Lipscomb et al. discloses how to copy a 2-dimensional bit map of a first image onto a second image made from part of a panoramic environment map.
OBJECTS AND ADVANTAGES OF THE INVENTION
It is a first object and advantage of this invention to provide an image-processing system that allows the display of panoramic images and embedded video images.
It is another object and advantage of this invention to provide an image-processing system that allows synchronizing the display of panoramic images and embedded video images.
It is a further object and advantage of this invention to provide an image-processing system that displays a scene of interest by synchronously combining a panoramic image of the scene and a continuously-received stream of video images.
Further objects and advantages of this invention will become more apparent from a consideration of the drawings and ensuing description.
SUMMARY OF THE INVENTION
The foregoing and other problems are overcome and the objects of the invention are realized by methods and apparatus in accordance with embodiments of this invention, wherein streaming video data is embedded within a panoramic image for display.
The present invention teaches a method for displaying an image of a scene of interest from a stored environment map of the scene and from video data of the scene. The method includes the steps of: (A) receiving and storing the video data of the scene; (B) selecting an orientation of the scene; (C) retrieving video data according to the selected orientation of the scene; (D) rendering the environment map according to the selected orientation of the scene to generate a first image for the selected orientation; (E) synchronously combining the retrieved video data and the first image to form a composite image for the selected orientation of the scene of interest; and (F) displaying the composite image.
In one embodiment the method for displaying the image of the scene of interest further includes a step of continuously receiving video data packets that include image data and sound data. In this embodiment the sound data is continuously played while selected portions of the image data are displayed.
In another embodiment, the method for displaying the image of the scene of interest further includes a step of layering the video data and the first image generated by rendering the environment map. In the present invention layering techniques include, for example, drawing order layering, z-buffer-hiding layering, and chroma-key layering.
The present invention further teaches an image processing system for displaying an image of a scene of interest from a stored environment map of said scene and from video data of said scene. The image processing system including an element for receiving and storing the video data of the scene, an element for selecting an orientation of the scene, an element for retr
Cheng Keh-shin Fu
Kumar Keeranoor G.
Lipscomb James Sargent
Menon Jai Prakash
Willebeek-LeMair Marc Hubert
August Casey P.
Britton Howard
Diep Nhon T
Perman & Green LLP
LandOfFree
Method and apparatus for displaying panoramas with streaming... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for displaying panoramas with streaming..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for displaying panoramas with streaming... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2845037