Computer graphics processing and selective visual display system – Display driving control circuitry – Controlling the condition of display elements
Reexamination Certificate
2001-01-02
2003-10-14
Mancuso, Joseph (Department: 2697)
Computer graphics processing and selective visual display system
Display driving control circuitry
Controlling the condition of display elements
C345S649000
Reexamination Certificate
active
06633317
ABSTRACT:
BACKGROUND
1. Technical Field
The invention is related to a virtual walkthrough system and process, and more particularly to an image-based walkthrough system and process that employs pictures, panoramas, and/or concentric mosaics captured from real scenes to present a photo-realistic environment to a viewer locally and/or over a network environment.
2. Background Art
Rapid expansion of the Internet has enabled a number of interesting applications related to virtually wandering in a remote environment, e.g. online virtual tours, shopping, and games. Traditionally, a virtual environment is synthesized as a collection of 3D geometrical entities. These geometrical entities are rendered in real-time, often with the help of special purpose 3D rendering engines, to provide an interactive walkthrough experience. The Virtual Reality Modeling Language (VRML) [1] is presently a standard file format for the delivery of 3D models over the Internet. Subsequently, efforts have been made to effectively compress and progressively transmit the VRML files over the Internet [2, 3, 4, 5].
The 3D modeling and rendering approach has several main problems. First, it is very labor-intensive to construct a synthetic scene. Second, in order to achieve a real-time performance, the complexity and rendering quality are usually limited by the rendering engine. Third, the requirement of certain accelerating hardware limits the wide application of the approach.
Recently developed image-based modeling and rendering techniques [6, 7] have made it possible to simulate photo-realistic environments. The advantages of image-based rendering methods are that the cost of rendering a scene is independent of the scene complexity and truly compelling photo-realism can be achieved since the images can be directly taken from the real world. One of the most popular image-based rendering software is Apple Computer's QuickTime™ VR [7]. QuickTime™ VR has its roots in branching movies, e.g., the movie-map [8], the Digital Video Interactive (DVI) [9], and the “Virtual Museum” [10]. QuickTime™ VR uses cylindrical panoramic images to compose a virtual environment, therefore provides users an immersive experience. However it only allows panoramic views at separate positions.
It is noted that in the preceding paragraphs, as well as in the remainder of this specification, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. Multiple references will be identified by a pair of brackets containing more than one designator, for example, [2, 3, 4, 5]. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.
SUMMARY
The present invention is directed toward an image-based walkthrough system and process that employs pictures, panoramas, and/or concentric mosaics captured from real scenes to present a photo-realistic environment to a viewer. This is generally accomplished by dividing a walkthrough space that is made available to the viewer for exploring into a horizontally sectioned grid. The viewer is allowed to “walk” through the walkthrough space and view the surrounding scene from a horizontal plane in the space. In doing so, the viewer “enters” and “exits” the various cells of the grid. Each cell of the grid is assigned at least one source of image data from which an image of a part or all of the surrounding scene as viewed from that cell can be rendered. Specifically, each cell is associated with one or more pointers to sources of image data, each of which corresponds to one of the aforementioned pictures, panoramas or concentric mosaics.
In the case where the image data is a picture, its pointer will be associated with the location in the cell corresponding to the viewpoint from which the picture was captured. Similarly, a pointer to panoramic image data will be associated with the location in the cell corresponding to the center of the panorama. And when the image data represents a concentric mosaic, the pointer is associated with the region of the cell corresponding to the portion of the wandering circle of the concentric mosaic contained within the cell. It should be noted that unlike a panorama and picture, a concentric mosaic could be associated with more than one cell of the walkthrough space since its wandering circle could encompass more than one cell.
Whenever the viewer moves into one of the grid cells, the pointers associated with that cell, as well as the pointers associated with the adjacent cells (e.g., the eight neighboring cells assuming square or rectangular-shaped cells), are considered. Specifically, the distance between the current location of the viewer, and each picture viewpoint, panorama center, and nearest wandering circle point, in the considered cells is computed. If the viewer's current location is within the wandering circle of a concentric mosaic, then no action is taken to shift the viewer's position. However, if the viewer's current position is not within such a wandering circle, the viewer is slotted onto the closest of these aforementioned points.
In general, the foregoing image-based rendering technique for providing a continuous walkthrough experience to a viewer would require a large number of images, and so the transfer of a large amount of image data between the device employed to store the data and the processor used to render the images. If the image data is stored locally, such as on a hard drive, or on a CD or DVD, which is directly accessible by the processor, then the requirement to transfer large amounts of image data is of little concern. However, walkthrough systems are often implemented in a network environment (e.g., the Internet) where the image data is stored in or directly accessible by a network server, and the processor used to render the images is located in a network client. In such a network environment the large amount of image data that needs to be transferred between the server and client is a concern as bandwidths are typically limited.
In order to overcome the bandwidth limitations in network environments, the present invention is additionally directed toward a unique image data transfer scheme that involves streaming the image data so that the viewer can move around in the virtual environment while downloading. Similar to other network streaming techniques, this new streaming technology cuts down the waiting time for the viewer. Furthermore, the viewer can interactively move in the environment, making the waiting less perceptible. In general the new transfer scheme allows the client to selectively retrieve image segments associated with the viewer's current viewpoint and viewing direction, rather than transmitting the image data in the typical frame by frame manner. Thus, the server is used to store the huge amount of image data, while the client is designed to interact with the viewer and retrieve the necessary data from the server. This selective retrieval is achieved by implementing a new client-server communication protocol. Additionally, cache strategies are designed to ensure a smooth viewing experience for the viewer by capturing the correlation between subsequent views of a scene.
In essence, the new transmission scheme characterizes pictures and panoramas similar to a concentric mosaic in that each is represented by a sequence of image columns. As with the concentric mosaics each image column can be of any width, but typically will be the width of one pixel, making the image column a column of pixels. To facilitate the transfer of the image data, whether it be in the form of a picture, panorama or concentric mosaic, a specialized server-side file structure is employed. This structure includes a file header, or a separate tag file, which provides descriptive information about the associated image data. It i
Li Jiang
Shum Heung-Yeung
Arnold Adam
Lyon Richard T.
Lyon & Harr LLP
Mancuso Joseph
Microsoft Corporation
LandOfFree
Image-based walkthrough system and process employing spatial... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image-based walkthrough system and process employing spatial..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image-based walkthrough system and process employing spatial... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3151858