Computer graphics processing and selective visual display system – Image superposition by optical means – Operator body-mounted heads-up display
Reexamination Certificate
2000-12-22
2004-08-10
Liang, Regina (Department: 2674)
Computer graphics processing and selective visual display system
Image superposition by optical means
Operator body-mounted heads-up display
C345S007000
Reexamination Certificate
active
06774869
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to computer-based teleconferencing, and more particularly to computer-based teleconferencing in a networked virtual reality environment.
2. Description of Related Art
Networked virtual environments allow users at remote locations to use a telecommunication link to coordinate work and social interaction. Teleconferencing systems and virtual environments that use 3D computer graphics displays and digital video recording systems allow remote users to interact with each other, to view virtual work objects such as text, engineering models, medical models, play environments and other forms of digital data, and to view each other's physical environment.
A number of teleconferencing technologies support collaborative virtual environments which allow interaction between individuals in local and remote sites. For example, video-teleconferencing systems use simple video screens and wide screen displays to allow interaction between individuals in local and remote sites. However, wide screen displays are disadvantageous because virtual 3D objects presented on the screen are not blended into the environment of the room of the users. In such an environment, local users cannot have a virtual object between them. This problem applies to representation of remote users as well. The location of the remote participants cannot be anywhere in the room or the space around the user, but is restricted to the screen.
Networked immersive virtual environments also present various disadvantages. Networked immersive virtual reality systems are sometimes used to allow remote users to connect via a telecommunication link and interact with each other and virtual objects. In many such systems the users must wear a virtual reality display where the user's eyes and a large part of the face are occluded. Because these systems only display 3D virtual environments, the user cannot see both the physical world of the site in which they are located and the virtual world which is displayed. Furthermore, people in the same room cannot see each others' full face and eyes, so local interaction is diminished. Because the face is occluded, such systems cannot capture and record a full stereoscopic view of remote users' faces.
Another teleconferencing system is termed CAVES. CAVES systems use multiple screens arranged in a room configuration to display virtual information. Such systems have several disadvantages. In CAVES systems, there is only one correct viewpoint, all other local users have a distorted perspective on the virtual scene. Scenes in the CAVES are only projected on a wall. So two local users can view a scene on the wall, but an object cannot be presented in the space between users. These systems also use multiple rear screen projectors, and therefore are very bulky and expensive. Additionally, CAVES systems may also utilize stereoscopic screen displays. Stereoscopic screen display systems do not present 3D stereoscopic views that interpose 3D objects between local users of the system. These systems sometimes use 3D glasses to present a 3D view, but only one viewpoint is shared among many users often with perspective distortions.
Consequently, there is a need for an augmented reality display that mitigates the above mentioned disadvantages and has the capability to display virtual objects and environments, superimpose virtual objects on the “real world” scenes, provide “face-to-face” recording and display, be used in various ambient lighting environments, and correct for optical distortion, while minimizing computational power and time.
SUMMARY OF THE INVENTION
In accordance with the teachings of the present invention, a teleportal system is provided. A principal feature of the teleportal system is that single or multiple users at a local site and a remote site use a telecommunication link to engage in face-to-face interaction with other users in a 3D augmented reality environment. Each user utilizes a system that includes a display such as a projective augmented-reality display and sensors such as a stereo facial expression video capture system. The video capture system allows the participants to view a 3D, stereoscopic, video-based image of the face of all remote participants and hear their voices, view unobstructed the local participants, and view a room that blends physical with virtual objects with which users can interact and manipulate.
In one preferred embodiment of the system, multiple local and remote users can interact in a room-sized space draped in a fine grained retro-reflective fabric. An optical tracker preferably having markers attached to each user's body and digital video cameras at the site records the location of each user at a site. A computer uses the information about each user's location to calculate the user's body location in space and create a correct perspective on the location of the 3D virtual objects in the room.
The projective augmented-reality display projects stereo images towards a screen which is covered by a fine grain retro-reflective fabric. The projective augmented-reality display uses an optics system that preferably includes two miniature source displays, and projection-optics, such as a double Gauss form lens combined with a beam splifter, to project an image via light towards the surface covered with the retro-reflective fabric. The retro-reflective fabric retro-reflects the projected light brightly and directly back to the eyes of the user. Because of the properties of the retro-reflective screen and the optics system, each eye receives the image from only one of the source displays. The user perceives a 3D stereoscopic image apparently floating in space. The projective augmented-reality display and video capture system does not occlude vision of the physical environment in which the user is located. The system of the present invention allows users to see both virtual and physical objects, so that the objects appear to occupy the same space. Depending on the embodiment of the system, the system can completely immerse the user in a virtual environment, or the virtual environment can be restricted to a specific region in space, such as a projective window or table top. Furthermore, the restricted regions can be made part of an immersive wrap-around display.
Further objects, features and advantages of the invention will become apparent from a consideration of the following description and the appended claims when taken in connection with the accompanying drawings.
REFERENCES:
patent: 5130794 (1992-07-01), Ritchey
patent: 5418584 (1995-05-01), Larson
patent: 5495576 (1996-02-01), Ritchey
patent: 5510832 (1996-04-01), Garcia
patent: 5572229 (1996-11-01), Fisher
patent: 5581271 (1996-12-01), Kraemer
patent: 5606458 (1997-02-01), Fergason
patent: 5621572 (1997-04-01), Fergason
patent: 5642221 (1997-06-01), Fischer et al.
patent: 5644324 (1997-07-01), Maguire, Jr.
patent: 5673059 (1997-09-01), Zavracky et al.
patent: 5684935 (1997-11-01), Demesa, III et al.
patent: 5708449 (1998-01-01), Heacock et al.
patent: 5708529 (1998-01-01), Togino et al.
patent: 5712732 (1998-01-01), Street
patent: 5739955 (1998-04-01), Marshall
patent: 5742263 (1998-04-01), Wang et al.
patent: 5751259 (1998-05-01), Iwamoto
patent: 5777794 (1998-07-01), Nakaoka
patent: 5777795 (1998-07-01), Colucci
patent: 5790311 (1998-08-01), Togino
patent: 5790312 (1998-08-01), Togino
patent: 5808589 (1998-09-01), Fergason
patent: 5812100 (1998-09-01), Kuba
patent: 5822127 (1998-10-01), Chen et al.
patent: 5853240 (1998-12-01), Tanaka et al.
patent: 5883606 (1999-03-01), Smoot
patent: 5886735 (1999-03-01), Bullister
patent: 5886823 (1999-03-01), Sugano
patent: 6064749 (2000-05-01), Hirota et al.
patent: 6124825 (2000-09-01), Eschenbach
patent: 6147805 (2000-11-01), Fergason
patent: 6278479 (2001-08-01), Wilson et al.
patent: 6317127 (2001-11-01), Daily et al.
patent: 6408257 (2002-06-01), Harrington et al.
patent: 2002/0075201 (2002-06-01), Sauer et al.
Billinghurst, M.; Weghorst, S.;
Biocca Frank
Rolland Jannick P.
Board of Trustees operating Michigan State University
Liang Regina
Nguyen Jennifer T.
LandOfFree
Teleportal face-to-face system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Teleportal face-to-face system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Teleportal face-to-face system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3276619