Computer graphics processing and selective visual display system – Plural display systems – Tiling or modular adjacent displays
Reexamination Certificate
1999-02-09
2001-04-03
Hjerpe, Richard (Department: 2674)
Computer graphics processing and selective visual display system
Plural display systems
Tiling or modular adjacent displays
C345S156000, C359S009000, C359S010000, C359S630000
Reexamination Certificate
active
06211848
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to holography, and in particular to dynamic holographic systems offering interactive capability.
BACKGROUND OF THE INVENTION
A wide variety of virtual reality (VR) application areas such as telesurgery, training, computer modeling and entertairnment employ computational haptics (i.e., computer-mediated manipulation and perception through tactile and kinesthetic senses) and high-quality computer graphics to study, interact with or modify data. Most present-day applications dissociate the visual and manual workspaces; that is, although the user can perceive an image of, for example, his or her hand in a virtual workspace, the user's actual hand is elsewhere. The result is a sensory disjunction: what the user sees is spatially separated from what the user feels. There have, however, been several efforts to conjoin eye and hand in interactive applications.
For example, the “Virtual Lathe” (Deering, “High Resolution Virtual Reality,”
Proc. SIGGRAPH '
92,
Computer Graphics,
26:195-202 (1992)) utilized a head-tracked stereo display showing a virtual stock, spinning about its long axis, which a person could interactively lathe using a 3D mouse in the shape of a rod. The demonstration used liquid-crystal display (LCD) shutter goggles for stereo viewing, and had no provision for force feedback.
Another interesting example is described in Yokokohji et al., “Vision-based Visual/Haptic Registration for WYSIWYF Display,”
International Conference on Intelligent Robots and Systems
(1996) pp. 1386-1393. The visual display behaves like a moveable “magic window,” interposed between the viewer's eyes and hand, and through which the hand can be seen interacting with a virtual, tangible scene. The work employs a six degree-of-freedom haptic manipulator and monographic visual rendering to combine three pieces of information in this final coincident display: a video image of the operator's hand/arm, the computationally rendered scene, and an accompanying force model. The visual display is a color LCD panel with a charge-coupled device (CCD) camera attached to its backplane. This display/camera unit can be moved with respect to the physical scene, with vision-based pose estimation employed to determine its new orientation. The visual display shows a computationally rendered view of the synthetic scene generated from the newly determined viewpoint, and composited with a live chroma-keyed image of the operator's hand/arm moving behind the display and interacting with the haptic device. This display cannot currently reproduce correct occlusion relationships between the hand/arm and virtual objects, however, and provides only monocular cues to scene depth (i.e., neither stereoscopic viewing nor head-tracked motion parallax is available).
Other systems employing a coincident workspace utilize a half-silvered mirror to combine an image displayed by a conventional monitor with that of the haptic workspace. One such project, the “Virtual Workbench,” is described in Wiegand, “The Virtual Workbench & the Electronics Training Task,” MIT internal communication (1994) (available at http://mimsy.mit.edu/). This system, used to study human sensorimotor capabilities and to develop training applications, employs a PHANTOM haptic interface and the half-silvered mirror technique for coincident stereoscopic display. It, too, does not represent correct occlusion relationships between the hand and simulated objects. Moreover, the workspace that can actually be shared by the visual display and the hand is depth-limited in stereoscopic systems; inherent in these displays is an accommodation-convergence mismatch—that is, a functional disengagement of several systems of the eye that normally function in cooperation. If scene depth is not designed well for the display's particular viewing geometry, eye strain, headaches and unfuseable stereo images can result. Of course, the very purpose of combining the manual and visual workspace is to visually monitor the hand (or hand-held tool) and its interaction with the object or material. Consequently, the ability to keep both the displayed object and the hand in visual focus is essential, and careful design must be employed to render it so.
Holographic displays eliminate this particular design problem by permitting a viewer to freely converge and accommodate to any point in the display volume. Indeed, throughout the history of holography, there has been considerable interest in building real-time, interactive holographic displays. The problem has been recognized as a difficult one and it is only very recently that quasi real-time holographic displays have made their appearance; see, e.g., Kollin et al., “Real-Time Display of 3D Computed Holograms by Scanning the Image of an Acousto-Optic Modulator,” in
SPIE Proc. Vol. #
1136,
Holographic Optics II. Principles and Applications
(1989), paper #1136-60; St. Hilaire, “Scalable Optical Architectures for Electronic Holography,” Ph.D. Thesis, MIT Program in Media Arts and Sciences, Massachusetts Institute of Technology, 1994 (hereafter “St. Hilaire”); and U.S. Pat. No. 5,175,251. Making these displays interactive has of proved to be a challenging engineering problem owing to the large computation, communication, and modulation bandwidths involved. The most recent incarnation of holovideo is capable of displaying up to three pre-computed 36-Mbyte holograms per second. Computing a single hologram still requires about five seconds on our fastest computing hardware. These computational and display update rates still fall short of those required for real-time interactivity.
The combination of haptics and holography was investigated for an object inspection task as described in Jones, “The Haptic Hologram,”
Proceedings of SPIE, Fifth International Symposium on Display Holography
2333:444-447 (1994). Visual display was provided by a reflection-transfer hologram which presented an aerial image of a control valve. A computer-ontrolled tactile glove (CCTG) provided coincident haptic display of the same data. Reflection holograms, however, allow the interacting hand to block the illuminating light and thereby interfere with image reconstruction.
That problem was addressed by employing full-parallax edge-illuminated holograms in combination with the PHANTOM for the inspection of static 3D models (see Plesniak, et al., “Tangible holography: adding synthetic touch to 3D display,” in
Proceedings of the IS&T/SPIE's Symposium on Electronic Imaging, Practical Holography XI
(1997). The edge-illuminated hologram format allowed hand movements in any part of the visual workspace. Thus a viewer could haptically explore the spatially registered force model while visually inspecting the holographic image details over a wide field of view. All of these displays were static, however; no dynamic modification could be made to the image presented.
DESCRIPTON OF THE INVENTION
Brief Summary of the Invention
The present invention combines the three-dimensional realism of holography with force feedback and dynamic interaction. In particular, by utilizing a limited-parallax computed holographic display and by making only local changes to the hologram, the present invention enables a user to rapidly interact with and modify an electronic holographic image using a force-feedback device and both general and special-purpose hardware. In general, hardware constraints favor generation of holograms with parallax (i.e., the change in the apparent relative orientation of the holographically depicted object as the viewing angle changes) extending along a single dimensional axis. For example, in a horizontal parallax only (HPO) system, the relative orientation of the depicted object changes only in response to the viewer's horizontal movement; vertical characteristics remain unchanged. Of course, as faster holographic rendering hardware becomes available, the invention can be straightforwardly adapted to provide full parallax.
Along the axis of parallax, the holographic natur
Benton Stephen
Pappu Ravikanth
Plesniak Wendy
Hjerpe Richard
Massachusetts Institute of Technology
Nguyen Kimnhung
Testa Hurwitz & Thibeault LLP
LandOfFree
Dynamic holographic video with haptic interaction does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Dynamic holographic video with haptic interaction, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dynamic holographic video with haptic interaction will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2503539