Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
2000-09-15
2004-09-28
Nguyen, Phu K. (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C348S042000, C348S051000
Reexamination Certificate
active
06798406
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates to the production of stereo images.
The invention has applications in the fields of natural image generation by film or digital photography, analogue or digital video, movie film generation, and synthetic Image generation using methods including computer graphics or image based rendering systems, and is particularly relevant to two view stereoscopic devices including electronic and hard copy devices where more than one image of a scene is generated to create a 3D effect by showing a different image to a viewer's left and right eyes.
2. Description of the Related Art
Applications of the invention include photogragphy, videography, movie production, electronic shopping kiosks, computer games systems for home or public use, multimedia packages e.g. encyclopaedias, medical imaging, CAD/CAM systems, scientific visualisation, remote manipulation, remote sensing, security systems and any other application where a benefit is found from a stereoscopic 3D image of a scene.
Many types of stereoscopic and auto-stereoscopic electronic displays and printing or photographic reproduction methods have been developed, for example see the following European and British patent applications: EP 0 602 934, EP 0 656 555, EP 0 708 351, EP 0 726 483, GB 9619097.0 and GB 9702259.4. The problem of image generation for these systems is less well understood and many existing stereoscopic images can be uncomfortable to view even on a high quality stereoscopic imaging device. (Where the term stereoscopic is used it should also be taken to imply multi-view systems where more than one image is generated and presented to the user even if only two of the images are viewed at any one time by the left and right eyes.)
As described in B. E. Coutant and G. Westheimer, “Population distribution of stereoscopic ability”, Opthal. Physiol. Opt., 1993, Vol 13, January, up to 96% of the population can perceive a stereoscopic effect and up to 87% should easily be able to experience the effect on desktop 3D display systems. The following summarises some problems inherent in previous approaches to stereoscopic image generation.
SUMMARY OF THE RELATED ART
Stereoscopic systems represent the third dimension, depth in front of and behind the image plane, by using image disparity As illustrated in FIG.
1
. The image disparity displayed on a screen has a physical magnitude which will be termed screen disparity. Crossed disparity, d
N
, results in a perceived depth, N, in front of the display plane while uncrossed disparity, d
F
, results in a perceived depth, F, behind the display plane as illustrated in FIG.
1
.
The screen disparities dn or df between homologous points in the left and right images are seen by the viewer as perceived depths N or F in front or behind the display plane. To see this effect the viewer must maintain focus on the display plane while verging their eyes off the display plane. This is thought to stress the visual image if the perceived depth value is too great and therefore limits are required for the values of N and F if comfortable images are to be produced.
These type of stereoscopic display systems do not exactly match the user's perception in the real world in that it requires the user to accommodate (focus) on the display surface while verging their eyes away from the display surface, see FIG.
1
. Since the accommodation and vergence mechanisms are linked in the brain (see D. B. Diner and D. H. Fender, “Human engineering in stereoscopic viewing devices”, 1993, Plenum Press, New York, ISBN 0-306-44667-7, and M. Mon-Williams, J. P. Wann, S. Rushton, “Design factors in virtual reality displays”, Journal of SID, Mar. 4, 1995) this requires some effort from the viewer and a greater effort the more depth is being perceived. The invention recognises that the key variable to control is perceived depth, the larger this value is the more stress is placed on the viewer's visual system.
It is now widely recommended (see L. Hodges, D. McAllister, “Computing Stereoscopic Views”, pp71-88, in Stereo Computer Graphics and Other True 3D Technologies, D. McAlister, Princeton University Press, 1993; A. R. Rao, A. Jaimes, “Digital stereoscopic imaging”, SPIE Vol 3639, pp144-154, 1999; and R. Akka, “Converting existing applications to support high quality stereoscopy”, SPIE Vol 3639, pp290-299, 1999) that images are captured using two cameras positioned so that the only difference between the two images is the image disparity due to a horizontal translation of cameras. This arrangement is normally referred to as a parallel camera system. This avoids viewer discomfort due to keystone distortion (this arises when the cameras are not parallel because the vertical dimensions of the two images vary from one side of each image to the other) and associated vertical disparity ie. when the two images are superimposed there is varying vertical disparity across the images. In addition, for physical cameras the optics and light sensitive media must be matched to avoid unnatural intensity or geometric distortions. The latter two issues are part of a specific camera design and are not considered further here.
As illustrated In
FIG. 2
, the parallel camera image must be processed to ensure the depth range captured in the image disparity fits both in front and behind the display plane. This requires the use of offset sensors or film behind the lens, skewed camera frustum (in the case of computer graphics) or image cropping. Without such adjustments all depth in the images will be perceived in front of the display plane.
FIG. 2
shows that the images from parallel cameras need to be adjusted, either to have the edge of the image cropped or by the use of an asymmetric camera fustrum. The latter is possible in many computer graphics systems, or can be achieved by offsetting the image sensitive material (e.g. CCD or film) to one side in a physical camera.
The factors which directly affect perceived depth are:
For depth behind the display surface, uncrossed disparity:
F=Z
/((
E/df
)−1
For depth in front of the display surface, crossed disparity:
N=Z
/((
E/dn
)+1)
From these equations it can be seen that perceived depth depends on the screen disparity, the viewer's eye separation, E, and the display viewing distance, Z. While other methods have approximated or ignored these variables the new method allows them to be fully accounted for.
The screen disparity (dn or df) is important as it is determined by image disparity which in turn is determined by the image capture environment, including the camera parameters and the depth in the scene. The invention seeks to control image disparity and therefore screen disparity and perceived depth by controlling the camera parameters. While various previous methods to control the camera parameters have been proposed none consider the issue of directly controlling perceived depth and often approximate the parameters such as the comfortable near and far perceived depth limits for the target display.
In. S. Kitrosser, “Photography in the service of Stereoscopy”, Journal of imaging science and technology, 42(4), 295-300, 1998, a slide rule type calculator is described allowing selection of camera separation given the camera details and scene near and far distances. It does not take into account different viewer's eye spacings and the maximum image disparity is set at a predetermined value, It cannot account for the perceived depth the user sees when using a particular display and as has been discussed earlier this is the key variable in assessing the comfort of a stereoscopic image.
In L. Lipton, “Foundations of the Stereoscopic Cinema, A Study in Depth”, Van Nostran Reinhold Company, 1982, Lipton examines the mathematics involved in positioning cameras and develops a set of tables for different film formats giving maximum and minimum object distances for a given convergence distance and lens focal length. He assumes converging cameras will be used and the maximum screen disparity
Holliman Nicolas Steven
Jones Graham Roger
Lee Delman
Nguyen Phu K.
Renner Otto Boisselle & Sklar
Sharp Kabushiki Kaisha
LandOfFree
Stereo images with comfortable perceived depth does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Stereo images with comfortable perceived depth, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Stereo images with comfortable perceived depth will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3213977