Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
2000-03-29
2004-05-18
Patel, Jayanti K. (Department: 2625)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S312000, C707S793000
Reexamination Certificate
active
06738533
ABSTRACT:
FIELD OF THE INVENTION
This invention relates generally to image-based rendering (IBR), and more particularly to determining a minimum sampling rate and/or a minimum sampling curve for IBR.
BACKGROUND OF THE INVENTION
Image-based rendering (IBR) in general simulates a continuous range of virtual camera viewpoints of a scene from a discrete set of input images of the scene. IBR can be used for a variety of different applications. For example, a currently popular application is to use IBR in conjunction with electronic commerce-type applications on the Internet. A user interested in seeing a car's interior or exterior from different viewpoints, for example, may be able to rotate inside a virtual camera inside the car's interior, and zoom in and out to see more or less of the interior, and may also be able to rotate a virtual camera around the car's exterior. Thus, using IBR in such instances enables consumers to better see physical real-world objects that they may desire to purchase, allowing those consumers to make better-informed decisions.
IBR can be classified as a continuum between two different approaches. At one end, traditional texture mapping relies on accurate geometrical models, but only a few images. For example, in an IBR system with depth maps, such as three-dimensional (3D) warping, view interpolation, view morphing, and layered-depth images, a model has a set of images of a scene and their associated depth maps. When depth is available for every point in an image, a new image can be rendered from any nearby point of view by projecting the pixels of the image to their appropriate 3D locations, and reprojecting them onto a new image.
At the other end of the continuum, light-field rendering uses many images, but does not require any geometrical information. For example, light-field rendering generates a new image of a scene by appropriately filtering and interpolating an already acquired set of sample images. An approach known as Lumigraph is equivalent to light-field rendering, but applies approximated geometry to compensate for non-uniform sampling to improve rendering quality. Light-field rendering, however, has a tendency to rely on oversampling to counter undesirable aliasing effects in the rendered new image. Oversampling results in intensive data acquisition, significant storage space required to store the acquired images, and a large degree of redundancy in the acquired images.
Little is known in the prior art as to the relationship between IBR techniques at the texture-mapping end of the continuum and IBR techniques at the light-field rendering end of the continuum. In particular, when a given depth is specified, the number of sampled images necessary to achieve optimal new image rendering is unknown. For light-field rendering such as Lumigraph, the number of sampled images necessary to achieve optimal new image rendering is also unknown, which results in prior art approaches oversampling the scene, as described in the previous paragraph. For these and other reasons, therefore, there is a need for the present invention.
SUMMARY OF THE INVENTION
The invention relates to determining a minimum sampling rate for light-field rendering specifically, and to determining a minimum sampling curve for image-based rendering (IBR) in general. In one embodiment, the minimum sampling rate for light-field rendering is determined in accordance with
Δ
t
max
=
1
2
K
Ω
v
fh
d
,
where K
&OHgr;
v
accounts for a light-field signal cut-off frequency, a sampling camera resolution and an output resolution, ƒ specifies a sampling camera focal length, and h
d
specifies a light-field depth range. In another embodiment, the minimum sampling curve for IBR is determined as constrained by N
d
=2K
&OHgr;
v
ƒh
d
&Dgr;t, N
d
≧1, where N
d
is the number of depth layers, &Dgr;t specifies a sampling interval along a t direction, K
&OHgr;
v
accounts for a signal cut-off frequency, a sampling camera resolution and an output resolution, ƒ specifies a sampling camera focal length, and h
d
specifies a depth range. Where IBR is to be performed under uncertain depth conditions, the minimum sampling curve is determined in one embodiment as
Δ
t
max
=
min
z
e
(
z
e
+
Δ
η
)
(
z
e
-
Δ
η
)
4
fK
Ω
v
Δ
η
,
where &Dgr;t
max
specifies a maximum sampling interval along a t direction, K
&OHgr;
v
accounts for a signal cut-off frequency, a sampling camera resolution and an output resolution, ƒ specifies a sampling camera focal length, z
e
specifies an estimated depth, and &Dgr;&eegr; specifies a depth error.
Embodiments of the invention provide for advantages over the prior art. In particular, embodiments provide for the number of image samples from a four-dimension (4D) light field and the geometrical and textural information needed to generate a continuous representation of a scene for which new images are to be generated. This is also referred to as plenoptic sampling, that is, the number of images needed for plenoptic modeling. Furthermore, the minimum sampling curve described in some embodiments of the invention serves as a guiding design principle for IBR systems, and bridges the gap between IBR and geometry-based rendering.
REFERENCES:
patent: 5313568 (1994-05-01), Wallace et al.
patent: 5365428 (1994-11-01), dePinto et al.
patent: 5751926 (1998-05-01), Kasson et al.
patent: 6166742 (2000-12-01), He
patent: 6466207 (2002-10-01), Gortler et al.
patent: 6502097 (2002-12-01), Chan et al.
patent: 6549308 (2003-04-01), Camahort
E.H. Adelson and J. Bergen. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing. Pp. 3-20. MIT Press, Cambridge, MA, 1991.
S.J. Gortler, R. Grzeszcuk, R. Szeliski, M.F. Cohen. The lumigraph. In Computer Graphics Proceedings, Annual Conference Series, pp. 43-54, Proc. SIGGRAPH'96 (New Orleans), Aug. 1996, ACM SIGGRAPH.
M. Levoy and P. Hanrahan. Light field rendering. In Computer Graphics Proceedings, Annual Conference Sseries, pp. 31-42, Proc. SIGGRAPH'96 (New Orleans), Aug. 1996, ACM SIGGRAPH.
B. Girod, Motion compression, visual aspects, accuracy, and fundamental limits. In Motion Analysis and Image Sequence Processing, Kluwer 1995, chapter 5.
Chai Jin-Xiang
Shum Heung-Yeung
Tong Xin
Azarian Seyed
Lyon & Harr LLP
Microsoft Corporation
Patel Jayanti K.
Watson Mark A.
LandOfFree
Minimum sampling rate and minimum sampling curve for... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Minimum sampling rate and minimum sampling curve for..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Minimum sampling rate and minimum sampling curve for... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3260008