Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
1999-01-15
2001-03-13
Couso, Yon J. (Department: 2623)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S154000, C382S106000
Reexamination Certificate
active
06201899
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates, in general, to image processing and, more specifically, to a method and apparatus for combining multiple source images of substantially identical fields of view but differing focal distances into a single image that appears to be in focus at all points in the field of view.
BACKGROUND OF THE INVENTION
It is difficult to produce a 2-dimensional image of a relatively deep 3-dimensional scene in which both relatively close and relatively distant objects within the scene appear in focus. For example, when a microscope is used to photograph a 3-dimensional specimen, portions of the specimen that are not in or near the focal plane are blurred or invisible.
Adelson in U.S. Pat. No. 4,661,986 (incorporated herein by reference) teaches a method for obtaining a focused 2-dimensional image from an assemblage of M separately focused 2-dimensional images of the same 3-dimensional scene. Adelson employs an image processing algorithm developed by Dr. Peter J. Burt (hereinafter referred to as the Burt Pyramid). Adelson's method separately analyzes each of the M separately focused images into N similar sets of pixel samples. The method then selects, on a pixel-by-pixel basis from each group of M corresponding sets of the assemblage, the best focused pixels, to derive a single analyzed image of N sets of improved-focus pixels. By employing the Burt Pyramid image synthesizing technique, Adelson synthesizes the final 2-dimensional image from the single analyzed image of N sets.
Adelson describes the selection of image pyramid coefficients from a collection of source image pyramids by taking those of maximum squared value or maximum absolute value, or by other methods that depend on the relative coefficient values themselves. Adelson thus constructs a pyramid transform for each source image. At each pyramid sample position, corresponding samples of each sub-group are selected. All the selected sub-groups are combined to form a single pyramid representing the composite image. The composite image is recovered through an inverse Burt pyramid transform.
The Burt pyramid transform of the improved-focus two-dimensional image is assembled octave by octave, choosing the corresponding octave of the original image having the highest intensity level. The improved-focus two-dimensional image is then generated from its Burt pyramid transform by performing an inverse pyramid transform procedure. A bibliography of the early literature concerning the Burt pyramid transform may be found in U.S. Pat. No. 4,661,986.
The Burt pyramid may be viewed as a set of low-pass or band-pass filtered copies of an original image in which both the band limit and the sample density are reduced in regular steps. Pyramid construction is outlined in FIG.
1
. The process begins with the formation of a low-pass filter or Gaussian pyramid through the steps shown on the top row of FIG.
1
. The original image, or source image, is designated as G(0). This original image, G(0), is first low-pass filtered (F1) then subsampled by a factor of two in the vertical and horizontal directions (F2) to obtain an image G(1), which is half as large in each dimension as G(0). G(1) is then filtered and subsampled in the same way to obtain G(2). These steps are iterated to generate the tapering sequence of images which constitute levels of the pyramid. Since both resolution and image size are decreased, the formation of the various levels of the Gaussian pyramid may be referred to as filtering and decimating steps. With each level of the pyramid, the bandwidth is reduced by one octave step.
The Gaussian pyramid is a set of low-pass filtered images. In order to obtain band-pass images, each level of the Gaussian pyramid may be subtracted from the next lowest level, as shown in the bottom row of FIG.
1
. Because these arrays differ in sample density, it is necessary to interpolate new samples between those of a given array before it is subtracted from the next lowest array. As shown, the Laplacian, for example L(0), is formed by restoring the subsampled data from G(1), for example (by inserting zero-valued samples between the given samples (F2′) then applying an interpolation filter (F1′)) and subtracting the data from the G(0) Gaussian. The Laplacian formed this way is known as the reduce-expand Laplacian. Alternatively, the Laplacian may be formed without subsampling and re-interpolation, as shown by the dotted line in FIG.
1
. This is known as a filter-subtract-decimate Laplacian. Thus, the Laplacian is a sequence of band-pass images L(0), L(1), L(2), etc. This is precisely how Adelson separates the visual frequencies of each source image into eight ranges.
SUMMARY OF THE INVENTION
To meet this and other needs, and in view of its purposes, the present invention provides a method for constructing an extended depth of field image from a plurality of source images taken of an object at substantially identical fields of view but at differing focal distances. In one embodiment the method includes the following steps:
a) receiving the plurality of source images,
b) filtering the plurality of source images to obtain relatively high frequency components in each of the filtered images,
c) estimating energy levels of the high frequency components,
d) selecting sub-regions in each filtered image having energy levels which are greater than energy levels of sub-regions in other filtered images,
e) mapping the selected sub-regions of each filtered image to corresponding sub-regions in each source image, and
f) merging the corresponding sub-regions of the source images to obtain the extended depth of field image.
The filtering may include constructing a Laplacian pyramid of various levels or using other digital filters, such as a discrete Fourier transform, a recursive or non-recursive filter.
It is understood that the foregoing general description and the following detailed description are exemplary, but are not restrictive, of the invention.
REFERENCES:
patent: 4584704 (1986-04-01), Ferren
patent: 4661986 (1987-04-01), Adelson
patent: 5325449 (1994-06-01), Burt et al.
patent: 5878152 (1999-03-01), Sussman
International Search Report for International (PCT) Application No. PCT/US99/23461 dated Feb. 2, 2000.
Burt et al. “Enhanced Image Capture Through Fusion,” David Sarnoff Research Center, Princeton, NJ, 1993.
Burt et al. “A Multiresolution Spline with Application to Image Mosaics” ACM Transactions on Graphics vol. 2, No. 4 (Oct. 1983).
Burke William J.
Couso Yon J.
Sarnoff Corporation
LandOfFree
Method and apparatus for extended depth of field imaging does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for extended depth of field imaging, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for extended depth of field imaging will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2536529