Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
1999-06-11
2003-10-21
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S190000, C345S629000
Reexamination Certificate
active
06636647
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a method and apparatus for smoothly merging data such as image data. The present invention also relates to a medium for providing a data merging program for the data merging method.
2. Description of the Related Art
Multi-resolution-based techniques are available as a method for splicing two or more images into a single image having a smooth splice. These techniques are described in a paper authored by Peter J. Burt et al., entitled “A Multiresolution Spline With Application to Image Mosaics,” ACM Trans. on Graphics, Vol. 2, No. 4, October 1983, pp. 217-236, and in a paper authored by Peter J. Burt, entitled “Fast Filter Transforms for Image Processing”, Computer Graphics and Image Processing, 16, 1981, pp. 20-51. The method using the multi-resolution techniques described in these papers include following three steps.
In a first step, each image is divided by spatial frequency band. A set of divided images is called a Laplacian pyramid. In a second step, the images of different frequency components are blended for splicing according to weighted average. In a third step, the set of images spliced according to the frequency component is summed to form a merged image.
The Laplacian pyramid is a set of images that are divided according to spatial frequency band. A method of creating the Laplacian pyramid is briefly discussed.
An original image is low-pass filtered using a reduce operation to produce a set of images called a Gaussian pyramid. The reduce operation here refers to an operation in which the original image is low-pass filtered to reduce a sample density of the image.
When the original image is subjected to the reduce operation once, image data at a first layer of the Gaussian pyramid is created. Another reduce operation, if executed, creates image data at a second layer of the Gaussian pyramid. The reduce operation is iterated to create image data of further layers. The original data is called a zero-layer image data. The higher the hierarchical layer in the Gaussian pyramid, the lower the frequency component of the image becomes as a result of low pass filtering.
Using the Gaussian pyramid thus constructed and an expand operation, the Laplacian pyramid is created. The expand operation is an operation reverse to the reduce operation. The “image data at the i-th layer of the Laplacian pyramid” is created by subtracting the “image data that is obtained by subjecting once the image data at (i+1)-th layer of the Gaussian pyramid to the expand operation” from the “image data at an i-th layer of the Laplacian pyramid”.
The image data at each layer in the Laplacian pyramid is the one, having a particular frequency band, picked up from the original image data. As the layer moves upward in the Laplacian pyramid, the image has a lower frequency band.
The Laplacian pyramid, the Gaussian pyramid, the reduce operation and the expand operation have been discussed. A detailed discussion about them is provided in the two above-cited papers.
The image merging using the multi-resolution techniques disclosed in the above-cited papers is now discussed.
The images are merged on the basis of the frequency component according to the cited merging method. The size of each merging area is different from frequency component to frequency component. In a low-frequency component, namely, a high layer in the Laplacian pyramid, two images are blended with respective weights in a wide merge area, while in a high-frequency component, namely, a low layer in the Laplacian pyramid, two images are blended with respective weights in a narrow merge area. The merge area in the low-frequency is set to be wide because of the following reason: if the images having a direct current component (the extreme case of low-frequency components) are spliced in a narrow area, a discontinuity stands out, and the images are thus allowed to mildly vary in a wide area for splicing. The merge area in the high-frequency component is set to be narrow. The high-frequency components at an edge or other features, if blended in a wide area, are blurred and look like double exposures.
In accordance with the merging method using the multi-resolution techniques, in the low-frequency component, namely, the high layers in the Laplacian pyramid, two images are blended with respective weights in a wide merge area. The lower the frequency component, the wider the merge area. At the lowest frequency band, namely, in the topmost layer in the Laplacian pyramid, the size of the merge area extends to the entire original image.
If images formed of elements having extremely varying aspect ratios, such as images of the hair, are merged using the above-referenced merging method, a natural looking merging is difficult.
Now, one single hair stands out among a plurality of hairs. The outstanding one single hair means that the edge of the one single hair looks distinct because of the lighting conditions. Such a hair contains a great deal of high-frequency components, and data about it is placed in a low layer in the Laplacian pyramid. The hair is thus blended in a narrow area in the center of the merge area. Specifically, the above merging method results in an unnatural-looking image in which the outstanding hair extending from one side disappears abruptly in the center.
Now, an original image shown in FIG.
1
A and an original image shown in
FIG. 1B
are merged at the center of the screen.
FIG.
1
A and
FIG. 1B
are images of elements such as the hair, featuring extremely varying aspect ratios. Some elements (hairs) are distinct while other elements (hairs) are less distinct.
Referring to
FIG. 1A
, black stripes (a
1
, a
2
, a
3
and a
4
) are distinct elements (hairs). The blank portion indicates less distinct elements, which are not shown in
FIG. 1A
because they are not important in the context of the present invention. Similarly, black stripes (b
1
, b
2
, b
3
, and b
4
) shown in
FIG. 1B
are distinct elements (hairs). The blank portion indicates less distinct elements, which are not shown in
FIG. 1B
because they are not important in the context of the present invention.
FIG. 1C
shows a merged image C, into which the original image shown in FIG.
1
A and the original image shown in
FIG. 1B
were merged in accordance with the conventional data merging method. Since the distinct elements (hairs) contain a great deal of high-frequency components in the merged image, the elements abruptly increase in number near the center (at a vertically aligned dotted line cc, which does not appear in the real image).
Elements a
1
, a
2
, and a
4
shown in
FIG. 1A
become ca
1
, ca
2
, and ca
4
, respectively, as shown in
FIG. 1C
, and ca
1
, ca
2
, and ca
4
appear only on the left half of the screen. Since a
3
does not appear on the left half as shown in
FIG. 1A
, it does not appear in
FIG. 1C
, either. Elements b
1
, b
2
, b
3
, and b
4
shown in
FIG. 1B
become cb
1
, cb
2
, cb
3
, and cb
4
as shown in
FIG. 1C
, and cb
1
, cb
2
, cb
3
, and cb
4
appear only on the right half.
SUMMARY OF THE INVENTION
Accordingly, it is an object of the present invention to provide a method and apparatus for merging a plurality of data in a natural looking manner.
It is another object of the present invention to provide a medium for providing a program for the data merging method.
According to one aspect of the present invention, a method for merging a plurality of data includes the extracting step of extracting at least one of the high-frequency component of first data and the high-frequency component of second data, and the synthesizing step of determining a composite value of the first data and the second data in accordance with at least one of the amount of the high-frequency component of the first data and the amount of the high-frequency component of the second data.
According to another aspect of the present invention, an apparatus for merging a plurality of data, includes an extracting unit for extracting at least one of the high-frequency component of first dat
Nakamura Kyoko
Ohki Mitsuharu
Totsuka Takashi
Bell Boyd & Lloyd LLC
Mehta Bhavesh M.
Patel Kanji
Sony Corporation
LandOfFree
Method and apparatus for merging data and medium for... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for merging data and medium for..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for merging data and medium for... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3127151