Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
1999-06-08
2003-03-04
Couso, Jose L. (Department: 2621)
Image analysis
Image transformation or preprocessing
Combining image portions
C348S234000, C359S227000, C396S063000
Reexamination Certificate
active
06529640
ABSTRACT:
This application claims the benefits of Japanese Application Nos. 9-050132, 10-175301 and 10-259362 which are hereby incorporated by reference.
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image processing apparatus such as an image pickup apparatus which performs an image pickup using a space pixel offset method and an image take-in apparatus taking a static image as a digital signal, more particularly to an image pickup apparatus which performs an interpolating processing prior to an image superposition and a technology for enlarging a dynamic range of a digital still camera and improving an image quality while reflecting an intention of a photographer.
2. Related Background Art
In general, a plural image sensor type image pickup apparatus using a color separation prism has been incorporated in video cameras and electronic cameras. In order to achieve a high resolution, this plural image sensor type image pickup apparatus adopts an image pickup method so called a space pixel offset method.
The known image pickup apparatus using the space pixel offset method, which was disclosed in Report by Television Technology Society, 17[51], pp. 1-6, 1993, will be described with reference to FIG.
39
and
FIGS. 40A
to
40
G.
Referring to
FIG. 39
, a dichroic prism A
52
is arranged on an optical axis of an taking lens A
51
, and the dichroic prism A
52
separates an incidence light from the taking lens A
51
into a mixed light of Rand B components (magenta component) and a light of two G components. The color separation for the G component is a separation by a half mirror, and performed irrespective of a wavelength of the incidence light.
A RB image pickup device A
53
a
is arranged in a travelling direction of the mixed light. A color filter array A
52
a
in which red and blue color filters are arranged in a stripe fashion or checkerwise is applied to a image pickup surface of the RB image pickup device A
53
a
. In the RB image pickup device A
53
a
, the R and B components of an optical image are subjected to a photoelectric conversion.
On the other hand, G image pickup devices A
53
b
and A
53
c
are individually arranged in a travelling direction of the lights of the two G components, and the two G components of an optical image are subjected to a photoelectric conversion in the respective devices.
Each of pixel outputs of the G image pickup device A
53
b
is indicated by a symbol ◯ of
FIG. 40A
, and each of pixel outputs of the G image pickup device A
53
c
is indicated by a symbol &Circlesolid; of FIG.
40
B. It should be noted that the G image pickup devices A
53
b
and A
53
c
are arranged so as to be offset in vertical and horizontal directions by ½ pixel from each other.
A signal processing circuit A
54
rearranges the pixel outputs of the G image pickup device A
53
b
(indicated by the symbol ◯ in
FIG. 40A
) and the pixel outputs of the G image pickup device A
53
c
(indicated by the symbol &Circlesolid; in
FIG. 40B
) alternately on corresponding superposition lattice points, thus forming a synthesized image as shown in FIG.
40
D. It should be noted that symbols × indicate positions that have no corresponding pixels.
Moreover, the signal processing circuit A
54
performs an interpolating processing for vacant lattice points having no corresponding pixels (symbols × in FIG.
40
D). As the interpolation method, there have been a upper and lower average interpolation in which the average of outputs of the pixels adjacent to each other in the vertical direction is calculated (see FIG.
40
E), a prior value interpolation method in which the interpolation is carried out using the output of the pixel disposed in the immediately prior or left-adjacent position (see FIG.
40
F), and a left and right average interpolation method in which the interpolation is carried out by calculating the average of the outputs between the left and right pixels (see FIG.
40
G).
By carrying out such an image processing, the image pickup apparatus using the space pixel offset method can achieve a resolution twice that of an image obtained by a single image sensor type image pickup apparatus.
FIG.
41
and
FIGS. 42A
to
42
D are figures for explaining the problems of the prior art by using concrete numerical values for the above described image processing.
As shown in (A) of
FIG. 41
, assuming that a vertically-striped lattice image is photographed, the vertically-striped image is formed on image pickup planes of the G image pickup devices A
53
b
and A
53
c
. Light receiving cell constituting the G image pickup device A
53
b
correspond to any of bright and dark portions of the stripe, so that the light receiving cell have a quantity of received light amounting to “0” or “4”. Furthermore, light receiving cells constituting the G image pickup device A
53
c
corresponds to a boundary portion between the bright and dark portions of the stripe, so that all of the light receiving cells have a quantity of received light amounting to “2”.
The signal processing circuit A
54
replaces the pixel output, which has been subjected to the photoelectric conversion by the G image pickup device A
53
b
, with a position on a synthesized lattice point, thus forming an image shown in (B) of FIG.
41
. Moreover, the pixel output of the image pickup device A
53
c
is replaced with a position on the synthesized lattice point, thus forming an image shown in (C) of FIG.
41
.
The signal processing circuit A
54
synthesizes these two images, thus forming a synthesized image shown in (D) of FIG.
41
. The superposition of the two images is conducted by rearranging each of the pixels of (C) in
FIG. 41
on corresponding one of the vacant lattice points of (B) in FIG.
41
. For example, the pixel “a” is rearranged on the position of the lattice point “a”.
Next, the signal processing circuit A
54
performs an interpolation processing for vacant lattice points of the synthesized image. For the vacant lattice points of the synthesized image, instead of the interpolation processing mentioned above, an adjacent evenness interpolation processing is sometimes performed. The adjacent evenness interpolation processing is the one in which a weighted sum of the matrix is calculated relative to each of the pixels adjacent to the vacant lattice points to be interpolated and the obtained value is used as the interpolation value. The interpolation results are shown in
FIGS. 42A
to
42
D. It should be noted that
FIG. 42A
is a figure showing the vertical average interpolation,
FIG. 42B
is a figure showing the prior value interpolation,
FIG. 42C
is a figure showing the horizontal average interpolation, and
FIG. 42D
is the adjacent evenness interpolation. For example, when the interpolation matrix as shown in
FIG. 42D
is used, each of the pixels located in the positions adjacent to the vacant lattice points vertically and horizontally is multiplied by ¼, and the sum of the multiplied values is used as the interpolation value.
Hereupon, when the edge portions of
FIGS. 42A
to
42
D are paid attention to, in all of the interpolation methods other than the vertical interpolation method, image degrading artifacts are generated in the edge portion, and a distortion of the image, which is mosaic-like, occurs. This is because pixel outputs of the G image pickup device A
53
c
that are other image information contribute to the interpolation in addition to the pixel output of the G image pickup device A
53
b
, so that the image degrading artifacts are generated.
Since when a horizontal stripe lattice image is photographed, the horizontal stripe lattice image is obtained by rotating the vertical lattice image by 90 degrees, image degrading artifacts are not generated in the horizontal average interpolation. However, the image degrading artifacts are generated in the vertical average interpolation, so that a distortion of an image, which is mosaic-like, occurs in the edge portion.
Specifically, when the interpolation processing is performed for th
Matsuda Hideaki
Utagawa Ken
Couso Jose L.
Miles & Stockbridge P.C.
Nikon Corporation
Patel Kanji
LandOfFree
Image processing apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3061202