Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
1998-12-30
2002-05-21
Boudreau, Leo (Department: 2621)
Image analysis
Image transformation or preprocessing
Combining image portions
C345S629000, C348S584000
Reexamination Certificate
active
06393162
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to an image synthesizing apparatus structured such that overlap portions are provided for each image, followed by dividing one composition into sections to be photographed by a plurality of photographing operations and followed by sequentially connecting the photographed images to one another so that a precise image or an image having a wide angle of view is formed or an all round panoramic photograph is taken.
Recently, reduction in the cost of high-performance personal computers has been realized, causing personal computers to widely be used in companies, schools and homes. Simultaneously, visual information apparatuses including digital still cameras and digital video cameras have been widely used because of easiness of capturing an image into the personal computer in place of conventional film cameras.
The digital camera is not required to perform a development process of a photographed image and is permitted to easily delete and edit photographed images. Therefore, digital cameras have widely been used mainly by users of the personal computers.
However, images photographed by a solid-state image device, such as CCD, suffer from inferior resolution to that of an image exposed to a film. Therefore, raising of the resolution has considerably be required.
The resolution can be raised by increasing the number of pixels of the image pickup device. A fact is known that the cost of the solid-state image device is excessively raised in proportion to the number of pixels. Thus, the overall cost of the camera is raised excessively.
Therefore, the applicant of the present invention has developed a technique for connecting, to one another, images photographed by a plurality of image pickup devices and another technique of photographing a subject in a division manner by, for example, moving a camera, to connect images photographed by one image pickup device, which has been filed as U.S. patent application Ser. No. 08/969,937.
An example of the image synthesizing techniques of a type in which a plurality of image pickup devices are disposed so as to be operated as one image pickup device will simply be described with reference to FIG.
18
A.
The foregoing image pickup apparatus incorporates image input portions
1
a,
1
b
and
1
c
each having an optical system
2
, an image pickup portion
3
including an image pickup device, such as a CCD, an A/D conversion portion
4
for converting analog data to digital data; an image synthesizing portion
6
for connecting the supplied images A, B and C; a monitor
7
for displaying a synthesized image; a printer
8
; and a storage medium
9
. Although each of the image input portions
1
a,
1
b
and
1
c
generally incorporates an image processing portion for adjusting white balance for compressing image data so as to efficiently store image data, the image processing portion is omitted from illustration.
The image of a subject
5
is divided into a plurality of images each having overlap portions, and then photographed by the image input portions
1
a,
1
b
and
1
c.
The photographed images are supplied to the image synthesizing portion
6
. The image synthesizing portion
6
uses image data of the overlap portions of the images to calculate the positional relationship among the images. Then, the image synthesizing portion
6
forms and transmits a connected image to the monitor
7
or the like.
FIG. 23
shows the structure of the image synthesizing portion
6
.
The images A, B and C photographed by the image input portions
1
a,
1
b
and
1
c
are temporarily stored in frame memories
10
a,
10
b
and
10
c.
Then, deviation detectors
11
a
and
11
b
calculate the positional relationships (amounts of parallel translation and quantities of rotations) of the adjacent images in accordance with image data of the overlap portions. The results of the calculations are supplied to interpolators
12
a
and
12
b.
The interpolator
12
a
interpolates the image B by using the output of the detector
11
a,
generating an interpolated image B. The interpolator
12
b
interpolates the image C by using the outputs of the detectors
11
a
and
11
b,
generating an interpolated image C. The image A used as a reference image in this case, the interpolated image B, and the interpolated image C are supplied to a coefficient determining device
13
. The pixels of the image A, those of the interpolated image B, and those of the interpolated image C are multiplied by a predetermined coefficient used in weighting addition. The three images subjected to the multiplication are supplied to an adder
15
, which adds the correct images, forming a combined image. The combined image is temporarily stored in a frame memory
16
and can be output therefrom whenever necessary.
As shown in
FIG. 18B
, the coefficient determining device
13
determines coefficients Ca, Cb and Cc in such a manner that the weights are gradually reduced (or enlarged) in the overlap portions.
The above-mentioned technique is able to effectively raise the resolution of a digital image. Moreover, the technique can be used to enlarge the angle of view. In addition, the technique can be applied to the following technique as well as the technique for simply enlarging the angle of view.
Software has been put into practical use so that an image photographed in an angular degree of 360° about a certain point, that is, a so-called panoramic image is used as an original image. Then, a required viewpoint is selected by operating a mouse or a keyboard to form an image photographed by a camera directed to the direction of the selected viewpoint. Thus, the image is displayed on a display unit. The foregoing software enables a user to arbitrarily observe subjects from a required direction. Therefore, the user is able to feel a virtual reality as if the user standing at the photographing point looks around the user.
To form the panoramic image, two methods have been widely used.
One of the two methods is a method using a panoramic camera having an aperture portion which is rotated in synchronization with the operation for winding a film up so that an image is photographed. Another method is a method of forming a panoramic image by rotating a usual camera on a horizontal plane to photograph divided sections of a subject by performing a plurality of photographing operations and by connecting the formed images to one another.
The latter method of the two methods will now be described.
The latter method uses the technique disclosed in the ITEC '91 theses, ITE Annual Convention, 1991, and shown in
FIGS. 19 and 20
.
Referring to
FIG. 19
, symbol R represents subject and O represents a reference point of a camera (in general which is a node of an optical system). Projection planes Q and Q′ are planes onto which the subject is projected when the subject is photographed. In the foregoing case, a cylinder P is a projection plane equivalent to the projection plane when a panoramic image is photographed by the first method, the cylinder P being a cylinder formed around the point O.
A plurality of images projected onto the projection planes Q and Q′ are converted into images projected onto the cylinder P, followed by connecting adjacent images to one another.
For example,
FIG. 20A
shows images photographed by a camera and projected onto the projection planes Q and Q′.
FIG. 20B
shows images formed by projecting photographed images from the projection planes Q and Q′ to the cylinder P. When the foregoing images are connected to each other by a method disclosed in U.S. Ser. No. 08/969,937 an image as shown in
FIG. 20C
can be obtained. The obtained image is equivalent to the image photographed by the former method of the two methods.
The technique is structured to perform a smoothing process when the adjacent images are connected to each other. Therefore, the overlap portions can smoothly be connected to each other. If a subject as shown in
FIG. 21
is photographed, the sky having a large ratio is photographed in the left-hand ima
Boudreau Leo
Frishauf, Holtz Goodman, Langer & Chick, P.C.
Kassa Yosef
Olympus Optical Co,. Ltd.
LandOfFree
Image synthesizing apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image synthesizing apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image synthesizing apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2850026