Computer graphics processing and selective visual display system – Computer graphics processing – Graph generating
Reexamination Certificate
1997-05-28
2001-06-05
Zimmerman, Mark (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Graph generating
C382S284000
Reexamination Certificate
active
06243103
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to an image synthesizing apparatus for synthesizing a plurality of images, recorded by an electronic camera, having overlapping portions, on a computer, an image sensing apparatus used in combination with the image synthesizing apparatus, and an image synthesizing system using these apparatuses. More specifically, the present invention relates to automatic determination of timing for recording an image or images taken in series (image or images to be combined with an image or images which have already been recorded).
Processing for obtaining a wide angle image by synthesizing a plurality of images having overlapping portions using a computer is called “panoramic image synthesizing processing”. This processing is to obtain a single image of stretched scenery or a single wide image.
The resolution of an electronic camera is lower than that of a silver halide camera or an image scanner (i.e., the number of pixels of the electronic camera is small), which is a defect of an electronic camera. Because of this defect, it is necessary to synthesize images recorded by an electronic camera into a panoramic image in order to obtain a wide image as well as an image of high resolution. An image of wider angle of view obtained in one shot with the electronic camera would provide lower the resolution than an image having normal width. In contrast, by sensing a single object (original image) in a plurality of shots, or sensing a scenery in a plurality of shots by shifting image sensing positions in the horizontal direction, the recorded image as a whole has as high resolution than that of an image sensed by a scanner or a silver halide camera.
Meanwhile, some panoramic image synthesizing systems have been configured with electronic cameras and personal computers connected to each other, which inputs images from the electronic cameras to the personal computers, and provided at affordable prices. In such systems, it is difficult for the computers to take full-size images on real-time basis because of the limitation of transmission paths, such as a parallel interface (I/F) or a serial I/F, connecting the electronic cameras and the computers. Therefore, in the conventional systems, images are reduced in size in order to secure a speed for transmitting images on almost a real-time basis and inputted by the computers in a short interval, then the images are provided for a user to confirm an image. Then, after the user confirms a scene which is desired to obtain a reduced image, a shutter button of the electronic camera is pressed. Accordingly, it is possible to obtain a full-size still image after confirming it. Note, the reduced image used for confirmation is called “confirmation image”, hereinafter.
The most difficult process in generating a panoramic image by synthesizing a plurality of images is to find overlapping portions, i.e., to find a group of corresponding pixels in the plurality of images (the process is called “corresponding point extraction”, hereinafter). Where corresponding points are to be found in two images, for example, position relationship between these two images is detected, thus it is possible to synthesize the images.
Upon performing corresponding point extraction, characterizing points and/or resemblance points are needed in these images. In a case where there are little characterizing points and/or resemblance points in the images, an error may occur when performing the corresponding point extraction. As a result, intervention by the user is required in many cases, and it is difficult to realize a full-automatic corresponding point extraction process with the conventional technique.
SUMMARY OF THE INVENTION
The present invention has been made in consideration of the above situation, and has as its object to automatically determine timing for recording an image or images to be used for panoramic image synthesis.
It is another object of the present invention to provide an image synthesizing apparatus, an image sensing apparatus, and an image synthesizing system capable of automatically determining timing for recording an image or images to be used for panoramic image synthesis by reduced images generated from sensed images, without requiring an intervention of a user.
It is still another object of the present invention to provide an image synthesizing apparatus which receives reduced images, used for determining timing for recording an image or images, from an image sensing apparatus.
It is still another object of the present invention to provide an image synthesizing apparatus which generates reduced images, used for determining timing for recording an image or images, by reducing an image or images transmitted from an image sensing apparatus.
According to an aspect of the present invention, a user interface is improved by providing display means for displaying reduced images.
According to another aspect of the present invention, the position relationship between the first and second images to be synthesized is recognized by detecting the first overlapping portion between the first and second reduced images. This is for increasing processing speed for determining image recording timing.
According to another aspect of the present invention, the second overlapping portion between the first and second images is detected from the position relationship between the first and second reduced images, thereby increasing the processing speed for synthesizing the images.
According to another aspect of the present invention, determination means for determining whether the size of the first overlapping portion is appropriate or not is provided, thereby improving precision for determining image recording timing.
According to another aspect of the present invention, determination means for determining whether the size of the second overlapping portion is appropriate or not is provided, thereby improving the synthesis precision.
According to another aspect of the present invention, notification means for notifying a user of a determination result by the determination means is provided, thereby improving the user interface.
According to another aspect of the present invention, the second overlapping portion is roughly obtained by magnifying the first overlapping portion, edge pixels are extracted in the roughly obtained second overlapping portion as corresponding points, a mapping relationship between the first and second images is obtained on the basis of a group of extracted corresponding points, and the first and second images are combined by converting either the first or second image on the basis of the obtained mapping relationship.
According to another aspect of the present invention, position relationship between the first and second reduced images is detected by comparing pixel values of the first predetermined number of pixels which are set in the first reduced image to pixel values of the second predetermined number of pixels which are set in the second reduced image.
According to another aspect of the present invention, an amount an overlapping portion between the first and second reduced images is calculated on the basis of the position relationship between the first and second reduced images, and whether the ratio of the amount the overlapping portion to the area of the first or second reduced image is less than a first threshold or not is determined, and, when the ratio is less than the first threshold, this information is notified to the user as a timing suitable for recording an image to be used for image synthesis.
According to another aspect of the present invention, when the amount of the overlapping portion becomes less than a second threshold which is smaller than the first threshold, t is information is notified to the user in that it is not a timing suitable for recording an image to be used for image synthesis, thereby preventing inappropriate image recording operation.
According to another aspect of the present invention, the image sensing apparatus and the image synthesizing apparatus are connected with a
Hatori Kenji
Katayama Tatsushi
Takiguchi Hideo
Yano Kotaro
Canon Kabushiki Kaisha
Morgan & Finnegan , LLP
Vo Cliff N.
Zimmerman Mark
LandOfFree
Panoramic image generation in digital photography does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Panoramic image generation in digital photography, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Panoramic image generation in digital photography will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2463308