Image analysis – Pattern recognition – Feature extraction
Reexamination Certificate
1999-06-01
2003-09-23
Dastouri, Mehrdad (Department: 2623)
Image analysis
Pattern recognition
Feature extraction
C382S236000, C382S248000, C382S199000, C348S586000
Reexamination Certificate
active
06625316
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image processing apparatus and method and, more particularly, to an image processing apparatus and method of extracting an object from an image and processing the extracted object.
2. Description of the Related Art
Coding systems such as MPEG-1, MPEG-2, and H.261 are known as conventional motion image coding systems. These coding systems can efficiently encode motion images by using an interframe correlation.
As a coding system having higher efficiency than those of the above conventional coding systems, standardization of a system, such as MPEG-4, has been examined which cuts out an object from an image, encodes the object separately from its background, and transmits only this object. When this coding system is used, an image region to be transmitted can be decreased, so motion images can be transmitted even by a low-bit-rate channel. Additionally, a receiving side can display suitable images by selectively displaying objects or changing the arrangement or sizes of objects. Furthermore, editing such as synthesizing an object with another background can be easily performed.
As a method of extracting an object from a motion image, an extraction technique using a chromakey used in broadcasting stations and the like is generally known. This technique is a method of photographing an object such as a person before a blue background and cutting out this person object from the image signal. Object photographing using the chromakey is usually performed in photographing studios and the like under well-ordered illuminating conditions, so no object shades are formed.
Also, the method can automatically separate an image region from a still image. To extract an object from a still image, a desired image region is cut out by manually designating the region by a user or uniting regions having similar colors.
Unfortunately, objects extractable by the chromakey are limited to relatively small ones which can be photographed only before a blue background. Extraction from motion images of natural images is one possible method of extracting relatively large objects. Known examples of the method are a method of previously inputting a background image and cutting out an object from a difference image of the background image and an input image and a method of previously acquiring color information and the like constructing a background and extracting a region having a different color from that of the color information from an input image (Picture Coding Symposium of Japan PCSJ97I-3.15).
A method of cutting out a helicopter
1051
as an object from an image
1050
as shown in
FIG. 21
will be described below. That is, a difference between the image
1050
shown in
FIG. 21 and a
previously photographed background image
1052
shown in
FIG. 22
is obtained. Processing such as noise reduction is performed for this difference to extract an object
1053
shown in
FIG. 23. A
new image shown in
FIG. 25
can be edited by synthesizing the object
1053
and a background image
1054
shown in FIG.
24
.
Unlike in object photographing for general chromakey synthesis, however, an object in a motion image of a natural image often has a shade because the object is photographed in natural light. Therefore, the shade of the helicopter appears in the sky in the image shown in
FIG. 25
, resulting in unnatural synthesis.
Likewise, an object
1057
shown in
FIG. 27
is obtained by cutting out a cattle
1056
grazing herbage on a lawn from an image
1055
shown in FIG.
26
. When this object
1057
is synthesized on an image of an urban district as shown in
FIG. 28
, the shade of the cattle contained in the object
1057
is also synthesized. An image of this cattle shade contains an image of the herbage on the lawn in the original image
1055
. Therefore, when this object
1057
is synthesized on the asphalt background as shown in
FIG. 28
, a considerably unnatural image results.
To obtain an object having no shade by using the chromakey or the like and synthesize this object, a three-dimensional positional relationship between the object and the synthesized background, a light source, and the like can be set by, e.g., computer processing. This setting is effective in a limited environment such as a studio. However, no shade can be formed for an object once two-dimensionally input as a motion image, so an unnatural synthetic image having no shade is formed.
FIG. 29
shows an image formed by synthesizing the cattle
1056
with no shade. As is evident from
FIG. 29
, this image is very unnatural. This unnaturalness of an image resulting from shadeless synthesis becomes conspicuous when the image is synthesized on a background image having large amounts of shades of other objects.
As described above, it is difficult to independently and appropriately process a main object extracted from an image and a secondary object attached to this main object.
The abovementioned MPEG-4 coding system is a method of separating a motion image into a background and a subject to be encoded, which is called an “object”, and separately encoding the background and the object. Unlike in encoding performed in units of frames such as in conventional MPEG-1, MPEG-2, h.261, and h.263, a background having no (or little) motion is encoded only once, so low-bit-rate encoding is possible. Additionally, a decoding side can easily perform editing such as selection, enlargement or reduction, and rotation of an object. This allows a user to perform desired decoding.
An example of coding in the MPEG-4 coding system will be described below. Note that a method of extracting a background and an object from an image is not a standard subject of MPEG-4, so any arbitrary method can be used. For example, a method as described in “Morphological Segmentation Using Advance Knowledge Information in Sports Programs” (1997 Image Media Processing Symposium (IMPS97) I-3, 15, Oct. 8th, 1997, Naemura et al.) This is a method of previously acquiring information of, e.g., a ground where no players as objects exist, as a background and, on the basis of this background information, extracting objects (players) from a motion image.
FIG. 47
is a block diagram showing the arrangement of a conventional motion image input apparatus. Referring to
FIG. 47
, motion image data obtained by image sensing by a TV camera
1001
is stored in a background memory
1002
. Assume that an image of a yacht and a battle ship cruising on the sea as shown in
FIG. 48
is to be processed. When a background region is extracted from this image by forming a color histogram, the sky in the upper half and the sea in the lower half can be detected as shown in FIG.
49
. Since this background image shown in
FIG. 49
is encoded by, a background image encoder
1003
.
Subsequently, an image containing the yacht and the battle ship as objects is sensed as a motion image. An object extractor
1004
extracts the objects by calculating the difference from the background image or extracting regions having different colors from that of the background image. The extracted objects are as shown in FIG.
50
. An object encoder
1005
encodes these objects.
FIG. 51
is a detailed block diagram of the object encoder
1005
. Shape information of an object is input from a terminal
1020
and stored in a shape memory
1022
. This shape information is represented by a binary image in which pixels indicating an object are white and other pixels are black. A boundary extractor
1023
extracts pixels where black and white are switched from this image as a boundary and inputs this boundary to a motion compensator
1024
. If a frame mode is an I frame, this motion compensator
1024
does not operate but the input shape information is stored to a boundary memory
1025
and arithmetically encodes at a arithmetic encoder
1026
. If the frame mode is a P frame or a B frame, the motion compensator
1024
performs motion compensation by comparing the input boundary with the contents of the boundary memory
1025
storing boundary condit
Canon Kabushiki Kaisha
Dastouri Mehrdad
Fitzpatrick ,Cella, Harper & Scinto
LandOfFree
Image processing apparatus and method, and image processing... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing apparatus and method, and image processing..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing apparatus and method, and image processing... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3015966