Image analysis – Image segmentation – Region labeling
Reexamination Certificate
2000-09-26
2004-09-21
Wu, Jingge (Department: 2623)
Image analysis
Image segmentation
Region labeling
C382S243000, C375S240160, C375S240240
Reexamination Certificate
active
06795578
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image processing method and apparatus, and a storage medium and, more particularly, to an image processing method and apparatus for extracting an object in a moving picture, and a storage medium which stores program software for executing the method.
2. Related Background Art
In recent years, a great deal of attention is paid to processing of separating and synthesizing image information in units of objects. In particular, an MPEG-4 encoding scheme as an international standard receives a great deal of attention as a moving picture compression/encoding scheme. The MPEG-4 scheme can separate a plurality of objects in image data of one frame forming moving picture data, and encode/decode each object. This can facilitate an increase in encoding efficiency, data distribution in accordance with a transmission path, and re-processing of images. Various applications, which are difficult to realize by a conventional scheme, are expected.
An object can take any arbitrary shape, and is made up of a combination of data called a “shape” representing shape information, and data called a texture representing the contents of an image.
The conventional object generation method includes a chromakey separation method using a studio set and the like, a generation method using computer graphics (CG), and an extraction method using a natural image.
One of methods of separating a background and object from a natural image is an area division method.
Area division methods for a still image can be roughly classified into two. One gives attention to the discontinuity of areas, and the other gives attention to the continuity of areas. As an example of the former method, an edge detection method is known. The edge detection method is disadvantageous in that a detected edge does not always form a closed area. As an example of the latter method, a region growing method is known. The region growing method sets a region serving as a seed in a still image, and uses pixels having features closer to the seed area among pixels adjacent to the seed, thereby forming the region.
Area division is influenced by noise on data. For example, in the edge detection method, an edge cannot be normally detected owing to noise. In the region growing method, a region which should grow is trapped to an extreme under the influence of noise.
To reduce the influence of noise, MRF (Markov Random Field) resistant to noise is often used for area division processing.
As a region division method using MRF, a heat bath method is known.
Region division by the heat bath method can be realized by, e.g., the following processing. The processing unit of area division is called a cell. A cell has the number of a cluster to which the cell belongs, as the area label (state value) of the cell.
In the first step, the initial value of the area label of each cell is determined using a random number. The parameter of a cluster is set in accordance with cells which are classified based on their initial values and belong to each cluster. For example, the parameter of the cluster is the average color of cells belonging to the cluster.
In the second step, the probability at which a cell belongs to each cluster is calculated for all the cells by an MRF probability function. The area label of the cell is determined in accordance with the calculated probability.
Assume that the number of a cluster exhibiting the highest probability is set as the area label of the cell. This area label determination method for cells is called ICM (Iterated Conditional Modes).
In the third step, the parameter of the cluster is set again in accordance with the area labels of cells determined in the second step.
In the fourth step, cells whose area labels have changed in the second step are checked. Processing from the second step is repeated until the ratio of cells whose area labels have changed decreases to a predetermined ratio to all the cells. This repetitive processing decreases a frequency of the change of the area label, and when the change ratio reaches the predetermined ratio, the area labels of cells are determined as final area labels.
This processing determines the area label of each cell, and cells can be classified based on their area labels.
The calculation amount of the heat bath method is determined by the number of cells, the number of clusters, and the repeat frequency of the second to fourth steps.
In area division of a moving picture, motion information is often added as a feature amount. The motion information is made from a two-dimensional motion amount (motion vector) obtained by finding out, e.g., the correspondence point of a pixel between frames.
Calculation of the motion vector generally adopts a method using block matching. A matching evaluation function is defined, and a vector value at which an evaluated value minimizes within a search range is selected.
As described above, the motion vector is an important parameter in extracting an object from a moving picture. However, a motion vector calculated by a conventional method does not always accurately reflect the motion of an actual block.
For example, when an image within a block is flat, a portion where matching is established cannot be determined, and a vector at which the evaluation function minimizes changes due to slight noise or changes in luminance. If an image does not contain any edge in both the horizontal and vertical search directions, an accurate motion vector is difficult to be detected. When a repetitive pattern similar to a block exists in the search range, an accurate motion vector is difficult to detect.
If an object is determined using as a determination criterion a motion vector which is not necessarily accurate, an erroneous extraction result (area division) is generated.
SUMMARY OF THE INVENTION
The present invention has been made in consideration of the above situation, and has as its object to provide an image processing apparatus and method capable of accurately extracting an object from the frame of moving picture data, and a storage medium which stores the image processing program.
To achieve the object, according to one aspect of the present invention, there is provided an image processing apparatus/method characterized by inputting successive image data, dividing the image data into blocks each formed from a plurality of pixels, detecting a motion vector of each block, calculating a probability at which the block has a predetermined area label, in accordance with the motion vector and an area label of a neighboring block, determining an area label of each block in accordance with a calculation result, and dividing the image data into areas of respective objects in accordance with the determined area label.
An image processing apparatus/method according to another aspect of the present invention is characterized by inputting successive image data, generating a cell serving as a processing unit of area division of the image data, initially dividing the image data to initialize the cell of the image data, calculating a probability at which the cell has a predetermined area label, thereby determining an area label of the cell from a calculation result, and dividing the image data into areas of respective objects on the basis of the determined area label, wherein initial division processing executes initial division of image data of a processing frame using an area division result of image data of a neighboring frame.
A recording medium according to still another aspect of the present invention is characterized by storing a code of the input step of inputting successive image data, a code of the block division step of dividing the image data into blocks each formed from a plurality of pixels, a code of the detection step of detecting a motion vector of each block, a code of the calculation step of calculating a probability at which the block has a predetermined area label, in accordance with the motion vector and an area label of a neighboring block, a code of the determination step
Itokawa Osamu
Kotani Takuya
Canon Kabushiki Kaisha
Fitzpatrick ,Cella, Harper & Scinto
Wu Jingge
LandOfFree
Image processing apparatus and method, and storage medium does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing apparatus and method, and storage medium, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing apparatus and method, and storage medium will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3260780