Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
2001-08-14
2004-10-26
Philippe, Gims (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C348S399100, C382S243000
Reexamination Certificate
active
06810079
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a computer-readable storage medium on which a program for executing image processing is stored, and specifically, to processing for extracting a particular object from frames of moving image data.
2. Related Background Art
In recent years, coding systems have been studied in which a moving picture is grasped as a synthesis of components called objects, and in which compression coding is performed on an object basis. Standardization of such systems known as Moving Picture Experts Group Phase 4 (MPEG-4) is being advanced. A moving picture object can have any shape and is expressed by a combination of data called “shape” representing information on the shape and data called “texture” representing the contents of the image.
Known object generation methods include a chroma key separation method using a studio set, etc., a method for generating a desired object by computer graphics (CG), and a method of extracting an object from a natural image.
The chroma key separation method adopts a process in which a uniform blue background called a blue back is prepared in a studio, an image including the blue back is photographed, and a desired object is extracted from the photographed image by removing the blue portion.
An essential feature of computer graphics resides in freely generating an image in any shape and, therefore, it is not necessary to pay a special attention to an extraction processing when computer graphics is used. Animation images can be processed in the same manner as CG images if each of cell images (images on celluloid sheets) is regarded as an object.
As a means for extracting an object from a natural image, the method of minimizing energy of an active contour model called “Snakes” (see, for example, Michael Kass, Andrew Witkin, and Demetri Terzopoulos, “Snakes: Active Contour Models”, International Journal of Computer Vision, Vol. 1, No. 3, pp. 321-331, 1988) is well known.
In snakes representation, an energy function which minimizes when a contour is extracted is defined and its minimal solution is obtained by iterative calculation from a suitable initial value. The energy function is defined as the linear sum of external energy provided as constraints on passing through edge points and internal energy provided as constraints on smoothness.
To use a snake, it is necessary to roughly designate a contour of an extraction target as an initial contour. In processing a moving picture, setting of an initial contour is required with respect to each frame. However, automatization of processing can be achieved by setting the results of extraction from the previous frame as initial values in the current frame. A technique for obtaining corresponding contour lines between frames is called tracking.
Each of the above-described extraction methods entails a problem difficult to solve. That is, the chroma key method requires a uniform background color and, hence, a large-scale studio set if a high extraction accuracy is specified. Also, if an object extraction target contains the background color, the target cannot be correctly extracted. Therefore the selection of colors of targets is restricted.
Computer graphics and animation can be used without extraction processing but have a fatal drawback of being incapable of processing natural images such as those in pictures taken by video cameras.
The methods of extracting objects from natural images have the advantage of reducing restrictions on image contents and having high versatility but have the drawback of requiring designation of initial contours with certain accuracy. This is because a minimal solution of an active contour model is largely influenced by an initial contour. That is, if an initial contour differs largely from the actual contour of an object, the convergent result of calculation does not match the actual contour of the object. Ordinarily, initial contour setting is made by a user using a graphical user interface (GUI) including a mouse or the like. The operability of the process of manually setting an initial contour is low and it is difficult to manually make a sufficiently reproducible initial contour setting. The burden on the user in manually setting an initial contour becomes larger as the shape of the contour becomes more complicated.
As a solution to this problem, a method for automatically generating an initial contour may be used. Use of information on movements between frames as well as information on colors and textures in frames is effective in automatically extracting an initial contour. To use such movement information, however, it is necessary that relative movements between the extraction target and the background be detected.
This problem will be described in more detail with reference to FIG.
1
.
FIG. 1
is a diagram schematically showing movements of objects between frames in a moving picture sequence. Frames F
1
to F
6
appear successively with passage of time. Frame F
1
is a start frame and frame F
6
is an end frame. Start frame F
1
and the next frame F
2
have completely the same internal configurations with no movement of the subject or the background. In this case, since the background and the object are equal in movement, the object cannot be extracted even if processing for obtaining the amount of movement between the frames is performed. Processing ends with the same result in a case where the camera is panned when both the background and the object are at rest.
If a frame remote from the start frame with respect to time, e.g., frame F
4
is referred to instead of the adjacent frame, the possibility of success in detecting a relative movement between the object and the background is increased. In such a case, however, it is difficult to recognize the correspondence between the images of the background and between the images of the object if the search range is not increased. This is because, if the time interval is increased, the possibility of increase of a change in shape is high. Consequently, the method of setting an initial contour by obtaining a movement from the start frame is rather low in versatility and entails difficulty in obtaining good results in subsequent tracking. Also, the method of setting an initial contour by obtaining a movement from the start frame does not always ensure good extraction results.
SUMMARY OF THE INVENTION
According to the above-mentioned background, an object of the present invention is to provide an image processing apparatus, which facilitates user operation and can extract objects more correctly from frames of moving image data, and an image processing method and a computer-readable storage medium on which a program for executing image processing is stored.
An image processing method according to a preferred embodiment of the present invention is characterized in that the method comprises: a selection step of selecting a particular frame from a plurality of frames of moving image data; a generation step of generating an initial contour corresponding to an extraction target in the frame selected in the selection step; a convergence step of converging the initial contour to the contour of the extraction target; and a control step of setting an initial contour corresponding to the extraction target in an other frame on the basis of the result of convergence in the convergence step, and effecting convergence to the contour of the extraction target on the basis of the set initial contour, wherein, in the control step, frames preceding and subsequent to the selected frame with respect to time can be set as the other frame.
Further, according to an another preferred embodiment of the present invention, there is provided an image processing method comprising: a generation step of generating an initial contour in each of a plurality of frames of moving image data; a selection step of selecting the initial contour in specific one of the frames from the initial contours generated in the frames in the gener
LandOfFree
Image processing apparatus, image processing method, and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing apparatus, image processing method, and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing apparatus, image processing method, and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3323182