Image analysis – Image segmentation
Reexamination Certificate
2000-12-26
2004-11-16
Wu, Jingge (Department: 2623)
Image analysis
Image segmentation
C382S166000, C358S464000
Reexamination Certificate
active
06819796
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a method of and an apparatus for segmenting a pixellated image into at least one foreground region and at least one background region. Such techniques may be used in the field of video compression in order to reduce the data rate and/or improve compression quality of foreground regions. Such techniques may also be used to compose new image sequences by replacing a segmented background with another background image or another sequence of background scenes. Further possible applications include video communication, video conferencing, television broadcasting, Internet multimedia applications, MPEG-4 applications, face detection applications and real time video tracking systems such as observer tracking autostereoscopic 3D displays. A specific application of such techniques is in digital video cameras and other digital image capture and recording devices for multimedia applications. An example of such a device is the Sharps® Internet ViewCam.
2. Description of the Related Art
Many known image processing and analysis applications involve image sequences which contain foreground objects, which are normally temporally active, and a background region, which is relatively static. Parts of the background scene may be covered and/or uncovered as the foreground objects move and/or change shape. It is very useful for these applications to have the capability to segment the images into foreground and background regions.
The Sharp® Corporation Internet ViewCam VN-EZ1 is an MPEG-4 digital recorder made for multimedia applications. This recorder enables computer users to incorporate moving pictures into their multimedia applications, such as home pages, Internet broadcasts, and e-mail communications. This recorder uses the MPEG-4 digital moving picture compression standard and Microsoft® Advanced Streaming Format to produce moving picture files that are small in size and thus more practical for Internet distribution. The video data are recorded onto SmartMedia™ memory cards, offering approximately one hour of recording time.
A successful segmentation, for example, would enable different compression techniques to be applied to the foreground and background regions. A higher compression ratio may then be achieved, enabling a longer recording time with an improved quality in the foreground regions. In addition, the background regions may be replaced with other scenes to produce a special effect to enhance attractiveness to consumers.
Earlier systems performed segmentation by using a carefully controlled background such as a uniformly coloured screen or a brightly illuminated backing behind the foreground objects. For example, U.S. Pat. No. 5,808,682 discloses a data compressing system which segments the foreground objects from a special background, which is illuminated uniformly by a known colour. Any colour may be used but blue has been the most popular. Therefore this type of coloured backing is often referred to as blue backing. The foreground objects can then be segmented using well known chroma key technology.
On large coloured backing, it is not a simple matter to achieve uniform illumination. U.S. Pat. No. 5,424,781 discloses a linear image compositing system which corrects for non-uniform luminance and/or colour of the coloured backing without incurring the penalties of edge glow, edge darkening, loss of edge detail and other anomalies.
For black-and-white images, it is known to use a controlled background so as to try to separate the foreground objects and the background scene into two different ranges of the grey scale. Typically the segmentation may be achieved by finding a deep valley in the histogram of the grey levels Nobuyuki Otsu “A threshold selection method from grey-level histograms”, IEEE Trans. on Systems, Man and Cybernetics, Vol. SME-9, No. 1, January 1979 pp. 62-66 discloses such a method to find an optimal threshold to segment the foreground objects from their background.
FIG. 1
of the accompanying drawings illustrates a histogram of this type in which h(t) represents the number of pixels and t represents the amplitude of the pixel values. The controlled background is such that the majority of the background pixels have relatively low levels whereas the foreground pixels have levels which tend to occupy a higher range. Otsu attempts to define a threshold T in the valley between the two ranges.
There are several problems with this technique, For example, although
FIG. 1
Indicates that a well-defined valley exists between the background and foreground grey level ranges, this is only the case for very carefully controlled backgrounds and possibly some but certainly not all foregrounds.
If this technique is not restricted to very carefully controlled conditions, then the problems become more severe. In particular, for many if not all images to be segmented, significant numbers of foreground pixels will have levels extending below the threshold whereas significant numbers of background pixels will have levels extending above the threshold. Thus, any threshold T which is chosen will lead to incorrect segmentation.
Another technique for segmenting an image is disclosed in T Fugimoto et al “A method for removing background regions from moving images”, SPIE vol. 1606 Visual communications and image processing 1991, imaging processing, pp. 599-606. This technique makes use of both the level and polarity of the pixel values in order to be resistant to lighting intensity fluctuations.
FIG. 2
of the accompanying drawings is a histogram with the same axes as
FIG. 1
but illustrating the effect of lighting intensity fluctuations. In the absence of such fluctuations, the distribution illustrated in the histogram has a narrow peak centred on the vertical axis with symmetrically sloping sides. When a lighting intensity fluctuation occurs, this peak becomes offset horizontally. The technique of Fugimoto et al is to derive asymmetrical positive and negative thresholds T
1
and T
2
by matching a Gaussian distribution to the actual position of the peak and simulating the remainder of the curve, which is assumed to represent foreground pixel levels, with a constant function. The intersection between the gaussian distribution and the constant function gives the threshold values T
1
and T
2
for the image being processed. It is then assumed that all pixel values between the thresholds represent noise.
This technique suffers from the same problems as Otsu. Although it may be resistant to lighting intensity fluctuations, the selection of the thresholds cannot be made in such a way that every image which is likely to be encountered will be correctly segmented.
U.S. Pat. No. 5,878,163 discloses an imaging target tracker and a method of determining thresholds that are used to optimally distinguish a target from its background. The target is assumed to occupy a gray level region which is identified from two histograms corresponding to the inner and outer regions of the target, respectively. Both histograms are recursively smoothed and a lookup table of actually observed pixel values is then computed. Two optimal thresholds are selected and are set at respective ends of histogram segments. The likelihood maps adapt over time to the signature of the target. The grey-level distribution of the target is used to select thresholds that pass a band of grey levels whose likelihood of their belonging to the target is high. It is not necessary for an accurate segmentation for this type of application.
While these methods may achieve reasonable results of segmentation for the desired applications and are usually computationally efficient, the requirement of having a carefully controlled background that can be distinguished from the target in either intensity or colour severely limits the range of the applications available.
A more challenging task is therefore how to segment the foreground objects from the background of a general scene. These methods often require the calculation of a difference image which characteri
Holliman Nicolas Steven
Hong Qi He
Renner , Otto, Boisselle & Sklar, LLP
Sharp Kabushiki Kaisha
Wu Jingge
LandOfFree
Method of and apparatus for segmenting a pixellated image does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method of and apparatus for segmenting a pixellated image, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method of and apparatus for segmenting a pixellated image will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3345770