Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
2000-03-06
2004-01-20
Couso, Yon J. (Department: 2723)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S170000
Reexamination Certificate
active
06681058
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention concerns image processing methods and apparatus and, in particular, a method and apparatus for processing a sequence of images to recover images of unchanging portions of the scene that are partially obscured by moving objects in the scene.
There are many applications in which it is desirable to identify unchanging portions of a sequence of video images. These include data compression systems, surveillance systems and image compositing systems. The unchanging portions of a sequence of images represents redundant information in all of the images except for the first image in which they appear. Sequences of video images often include a considerable amount of redundant temporal information. For example, in a television news broadcast the background and even much of the news announcer remains substantially the same from frame to frame. In a video surveillance system, the image is substantially the same from frame-to-frame except for possible intruders.
This temporal redundancy may be used to advantage when processing a sequence of images. For example, in image data compression systems, if the static portions of a sequence of images can be identified, they may be transmitted separately from the changing portions of the image, thus reducing the total bandwidth needed to transmit the sequence of images. In a surveillance system, the unchanging portions of the image may be subtracted from newly received images to provide an output image that contains only the inconsistencies in the newly received images with respect to the preceding background images.
Depending on the application, however, it may be difficult to identify the unchanging or background portions of the sequence of images. This difficulty occurs when foreground objects frequently obscure portions of the background image. One example of such in application is a traffic monitoring system. An exemplary system of this type may produce an image such as the one shown in FIG.
1
. The image includes three automobiles
110
,
112
, and
114
travelling along a road
116
. In order to monitor the traffic flow within the field of view of the image
100
, it is desirable for the traffic monitoring system to recover an image of the road
116
without the cars
110
,
112
, and
114
. The image of the road
116
is not constant, it changes over the course of the day, for example, with changes in solar illumination. Although the traffic monitoring system may need to obtain an image of the road in the absence of the automobiles, this may be difficult to do, especially if the road is very heavily traveled, as the image of the road will be visible only between the times that the cars are visible.
One solution to this problem is to pre-store images of the road surface corresponding to different times of day and different weather conditions. This method, however, is subject to error in the selection of the appropriate background image and may require the storage of a relatively large number of images in order to cover all times of day in all seasons of the year.
SUMMARY OF THE INVENTION
The present invention is embodied in a method and apparatus which robustly estimates the brightness or other feature values of an image region in a sequence of video images even when the region is obscured by objects over large portions of the video sequence. In accordance with the present invention, a histogram is generated for each image region over a plurality of image frames in the sequence. The mode, or most frequently occurring value, of each region as indicated by the histogram is selected as representing the unchanging portion of the image. The mode values of all of the pixels in the regions are then assembled to form a background image.
According to one aspect of the invention, the histogram is generated using a recursive filter.
According to another aspect of the invention, the images in the video sequence are aligned before generating the histogram.
According to another aspect of the invention, each image frame is filtered prior to the histogramming operation to remove image artifacts caused by the camera.
According to yet another aspect of the invention, the region of the image is processed at regular intervals, skipping interstitial frames, in order to obtain the histogram.
According to yet another aspect of the invention, the region of the image is processed at random irregular intervals in order to obtain the histogram.
According to yet another aspect of the invention, the histogram is applied over a small number of frames in order to generate a noise reduced image.
REFERENCES:
patent: 5635982 (1997-06-01), Zhang et al.
patent: 5649032 (1997-07-01), Burt et al.
patent: 5748775 (1998-05-01), Tsuchikawa et al.
patent: 5847755 (1998-12-01), Wixson et al.
patent: 5872867 (1999-02-01), Bergen
patent: WO9716921 (1997-05-01), None
patent: WO9821688 (1998-05-01), None
K. Hanna and N. Okamoto, Combining Stereo and Motion Analysis for Direct Estimation of Screen Structure, IEEE Computer Society, May 1993, pp. 357-365.
K. Hanna, “Direct Multi-Resolution Estimation of EGO-Motion and Structure from Motion,” IEEE Computer Society, Oct. 1991, pp. 156-162.
J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, Hierarchical Model-Based Motion Estimation, pp. 1-21.
A. Pope et al.; Video Abstraction: Summarizing Video Content for Retrieval and Visualization, 1998, IEEE, pp. 915-919.
M. Irani et al.; Video Compressin Using Mosaic Representations; Signal Processing, Image Communication, NL, Elsevier Science Publishers, Amsterdam, vol. 7, No. 4, Nov. 1, 1995, pp. 529-552.
European Search Report, Jun. 13, 2001.
Hanna Keith
Kumar Rakesh
Burke William J.
Couso Yon J.
Sarnoff Corporation
LandOfFree
Method and apparatus for estimating feature values in a... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for estimating feature values in a..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for estimating feature values in a... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3250187