Computer graphics processing and selective visual display system – Display peripheral interface input device
Reexamination Certificate
1999-05-24
2001-09-11
Luu, Matthew (Department: 2672)
Computer graphics processing and selective visual display system
Display peripheral interface input device
C345S111000, C345S421000, C348S586000, C348S650000, C382S163000, C382S167000
Reexamination Certificate
active
06288703
ABSTRACT:
BACKGROUND
Motion picture Travelling Matte
In the motion picture industry, the travelling matte process photographs a subject before a backing of uniform color and luminance. A matte signal is generated from the color differences between the colored backing and the subject. This matte signal permits removal of the colored backing, and the subsequent insertion of a different background.
The matte signal is simply a measure of the visibility of the background in the photographed scene. It can also be expressed as the contribution of the background to each pixel in the image frame. Since the background cannot be seen behind an opaque foreground subject, its contribution (matte) is zero. In all areas of the background not obscured by the subject, the background is fully visible, and the matte is therefore 1.0.
The matte is a fraction between 0.0 and 1.0, at the edge of moving or out-of-focus subjects where a semitransparent blur occurs, and for semitransparent and translucent objects such as glassware, smoke, or fog. Semitransparent subject areas contain a mix of the subject color and the color of the background since both occupy the same area. It is the background contribution that must be removed from these semitransparent areas. The matte signal is a measure of that contribution, and is used for its removal.
The above process has been well developed and is literally flawless for most subjects. See U.S. Pat. Nos. 4,100,569, 4,344,085; 4,625,231, 5,032,901, 5,343,252, and 5,424,781.
In both motion pictures and graphic arts, it is often necessary to select a foreground subject in an existing scene and remove its background, so that a different background may be inserted. The well developed matte equations for uniform colored backings do not apply to a background consisting of multiple colors and objects such as people, streets, trees, buildings and blue sky.
The development of a workable background extraction method for such complex backgrounds is described in U.S. patent applications Ser. Nos. 09/008,270 and 09/141,703. This extraction method employs a computer, cursor and monitor for manipulation of picture elements.
Application Ser. No. 09/008,270 is briefly reviewed here, since reference to this method will be made in the disclosure of the present invention. A boundary line is drawn on the background just outside the subject, on the original image, to isolate the background from the foreground subject. The isolated background is assigned a matte level of 1.0 since it is unobscured by the subject. A second boundary line is drawn just inside the subject to isolate the opaque subject, and this isolated subject area is assigned a matte level of zero. Between these two boundary lines is the transition area between subject and background. Semitransparent subject edges due to subject motion, or lack of sharp focus, are included in the transition area.
A background reference frame is computer generated by extrapolation and interpolation of the RGB levels of background pixels into and through the area occupied by the subject. The background reference frame is what one would have seen in the absence of the subject. This reference frame need be relatively accurate only through the transition area between the two outlines.
A foreground subject reference frame is generated by extrapolating the subject RGB levels near the subject edges to stretch the subject out through and beyond the outlined transition area.
Each pixel in the original image frame has a corresponding address in each of the two reference frames. These three frames each have separate colors for the same pixel address. These three colors can be shown as three points in a three dimensional color space as noted in
FIG. 2
of the referenced application Ser. No. 09/008,270. The position of the image pixel color with respect to the background and foreground reference frame colors determines the contribution (matte level) of the background to the image pixel.
The above method requires the drawing of a line on each side of the transition between subject and background. While this method worked very well for most subjects and backgrounds, consider a complex subject such as a leafy tree covering much of the image frame. The background shows blue sky, white clouds and mountains. Portions of the leaves are illuminated by the sun, and the sky peeks through between clumps of leaves.
There is no expedient way to separate foreground and background by drawing boundary lines. These kinds of busy subjects and backgrounds led to the invention of an alternative method, described below, in which the matte is generated from multiple conditional and Candidate mattes.
SUMMARY OF THE INVENTION
Each observed background color is identified to the computer by using a cursor to select the dominant colors. Each foreground color is also identified to the computer by using the cursor as a selector. For each pixel, several Candidate mattes are computed—one for each background color. Each Candidate matte is computed from a single background color and the set of selected foreground colors using any known method such as the method described in U.S. Pat. No. 5,355,174 (Mishima, Oct. 11, 1994). Of the several Candidate mattes, the greater of these Candidates becomes the Matte Signal for a given pixel. If this matte is 1.0, then the pixel is part of the background. If the matte is zero, then the pixel is part of the subject. If the matte is between 1.0 and 0.0, then the pixel is in a transition area and the contribution of the background to the pixel is that pixel's matte level.
A ‘Processed Foreground’ is generated by removing the contribution of the background from each pixel in the image, resulting in the subject appearing against a black field. The above generated Candidate mattes permit the generation of Processed Foreground Candidates. Of these Processed Foreground Candidates, the smallest is the final Processed Foreground for a given pixel.
REFERENCES:
patent: 4100569 (1978-07-01), Vlahos
patent: 4344085 (1982-08-01), Vlahos
patent: 4589013 (1986-05-01), Vlahos
patent: 4625231 (1986-11-01), Vlahos
patent: 5032901 (1991-07-01), Vlahos
patent: 5343252 (1994-08-01), Dadourian
patent: 5355174 (1994-10-01), Mishima
patent: 5424781 (1995-06-01), Vlahos
patent: 5469536 (1995-11-01), Blank
patent: 5519826 (1996-05-01), Harper
patent: 5907315 (1999-05-01), Vlahos
patent: 6134345 (2000-10-01), Berman
patent: 6134346 (2000-10-01), Berman
Berman Arie
Dadourian Arpag
Vlahos Paul
Blakely & Sokoloff, Taylor & Zafman
Luu Matthew
Sajous Wesner
Ultimatte Corporation
LandOfFree
Method for removing from an image the background surrounding... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for removing from an image the background surrounding..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for removing from an image the background surrounding... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2439944