Computer graphics processing and selective visual display system – Computer graphics processing – Graphic manipulation
Reexamination Certificate
1997-10-15
2003-03-11
Nguyen, Thu (Department: 3661)
Computer graphics processing and selective visual display system
Computer graphics processing
Graphic manipulation
C382S283000
Reexamination Certificate
active
06532022
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of Invention
The present invention relates generally to methods and apparatus for compositing objects from an input image to a destination image. More particularly, the present invention relates to reducing extraneous clutter and holes from the object being composited and, in addition, reducing the effect of shadows emanating from the composited object in the destination image.
2. Background
An important procedure in the field of film and video editing is taking an object in one image, e.g. an input image, and compositing that object onto another image, e.g. a destination image, with minimal distortion of the object. For example, this procedure could be used for taking an object such as a human figure in one video and placing that human figure into another video without distorting or altering the image of the person. One video image may have a person standing or moving around in a typical living room and another video can be of an outdoor scene such as a jungle or desert. The compositing procedure would take the image of the human figure in the living room and place the figure in the other video thereby providing the effect of the human figure standing or moving around in the given outdoor setting.
One well-known and widely used prior art method for compositing objects from a input image to a destination image is from the field of digital effects and chroma-keying. This method is commonly referred to as blue-screening and involves placing a blue or other fixed-color screen behind the object being composited, typically the image of a person (the color blue is a hue that strongly contrasts all colors of human skin). In blue-screening, the system checks to see which pixels in the input image are not blue and labels those pixels as foreground pixels. Normally, the foreground pixels will only be those pixels that are part of the object being composited since there are typically no other objects in the image and the background is solid blue. The system then composites, or blends, the object (i.e. collection of all foreground pixels) onto a destination image. One of the disadvantages of using blue-screening for object compositing is that it requires a fixed color screen behind the object. Another disadvantage is that if any of the colors on the object, such as an item of clothing, is blue, holes will appear in the object in the destination image. This occurs because the pixels in the blue areas on the object will not be labeled as foreground pixels and thus will not be composited with the rest of the object, resulting in the object having holes when composited onto the destination image.
Other prior art background subtraction procedures, from the field of computer vision, are used to eliminate the fixed color screen requirement. One procedure involves building an average background image by taking a predetermined number of sample images of a (multi-colored) background and creating a background model. For each new sample image taken, each pixel in the new image is compared to its corresponding pixel in the background model being formed. This is done to determine whether the pixel in the current sample image is a foreground pixel, i.e. an object pixel. Pixels that are determined to be part of the foreground are then blended or composited onto the destination image. One disadvantage with this procedure is if a foreground pixel happens to match its corresponding background model pixel color, it will not be considered a foreground pixel. This will introduce holes into the composited object. Another disadvantage is that shadows cast by the object often make the object, when composited, appear to have its original form plus extraneous appendages (as a result of the shadows). The procedure mistakenly labels the “shadow” pixels as foreground pixels. Yet another disadvantage is if any portion of the background changes or if the camera is moved while the background model is being built, certain portions of the background (e.g. the portions that moved) will be incorrectly labeled as part of the foreground and be composited onto the destination image. Although there are prior art techniques for updating the background model to reflect changes, they cannot account for a constantly changing background such as one that includes a changing television screen or a window looking out onto a busy street.
Another prior art method of compositing objects, taken from the field of computer vision and z-keying, involves the use of stereo cameras. The method involves calculating, or extracting, a depth value for each pixel. Pixels that are closer than a certain depth from the camera are labeled as foreground pixels and eventually composited onto a destination image. The image processing algorithms involved in computing the depth values in real time require immense computation making them impractical to implement on typical home personal computers. In addition, the procedure requires the use of two cameras.
An important sub-function of the broader procedure of compositing objects from an input image to a destination image is reducing the effect of shadows emanating from the object in the input image in the compositing procedure. For example, if the object is a person standing in a room in which the lighting causes the person to cast shadows on the floors or walls around him, the goal is to reduce the effect of the shadow in the destination image (i.e., the shadow should not appear as part of the person in the destination image). The procedure should determine which pixels belong to the actual object and which pixels make up a shadow.
One prior art method of reducing the effect of shadows in the destination image, referred to as intensity-based shadow filtering, involves building an average model of the background image. Once this average background image is entered, the system knows the approximate brightness of each pixel in the background. Thus, if a pixel becomes somewhat darker in the input image, the system assumes that the pixel is now within a shadow. However, if the pixel in the average background image is now “covered” by the object being composited and also happens to be darker than the “covered” pixel, the same method will create a hole in the object once composited onto the destination image (the size of the hole depending on the number of pixels that are darker and are part of the object). Thus, the problem with the prior art method of reducing the undesirable effect of shadows in the destination image is that the shadow removal process itself may create more holes in the body of the object being composited.
Therefore, it would be desirable to have a method and apparatus for compositing objects from an input image to a destination image such that the object is composited with the least amount of distortion from shadows or a constantly changing background, and has a reduced number of holes and gaps after composited onto the destination image.
SUMMARY OF THE INVENTION
The present invention provides an object compositing system for compositing an object from an input image onto a destination image. In a preferred embodiment, the object is composited from an image having an arbitrary or non-uniform colored background containing some non-static elements onto a destination image with minimum effects from shadows cast by the object and with minimum gaps or holes within the object. Various improvements in the compositing procedure such as shadow reduction and hole filling, and less restrictive requirements regarding the object's surroundings are described herein.
In a preferred embodiment, an object compositing method of extracting an object from an image model and blending the object onto a destination image is described. A background model is created by examining several frames of an average background image before the object being composited enters the image. A frame of the input image containing the object is obtained after the background image model has been created. An alpha image is created in which each pixel ranges from “0” indicating it is not part of the
Electric Planet, Inc.
Nguyen Thu
Van Pelt & Yi LLP
LandOfFree
Method and apparatus for model-based compositing does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for model-based compositing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for model-based compositing will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3063000