Computer graphics processing and selective visual display system – Computer graphics processing – Attributes
Reexamination Certificate
1999-05-28
2002-04-23
Luu, Matthew (Department: 2672)
Computer graphics processing and selective visual display system
Computer graphics processing
Attributes
C345S590000, C345S592000, C345S620000, C345S626000, C348S584000, C348S578000, C382S162000
Reexamination Certificate
active
06377269
ABSTRACT:
BACKGROUND
1. Field of the Invention
The present invention relates generally to the field of photography, and in particular to the manipulation of multiple images in order to isolate a particular subject of those images for use in electronic photo-compositing. More specifically, the present invention relates to a method for electronically processing multiple images of a foreground subject taken on different backgrounds in order to generate a new image of the foreground subject completely removed from the backgrounds, said new image of foreground subject being true in color and transparency for use in photo-realistic, composite images.
2. Description of the Related Art
A number of existing methods allow users of electronic imaging systems to extract a foreground subject from a scene in an image, isolating said subject from the background of the scene so that the subject may later be composited into a different scene in another image. However, existing methods have a number of deficiencies, including being mathematically inaccurate at determining correct colors and transparencies, requiring excessive user skill and attention, requiring costly, colored background materials, and generally producing results that are only marginally acceptable in many cases.
A first existing method requires the foreground subject to be photographed in front of a uniformly colored background so that a “chroma key” method or “blue screen” method may be used to isolate the foreground subject from the uniformly colored background. An example of this method is described in U.S. Pat. No. 5,424,781. This method can yield acceptable results, but it commonly produces resulting images with fringes of color around the foreground subject's edges, poor reproduction of small features, inaccurate color values in transparent areas of the subject, and it typically requires a high level of user skill and attention. In addition, a carefully chosen background color must be used because if the chosen background color appears anywhere in the foreground subject, this method will also assume that these colored areas are part of the background and will incorrectly treat these areas as transparency in the resulting mask even though the foreground subject may have been that color and fully opaque at that point.
A second existing method uses two images, a first image of a foreground subject on a background and a second image of the said background alone. In this method, a difference image is calculated from the first and second image to be reapplied to the first image to isolate the foreground subject from the background. Unfortunately, this method suffers from some of the same problems as the “blue screen” method above. If the foreground subject contains a color at a location in the image that is similar to the color of the background at that same location in the image, this method will interpret the color at this location as part of the background and will incorrectly assume it is the background, resulting in transparency in the mask even though the foreground subject may have been fully opaque at that point.
A third existing method allows the subject to be photographed on any background and electronically “cut” out of the background using software tools available in such digital image editing programs as Adobe Photoshop. However, if the foreground subject has more than a small amount of detail, it usually becomes a time-consuming and difficult process that yields poor results with high-contrast edges that do not blend smoothly in composite photographs.
SUMMARY OF THE INVENTION
The present invention is directed to overcoming the problems set forth above. This invention is a new and unique method for generating masks required for electronic photo-compositing of multiple images. The method described herein uses two images of a foreground subject to calculate an accurate mask, each image captured from the exact same viewpoint but each image having a unique background. By using two images of the same foreground subject on at least two different backgrounds, it is possible to accurately calculate the exact mathematical solution to the amount-of-transparency problem. In fact, this method can accurately calculate the transparency as multiple color channels of transparency, rather than as a single monochromatic channel as all existing methods do. This allows reproduction of the true color filter effects of the transparent areas of the foreground subject.
It is necessary to set realistic goals for the functionality of the present invention. Inconsistent lighting, automatic exposure cameras, and camera noise will always interfere with an ideal, fully automated solution, but it is possible to automate the most difficult part of generating a mask with edges that are free of fringes and artifacts. The resulting image and mask can be used in many popular imaging and animation software packages, usually with little or no additional editing required.
It is a goal of the present invention to provide a 100% true and accurate solution to the amount-of-transparency problem. Photographers and designers want to minimize the destructive effects of any image manipulation in order to maintain the original image fidelity. All previous methods used to generate masks have deficiencies such as hard edges, edges with fringes of unwanted colors, the inability to produce semi-transparent areas of the proper colors, and the inability to produce color filter effects. These deficiencies hinder the reuse of the resulting image because the image typically has visible artifacts that are unacceptable on all but a select number of backgrounds that minimize the visible artifacts. As the accuracy and fidelity of the masked image approaches the true transparency values, the visible artifacts are reduced or eliminated and the ability to reuse the image in multiple ways increases dramatically. With no visible artifacts or flaws, the image may be re-composited with any choice of backgrounds, or used in ways that other masking methods would not allow. This reuse has many benefits. The need to take and store multiple images, each tailored or made suitable for a particular application or use, is eliminated. It gives the designer greater freedom of design and the ability to make rapid design changes. It also reduces the amount of film and prints and the number of digitized images which must be stored, which in turn, saves money and resources.
It is a further goal of the present invention to solve the most difficult parts of mask generation in an automated way, drastically reducing the amount of editing required before the image can be used. It is very likely that future hardware and software improvements will provide the operator with even more advanced tools than are currently available to improve the software-generated mask.
It is a further goal of the present invention to minimize user effort. With a goal of reducing the overall time and skilled effort required to produce a finished image and mask, the entire production process was examined, from photography to final image placement. Current blue/green screen techniques and other image editing tools only focus on the post-processing, or editing stage that occurs after the images have been captured and digitally transferred into a computer. Understanding that additional information can be gathered at the photography stage that would virtually eliminate the need for post-processing or editing, the method described herein substitutes a small additional level of effort during photography to greatly reduce the amount of skilled effort required during post production. The result is a method that requires two unique photographs to be taken during the photography stage, but that requires little or no image editing before the resulting image with transparency mask can be used. Overall, the total amount of user effort, from photography to final image placement, is drastically reduced. Even though this method requires the capture of one additional image, the process of acquiring this additional image is rapid and does not pla
Brandenberg Carl B.
Kay Robert L.
Hunn Mel A.
Luu Matthew
Sajous Wesner
LandOfFree
Automated generation of masks for photo-compositing does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Automated generation of masks for photo-compositing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Automated generation of masks for photo-compositing will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2820221