Television – Special applications – Object or scene measurement
Reexamination Certificate
1997-04-01
2001-02-20
Peng, John K. (Department: 2714)
Television
Special applications
Object or scene measurement
C348S587000, C348S586000, C348S135000, C348S136000, C348S137000, C348S142000, C348S169000
Reexamination Certificate
active
06191812
ABSTRACT:
FIELD AND BACKGROUND OF THE INVENTION
The present invention relates to video compositing and, more particularly, to a method for providing a background pattern so that the position and orientation of a moving camera can be determined in real time for the realistic compositing of images of a foreground object with a virtual background.
The technique of “chroma-key” compositing has long been used in video production to combine images of a foreground object with a virtual background stored in a digital data base. In this technique, the foreground object is photographed against a background of a “chroma-key” color, typically a particular shade of blue. In the digital images thus captured, all the pixels colored with the chroma-key color are replaced with pixels derived from the virtual background. In order for the background pixels to be rendered in a way that provides a realistic composite image, the position and orientation of the camera must be known.
Several method are known in the art for determining the position and orientation of the camera. These methods can be classified in two categories. In methods of the first category, the position and orientation of the camera are tracked explicitly. In these methods, the camera must be provided with special tracking devices that are rigidly attached thereto. In some of the methods of the first category, the tracking devices are encoders, and lateral movement is tracked by encoders that are in physical contact with the floor of the studio. Another example of the methods of the first category, a magnetic tracker system, is taught by Loftus et al., in PCT application no. US96/04846. Loftus et al.'s special tracking device is a magnetic tracker receiver rigidly attached to the camera. In the methods of the other category, the position and orientation of the camera are inferred from the captured images. For example, Graham, in U.S. Pat. No. 5,502,482, which is incorporated by reference for all purposes as if fully set forth herein, teaches the use of a chroma-key background of two or more shades of blue in a predetermined pattern, for example a checkerboard pattern. The position and orientation of the camera are inferred from the locations of features of the pattern on the captured images.
Methods such as Graham's have the advantage of allowing the use of conventional video cameras, without special tracking equipment. This is particularly convenient for video compositing with hand-held cameras, to avoid the necessity of contacting lateral motion encoders with the floor of the studio, or to avoid the extra bulk and weight of tracking equipment such as that of Loftus et al. Nevertheless, these methods have limitations of their own. One limitation is that the pattern is fixed, in both level of detail and location. If the camera zooms in too closely on the foreground object, there may be too few features in the portion of the image occupying the camera's field of view for the location and orientation of the camera to be determined accurately. Conversely, if the camera is too far from the background pattern, the pattern features may be sufficiently crowded to make it difficult to distinguish between a panning motion of the camera and a lateral translation. In principle, the pattern can be provided with multiple levels of detail, to preserve tracking resolution at all required distances; but this adds to the complexity of the pattern. Similarly, if the foreground object is a live actor who moves laterally with respect to the pattern, and the camera follows the actor, the actor may move so far laterally that not enough of the pattern is left in the camera's field of view to allow the camera's location and orientation to be determined accuracy. A second limitation is that the multiplication of chroma-key colors has been found to degrade the realism of the composite image. The narrower the spectral band of the pixels that are replaced with virtual background pixels, the more realistic the resulting composite image. The loss of realism associated with the use of a multi-shade chroma-key background persists even while the camera is in a fixed position and only one chroma-key color would suffice. A third limitation is that the pattern is fixed in place in the studio, typically being painted on a wall or a fixed partition. This limits the flexibility of the method and precludes its use, for example, in remote locations.
There is thus a widely recognized need for, and it would be highly advantageous to have, a method for providing an image for dynamic chroma-key compositing that is free of the limitations of the methods known in the art.
SUMMARY OF THE INVENTION
According to the present invention there is provided an improved video production method of the type in which a plurality of images of an object in a foreground volume are captured successively by a camera and superposed on a virtual background, the position and orientation of the camera being determined using a first pattern featuring a first scale length and located at least partially beyond the object with respect to the camera, the improvement including the steps of: (a) positioning a screen at least partially beyond the object with respect to the camera; and (b) projecting the first pattern onto the screen.
Simply stated, according to the present invention, the pattern is projected on a screen behind the foreground object. By “screen” is meant herein any suitable flat or curved surface on which the pattern may be projected, although this surface preferably is flat. As is noted below, the screen may be either opaque or transparent. The screen and the projector may be easily configured to be portable, enabling the method to be used in remote locations. In one preferred embodiment, the two colors of the pattern are two shades of a base color, with the screen being colored with the base color and the pattern being provided by projecting white light on the screen through a mask, the shadow of the mask thus creating an area on the screen characterized by a darker shade of the base color than the part of the screen that is illuminated by the white light. If the camera is stationary for a prolonged period of time, or if a scene is shot in which a hand-held camera is not needed, so that a camera equipped with one of the special explicit tracking devices described above may be used, or if a scene is shot in which the camera need not move at all, the projector is simply turned off. The color of the screen then is used as a single chroma-key color, preserving the enhanced realism of the composite image that is provided by the use of only one chroma-key color.
Multiple projectors are used to project multiple patterns on the screen. In this way, a pattern having multiple levels of detail is emulated. For example, if the camera moves so close to the screen that the level of detail of the first pattern is lost, a second pattern having a higher level of detail is superposed on the first pattern. Similarly, if the foreground object moves laterally, a second pattern is projected to the side of the first pattern, to provide continuous lateral coverage.
REFERENCES:
patent: 2651233 (1953-09-01), Tondreau et al.
patent: 2753395 (1956-07-01), Lawrence
patent: 2870672 (1959-01-01), Gate et al.
patent: 2886636 (1959-05-01), Grace
patent: 2969428 (1961-01-01), Wittlig
patent: 4393394 (1983-07-01), McCoy
patent: 4394680 (1983-07-01), Watanabe
patent: 4412121 (1983-10-01), Kremers et al.
patent: 4621292 (1986-11-01), Hirao et al.
patent: 4623237 (1986-11-01), Kaneda et al.
patent: 4771307 (1988-09-01), Kuno et al.
patent: 4965840 (1990-10-01), Subbarao
patent: 5117283 (1992-05-01), Kroos et al.
patent: 5223891 (1993-06-01), Fierstein et al.
patent: 5231443 (1993-07-01), Subbarao
patent: 5434617 (1995-07-01), Bianchi
patent: 5502482 (1996-03-01), Graham
patent: 5694203 (1997-12-01), Ogawa
patent: 5696892 (1997-12-01), Redmann et al.
patent: 5754225 (1998-05-01), Naganuma
patent: 5764291 (1998-06-01), Fullam
Tzidon Aviv
Tzidon Dekel
Desir Jean W.
Peng John K.
RT-SET Ltd.
LandOfFree
Method of providing background patterns for camera tracking does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method of providing background patterns for camera tracking, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method of providing background patterns for camera tracking will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2584014