Television – Image signal processing circuitry specific to television – Special effects
Reexamination Certificate
1997-07-02
2001-10-16
Eisenzopf, Reinhard J. (Department: 2614)
Television
Image signal processing circuitry specific to television
Special effects
C348S590000, C348S586000, C348S140000, C348S025000, C348S700000
Reexamination Certificate
active
06304298
ABSTRACT:
The present invention relates to methods and apparatus for creating virtual images and for determining the relative position of a TV camera.
Cross Reference to Related Applications
This application was filed under 35 U.S.C. §371 and claims priority to PCT application Ser. No. PCT/GB/02227 filed on Sep. 9, 1996. This application also claims priority under 35 U.S.C. §119 to United Kingdom Pat. application Ser. No. 9518432.1 filed on Sep. 8, 1995.
Chroma Key panels are known for use in TV studios. By focusing a TV camera onto a chroma-key background (or panel) and positioning a foreground object in front of the panel a combined picture can be created in which the foreground object appears against a virtual background which can be, for example, a still picture or a video sequence.
A problem which arises from this basic technique is that the camera cannot be allowed to move because the virtual background and the foreground object (possibly a TV presenter) will not move synchronously as in real life.
In JP 57-93788 a chroma-key panel is used which includes a series of equidistant parallel lines,
FIG. 11
, of two different shades of backing colour to monitor any changes in zoom which are manifested as changes in the frequency of the video signal. The boundaries of a chroma-key window are detected in order to fit the inserted image in size and position to the chroma-key window.
Perspective can be solved by using a two shade pattern with characteristic features etc. Such features may include characters, symbols, vertices of polygons etc. Whenever at least the image features can be matched with the physical pattern the perspective can be solved.
For the purpose of the present invention, the description will generally be confined to the use of a TV camera within a virtual studio but it is to be understood that the invention can be used for general tracking of a TV camera or an object on which it is positioned.
In co-pending Israeli Patent Application No. 109,487 to the same applicant, the use of chroma-key patterned panels is disclosed. These panels have a defined pattern which allows the video signals generated by the TV camera to be processed to ascertain the position of the camera.
A problem which arises in the above prior art systems is that for large zoom in factors the features in the Field of View (FOV) are reduced in number. Also, for a substantial occlusion the recognition of robust features may be difficult. Since the present invention movement and zoom of the camera are permitted and also the foreground object is allowed to move, these circumstances are very likely to occur.
In addition large perspective distortion makes the recognition of features very difficult in particular when said features comprise characters, graphical symbols etc.
If the camera loses synchronism between the foreground real object and the virtual background then the effect will be a loss of reality in the composite picture. Thus, as explained above, early previous systems were limited to a static camera and the later systems, although allowing camera movement, may still be subjected to a loss of synchronism between foreground and background.
Obviously if none of the patterned chroma-key background is visible then synchronism cannot be maintained but also is not necessary since no virtual background will be shown.
As the camera zooms in to the foreground object, the background chroma-key panel will become more occluded by the foreground object and the characteristic pattern will be broken and/or distorted in the case of large perspective views.
It is an object of the present invention to provide a TV camera position determination apparatus and method for measuring the position of a TV camera relative to a panel when part of the panel is occluded by a foreground object.
It is also an object of the present invention to provide a virtual studio system in which the TV camera is able to be moved laterally with respect to a foreground object and to a background chroma-key panel; in which the camera is able to zoom in and out with respect to the foreground object without losing synchronism between the foreground object and the virtual background even when the chroma-key panel is substantially completely occluded by the foreground object.
It is also a further object of the present invention to provide a camera positioning apparatus in which the position of a TV camera relative to a patterned panel can be determined even when a substantial part of the panel is obscured by an occluding object.
The present invention therefore provides a method of determining the position of a TV camera relative to a patterned panel being viewed by the TV camera including the steps of identifying a plurality of edge points of the pattern from the video signal produced by said camera and using these edge points to calculate the perspective of the pattern relative to the camera.
Preferably the method comprises the steps of identifying a plurality of said first edge points and a plurality of said second points; and producing an edge image.
Preferably two or more families of edges are used such that the edges of each family lie on a set of parallel lines comprising at least two lines. Preferably the orientations of the families are sufficiently far apart such that an edge point can be assigned to a specific family by means of its orientation only.
In a specific embodiment said patterned panel comprises a pattern of vertical and horizontal straight edges defining lines delineating a colour difference and in which each edge point is situated on one of said horizontal or vertical straight lines.
In a first embodiment said plurality of first edge points are clustered to associate edge points to specific lines using a slope and intercept process.
In a second embodiment steps of processing the video signal relating to said first and said second plurality of edge points comprise the steps of analysing all detected edge points and grouping together edge point into a first plurality of groups corresponding to horizontal lines and a second plurality of group corresponding to vertical lines.
Preferably the edge points in the first and second plurality of groups are allocated preliminarily to specific horizontal and vertical lines.
Preferably the step of allocation is followed by computation of the vanishing points of the horizontal and vertical lines, said vanishing points being computed within a defined location error.
The perspective projection of any set of parallel lines which are not parallel to the image plane, will converge to a vanishing point. In the singular case where the lines are parallel to the image plane, the vanishing point is at infinity.
Preferably the method also includes the step of projecting the edges corresponding to horizontal edges to obtain an edge projection profile map comprising peaks and troughs.
Preferably in the projection process a vertical accumulator array H[y] is cleared to zero. Then for each horizontal edge, the line connecting the vanishing point (previously computed for horizontal edges) with the edge is computed. That line is then intersected with the vertical axis (x=0). The cell of the accumulator array which corresponds to the intersection point is then incremented. Peaks in that array correspond to candidate lines.
Preferably the method further includes the step of assigning each horizontal edge to a most probable peak and producing a list of edges for each of a plurality of candidate lines indicated by the peak.
Preferably a line is specified for each list of edges, edges not corresponding to any specified line being disregarded.
The method steps are then preferably repeated for vertical edges and lines.
In the method an accurate video image edge line pattern is produced and in which the known pattern on the panel is compared with the edge line pattern.
This comparison preferably comprises a first step of identifying a first horizontal line in the accurate video image edge pattern, identifying a second horizontal line in the accurate video image pattern, calculating the distance bet
Aufhauser David
Livshits Zinovy
Nissim Moshe
Sharir Avi
Steinberg Alexander
Eisenzopf Reinhard J.
Lo Linus H.
Orad Hi Tec Systems Limited
Woodcock Washburn Kurtz Mackiewicz & Norris LLP
LandOfFree
Method and apparatus for determining the position of a TV... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for determining the position of a TV..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for determining the position of a TV... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2591656