Image analysis – Image transformation or preprocessing – Measuring image properties
Reexamination Certificate
1999-11-30
2003-04-29
Couso, Yon J. (Department: 2723)
Image analysis
Image transformation or preprocessing
Measuring image properties
C382S151000, C348S094000
Reexamination Certificate
active
06556722
ABSTRACT:
The present invention relates to position determination and is particularly, but not exclusively, concerned with derivation of camera position for so-called “virtual studio” television production. In such applications, a virtual scene, for example a computer-generated background, is superimposed onto a real scene, for example actors. It is important to know the exact position and orientation of the television camera in the studio, so that the virtual scene can be computed to correspond to the viewpoint of the real camera, to ensure correct registration between real and virtual elements in the scene.
There are a number of commercially-available systems that can provide camera position information. Most are based on mechanical camera mountings, such as robotic pedestals, pedestals mounted on tracks, or robot arms, which have sensors to measure position and orientation. These systems cannot be used with hand-held cameras, and can be bulky and difficult to use.
There are also methods that work without mechanical sensors, but instead use a patterned blue background, which is visible in the camera image. By analysing the video signal, these methods can deduce the orientation and position of the camera. One example of such a method is described in our earlier GB-A-2271241, which derives pan, tilt and zoom using an arbitrary patterned background. A further example using the same technique is described in WO-A-95/30312, which uses a particular type of pattern to enable pan, tilt, zoom and position to be determined. However, these methods rely on the presence of a two-tone blue background. This is inappropriate in some situations, for example when it is desired to extract shadows of objects from the foreground image. Also, in some situations there may be little or no blue background visible, for example during a close-up shot of an actor. Sometimes it may also be required to place a virtual object against a real background, in which case there will be no blue background at all.
WO-A-9711386 discloses apparatus for determining position and orientation in which a camera is pointed at an optically modulated target.
EP-A-706105 discloses a navigation system for an autonomous mobile robot in which coded signs are placed at various locations. The markers are distinguished based on the ratio of radii of rings.
A method for locating the position of a head-mounted display that has been used in the field of Augmented Reality is described in the paper by Azuma et al entitled “A Demonstrated Optical Tracker with Scalable Work Area for Head-Mounted Display Systems” published in ACM Computer Graphics: Proceedings of the 1992 Symposium on Interactive 3D Graphics (Cambridge, Mass., April 1992), pp. 43-52. This method uses a number of infra-red LEDs mounted in ceiling panels, viewed by four upward-looking sensors mounted on the user's headset. The sensors each provide information of the co-ordinates of bright points in the image (the LEDs) and from these co-ordinates, the known position of the LEDs and the geometry of the sensors, the position of the headset is computed. The inventors contemplated applying Azuma's method to the problem of determination of camera position. However, Azuma's method is not intended to be used for determining the position of a camera in a large studio, but is designed instead to work in a relatively small volume, where the total number of LEDs is small. The inventors identified several potential problems in applying such a technique to the field of camera position determination.
In a television studio, the camera could move many tens of meters, so the method of Azuma et al would require a very large number of ceiling markers of which only a small proportion would be visible at any one time. Azuma's method relies on active control of the LEDs to identify the LEDs. The control electronics and wiring required to implement this in a large studio would be complex and impracticable. Furthermore, the ceiling of a television studio generally contains a number of lights and other objects that could make identifying the markers much more difficult than in the carefully controlled conditions in which Azuma's method is designed to function; the bright spot sensors used by Azuma would generate spurious signals. A still further problem the inventors have identified is that the set of markers that the camera can see will change, not only due to markers coming into and out of the field of view as the camera moves, but also due to markers being obscured by objects such as microphone booms and studio lights. Using Azuma's method, each time a marker appears or disappears, there is likely to be a small but sudden change in the computed position and orientation of the camera, since any errors in the camera calibration or the measurement of the marker positions will lead to results that depend on which markers are being used.
Thus, Azuma's method cannot be directly applied to the present problem. The inventors have developed novel techniques which are particularly suited to the exacting requirements of determining camera position in a television studio, which overcome or alleviate the drawbacks of conventional techniques. References in this specification to a video camera are intended to include any camera capable of obtaining video images; although the video camera is advantageously a conventional television studio camera, no limitation to a particular type of camera or intended use is implied.
In a first aspect, the present invention provides a method of determining the position of an object comprising:
providing a plurality of markers at respective reference positions, at least some of the markers being patterned to encode identification information, the pattern providing a reference point for each marker;
storing information including a measure of the positions of the markers and information identifying the patterned markers;
obtaining an image of at least a sub-set of said plurality of markers from a camera associated with the object;
processing the image to identify the positions of said markers in the image and, for each patterned marker in the image, decoding said identification information;
determining a measure of the position of the object based on said processing and decoding and based on said stored information.
Thus, with the invention, at least some, and preferably all, markers are patterned to encode information identifying the marker. In this way, it becomes possible to determine not only relative movements of the object but also the absolute position by “looking up” the positions of each patterned marker. This may enable much greater freedom of movement of the object over a wider area whilst allowing absolute position to be measured accurately. In addition, determination of relative movement may be simplified or improved.
Preferably, the pattern comprises concentric shapes, preferably substantially closed rings; the use of concentric rings to encode the information facilitates location of the markers, as the centre of the rings can conveniently be located and provides a convenient reference position. Preferably, identifying the positions of the markers includes identifying the centres of the concentric rings as a reference point for each patterned marker. Although the rings are most preferably concentric to within the resolution of the camera means in use as this greatly facilitates identification and decoding, they need not be exactly concentric; in such a case, eccentricity may be used to provide a measure of angular orientation, and the reference point may be determined based on, for example, the centre of the innermost ring.
Preferably, the patterned markers encode information as a series of light and dark regions. This enables a monochrome camera to be used to identify the markers, allowing higher resolution to be obtained at 756° lowest. The camera may operate in the visible region of the spectrum, the patterned markers having visible-markings; this facilitates identification of the markers during setting up by a user. However, infra-red
Russell Richard Thomas
Thomas Graham Alexander
British Broadcasting Corporation
Couso Yon J.
McDermott & Will & Emery
LandOfFree
Position determination does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Position determination, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Position determination will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3043395