Television – Monitoring – testing – or measuring – Testing of camera
Reexamination Certificate
1999-04-30
2002-08-20
Hsia, Sherrie (Department: 2614)
Television
Monitoring, testing, or measuring
Testing of camera
C348S188000
Reexamination Certificate
active
06437823
ABSTRACT:
BACKGROUND
1. Technical Field
The invention is related to a computer-implemented system and process for calibrating digital cameras, and more particularly, to a system and process for calibrating digital cameras using only images of a pattern on a planar surface captured by the camera.
2. Background Art
Camera calibration is a necessary step in 3D computer vision in order to extract metric information from 2D images. Much work has been done, starting in the photogrammetry community [2, 4], and more recently in computer vision [9, 8, 23, 7, 26, 24, 17, 6]. These techniques can be classified roughly into two categories: photogrammetric calibration and self-calibration.
Photogrammetric camera calibration is performed by observing a calibration object whose geometry in 3-D space is known with very good precision. Calibration can be done very efficiently [5]. The calibration object usually consists of two or three planes orthogonal to each other. Sometimes, a plane undergoing a precisely known translation is also used [23]. These approaches require an expensive calibration apparatus, and an elaborate setup.
Self-calibration techniques do not use any calibration object. Just by moving a camera in a static scene, the rigidity of the scene provides in general two constraints [15, 17] on the cameras' internal parameters from one camera displacement by using image information alone. Therefore, if images are taken by the same camera with fixed internal parameters, correspondences between three images are sufficient to recover both the internal and external parameters which allow the reconstruction of 3-D structure up to a similarity [16, 13]. While this approach is very flexible, it is not yet mature [1]. Because there are many parameters to estimate, reliable results cannot always be obtained. Other related techniques also exist such as vanishing points for orthogonal directions [3, 14], and calibration from pure rotation [11, 21].
More recently a self-calibration technique [22] was developed that employed at least 5 views of a planar scene to calibrate a digital camera. Unfortunately, this technique has proved to be difficult to initialize. Liebowitz and Zisserman [14] described a technique of metric rectification for perspective images of planes using metric information such as a known angle, two equal though unknown angles, and a known length ratio. They also mentioned that calibration of the internal camera parameters is possible provided at least three such rectified planes, although no experimental results were shown.
It is noted that in the preceding paragraphs, as well as in the remainder of this specification, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. Multiple references will be identified by a pair of brackets containing more than one designator, for example, [2, 4]. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.
SUMMARY
The present invention relates to a flexible new technique to easily calibrate a camera that overcomes the problems of current methods. It is well suited for use without specialized knowledge of 3D geometry or computer vision. In general terms, the technique involves capturing images of a planar pattern from at least two different non-parallel orientations. It is not necessary that any particular pattern be used, or even the same pattern be used for each of the captured images. The only requirement is that the coordinates of a number of feature points on the pattern plane are known. The pattern can be printed on a laser printer and attached to a “reasonable” planar surface (e.g., a hard book cover). Compared with classical techniques, the proposed technique is considerably more flexible. Compared with self-calibration, it gains a considerable degree of robustness.
This new calibration technique is generally focused on a desktop vision system (DVS). Such cameras are becoming cheap and ubiquitous. However, a typical computer user will perform vision tasks only from time to time, so will not be willing to invest money for expensive calibration equipment. Therefore, flexibility, robustness and low cost are important.
The calibration process embodying the present invention begins by determining the 2D coordinates of at least 4 feature points on the planar pattern. Next, at least two, and preferably three or more, images of the planar pattern (if the same one is used for all the images) are captured at different (non-parallel) orientations using the digital camera being calibrated. It is noted that either the pattern can be reoriented and the camera held stationary, or the pattern held stationary and the camera moved. The specifics of the motion need not be known. The image coordinates of the aforementioned 2D feature points of the planar pattern are then identified in the captured images using conventional image processing techniques. If three or more images have been captured, a closed form solution exists that can be solved to derive all the intrinsic and extrinsic parameters needed to provide the camera calibration. Essentially, the known pattern coordinates and corresponding image coordinates are used to compute a homography for each image. Then, a process is employed that estimates the intrinsic camera parameters by analyzing the homographies associated with each image. Finally, the extrinsic parameters for each image are computed from the intrinsic parameters and the homographies.
Of course, the images can be effected by various noise sources. For example, the camera resolution can be a source of noise. The process used to identify the coordinates of the feature points as they appear in the images may also not provide absolutely accurate results. Thus, the closed form solution may not be exact. If higher accuracy is called for, steps can be taken to provide a more precise estimate of camera calibration. For example, a maximum likelihood inference process (which is essentially a non-linear minimization procedure) can be employed to either provide a more accurate first estimate, or to refine the estimates derived from the closed form solution algorithm. In the former case, an initial guess is used for the unknown intrinsic and external parameters, while in the latter case the parameters estimated using the closed form solution procedure are employed.
If lens distortion is also a concern, as it is with most inexpensive desktop digital cameras, the camera parameters can be further refined by taking into account this distortion. As it is believed radial distortion is the dominant component of lens distortion, all other sources can be ignored. It is possible to first determine the camera parameters using any of the foregoing techniques and then employ what amounts to a least squares analysis to ascertain the radial distortion coefficients. These coefficients can then be used to compensate for the radial distortion in the camera parameter estimates. However, it can be more efficient and faster to determine the radial distortion coefficients along with the camera parameters using an expanded maximum likelihood inference process.
In addition to the just described benefits, other advantages of the present invention will become apparent from the detailed description which follows hereinafter when taken in conjunction with the drawing figures which accompany it.
REFERENCES:
patent: 5699440 (1997-12-01), Carmeli
S. Bougnoux. From projective to euclidean space under any practical situation, a criticism of self-calibration. In Proceedings of the 6th International Conference on Computer Vision, pp. 790-796, Jan. 1998.
D. C. Brown. Close-range camera calibration. Photogrammetric Engineering, 37(8):855-866, 1971.
B. Caprile and V. Torre. Using Vanishing Points for Camera Calibra
Hsia Sherrie
Lyon Richard T.
Lyon & Harr LLP
Microsoft Corporation
LandOfFree
Method and system for calibrating digital cameras does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and system for calibrating digital cameras, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and system for calibrating digital cameras will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2969130