Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
1998-12-31
2001-10-09
Au, Amelia M. (Department: 2623)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S151000, C700S254000
Reexamination Certificate
active
06301396
ABSTRACT:
The disclosure of this patent document contains material which is subject to copyright protection. The owner thereof has no objection to facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
BACKGROUND OF THE INVENTION
The invention pertains to machine vision and, more particularly, to methods for calibrating the reference frame of a camera to that of a moveable object (e.g., a motion stage) imaged by the camera.
Machine vision refers to the automated analysis of an image to determine characteristics of objects and other features shown in the image. It is often employed in automated manufacturing lines, where images of components are analyzed to determine placement and alignment prior to assembly. Machine vision is also used for quality assurance. For example, in the semiconductor device industry, images of chips are analyzed to insure that leads, solder paste and other components do not overrun designated boundaries.
In many machine vision applications, it is essential to correlate locations in the “real world,” e.g., on a motion stage or conveyor belt, with coordinates in an image. For example, a camera image of a part being assembled on a robotic assembly line may reveal that a component is misplaced by several pixels. In order to move the motion stage so that the object can be properly repositioned, the relationship between coordinates in the image and on the motion stage must be known. That relationship is known as the calibration relationship.
The prior art suggests the use of so-called calibration plates to determine the calibration relationship between a camera and an object. A typical calibration plate consists of a complex “target,” such as an array of dots, a checkerboard, a bulls-eye of concentric circles, or a set of parallel stripes, that is placed on the object. Traditionally, it has been important to construct and reproduce these plates carefully because any error in the target will be wrongly “corrected for” when the camera is calibrated. For example, if the circles in a bulls eye-type target are slightly eccentric, the resulting calibration may produce an incorrect aspect ratio (i.e., the ratio of width to height).
It is sometimes quite difficult to construct an accurate calibration target. This is particularly true when the camera magnification is very large and the corresponding calibration target is very small. Here, even a small deviation in the target will result in an incorrect calibration relationship. This problem is exacerbated in machine vision systems that utilize multiple cameras to image a single target, e.g., systems of the type used in semiconductor chip manufacture, in which two or more high resolution cameras are used to inspect, simultaneously, multiple disparate regions on the chip surface. In addition to the difficulties associated with calibrating the reference frame of a single camera to the real world reference frame of the chip surface (or motion stage), are those associated with calibrating the reference frames of the cameras to one another.
An object of this invention is to provide improved machine vision systems and, particularly, improved machine visions methods for calibrating the reference frame of a camera to that of a moveable object (e.g., a motion stage) imaged by the camera.
Another object of the invention is to provide such methods as can calibrate the reference frames of multiple cameras to each other, as well as to that of a moveable object imaged by the cameras.
Yet another object of the invention is to provide such methods as minimize reliance on precisely machined calibration targets and plates.
Yet still another object of the invention is to provide such methods as can be implemented on conventional digital data processors or other conventional machine vision analysis equipment.
Still yet another object of the invention is to calibrate the reference frames of multiple cameras with respect to the motion stage's center of rotation.
Yet still another object of the invention is to provide such methods that can rapidly determine calibration relationships without undue consumption of resources.
SUMMARY OF THE INVENTION
The aforementioned objects are among those met by the invention, which provides in one aspect a method of determining a calibration relationship between a reference frame of motion of an object and a reference frame of a camera (or other image acquisition device) that generates images of the object. The method includes the steps of coupling a target to the object and placing the object at each of plural locations (and orientations) that are known with respect to the motion reference frame of the object. The location and orientation of the target with respect to the object need not be known.
An image of the object and target is generated while the object is at each of those locations/orientations. From each those images, the method determines the location and, optionally, the orientation, of the target with respect to the reference frame of the camera. The method then calls for determining the calibration relationship between the reference frame of motion of the object and the camera reference frame as a function of the known locations/orientations of the object (with respect to its motion reference frame) and the locations (or locations and orientations) of the target in the corresponding images (with respect to the reference frame of the camera).
In another aspect, the invention provides a method for determining a calibration relationship between a reference frame of motion of an object and a reference frame of each of plural cameras that generate images of the object. The method includes the steps of coupling plural targets to the object and placing the object at plural locations/orientations that are known with respect to the motion reference frame of the object. At least one target remains in the field of view of each respective camera as the object's placement is changed. Any given target need not remain in the field of view of its respective camera for all of the placement changes but, merely, for plural changes. As above, the location and orientation of the targets with respect to the object need not be known.
An image of the object and target is generated by each camera while the object is at each of the locations/orientations. From each those images, the method determines the location (or location and orientation) of the target with respect to the reference frame of the respective camera. As above, the calibration relationship between the reference frame of motion of the object and the camera's reference frames is determined as a function of the known locations/orientations of the object (with respect to its motion reference frame) and the location (or location and orientation) of the target in the corresponding images (with respect to the reference frame of the camera).
A related aspect of the invention provides a method as described above, in which the plural targets are coupled to the object such that the relative locations and/or orientations of the targets with respect to one another are known. With this information, the method can determine the calibration relationship between the reference frames of the cameras themselves as a function of known locations/orientations of the object, the location (or location and orientation) of the targets in the corresponding images of the object, and the known relative locations/orientations of the targets with respect to one another.
Further aspects of the invention provide methods as described above in which the method determines the calibration relationships between the motion reference frame of the object and the motion reference frame of the camera(s) by solution of a linear equation and particularly, for example, by a linearized least squares fit that minimizes an error between known locations/orientations of the object and estimates of those locations/orientations based on cand
Michael David J.
Wallack Aaron S.
Au Amelia M.
Cognex Corporation
Powsner David J.
Wu Jingge
LandOfFree
Nonfeedback-based machine vision methods for determining a... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Nonfeedback-based machine vision methods for determining a..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Nonfeedback-based machine vision methods for determining a... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2607919