Image analysis – Image transformation or preprocessing – Measuring image properties
Reexamination Certificate
2000-12-22
2004-09-28
Chang, Jon (Department: 2623)
Image analysis
Image transformation or preprocessing
Measuring image properties
C382S151000, C356S620000
Reexamination Certificate
active
06798925
ABSTRACT:
FIELD OF THE INVENTION
This invention relates to image acquisition systems for object location and inspection and, more particularly, to techniques for calibrating the optical system in an image acquisition system.
BACKGROUND OF THE INVENTION
In many applications, it is necessary to determine the two-dimensional location or translation and angular orientation of an object of interest. Depending on the application, a determination of one or more of these properties is called determining the “alignment” of an object. In some applications, alignment also may include additional degrees of freedom, such as scale, aspect, shear or even various types of non-linear deformations. For example, in a robotic assembly system, it may be necessary to determine the alignment of a printed circuit board so that a robotic arm can place an integrated circuit onto the board at a precise location. One way to perform this alignment is to mechanically constrain the board at a predetermined location and angular orientation. Guides, brackets and many other well-known mechanical arrangements can be used to accomplish this alignment. However, in other applications, it is not feasible to mechanically constrain each object. In these latter systems, machine vision systems are often used to determine the alignment of objects.
Machine vision systems acquire images of an environment, process the images to detect objects and features in the images and then analyze the processed images to determine characteristics of objects and other features detected in the images. The system generally includes a camera/frame grabber system that generates an image that consists of a plurality of digitized image pixels. The image pixels are then processed with an algorithm implemented in software and/or hardware typically called a vision “tool.”
In order for the machine vision system to operate with other equipment, such as a materials handing system or a robotic assembly arm, the output of the machine vision system must converted into a frame of reference that can be used to control the materials handing system. For example, the output of the machine vision system is typically expressed in pixel dimensions, since the underlying image is composed of pixels. The output in pixels must be converted into other metrics, such as millimeters or inches, which can be used to control the materials handing system. Further, it is necessary to convert the output of the machine vision system so that a given point in the image corresponds to a known point in the frame of reference in which the handling system is located and to which the handing system can move.
Conventionally, this conversion is performed in connection with a process called “calibration.” In order to calibrate a machine vision system, a special calibration target, or calibration plate, is mechanically fixed in a known position with respect to the frame of reference. The calibration target includes objects that are located at predetermined points and spaced apart at known distances. In addition, a unique “fiducial” mark may be included in order to determine absolute position. The machine vision system then acquires images of the calibration plate objects and the fiducial mark. Since the spacing of the objects is known, the output of the machine vision system in pixels can be directly converted into the necessary metrics for controlling the materials handing portion of the system. The position of the fiducial mark, as determined by the machine vision system, can be used to determine the position of the calibration plate with respect to the frame of reference.
The calibration operation not only permits the necessary conversions, but also can be used to compensate for various distortions introduced by the optical path of the machine vision system. For example, the image acquisition camera may be mounted so that the resulting image is rotated with respect to the frame of reference. Distortions in size, shape, position and aspect ratio of objects may be introduced by the other components of the image acquisition optical system. Since the spacing of the calibration objects are known and the position of the fiducial mark is known, during the calibration process the output of the image acquisition system can be adjusted accordingly in order to remove these distortions.
The manner of determining the alignment of the calibration objects depends on the type of vision tool used to locate the objects. Some tools can tolerate variations in size, shape and angular orientation during the location procedure, but have other deficiencies. For example, the earliest vision tool widely used for object location and inspection in industry was blob analysis. In this type of tool, image pixels are classified as object or background pixels by some means, the object pixels are joined to make discrete sub-objects using neighborhood connectivity rules, and various moments of the connected sub-objects are computed to determine object position, size, and orientation. Blob analysis tools can tolerate and measure variations in orientation and size.
However, such tools cannot tolerate the presence of various forms of image degradation. A more serious problem was that the only generally reliable method ever found for separating object pixels from background pixels was to arrange for the objects to be entirely brighter or entirely darker than the background. This requirement is difficult to achieve in other than the most controlled conditions, although it typically can be achieved if a specially manufactured calibration plate is used.
However, it is very desirable to eliminate the need for a special calibration plate and thus to overcome the limitations of blob analysis tools. In order to overcome such limitations, techniques called “template matching” tools were developed to locate objects based on their pattern rather than grayscale intensities. A template matching tool typically starts with a training step. In this step, a software representation called a pattern, or template, of an image or synthetic description of an ideal object is created and stored. At run-time, the template is moved in various positions over the digitized image and compared to like-sized pixel subsets of the image. The position where the best match between the template and image sub-set pixels occurs is taken to be the position of the object. Because a “search” is performed for the best match, this type of tool is often called a search tool. The degree of match (a numerical value) can be used for inspection, as can comparisons of individual pixels between the template and image at the position of best match.
The first template matching tools used a brightness threshold to reduce the pixels of the template and image to two states: “bright” and “dark.” This reduced the computation necessary for the comparison operation to a reasonable level for the available computation facilities. Unfortunately, the thresholding step made sub-pixel accuracy impractical and made the results highly susceptible to the selection of the threshold and variations in illumination and object reflectivity.
Later tools overcame the thresholding problem by using a normalized correlation operation for the template and image comparison step, albeit at the cost of considerable additional computation. Normalized correlation template matching tools also overcame many of the limitations of blob analysis tools—they can tolerate touching or overlapping objects, they perform well in the presence of various forms of image degradation, and the normalized correlation match value is useful in some inspection applications. Most significantly, perhaps, in order for the tool to operate properly, objects need not be separated from background by brightness, enabling a much wider range of applications.
Unfortunately, while normalized correlation template matching tools work well in determining the location of objects that are translated, they will tolerate only small variations in angular orientation and size: typically a few degrees and a few percent (depending on the specific templa
Chang Jon
Cognex Corporation
Kudirka Paul
Tucker Wesley
Weinzimmer Russ
LandOfFree
Method and apparatus for calibrating an image acquisition... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for calibrating an image acquisition..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for calibrating an image acquisition... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3241345