Image analysis – Applications – Manufacturing or product inspection
Reexamination Certificate
1997-12-30
2001-01-09
Couso, Jose L. (Department: 2721)
Image analysis
Applications
Manufacturing or product inspection
C382S154000, 36
Reexamination Certificate
active
06173070
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to locating and inspecting features on an object, and more particularly to automatically locating and inspecting features on electronic components.
BACKGROUND OF THE INVENTION
Machine vision systems are frequently used in contexts that require the system to capture a two dimensional image of a physical object and locate within that image some aspects or features that are to be analyzed, such as to determine the position of the features, inspect the integrity of the feature, or effect alignment or a particular orientation of the physical object based on the features found.
For example, U.S. Pat. No. 5,550,763 (“the '763 patent”) describes a method and apparatus for automatically locating the center of a ball bond, in a digitized two dimensional binary or grey-scale image of the ball bond formed on a bonded semiconductor pad. The '763 patent works with a two-dimensional grey-scale image representation, and constructs a flattened synthetic cone-shaped model which is used to produce a two-dimensional grey-scale image model. The grey-scale image model is used for a normalized correlation search in the area of the image at the nominal or expected location of the ball bond. The normalized correlation search, which is well known in the art, measures the similarity between a captured image of the ball bond and the synthetic grey-scale image model independent of linear differences in the image or model brightness. That is, the normalized correlation search effects a template matching, scanning the captured input image for a match to the synthetic image model to determine how closely the model matches, i.e. correlates to, the found object, with a high degree of tolerance for optical issues such as poor focus and poor lighting. The search results yield a correlation coefficient indicating the extent to which the image matches the model, and an X, Y location in the image indicating the center of the found matching object.
Disadvantageously, the '763 patent uses a two-dimensional grey-level image data set both as input and for the image model, which limits the inspection process to inspection of the desired feature only in terms of visually discernable artifacts as viewed by reflected light. Features that have a height or depth, i.e. a third dimension, are impossible to inspect with the disclosed process. The '763 patent can not discern any surface features other than by differing reflectance of light off the surface of the device. Any measurements and inspection based on spatial characteristics, such as volume, are impossible. In short, the method and apparatus disclosed in the '763 patent has no utility in three dimensional inspection applications.
Methods have been proposed for using machine vision systems to determine characteristics of three dimensional objects in a single pass. One proposed method implements a machine vision process that interprets “height images” to provide information that permits three-dimensional objects, such as solder bumps, to be distinguished from two-dimensional objects, such as solder pads. Spatial characteristics of a selected object in a “height image”, that has pixel values representative of heights of corresponding portions of the selected object, are determined by a process that first determines a height for each of a plurality of portions of the object. A volume of the object above a selected height, i.e. threshold, is determined by summing a count representing the cross-sectional area of the object at the selected height and a count representing the volume at each of the plurality of portions of the object above the selected height.
This proposed methodology for determining the volume of an object as a function of three-dimensional (i.e. height) data effectively determines a center of mass of the object above the particular threshold (i.e. selected height). The methodology sets a threshold with respect to a particular base height of an object, uses vision tools that effect a contour plot or connectivity of points in the image to identify the object, and then merely looks for groups of points above the base height of the two-dimensional contour.
Such an implementation is of questionable accuracy when there is any tilt of the plane above which the base height is defined, or when the base height is indeterminate. The threshold height is constant and does not accommodate tilting of the plane on which the objects are positioned, so that volumes that appear below the base height as a result of tilt will not be determined. When the base plane height is indeterminate, volumes that appear below the incorrectly assumed base height will not be determined. The previous methodology does not process the three-dimensional data set in a manner that considers asymmetry of the object(s) in that the vision tools used yield only information relating to the center of mass of the object above the threshold. It does not take advantage or use all the information in the three-dimensional data set input in that information below the base height is effectively ignored. This disadvantageously leads to less accurate inspection of three dimensional objects.
This previous methodology is also more susceptible to inaccuracies and errors resulting from noise spikes in that it relies heavily on information associated with areas of the object that are most noisy. Inspection of an object in three dimensions inherently involves significant amounts of noise in the captured image data, due to the manner in which 3D data is ascertained and typically resulting from issues associated with optics and lighting in real world conditions. Furthermore, many 3D sensors and image acquisition devices produce merit scores (x,y) for each z(x,y) measurement, and the previous methodology ignores such merit scores. For example, reflections and shading on the sides of objects may show up in the height image data set as noise spikes (with low merit scores). In merely establishing a threshold above which groups are analyzed, according to this prior method, there is no mechanism for tolerating and/or discounting noise spikes that exceed the selected base threshold. Accordingly, this prior method is relatively slow and inaccurate.
SUMMARY OF THE INVENTION
The present invention provides a robust method and apparatus for inspecting three-dimensional objects, represented as a captured three-dimensional image data set, via a normalized correlation search implementation which considers asymmetry of the object(s) and accommodates for tilt in the plane in which the objects are disposed.
According to the invention, a three-dimensional (3D) synthetic model of an object of interest, such as a solder ball, is constructed in a manner that facilitates discounting of areas of the object that are highly susceptible to noise. The 3D synthetic model is subjected to a mask that is derived in order to discount high noise areas as a function of the shape of the object of interest and the context of the object inspection, to provide a masked 3D model. The masked 3D model and a captured 3D image data set of the object of interest are processed via normalized correlation processing to provide an indication of location of the object of interest in the captured 3D image and a correlation coefficient indicative of the degree to which a found object of interest correlates to the synthesized 3D model.
In further accord with the invention, points proximate to the object of interest are sampled and a common two dimensional plane, surface or contour is fit on which the points lie in the 3D image data set. Accordingly, a location of a surface in the 3D captured image upon which the object of interest lies is determined and can be used for further inspection and/or processing, for instance to ascertain planarity of the surface, to inspect the condition of the surface or the like.
Features of the invention include provision of a location and inspection implementation for 3D captured image data that uses an optimized synthesized 3D model to enhance the speed of process
Michael David J.
Wallack Aaron S.
Cognex Corporation
Couso Jose L.
Dang Duy M.
Michaelis Brian L.
LandOfFree
Machine vision method using search models to find features... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Machine vision method using search models to find features..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Machine vision method using search models to find features... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2539616