Image analysis – Applications – Manufacturing or product inspection
Reexamination Certificate
1999-09-20
2004-06-15
Dastouri, Mehrdad (Department: 2623)
Image analysis
Applications
Manufacturing or product inspection
C382S218000, C382S226000, C382S195000
Reexamination Certificate
active
06751343
ABSTRACT:
CROSS REFERENCE TO RELATED APPLICATIONS
(Not Applicable)
BACKGROUND OF THE INVENTION
1. Technical Field
This invention relates to the field of content-based image retrieval and more particularly to a method and apparatus for indexing and retrieving manufacturing-specific digital imagery based upon image content.
2. Description of the Related Art
Images are being generated at an ever-increasing rate by sources such as defense and civilian satellites, military reconnaissance and survellance flights, fingerprinting and mug-shot-capturing devices, scientific experimentation, biomedical imaging, architectural and engineering design, and industrial manufacturing. In consequence, image knowledge structure and an access method for retrieving images are two significant problems in the design of large intelligent image database systems. Content-based image retrieval [CBIR] represents a promising and cutting-edge technology useful in addressing the problem of high-speed image storage and retrieval. Specifically, CBIR refers to techniques used to index and retrieve images from databases based on their pictorial content.
Typically, pictorial content is defined by a set of features extracted from an image that describes the color, texture, and/or shape of the entire image or of specific objects. This feature description is used in CBIR to index a database through various means such as distance-based techniques, rule-based decision making, and fuzzy inferencing. Yet, to date, no significant work has been accomplished to apply these technologies to the manufacturing environment. Notwithstanding, imagery collected from the manufacturing processes have unique characteristics that can be taken advantage of in developing a manufacturing-specific CBIR approach.
The manufacturing environment represents an application area where CBIR technologies have not been extensively studied. The low cost of computer systems, memory, and storage media have resulted in manufacturers collecting and storing more information about the manufacturing process. Much of the data being stored is product imagery collected from automated inspection tools. This imagery contains an historical record of manufacturing events that cause a reduction in product quality. Under the proper circumstances this data can be used to rapidly source product quality issues and improve product yield.
Semiconductor manufacturing is representative of an industry that has a mature computer vision component for the inspection of product. Digital imagery for failure analysis is generated between process steps from optical microscopy and laser scattering systems and from confocal, SEM, atomic force microscope and focused ion beam imaging modalities. This data is maintained in a yield management database and used by fabrication engineers to diagnose and source manufacturing problems, verify human assertions regarding the state of the manufacturing process, and to train inexperienced personnel on the wide variety of failure mechanisms observed. Yet, the semiconductor industry currently has no direct means of searching the yield management database using image-based queries. The ability to query the fabrication process is based primarily on date, lot, and wafer identification number. Although this approach can be useful, it limits the user's ability to quickly locate historical information. For example, if SEM review has determined that a particular defect or pattern problem exists on a wafer, the yield engineer must query on dates, lots, and wafers to find similar historical instances. Although roughly 70% of all space occupied in the database consists of imagery, queries to locate imagery are manual, indirect, tedious, and inefficient. Therefore, this becomes an iterative and slow process that can prove unwieldy in the modern semiconductor environment where a single manufacturing campus having multiple fabrication facilities at one site can generate thousands of images daily. If a query method can be designed that allows the user to look for similar informational content, a faster and more focused result can be achieved. A process for locating similar imagery based on image content, for example the image structure rather than lot number, wafer identification, and date, would result in a reduced time-to-source. Hence, what is needed is a method for manufacturing-specific CBIR that addresses defect analysis, product quality control, and process understanding in the manufacturing environment.
SUMMARY OF THE INVENTION
A method and apparatus for indexing and retrieving manufacturing-specific digital imagery based on image content in accordance with the inventive arrangement satisfies the long-felt need of the prior art by providing manufacturing-specific, context based image retrieval in an industrial environment. In response to an industrial event, the inventive method can afford fast access to historical image-based records of similar industrial events so that a corrective action can be quickly taken. Thus, the inventive arrangements provide a method and apparatus for employing an image-based query-by-example method to locate and retrieve similar imagry from a database of digital imagery. The inventive arrangements have advantages over all content-based image retrieval systems, and provides a novel and nonobvious system and method for indexing and storing content-based, manufacturing-specific digital imagery for subsequent fast retrieval.
A method for indexing and retrieving manufacturing-specific digital images based on image content comprises three steps. First, at least one feature vector can be extracted from a manufacturing-specific digital image stored in an image database. In particular, each extracted feature vector corresponds to a particular characteristic of the manufacturing-specific digital image. The extracting step can comprise extracting three independent feature vectors, the three independent feature vectors corresponding to a digital image modality and overall characteristic, a substrate/background characteristic, and an anomaly/defect characteristic, respectively. Notably, the extracting step includes generating a defect mask using a detection process selected from the group consisting of thresholding the manufacturing-specific digital image, comparing the manufacturing-specific digital image with a golden template, and comparing the manufacturing-specific digital image with a digital image of a neighboring product; and, extracting a feature vector for the substrate/background characteristic or the anomaly/defect characteristic using the defect mask. Moreover, the extracting step can comprise the steps of: distinguishing a defect-region from a non-defect region in the manufacturing-specific digital image; rendering the defect-region similar to the non-defect region based on an estimate derived from a region surrounding the defect-region, the removal of which forms a modified manufacturing-specific digital image; and, extracting a feature vector corresponding to the substrate/background characteristic from the modified manufacturing-specific digital image.
Second, using an unsupervised clustering method, each extracted feature vector can be indexed in a hierarchical search tree. Specifically, the hierarchical search tree includes data encapsulating nodes. The nodes can be leaf nodes or level nodes. Each leaf node encapsulates a feature vector. In contrast, each level node references at least one additional node. Moreover, each level node encapsulates a vector average defined by the average value of all feature vectors and vector averages encapsulated by nodes referenced by the level node.
The using step can comprise the steps of: forming a subset of feature vectors, the subset including at least one feature vector; establishing a main branch in the hierarchical search tree, the main branch having at least one leaf node encapsulating a feature vector included in the subset of feature vectors; and, adding leaf nodes to the hierarchical search tree using a top-down algorithm. In the adding step, each additional leaf node
Ferrell Regina K.
Karnowski Thomas P.
Tobin, Jr. Kenneth W.
Akerman & Senterfitt
Dastouri Mehrdad
UT-Battelle LLC
LandOfFree
Method for indexing and retrieving manufacturing-specific... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for indexing and retrieving manufacturing-specific..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for indexing and retrieving manufacturing-specific... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3359421