Image analysis – Color image processing – Image segmentation using color
Reexamination Certificate
2000-09-29
2003-12-23
Johns, Andrew W. (Department: 2621)
Image analysis
Color image processing
Image segmentation using color
C382S218000, C382S190000, C382S110000, C356S237100, C356S240100
Reexamination Certificate
active
06668078
ABSTRACT:
FIELD OF THE INVENTION
This invention relates to the field of image processing and computer vision. More specifically, the invention relates to an apparatus and method for taking images of objects independent of the background and/or of the ambient illumination even if these objects are surrounded by plastic that can be somewhat translucent.
BACKGROUND OF THE INVENTION
There are various prior art image processing and computer vision systems which acquire and/or process images of a scene. (Generally, a scene includes a background and one or more objects that are of interest.) Typically, in these systems, an analog image from a camera (image acquisition unit) is converted to a discrete representation by dividing the picture into a fixed number of locations called picture elements, or pixels, and quantizing the brightness or color of the image at those picture elements into a fixed number of values. Usually, color is represented as three different images, the red, the green and the blue image where the color of the pixels is quantized in a fixed number of values. The red, green and blue are referred to as the color channels or the spectral bands. Thus, much of the prior art develops a digital image of the actual image or scene and then processes the digital image using a computer. This processing, also called image processing or computer vision, includes modifying the scene image or obtaining properties from the scene image such as the identity or location of the objects in the scene.
Objects in the scene are illuminated when light falls on the object(s). Ambient illumination is the illumination due to light sources occurring in the environment such as the sun outdoors, room lights indoors, or a combination of artificial light and sunlight indoors. In general, the light reflected from an object patch, resulting in a brightness of the corresponding image pixels, is a mixture of a matte plus a glare (or specular) component, although at a given image pixel either the matte component or the glare component tends to dominate. The color of a matte reflection is a function of the natural color of the object and the color of the illuminating light (in the spectral domain, the illumination function and the reflection function are multiplied). Specular reflections (also called glare) are the bright highlights reflected off the surface of a shiny object. The color of the glare is mostly the color of the illuminating lights (as opposed to the natural color of the object).
The glare component is mostly unrelated to the object's intrinsic surface properties and, therefore, is of little use for object segmentation or recognition purposes. The matte reflection, on the other hand, is a function of the color of the object as well as the illuminating light. To produce a image which is more characteristic of an object's intrinsic color it is desirable to remove or suppress the specular component of reflection. One way to do this is by the use of polarizers. Because of diffusion in the surface layer of an object, matte reflection is not polarized. Specular reflection, on the other hand, is often polarized, especially as the viewing angle becomes more tangential to the surface. Thus adding a properly oriented polarizing filter to the camera will remove a certain portion of the glare. If all the illumination can be controlled, even better results can be obtained by deliberately polarizing the outgoing, illuminating light and then only sensing returned light with an altered polarization angle.
For these reasons an object's color, an important object property for object recognition, depends on the ambient light. In order to compensate for this effect, prior art solutions use the reflection of a white or gray patch in the scene. Color correction is then performed by transforming the image so that the color of the gray patch is transformed to a standard predetermined value. For instance, the patch image color spectrum could be transformed such that the spectrum of the patch image is uniform in the red, green and blue channels (spectral bands) and has a certain, preset reflectance. Indeed, the whole image including the object image is transformed in such a fashion. A representation of the object's color is thus represented for recognition purposes by its image color spectrum normalized by the standard color spectrum of the image of the gray patch. Such techniques, known as color constancy, are well known. An early example for gray scale images can be found in U.S. Pat. No. 4,314,281 to Wiggens and Elie, which is hereby incorporated by reference in its entirety.
Now consider the case where the object to be recognized is surrounded by a plastic bag. It is assumed that the transparency of the bag is high enough that a human can recognize the object. A part of the scene image (e.g., where the bag is flat) contains object image portions as would be seen as if there were no surrounding bag. However, even for those image parts the illuminating light passes through the bag then the reflected light passes through the bag again. Thus the color of the reflected light by any subtle tint to the bag, as well as by the bag's intrinsic diffuse reflectance properties is influenced to an extent depending on the level of translucency and tint of the bag. Other parts of the bag may completely obscure the underlying object image due to specular reflection off the bag surface and, to a lesser extent, due to fact that the bag is seen as opaque depending on the surface normal of the bag or folds in the bag. These phenomena make it difficult to gauge the true surface properties of an item enclosed by a bag.
During the image processing of the scene, the object (or objects) that is (are) of interest is (are) imaged along with the scene surroundings. These surroundings are called the background. The background is usually behind the object(s) of interest. In some types of image processing, it is necessary to separate the object(s) image from the background image of the scene. This separation is called figure/ground separation or segmentation. In such applications it is important that the segmented foreground portion accurately represents the properties of the object to be identified, and not be contaminated by illumination or other environmental artifacts.
This figure/ground separation is most often performed for the purposes of object recognition. U.S. Pat. No. 5,546,475 to Bolle et al. gives an example, where in combination with the segmentation techniques of U.S. Pat. No. 5,631,976 to Bolle et al., the object(s) in the segmented image are recognized using color features (in combination with other features). A segmentation of an image, may, therefore, be denoted as a mapping s of pixels (x, y) into some space s, e.g., S: (x, y)→s, where S(x, y) is set to some value X if pixel (x, y) is not a part of the segment, and S(x, y) is set to the original pixel value I(x, y) if (x, y) is part of the segment. An alternate segmentation of an image, could be a mapping (x, y)→{0, 1}, where an image point (x, y) is labeled ‘1’ if (x, y) is part of the segment and ‘0’ otherwise. Other variations are also possible, where s=[0, 1], the membership of pixel (x, y) of the segmentation is expressed as a degree of membership. The set s could also take on a set of n (greater than two) discrete numbers.
Figure/ground separation of some sort is required when using computer vision technology to recognize produce (fruit and vegetables) at the point of sale (POS) in supermarkets and grocery stores. The ability to automatically recognize produce at the checkout counter has many advantages, among which:
There is no need to affix the PLU (price lookup) stickers to the produce.
There is less need for prepackaging the produce, thereby saving solid waste.
The checkout of produce will be speedier because the checkers do not have to recall or lookup the PLU numbers.
Produce inventory control can be done more accurately.
Pricing can be done more consistently and accurately.
Allows more convenient self-checko
Bolle Rudolf M.
Connell Jonathan
Alavi Amir
Johns Andrew W.
Percello Louis J.
LandOfFree
System and method for segmentation of images of objects that... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for segmentation of images of objects that..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for segmentation of images of objects that... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3148957