Image analysis – Applications – Personnel identification
Reexamination Certificate
1999-11-09
2003-05-27
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Applications
Personnel identification
C382S278000, C382S296000, C340S575000
Reexamination Certificate
active
06571002
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to a face-image processing apparatus for use in an apparatus which extracts an image of the eye so as to detect a state of opening/closing of the eyes of a person which must be photographed in accordance with the state of the image of the eye.
Hitherto, a state of a driver who drives a vehicle including inattentive driving and driving the vehicle asleep has been detected by a face-image processing apparatus. The apparatus is structured to photograph the face of the driver with a camera disposed in a cabin of the vehicle. Then, the obtained image of the face is processed to extract the eyes which are characteristic points of the face so as to detect opening/closing of the eyes. As a conventional apparatus of the foregoing type, an apparatus has been disclosed in the Unexamined Japanese Patent Application Publication No. Hei 9-44684 having a structure that a density gradient image is converted into a binarized image. Then, the eyes are extracted from black regions in the profile of the extracted face. Then, inclinations of the upper portions of the corners of the eyes in the extracted image of the eyes are detected by obtaining primary correlation. In accordance with change in the inclinations, opening/closing of the eyes is determined.
FIG. 19
is a schematic view showing detection of the state of a driver which is performed by the structure disclosed in the Unexamined Japanese Patent Application Publication No. Hei 9-44684.
Referring to
FIG. 19
, reference numeral
1
represents a camera for photographing the driver;
2
represents an image input means to which a density gradient image is input and which A/D-converts the image into a digital gradation image;
3
represents a binarizing means for binarizing the digital gradation image obtained by conversion performed by the image input means
2
;
4
represents an eye-extraction means for extracting a region in which the eyes exists from the binarized image;
5
represents an evaluation-function calculating means for calculating a shape function from the region and further calculating an evaluation function such that attention is paid to circular-arc shapes; and
6
represents opening/closing determining means for determining opening/closing of the eyes in accordance with the value of the evaluation function.
FIG. 20
is a diagram showing the operation of the structure shown in
FIG. 19
in which opening/closing of the eyes is determined by using the binarized image of the region, in which the eyes exist, and which has been extracted from the image.
The conventional detection of the state of the driver structured as described above is performed such that a density gradient image supplied from the camera
1
, which has photographed the face of the driver, is, as an image signal, input to the image input means
2
so as to be A/D-converted. A digital gradation image obtained by the conversion is, by the binarizing means
3
, binarized with a predetermined threshold, that is, converted into black pixels and white pixels. Then, the eye-extraction means
4
extracts a region in which the eyes exist. The eye-extraction means
4
, for example, obtains the center of gravity from a mean coordinate of the black pixels in the binarized image. A rectangular region in a lateral predetermined range in the direction X in which a cluster of black pixels nearest the foregoing center of gravity is extracted as the region in which the eyes exist. Then, the evaluation-function calculating means
5
calculates the shape of the overall portion of the binarized image of the eyes in the region in which the eyes exist, that is, a shape function showing the characteristic. In accordance with the shape function, an evaluation function in which attention is paid to the circular-arc shape is calculated so that the value of the function is recorded in a memory. Then, the opening/closing determining means
6
determines a threshold from the value K of the evaluation function which is changed as time elapses so that opening/closing of the eyes is determined.
An example of the shape function will now be described with reference to FIG.
20
.
FIG. 20
shows transition of time (TA at which the eyes are opened, TC at which the eyes are closed and TB between TA and TC) and binarized images in the region in which the eyes exist corresponding to the transition of time. The upper pixels in predetermined ranges in the vicinity of the corners of the eyes of the binarized image of the eyes are approximated to straight lines by using the least-square method.
Thus, the inclination is enlarged when the eyes are opened and the inclination is reduced as the eyes are closed as illustrated. That is, assuming that the coordinates of the upper pixels of the corners of the eyes in a predetermined range (n pixels) are (xi, yi), the inclinations are calculated as follows:
K
=(
n&Sgr;xy−&Sgr;x&Sgr;y
)/(
n&Sgr;x
2
−(&Sgr;
x
)
2
)
Although the inclinations of the corners of the eyes are changed when the face is inclined, an average value of the inclinations of the two corners of the eyes is obtained to compensate the influence of the inclination. That is, assuming that the inclination of the corner of the left eye is KL and the inclination of the corner of the right eye is KR, opening/closing of the eyes is evaluated by the following equation:
K
=(
KL+KR
)/2
Then, the opening/closing evaluating means
6
sets threshold KB for determining opening/closing so as to determine opening/closing in accordance with the value of K.
Finally, driving the vehicle asleep is determined in accordance with the state of opening/closing of the eyes detected by a nictation detecting means.
The foregoing conventional apparatus, however, encounters change in the image of the eyes which is binarized by the binarizing means
3
when the environment in terms of light for the person who is photographed has rapidly been changed or owning to dispersion occurring when the A/D conversion is performed. In particular, a considerable influence of the change is exerted on the lines of the eyelids, and in particular, on the corners of the eyes. Therefore, when the inclinations of the corners of the eyes are calculated by the above-mentioned least-square method, the values of the inclinations are undesirably changed owning to the foregoing factors. Therefore, there arises a problem in that an error is easily caused in detection.
If the environment in terms of light for the right eye and that for the left eye of the person who is photographed are different from each other, the states of binarization for the right eye and the left eye are unbalanced. Thus, there arises a problem in that opening/closing of the eyes cannot accurately be evaluated.
SUMMARY OF THE INVENTION
To solve the above-mentioned problems, an object of the present invention is to obtain a face-image processing apparatus which is capable of stably determining a state of opening/closing of the eyes if a state of binarization of an image of the eyes of a photographed person which is performed by an eye-image binarizing means is changed.
A face-image processing apparatus according to the present invention comprises: image input means to which an image of the face photographed by a camera is input; eye-image extracting means for extracting a binarized image of the eye from the image of the face input by the image input means; coordinate rotating means for rotating the coordinates of the image of the eye extracted by the eye-image extracting means for a predetermined angle; correlation calculating means for calculating primary correlation by using the image of the eye rotated by the coordinate rotating means; and opening/closing determining means for determining opening/closing of the eye in accordance with a correlation coefficient obtained owning to the calculation performed by the correlation calculating means.
The eye-image extracting means incorporates an eye-region determining means for specifying the eye region of the image of the face input by the image input me
Mehta Bhavesh M.
Mitsubishi Denki & Kabushiki Kaisha
Sughrue & Mion, PLLC
Sukhaphadhana Christopher
LandOfFree
Eye open/close detection through correlation does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Eye open/close detection through correlation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Eye open/close detection through correlation will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3083766