Communications: electrical – Condition responsive indicating system – Specific condition
Reexamination Certificate
2001-02-09
2004-04-06
Mullen, Thomas (Department: 2632)
Communications: electrical
Condition responsive indicating system
Specific condition
C348S143000, C382S117000
Reexamination Certificate
active
06717518
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to an image processing system, and more particularly to the use of a generic image processing system to detect drowsiness.
2. Description of the Related Art
It is well known that a significant number of highway accidents result from drivers becoming drowsy or falling asleep, which results in many deaths and injuries. Drowsiness is also a problem in other fields, such as for airline pilots and power plant operators, in which great damage may result from failure to stay alert
A number of different physical criteria may be used to establish when a person is drowsy, including a change in the duration and interval of eye blinking. Normally, the duration of bilking is about 100 to 200 ms when awake and about 500 to 800 ms when drowsy. The time interval between successive blinks is generally constant while awake, but varies within a relatively broad range when drowsy.
Numerous devices have been proposed to detect drowsiness of drivers. Such devices are shown, for example, in U.S. Pat. Nos. 5,841,354; 5,813,993; 5,689,241; 5,684,461; 5,682,144; 5,469,143; 5,402,109; 5,353,013; 5,195,606; 4,928,090; 4,555,697, 4,485,375; and 4,259,665. In general, these devices fall into three categories: i) devices that detect movement of the head of the driver, e.g., tilting; ii) devices that detect a physiological change in the driver, e.g., altered heartbeat or brething, and iii) divice that detect a physical result of the driver falling asleep, e.g., a reduced grip of the steering wheel. None of this divice is believed to have met with commercial success.
The German patent application DE 19715519 and the corresponding French patent application FR-2.747.346 disclose an apparatus and process of evaluation of the drowsiness level of a driver using a video camera placed near the feet of the driver and a processing unit for processing the camera image with software detecting the blinks of the eyes to determine the time gap between the beginning and the end of the blinks. More particularly, a unit of the processor realizes
a memorization of the video image and its treatment, so as so determine an area comprising the driver's eyes,
the dectetion of the time gap between the closing oft he driver ayelids and their full opening and
a treatment in a memo and a processor in combination with the unit to calculate a ratio of slow blink apparition.
The object of the international patent application published WO-97/01246 is a security system comprising a video camera placed within the rear-view mirror of a car and a video screen remotely disposed for the analysis of what is happening in the car and around it, as well as of what happened due to the recording of the output video signal of the camera This is in fact a concealed camera (within the rear-view mirror), so that it is imperceptible to vandals and thieves and which observes a large scope including the inside of the car and its surroundings, the record allowing one to know later what has happened in this scope (page 6, lines 13 to 19), this is not a detector whose effective angle is strictly limited to the car driver face in order to detect its eventual drowsiness and to make him awake.
Commonly-owned PCT Application Ser. Nos. PCT/FR97/01354 and PCT/EP98/05383 disclose a generic image processing system that operates to localize objects in relative movement in an image and to determine the speed and direction of the objects in real-time. Each pixel of an image is smoothed using its own time constant. A binary value corresponding to the existence of a significant variation in the amplitude of the smoothed pixel from the prior frame, and the amplitude of the variation, are determined, and the time constant for the pixel is updated. For each particular pixel, two matrices are formed that include a subset of the pixels spatially related to the particular pixel. The first matrix contains the binary values of the subset of pixels. The second matrix contains the amplitude of the variation of the subset of pixels. In the first matrix, it is determined whether the pixels along an oriented direction relative to the particular pixel have binary values representative of significant variation, and, for such pixels, it is determined in the second matrix whether the amplitude of these pixels varies in a known manner indicating movement in the oriented direction. In domains that include luminance, hue, saturation, speed, oriented direction, time constant, and x and y position, a histogram is formed of the values in the first and second matrices falling in user selected combinations of such domains. Using the histograms, it is determined whether there is an area having the characteristics of the selected combinations of domains.
It would be desirable to apply such a generic image processing system to detect the drowsiness of a person.
SUMMARY OF THE INVENTION
The present invention is a process of detecting a driver falling asleep in which an image of the face of the driver is acquired. Pixels of the image having characteristics corresponding to characteristics of at least one eye of the driver are selected and a histogram is formed of the selected pixels. The histogram is analyzed over time to identify each opening and closing of the eye, and from the eye opening and closing information, characteristics indicative of a driver falling asleep are determined.
In one embodiment, a sub-area of the image comprising the eye is determined prior to the step of selecting pixels of the image having characteristics corresponding to characteristics of an eye. In this embodiment, the step of selecting pixels of the image having characteristics of an eye involves selecting pixels within the sub-area of the image. The step of identifying a sub-area of the image preferably involves identifying the head of the driver, or a facial characteristic of the driver, such as the driver's nostrils, and then identifying the sub-area of the image using an anthropomorphic model. The head of the driver may be identified by selecting pixels of the image having characteristics corresponding to edges of the head of the driver. Histograms of the selected pixels of the edges of the driver's head are projected onto orthogonal axes. These histograms are then analyzed to identify the edges of the driver's head.
The facial characteristic of the driver may be identified by selecting pixels of the image having characteristics corresponding to the facial characteristic. Histograms of the selected pixels of the facial characteristic are projected onto orthogonal axes. These histograms are then analyzed to identify the facial characteristic. If desired, the step of identifying the facial characteristic in the image involves searching sub-images of the image until the facial characteristic is found. In the case in which the facial characteristic is the nostrils of the driver, a histogram is formed of pixels having low luminance levels to detect the nostrils. To confirm detection of the nostrils, the histograms of the nostril pixels may be analyzed to determine whether the spacing between the nostrils is within a desired range and whether the dimensions of the nostrils fall within a desired range. In order to confirm the identification of the facial characteristic, an anthropomorphic model and the location of the facial characteristic are used to select a sub-area of the image containing a second facial characteristic. Pixels of the image having characteristics corresponding to the second facial characteristic are selected and histograms of the selected pixels of the second facial characteristic are analyzed to confirm the identification of the first facial characteristic.
In order to determine openings and closings of the eyes of the driver, the step of selecting pixels of the image having characteristics corresponding to characteristics of an eye of the driver involves selecting pixels having low luminance levels corresponding to shadowing of the eye. In this embodiment, the step of analyzing the histogram
Binford Thomas
Pirim Patrick
Holding B.E.V.S.A.
Mullen Thomas
LandOfFree
Method and apparatus for detection of drowsiness does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for detection of drowsiness, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for detection of drowsiness will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3193900