Photography – Attitude sensing
Reexamination Certificate
2001-09-17
2003-01-14
Perkey, W. B. (Department: 2851)
Photography
Attitude sensing
C396S051000, C351S210000
Reexamination Certificate
active
06507702
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an improvement in a device having a sight line detecting function for use in cameras or the like.
2. Description of the Related Art
There has been proposed various devices for detecting which position an observer is observing on an observed surface, so-called devices for detecting sight line (visual axis), such as eye cameras.
For example, in U.S. Pat. No. 5,486,892 and No. 6,014,524, a parallel light flux from a light source is projected to the anterior ocular segment of the eyeball of an observer, and the sight line of the observer is determined making use of the cornea reflection images by the light reflected from the cornea and image-forming position of the pupil.
Also, U.S. Pat. No. 5,598,248 proposes an optical device (camera) having a sight line detecting device arranged to perform various types of photographing using a sight-line calibration method wherein personal differences in the sight lines of photographers are corrected.
FIG. 25
is a diagram explaining the principle of sight line detection.
In
FIG. 25
, reference numerals
13
a
and
13
b
each denote an infrared-emitting diode (hereinafter abbreviated as IRED) projecting an infrared light. The IREDs
13
a
and
13
b
are disposed along the X-axis direction so as to be substantially symmetrical with respect to the optical axis (z-axis) of the light-receiving lens
12
, and illuminate a user's eyeball
17
from the lower sides (positions offset in the y-axis direction). A portion of the illuminating light reflected from the eyeball is converged on a CCD
14
through the light-receiving lens
12
. Here, reference numeral
17
a
denotes the optical axis of the eyeball,
17
b
denotes cornea, and
17
c
denotes iris.
FIG. 26A
is a schematic diagram showing an eyeball image projected to the CCD
14
, and
FIG. 26B
is a diagram showing an intensity distribution of the signal from the output line of the CCD
14
. Hereinafter, descriptions of the sight line detection will be made with reference to
FIGS. 25
,
26
A, and
26
B.
The infrared light projected from the IRED
13
b
illuminates the cornea
17
b
of the user's eyeball
17
. Herein, the image d reflected from the cornea (virtual image; hereinafter referred to as a “Purkinje image” or a “P-image”), which image d is formed of a portion of the infrared light reflected from the surface of the cornea
17
b
, is condensed by the light-receiving lens
12
, and forms an image at the position d′ on the CCD
14
. Likewise, the infrared light projected from the IRED
13
a
illuminates the cornea
17
b
of the user's eyeball
17
. Herein, the Purkinje image e formed of a portion of the infrared light reflected from the surface of the cornea
17
b
, are condensed by the light-receiving lens
12
, and forms an image at the position e′ on the CCD
14
.
The light flux from the end portions a and b of the iris
17
c
form images of the end portions a and b at the positions a′ and b′ on the CCD
14
, respectively, through the light-receiving lens
12
. When the rotational angle &thgr; of the optical axis of the eyeball
17
with respect to the optical axis of the light-receiving lens
12
is small, letting the x-coordinates of the end portions a and b of the iris
17
c
be xa and xb, respectively, the coordinate xc of the center position of the pupil
17
d
is expressed by the following equation.
xc
≅(
xa+xb
)/2 (1)
Also, the x-coordinate of the midpoint between the Purkinje images d and e substantially coincides with the x-coordinate X
0
of the center of curvature o of the cornea
17
b.
Therefore, if the x-coordinates of the occurrence positions d and e of the Purkinje images are represented by (Xd′, Xe′), and the standard distance from the center of curvature o of the cornea
17
b
to the center c of the pupil
17
d
is represented by Loc, the rotational angle &thgr;x of the eyeball optical axis
17
a
of the eyeball
17
satisfies the following relation.
Loc
×sin &thgr;
x
≅(
Xd′+Xe
′)/2
−xc
(2)
Therefore, by detecting the positions of each of the characteristic points of the eyeball
17
(the centers of each of the Purkinje images and the pupil) projected on the CCD
14
, the rotational angle &thgr; of the eyeball optical axis
17
a
of the eyeball
17
can be determined.
The rotational angle &thgr; of the eyeball optical axis
17
a
are determined based on the above equation (2) as follows:
&bgr;×
Loc
×sin &thgr;
x
≅{(
Xp
0
−&dgr;x
)−
Xic}×Ptx
(3)
&bgr;×
Loc
×sin &thgr;
y
≅{(
Yp
0
−&dgr;y
)−
Yic}×Pty
(4)
Here, &bgr; denotes an image-forming magnification determined by the position of the eyeball
17
with resect to the light-receiving lens
12
, and is virtually obtained as a function of the distance |Xd′−Xe′| between the two Purkinje images.
Also, &thgr;x and &thgr;y denote the rotational angles of the eyeball optical axis on the z-x plan, and y-z plan, respectively. (Xp0, Yp0) represents the coordinates of the midpoint between the two Purkinje images on the CCD
14
, and (Xic, Yic) represents the center coordinates of the pupil. Ptx and Pty denote pixel pitches in the direction of the x-axis and the y-axis, respectively. &dgr;x and &dgr;y denote correction terms for correcting the coordinates of the midpoint of a Purkinje image. The correction terms include a component for correcting errors occurring by the user's eyeball being illuminated not by parallel light but by diverging light. Here, &dgr;y includes also a component for correcting an offset component occurring by the user's eyeball being illuminated by diverging light from the lower eyelid.
When calculating the rotational angle (&thgr;x, &thgr;y) of the optical axis
17
a
of the user's eyeball, the user's gazing point (x, y) on the observed surface is determined by the following expression.
X[mm]=m×ax×
(&thgr;
x+bx
) (5)
Y[mm]=m×ay×
(&thgr;
y+by
) (6)
Here, the x-axis direction means the horizontal direction with respect to the observer, and the y-axis direction means the vertical direction with respect to the observer. Coefficient m denotes the transformation coefficient for performing a transformation from the rotational angle of the eyeball
17
to the coordinates of the observed surface. Coefficients ax, bx, ay, and by each denote gazing point calibration coefficient, and each correspond to correction coefficient for making the rotational angle of the user's eyeball coincide with the gazing point on the observed surface.
Now, a calculation method for the correction terms &dgr;x and &dgr;y for the midpoint position between Purkinje images will be described with reference to
FIGS. 27
to
29
.
The intersection point between the straight line connecting the IRED
13
a
and the IRED
13
b
and the optical axis (z-axis) of the sight line detecting optical system is set to the origin point. The IREDs
13
a
and
13
b
are disposed along the X-axis direction so as to be substantially symmetrical with respect to the origin point, and the x-ordinates and y-ordinates thereof are each the same. Let the ordinates of the IRED
13
a
be (−Sxi, Syi, Szi), the ordinates of the IRED
13
b
be (Sxi, Syi, Szi), and the ordinates of the center o of curvature of the cornea of the photographer's eyeball be (Sxc, Syc, Szc). Also, let the center coordinates of the CCD be (Xs, Xy).
The midpoint P0 between the Purkinje images d and e is equivalent to the position of the Purkinje image occurring due to a single IRED disposed at the midpoint between the IREDs
13
a
and
13
b
. Since the sight-line calculation equation is based on the coordinates of the midpoint P0 between the two Purkinje images, if the distance from the midpoint (0, Si, Zi) between th
Canon Kabushiki Kaisha
Fitzpatrick ,Cella, Harper & Scinto
Perkey W. B.
LandOfFree
Device having sight line detecting function does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Device having sight line detecting function, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Device having sight line detecting function will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3073417