Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
1999-02-03
2002-02-05
Mancus, Joseph (Department: 2621)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S295000, C382S294000, C348S239000
Reexamination Certificate
active
06345129
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to correcting distortion in images by computer, and to scanning wide fields of view by moving a lens over a detections system.
REVIEW OF THE RELATED TECHNOLOGY
An electronic camera uses a conventional lens but in place of film employs an array of electronic light sensor elements such as CCD's (charge coupled devices). Typically the array is rectangular. Signals from the respective sensors may be converted directly into an image on a display, for example a corresponding rectangular array of LED's (light emitting diodes). The sensor signals may also be stored in a computer memory, or processed by a computer CPU.
Conventional cameras using electronic pickup arrays are known in which a small sensor array is movable about the image plane. Such cameras cover a wider field of view, with a given lens, than would otherwise be possible. One example is described by Topps in U.S. Pat. No. 5,206,503. The sensor array, smaller than the usable focal plane area of the lens, is movable within the image plane in two directions, denoted X and Y. If the sensor array is coupled to a TV, the image translates as the sensor undergoes X-Y motion, even when the lens is stationary.
Very similar to Topps is U.S. Pat. No. 4,928,174 to Smith. Smith discloses a surveillance camera providing a narrow field of view, i.e. a relatively high magnification, with a lens focussing on a relatively small CCD array for video pickup. The CCD array is movable, within the focal plane, to change the view. FIGS. 6
a
-6
c
of Smith show the basic idea. The movable video pickup (CCD array) provides a mechanical improvement over the more conventional method of moving the entire camera body around because the sensor array is smaller, better protected, etc.
Although it is not of concern to Smith, the images from Smith's movable-pickup camera are not exactly the same as those provided by the more conventional swivelling-body camera in which the lens, and its optical axis, change direction. This is because of perspective.
When an amateur photographer takes a picture of a tall building, invariably the camera is tilted up: both the optical axis of the lens and the film at an angle. On the resulting photograph the building is in the shape of a triangle, with its parallel sides inclined on the paper. (The same effect is seen in a photograph taken of railroad tracks when the camera lens optical axis is parallel to the tracks: they converge to the horizon on the photographic print, even though they are of course actually parallel.)
In contrast, an architectural photographer may take a picture of the same building, from the same vantage point, but using a view camera having adjustments not available on other cameras. These adjustments allow the lens's optical axis to remain horizontal and permit the film to move downward while the film plane remains perpendicular to the lens axis. In the resulting photograph the sides of the building appear as they are, parallel.
The more conventional surveillance camera, in which the axis tilts, provides a perspective like that of the nonadjustable camera, while Smith's movable-pickup camera mimics the professional view camera and provides a different perspective.
Smith does not comment on this perspective difference, and for simple surveillance purposes with moderate lenses the perspective is not crucial. However, in the case of an extreme-wide angle lens and a pickup movable to positions far off the optical axis, the distortion produced would become noticeable. Photographs taken with extreme wide-angle lenses, e.g. a 15-mm lens on a 35-mm camera, appear distorted even when the lens actually has no “distortion” in the sense used by opticians or lens designers. (An optician considers a lens to distort when the rays from objects to the corresponding points on the image plane does not all pass through a single point inside the lens, that is, when the angle of the image point off the optical axis is the same as the angle of the object point off the optical axis. Such distortion causes a straight object to have a curved image. “Pin-cushion” and “barrel” distortions are examples.)
Moreover, even if the lens is optically perfect (without opticians' distortion) and the angle is not extreme, the images from different lens positions do not correlate. For example, suppose a person is photographing a mountain scene with a camera having a horizontal field of view of 30°. She takes a picture with the camera pointed north, then another with the camera pointed 30° to the east, then another pointed 60° to the east, and so on. She prints the pictures and then constructs a panorama by aligning the edges of the pictures. It will be found that the photograph images do not correlate correctly at the photograph borders.
To be more precise, there will be a point-to-point matchup at the border between two photographs, so that a line crossing the border (e.g. a telephone wire) will meet there. But the angles of the line in the two photos will be different: objects which are actually straight will appear to bend at the border. This effect will be referred to as “angular distortion”.
It will be found that angular distortion becomes worse as the field of view increases If the photographer in the example above had used a camera with a 45° field of view and shoot pictures at 45° increments, the distortion would be worse. If a telephoto lens having a narrow field were used, the effect would be negligible and the photographs could be juxtaposed without noticeable angular distortion.
Because of this effect, panoramic cameras confine the film to one vertical strip at the image plane and rotate the camera to provide a continuous change in the azimuthal angle of the lens optical axis. This avoids the sharp bends in the images of such objects as roof lines which occur with panoramas made of pasted-together photographs. In a panoramic picture the image of a straight object such as a roof line often appears as a curved line on the photograph.
The origin of angular distortion is two-fold.
First, the outer portions of an image are magnified as compared to the inner portions, because the film is farther from the optical center of the lens. Referring to
FIG. 1
, the optical center C of the lens L is the point inside the lens which corresponds to the position of a pin-hole lens producing the same image. The distance from the lens center C to the plane of the sensor-element pixel array SP is clearly greater when the image points are away from the optical axis A.
As in a telephoto lens, which has a long focal length, the image at the corners of the image plane (coinciding with the pixel array SP) is enlarged because the effective focal length of the lens L is greater at the corners than in the center.
Consider the two point objects OP separated by a small angle &agr;. The distance between their image points on the sensor array SP, denoted as image pair IP, is a function of &thgr;. When the two close-set points OP are both near to the optical axis their separation on the film is almost exactly equal to the focal length of the lens times the angular separation between them, measured in radians; but when offset from the axis as illustrated in
FIG. 1
their two image points IP will be separated by a greater amount, namely the on-axis distance divided by the cosine of &thgr;.
In other words, the magnification factor is 1/(cos &thgr;). Because the function cosine &thgr; is approximately constant and equal to one in the neighborhood of &thgr;=0°, there is little angular distortion there. But as the off-angle increases, so does distortion.
The second origin of angular distortion has to do with the tilt of the focal or film plane with increasing &thgr;. The distance between the image pair of points IP will move farther apart from one another as &thgr; increases (while the angular separation &agr; of the object pair OP stays constant), but will move apart still faster if the line between them passes through the optical axis, as compared to the case where the line be
Browdy and Neimark
Kassa Yosef
Mancus Joseph
LandOfFree
Wide-field scanning tv does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Wide-field scanning tv, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Wide-field scanning tv will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2975797