Image analysis – Image compression or coding
Reexamination Certificate
1997-12-23
2001-06-26
Johns, Andrew W. (Department: 2621)
Image analysis
Image compression or coding
C382S236000, C382S240000, C382S248000, C345S205000, C348S387100, C348S390100, C358S426010
Reexamination Certificate
active
06252989
ABSTRACT:
1.1. FIELD OF THE INVENTION
The present invention relates generally to the field of image data compression. More specifically, it relates to a foveated imaging system which can be implemented on a general purpose computer and which greatly reduces image transmission bandwidth requirements.
1.2. DESCRIPTION OF THE RELATED ART
The human visual system provides detailed information only at the point of gaze, coding progressively less information farther from this point. This provides an efficient means for the visual system to perform its task with limited resources. Processing power is thus devoted to the area of interest and fewer neurons are required in the eye. Remarkably, we rarely notice the severely degraded resolution of our peripheral visual field; rather, we perceive the world as a single high resolution image that can be explored by moving our eyes to regions of interest. Imaging systems utilizing this fact, termed foveation, greatly reduce the amount of information required to transmit an image and therefore can be very useful for a number of applications including telemedicine, remote control of vehicles, teleconferencing, fast visual data base inspection, and transmission of pre-recorded video.
Recently, there has been substantial interest in such foveated displays. The U.S. Department of Defense has studied and used so-called “area-of-interest” (AOI) displays in flight simulators. These foveation schemes typically consist of only 2 or 3 resolution areas (rather than the superior continuous resolution degradation) and the central area of high resolution is often quite large, usually between 18° and 40° (see, for example, Howard, 1989; Warner, et al., 1993). Other researchers have investigated continuous, variable-resolution methods using a log polar pixel configuration (Weiman, 1990; Juday and Fisher, 1989; Benderson et al., 1992). The log polar configurations are particularly advantageous when rotation and zoom invariance are required, but their implementations have necessitated special purpose hardware for real-time operation.
Juday and Sampsell (U.S. Pat. No. 5,067,019) describe an apparatus which will encode a number of space-variant schemes. Fisher later refined the apparatus (U.S. Pat. No. 5,208,872). While otherwise effective, these systems are limited in that a specific hardware apparatus is necessary to perform the foveation operations. Further, these systems have utilized an Archimedes spiral to represent the falloff function applied to the image in their descriptions of the foveation method. While this may be computationally efficient for their specific hardware implementation, it does not as accurately represent the actual resolution falloff parameters of the human visual system, and does not provide the degree of flexibility in controlling the resolution function as the methods proposed here. Optimal compression and image quality are obtained by closely representing the actual resolution falloff parameters of the human visual system. The system of Weimans (U.S. Pat. No. 5,103,306), is more closely related to the apparatus described herein. However, implementation of the Weimans system requires specific hardware, and the foveation occurs in log polar coordinates, rather than Cartesian coordinates. Further, the algorithms employed during the compression and reconstruction of the foveated image create “cartoon-like” images (page 6, line 52, U.S. Pat. No. 5,103,306). Wallace, Benderson and Schwartz have also done work in the area of space-variant image processing (U.S. Pat. No. 5,175,617). They too, used a long polar mapping scheme, but because of the algorithms they utilized to perform the compression, the transmission rates were restricted to 4 frames per second, well below the rates required for near perceptually loss-less encoding.
2.0. SUMMARY OF THE INVENTION
The present invention overcomes the limitations in the prior art by providing a system that accomplished real-time foveated image compression and display, using general purpose computer processors, video hardware, and eye-tracking equipment. The foveated imaging system proposed here generates images that are nearly imperceptible from uncompressed images to the human observer. Further, the present invention makes use of a novel application of modified pyramid coding methods for creating foveated images. These coding methods provide higher quality images, at high compression rates, with little or no blocking or aliasing artifacts. The system can utilize one or more computer processors; the number of processors used depends upon the particular application and image-processing requirements.
2.1. Two-Processor FIS With Eye Tracking
In the inventors' Foveated Imaging System (FIS), eye movements are recorded in real time and are used to construct and display variable resolution images centered at the current point of gaze (fixation point). Typical operation of an exemplar FIS
10
proceeds as follows with reference to
FIG. 1
for components and
FIG. 2
for processes. First, the location of an observer
11
fixation point
12
on a display monitor
14
is measured with an eye tracking device
16
as depicted in process block
30
of FIG.
2
. Second, the eye position coordinates are transmitted to a remote computer
18
as depicted in process block
32
. Third, as represented by process block
34
of
FIG. 2
, remote computer
18
captures an image from camera
20
. Fourth, the camera image is foveated (i.e., encoded and compressed so that the resolution of the image decreases from the point of fixation) as shown in process block
36
. In other words, the degree of data compression increases with the distance from the point of fixation. Fifth, the encoded image is transmitted by communications channel
21
to a local computer
22
as shown in process block
38
. Sixth, as depicted by process block
40
, the received image is decoded and displayed on video monitor
14
such that the highest resolution region in the displayed image is centered at the fixation point
12
. These six steps are repeated continuously in a closed loop. The system has been implemented in C
++
for execution on standard PC compatible processors, including Intel Pentium® and Pentium® Pro, but other general purpose processors could also be used.
FIS
10
can operate in any one of several different modes. The different modes correspond primarily to different methods of foveated image encoding and decoding. Mode 1 is the simplest and quickest, and hence is most appropriate for large images and/or high video frame rates. Mode 2 produces substantially better data compression, for the same image quality, but is somewhat slower. Mode 3 produces the greatest data compression, but is even slower, and hence is most appropriate for smaller image sizes and/or low frame rates. Mode 4 is also slower and produces somewhat poorer image quality than Modes 2 and 3; it is included because of its wide-spread use. As general-purpose processors become more powerful, there will be an increasing number of situations where the high compression modes will be appropriate.
Mode 1: Foveated Pixel Averaging
In Mode 1, variable size pixels (SuperPixels) are created by simple averaging (Kortum and Geisler, 1996). The sizes of the SuperPixels increase smoothly away from the point of fixation in a manner that matches the decline in resolution of the human visual system away from the point of fixation. The color of each SuperPixel is obtained by averaging the RGB (or gray level) values of the image pixels that fall within the boundary defined by the edges of the SuperPixel. The collection of integration boundaries defined by the edges of the SuperPixel is called the ResolutionGrid.
FIG. 3
illustrates one of the ResolutionGrids used in the system. ResolutionGrid
50
consists of a series of concentric rings of SuperPixels that increase in size away from the point of fixation, x
0
, y
0
. Because each of the SuperPixels is rectangular, SuperPixels can be represented with two pairs of coordinates (which define the summation bounds). This allows fast i
Geisler Wilson
Kortum Philip
Board of the Regents, The University of Texas System
Fulbright & Jaworski
Johns Andrew W.
Mariam Daniel G.
LandOfFree
Foveated image coding system and method for image bandwidth... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Foveated image coding system and method for image bandwidth..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Foveated image coding system and method for image bandwidth... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2501590