Image analysis – Image transformation or preprocessing – Selecting a portion of an image
Reexamination Certificate
1999-04-21
2002-12-17
Couso, Jose L. (Department: 2721)
Image analysis
Image transformation or preprocessing
Selecting a portion of an image
C382S224000, C382S232000
Reexamination Certificate
active
06496607
ABSTRACT:
The invention relates generally to a system for processing images and, more particularly, to an apparatus and a concomitant method for identifying and using region(s) of interest to provide functionalities such as zooming, composition, selective input image formation and adaptive allocation of processing resources, e.g., bit allocation.
BACKGROUND OF THE INVENTION
An image sequence, such as a video image sequence, typically includes a sequence of image frames or pictures. The reproduction of video containing moving objects typically requires a frame speed of thirty image frames per second, with each frame possibly containing in excess of a megabyte of information. Consequently, transmitting or storing such image sequences requires a large amount of either transmission bandwidth or storage capacity. To reduce the necessary transmission bandwidth or storage capacity, the frame sequence undergoes image processing, e.g., compression, such that redundant information within the sequence is not stored or transmitted. Television, video conferencing and CD-ROM archiving are examples of applications, which can benefit from efficient video sequence encoding.
Additionally, in an image processing environment where processing resources are limited or constrained by the requirements of a particular application, it is necessary to carefully allocate the available resources. Namely, although many powerful image processing methods are available, powerful image processing methods are not practical or must be sparingly and selectively applied to meet application requirements.
For example, in real-time application such as videophone or video conferencing, the talking person's face is typically one of the most important part of an image sequence. The ability to detect and exploit such regions of importance will greatly enhance an encoding system.
For example, the encoding system in a low bitrate application (e.g., real-time application) must efficiently allocate limited bits to address various demands, i.e., allocating bits to code motion information, allocating bits to code texture information, allocating bits to code shape information, allocating bits to code header information and so on. At times, it may be necessary to allocate available bits such that one parameter will benefit at the expense of another parameter, i.e., spending more bits to provide accurate motion information at the expense of spending less bits to provide texture information. Without information as to which regions in a current frame are particularly important, i.e., deserving of more bits from a limited bit pool, the encoder may not allocate the available bits in the most efficient manner.
Furthermore, although the encoder may have additional resources to dedicate to identified regions of importance, it is often still unable to improve these regions beyond the quality of the existing input image sequence. Namely, changing the encoding parameters of the encoder cannot increase the quality of the regions of importance beyond what is presented to the encoder.
Therefore, there is a need in the art for an apparatus and a concomitant method for classifying regions of interest in an image, based on the relative “importance” of the various areas and to adaptively use the importance information to allocate processing resources and to control manipulation of the input image sequence prior to encoding.
SUMMARY OF THE INVENTION
An embodiment of the present invention is an apparatus and method for classifying regions of an image as important or region(s) of interest. The parameters that contribute to such classification may initially be derived from a block classifier that detects the presence of facial blocks, edge blocks and motion blocks. Such detected blocks can be deemed as important blocks and is then collected and represented in an “importance map” or “class map”.
Additionally, other parameters can be used in the generation or refinement of the importance map. Namely, a voice detector can be employed to detect and associate a voice to a speaker in the image sequence, thereby classifying the region in the image that encompasses the identified speaker as important or a region of interest. Furthermore, additional importance information may include user defined importance information, e.g., interactive inputs from a user that is viewing the decoded images.
Once the importance information is made available, the present invention allocates processing resources in accordance with the importance information. For example, more bits are allocated to “important” regions as compared to the less “important” regions; more motion processing is applied to “important” regions; coding modes are changed for “important” regions; and/or segmentation processing is refined for “important” regions as well.
In another embodiment, the formation of the input image sequence is also accomplished in accordance with the importance information. Namely, a higher resolution for the identified regions of interest is acquired from a higher quality source, e.g., directly from an NTSC signal, to form the input image sequence prior to encoding. Such input image sequence formation allows functionalities such as zooming and composition. Thus, the relative “importance” of the various areas of a frame is rapidly classified and used in resource allocation and input image formation.
REFERENCES:
patent: 5432871 (1995-07-01), Novik
patent: 5550580 (1996-08-01), Zhou
patent: 5675374 (1997-10-01), Kohda
patent: 5737011 (1998-04-01), Lukacs
patent: 5844599 (1998-12-01), Hildin
patent: 5973738 (1999-10-01), Srivastava
patent: 6011901 (2000-01-01), Kirsten
patent: 6023296 (2000-02-01), Lee et al.
patent: 6044115 (2000-03-01), Horiike et al.
Patent Abstracts of Japan vol. 1997, No.-, Apr. 30, 1997 & jP 08 336505 A 9Hitachi Medical Corp), Dec. 24, 1996 abstract.
Schumeyer R P et al: “Region of Interest priority coding for sign language videoconferencing” 1997 IEEE First Workshop of Multimedia Signal Processing (Cat. No. 07IH8256), Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing, Princeton, NJ, USA, Jun. 23-35, 1997, pp. 531-536, XP9002116570 1997, New York, NY, USA, IEEE, USA ISBN: 0-7803-3780-8 p. 533, paragraph 4-page 534, paragraph 3.
Vlahakis V et al: “ROI approach to wavelet-based, hyubrid compression of MR images” Sixth International Conference on Image Processing and Its Applications (Conf Publ. No. 443), Proceedings of 6th International Conference on Image Processing and Its Applications (Conf. Publ. No.443), Dublin Ireland, Jul. 14-17, 1997, London, UK, IEE, UK ISBN: 0-85296-692-x Abstract p. 834, right-hand column, line 27-line 42 page 833, right-hand column, line 19-line 25.
PCT International Search Report PCT/US99/14543 corresponding to U.S. application No. 09/295,526.
“A Color-Based Classifier for Region Identification in Video”, R. Schumeyer and K. Barner, SPIE Proceedings, vol. 3309, Jan. 1999, pp. 189-200.
Krishnamurthy Ravi
Sethuraman Sriram
Zhang Ya-Qin
Burke W. J.
Couso Jose L.
Patel Kanji
Sarnoff Corporation
LandOfFree
Method and apparatus for region-based allocation of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for region-based allocation of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for region-based allocation of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2935286