Television – Basic receiver with additional function – For display of additional information
Reexamination Certificate
1999-06-10
2001-03-20
Hsia, Sherrie (Department: 2714)
Television
Basic receiver with additional function
For display of additional information
C348S568000, C348S596000, C348S588000
Reexamination Certificate
active
06204887
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to methods and apparatus for decoding and displaying images and, more particularly, to methods and apparatus for decoding multiple images to be displayed at the same time, using limited resources.
BACKGROUND OF THE INVENTION
Television sets and many other video display devices, e.g., computers, support the ability to simultaneously display multiple images. In the case of television sets, the ability to display multiple images at the same time is often called picture-in-picture (PIP) capability. In the case of computer systems, the display of multiple images at the same time often occurs when multiple windows are open simultaneously in a system which supports windows based user interfaces.
In order to reduce data storage and transmission bandwidth requirements, video signals and images, e.g., digital television signals, are normally compressed prior to transmission and/or storage. Even in compressed, e.g., encoded form, video images can comprise a large amount of data. Before being displayed they must normally be decoded. Decoding of images can require a considerable amount of processing resources.
When video programs comprising sequences of images are being displayed, the images in each window need to be updated as the image sequence being displayed in a given window changes, e.g., due to motion in the images being displayed. Accordingly, to display a video sequence including motion, multiple images represented by, e.g., frames, need to be decoded and displayed over a period of time.
The number of images which are to be displayed in a given period of time for a given video image is often called the frame rate. It is frequently expressed in terms of frames per second. The time used to display a single image is called a frame time.
In the case of a television, decoder circuitry is often subject to real time processing constraints. In other words, the video images must be decoded in an amount of time equal to, or less than, the time that will be used to display the images. Failure to meet real time decoding requirements can result in the loss of data, e.g., frames, which may give the appearance of jerky motion. It may also cause other image errors and/or distortions which can result in an unpleasant and/or undesirable viewing experience.
When video sequences are being displayed in multiple windows at the same time, to support real time decoding of all the images in the video sequences being displayed, a decoder must be able to decode, on average, a number of images at a rate equal to the sum of the frame rates for each of the individual video sequences being displayed.
In addition to decoder processing resources, supporting the simultaneous display of multiple images and/or video sequences can place a considerable burden on other system resources as well. The large amount of data associated with video images which must be transferred for decoding, in some cases, can exceed a system's available bus bandwidth. This is particularly the case when multiple images are to be decoded and displayed simultaneously.
The demand for decoder processing and/or bus resources is a function of the number of images which must be decoded and displayed in a fixed amount of time. In the case of a PIP embodiment, such as a PIP capable television system, the demands on video decoder processing and bus resources can be considerable.
When more than two video sequences are displayed on a system, e.g., a television or computer, if the total load on the system resources, e.g., CPU and/or bus, required to decode and display the video sequences exceeds the available resources, images in the sequences will normally be degraded and/or discarded. The inability to fully decode images to be displayed in all of a systems open windows may result in jerky motion in one or more image sequences.
This may cause a viewer of the decoded images to feel uncomfortable. In addition, it might also make it difficult for a viewer to follow the video sequences, e.g., TV programs, being displayed. Such consequences of limited system resources could potentially defeat the purpose of displaying multiple images on a single screen.
In view of the above remarks, it is apparent that there is a need for methods and apparatus for allocating resources when multiple images, or multiple sequences of images, are to be decoded and displayed on a single display device. There is also a need for reducing or limiting the demand for decoder and display resources, e.g., to levels which are available in a given system.
SUMMARY OF THE PRESENT INVENTION
The present invention relates to methods and apparatus for processing data representing multiple image sequences, e.g., TV programs, and displaying multiple images from different video sequences in different windows of a single display screen. The methods and apparatus of the present invention can be used to implement televisions, computers and various other systems which display images.
The amount of system resources required to process and display image data representing, e.g., TV programs, is a function of the size of the windows in which the program images are displayed. In accordance with the present invention demand for system resources is assessed and compared to the amount of available system resources. If the demand for system resources will exceed the available system resources, the size of one or more of the windows used to display secondary images, e.g., the small PIP windows, is reduced to reduce the burden, e.g., processing or bus bandwidth burden, on one or more system resources. The evaluation of the demand on processing resources and adjustment of display window sizes may be, and in various embodiments is, performed periodically e.g., at pre-selected time intervals corresponding to multiple frame times.
In one embodiment, the system suggests a set of window sizes to the user. The suggested window sizes allow for the decoding and display of programs without exceeding the available system resources. The user can accept or modify the suggested program window sizes. When the user input results in the use of window sizes that will result in a demand for system resources that exceeds the available resources, data corresponding to inter-coded frames is ignored and/or decoding and display of images in one or more secondary programs is limited to images with significant scene changes e.g., images with more than 40% new content from the previous frame.
In one PIP embodiment, the secondary programs are the programs displayed in the smaller windows as opposed to the main picture window.
Thus, in accordance with the techniques of the present invention, the processing of data to be displayed in multiple windows at the same time can be done in a manner that does not exceed the available system resources, e.g., processor or bus bandwidth resources.
In addition to the above described embodiments and features, numerous additional features, embodiments and advantages of the methods and apparatus of the present invention are discussed below.
REFERENCES:
patent: 4238773 (1980-12-01), Tsuboka et al.
patent: 4845644 (1989-07-01), Anthias et al.
patent: 4931879 (1990-06-01), Koga et al.
patent: 5091785 (1992-02-01), Canfield et al.
patent: 5237460 (1993-08-01), Miller et al.
patent: 5313303 (1994-05-01), Ersoz et al.
patent: 5335117 (1994-08-01), Park et al.
patent: 5361098 (1994-11-01), Lucas
patent: 5367318 (1994-11-01), Beaudin et al.
patent: 5386241 (1995-01-01), Park
patent: 5390052 (1995-02-01), Kato et al.
patent: 5398072 (1995-03-01), Auld
patent: 5408270 (1995-04-01), Lim
patent: 5420641 (1995-05-01), Tsuchida
patent: 5433677 (1995-07-01), Do
patent: 5463422 (1995-10-01), Simpson et al.
patent: 5469223 (1995-11-01), Kimura
patent: 5477397 (1995-12-01), Naimpally et al.
patent: 5614957 (1997-03-01), Boyce et al.
patent: 5642498 (1987-06-01), Kutner
patent: 0 537 030 A2 (1993-04-01), None
patent: WO 94/09595 (1994-04-01), None
A. Hoffmann, B. Macq and J.J. Quisquater, “Future Prospects of the Cable TV Networks, New Technologies and New Services
Hitachi America Ltd.
Hsia Sherrie
Straub Michael P.
Straub & Pokotylo
LandOfFree
Methods and apparatus for decoding and displaying multiple... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Methods and apparatus for decoding and displaying multiple..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Methods and apparatus for decoding and displaying multiple... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2483458