Multiplex communications – Special services – Conferencing
Reexamination Certificate
1998-01-28
2001-09-04
Nguyen, Chau (Department: 2663)
Multiplex communications
Special services
Conferencing
C348S014090
Reexamination Certificate
active
06285661
ABSTRACT:
BACKGROUND
This invention relates generally to real-time multipoint video conferencing.
Video teleconferencing systems allow for the simultaneous exchange of audio, video and data information among a plurality of audio-video terminals. In multipoint video conferencing, typically three or more participants are involved in a video conference. The audio, video and data signals associated with each participant are typically compressed by a user audio-video terminal (AVT) and sent to a multipoint control unit (MCU) for further processing. The multipoint control unit performs switching functions to allow all of the three or more audio-video participants to communicate in a video conference. A principal function of an MCU is to process the received signals and transmit the processed, received signals back to the user terminals. The MCU links multiple video conferencing sites together by receiving data units of digital signals from the audio-video terminals, processes the received data units and retransmits the processed data units to appropriate audio-video terminals as data units or frames of digital signals.
The digital signals include audio information, video information, data and control information. The audio signals from two or more audio-video terminals are mixed to form a composite audio signal. The audio processing typically is relatively straightforward. The audio signals are decoded and summed to provide a composite signal. The composite signal is re-encoded as one audio signal. The re-encoded, summed audio signal is transmitted to those terminals whose audio is not contained in the summed signal. Thus, the participants at each of the terminals can hear what the other participants are saying. Audio encoding can be selective, for example, audio encoding can encode the two or three loudest audio signals in the videoconference. Other arrangements are possible.
Video processing, however, is more difficult since there is no simple way to sum several video signals. There are two ways for a multipoint control unit to handle a video signal. In the so-called “switched video mode” one video source is selected as the broadcaster and is sent to all of the terminals. Typically, the broadcaster is the current speaker who receives video from a previous speaker. In this mode, essentially no video processing is needed except for switching the video source. In a second mode, the so-called “continuous presence” mode, multiple, compressed video bit streams are received by the MCU. These bit streams are processed and combined into one video bit stream so that participants can view multiple persons simultaneously. The combination of several digital bit streams into one stream is also known as “digital video mixing.” While non-realtime video mixing is relatively easy, real-time video mixing presents significant challenges because it requires highly complex processing. For example, the delay incurred in processing the video bit streams has to be as small as possible so as to facilitate desirable interaction among the conference participants.
SUMMARY
In one aspect of the invention, a method of operating a multipoint control unit for use in a video conference includes extracting segment data from a plurality of video streams, storing the segment data in a corresponding one of a plurality of data queues and determining whether there is data in any of the plurality of data queues that can be included in a new picture. The method combines data from said plurality of data queues in accordance with the determination of whether there is data in the data queues. The output picture rate after combining is related to a maximum input picture rate of said video streams.
In a further aspect of the invention, a method of operating a multipoint control for use in videoconferencing of a plurality of audio-video terminals includes feeding a plurality of video streams to the multipoint control unit, and for each one of said streams, extracting segments data. The segments of data are output to a data queue, and a new picture is formed from the segments of data stored in each of the data queues by including in the new picture data from those queues that have a complete picture. If no queue has a complete picture, the method determines for each of the queues whether the queues have a complete segment of data and, for those queues having a complete segment of data, the method includes data from those queues in the new picture.
In a still further aspect of the invention, a method of operating a multipoint control unit for use in videoconferencing of a plurality of audio-video terminals includes feeding a plurality of video streams to the multipoint control unit. For each one of said streams, extracting segments of data, and adaptively reducing the output bit rates. The segments of data are output to a data queue and a new picture is formed from the segments of data stored in each of the data queues by including in the new picture data from those queues that have a complete picture. If no queue has a complete picture, the method determines for each of the queues whether the queues have a complete segment of data and, for those queues having a complete segment of data, the method includes data from those queues in the new picture.
In a still further aspect of the invention, a computer program product residing on a computer readable medium includes instructions for causing a multipoint control unit to extract segment data to provide a plurality of extracted segment streams, store each one of the segment data streams in a corresponding one of a plurality of data queues, determine which of said data queues has data for inclusion in a new picture, and combine data from said plurality of data queues in accordance with the determination of which data queues have data for inclusion in a new picture.
In a still further aspect of the invention, a multipoint control unit includes a plurality of segment processors arranged to extract segment data from a corresponding plurality of input video streams. The multipoint control unit further includes a plurality of data queues, each one of said data queues being coupled to a corresponding one of said plurality of segment processors for storing segment data provided from said segment processors, a segment combiner coupled to said data queues for combining output data from said data queues, and an optional controller coupled to said combiner for selectively providing rate control data to the segment processors.
One or more of the following advantages are provided by the above. The output picture rate after combining is related to a maximum input picture rate of the video streams. The queue fullness of each of the queues is used to determine from which queues to provide data. Alternatively, the number of segments of data awaiting service for each of the queues is used to determine from which queues to provide data or to determine whether any of the queues has a complete picture of data awaiting transmission.
REFERENCES:
patent: 5453780 (1995-09-01), Chen et al.
patent: 5600646 (1997-02-01), Polomski
patent: 5936662 (1999-08-01), Kim et al.
Kerofsky Louis J.
Zhu Qin-Fan
Boakye Alexander D.
Nguyen Chau
PictureTel Corporation
Ropes & Gray
LandOfFree
Low delay real time digital video mixing for multipoint... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Low delay real time digital video mixing for multipoint..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Low delay real time digital video mixing for multipoint... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2512213