Intelligent streaming framework

Details Intelligent streaming framework Intelligent streaming framework

2001-03-27

2004-07-20

Myers, Paul R. (Department: 2112)

Electrical computers and digital data processing systems: input/

Intrasystem connection

Bus interface architecture

C709S241000

Reexamination Certificate

active

06766407

ABSTRACT:

TECHNICAL FIELD
This invention relates generally to electronic data processing, and, more particularly, relates to managing the flow of streaming data in a network environment.
BACKGROUND OF THE INVENTION
Digitally based multimedia, the combination of video and audio in a digital format for viewing on a digital device is rapidly increasing in capacity and proliferation. Nearly every new personal computer manufactured today includes some form of multimedia. Sales of digital products such as cameras, video recorders, phones and televisions are steadily increasing. Multimedia is also becoming increasingly prevalent in the Internet realm as the growth of the Internet steadily and rapidly continues. Along with this growth has come increased performance expectations by the users of such computer equipment. These increased user expectations extend not only to hardware capability, but also to the processing capability of the data itself.
A technique known as streaming has been developed for multimedia applications to satisfy these increasing expectations. Streaming allows data to be transferred so that it can be processed as a steady and continuous stream. This has the benefit that data can be displayed or listened to before the entire file has been transmitted, a must for large multimedia files.
Initially, the streaming framework consisted of a chain of data processing modules (e.g., capturing filters, transformation filters, and rendering filters) with very little intelligence from the chain manager. The data processing modules, also called filters, make the decisions on how to get connected, what data formats to use, and how to control each other. During connection of filters in a chain, protocols define a predefined fixed sequence of data flow and control connection negotiations. A typical negotiation sequence is to negotiate the following in order: the interface, the medium, the data format, the allocators, and the master clock. The data processing chain provides an end to end solution within a computer system.
As the complexity of streaming increased, the industry recognized that it became necessary to optimize the processing chains that were processing data with real-time constraints such as video and audio processing chains. The Microsoft Windows operating system by Microsoft Corporation, for example, separates user mode and kernel mode. Filters operated entirely in user mode and called into the kernel mode to control the hardware to perform many of the functions needed to stream data. A stream class driver was developed to enable kernel mode streaming, which allowed greater efficiency and reduced latency over the previous generation of filters that operated entirely in user mode. As client and application requirements continued to change, it became clear that an even smarter framework was needed.
Industry responded in several ways. One way is the development of high-speed serial buses to interconnect between components on the same backplane, components on other backplanes, and external peripherals. These high-speed buses are used to increase data transfer speeds. One of the most prevalent buses is the IEEE 1394 standard (the “1394 bus”). Architectures such as the Home Audio/Video Interoperability Architecture (also called the HAVi Architecture) were developed to utilize the 1394 bus. The HAVi Architecture uses the JAVA programming language and connects to 1394 nodes first. However, these architectures introduced unnecessary limitations. For example, if the 1394 bus will not work for what is required (e.g., IP based streaming), the HAVi Architecture defers to a bridge to take control and handle what is needed. The preference for first connecting to the 1394 bus can result in situations where the 1394 bandwidth is being used by clients that do not need the speed of 1394 while clients that do need the speed are unable to access the bandwidth required for their application.
Another way that industry responded was the development of the JAVA Media Framework. The Java Media Framework was developed to incorporate time-based media processing into Java applications. The JAVA Media Framework requires a single master clock to be used, which places constraints on the types of “building blocks” one can use.
Another way that industry responded was the development of new standards and topologies to better control streaming data. Commonly assigned patent application Ser. No. 09/310,610 “Improving the Flow of Streaming Data through Multiple Processing Units,” filed May 12, 1999, introduces the concept of data pipes for enhancing the data flow of streaming-data frames through a chain of interconnected modules in streaming-data environments in a personal computer. The data pipes avoid redundant storage and copying of data as a number of modules process the data frames, and streamline allocation of the frames in which the data is packaged. Another commonly assigned patent application Ser. No. 09/310,597, “Improving the Control of Streaming Data through Multiple Processors,” filed May 11, 1999, provides a mechanism for controlling the flow of frames through multiple modules in a personal computer by improving the control from a chain-wide perspective, rather than optimizing each individual module separately. Any control component in the chain that is unnecessary to the overall operation of the graph is removed and the remaining components are then connected directly to each other. Commonly assigned application Ser. No. 09/310,596 “Efficient Splitting and Mixing of Streaming-Data Frames for Processing Through Multiple Processor Modules”, filed May 11, 1999, presents a mechanism for splitting a single frame of streaming data into multiple frames and for combining, merging, or mixing multiple streaming data frames into a single frame. Commonly assigned application Ser. No. 09/511,457 “Adaptive Control of Streaming Data in a Graph”, filed Feb. 23, 2000, provides a mechanism in a personal computer that provides timing and synchronization of streaming data flowing through a chain that has multiple modules and multiple clocks.
However, a wide variety of different formats and buses are now available to stream the data making it difficult to uniformly process this data. Additionally, a wide variety of different methods of compression and decompression of audio and video data and software have been created, which further complicates the processing of streaming data. For example, video data might be in ASF, WMA, AVI, CIF, QCIF, SQCIF, QT, DVD, MPEG-1, MPEG-2, MPEG-4, RealVideo, YUV9, or any other type of format. Audio data might be in MP3, AIFF, ASF, AVI, WAV, SND, CD, AU or other type of format. In many scenarios, different types of modules and devices within the network system need to be connected together to process the streaming data. For example, an audio and video clip might initially require MPEG decoding in a dedicated hardware module residing on a first network based on a first clock (e.g., 1394), rasterizing of the video fields in another hardware module, digital filtering of the audio in a software module, insertion of subtitles by another software module, parsing of the audio data to skip silent periods by a software module, D/A conversion of the video in a video adapter card residing on a second network (e.g., laptop) based on a second clock, and D/A conversion of the audio in a separate audio card on the second network based on a third clock. Users now expect these requirements to be implemented quickly and with minimum interruption.
Accordingly, there exists a need for a multimedia data streaming system that is capable of handling multiple networks (e.g., transports), data formats, control protocols, and clocks and that is capable of configuring modules and devices with minimal user input.
SUMMARY OF THE INVENTION
In view of the above described problems existing in the art, the present invention provides a system that analyzes the user requirements and searches for a solution based on the user requirements, end-to-end latency requirements (requested or derived), data formats, buses, gateways

Affiliated with

Also associated with

Rating

Intelligent streaming framework does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Intelligent streaming framework, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Intelligent streaming framework will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3221020