Computer graphics processing and selective visual display system – Computer graphics processing – Graphic manipulation
Reexamination Certificate
2001-02-14
2004-12-21
Bayerl, Raymond J. (Department: 2173)
Computer graphics processing and selective visual display system
Computer graphics processing
Graphic manipulation
C345S619000, C715S252000, C715S252000, C382S240000, C382S232000
Reexamination Certificate
active
06833840
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to computer network based multimedia application in general, and more particularly to PROTO implementation in MPEG-4.
BACKGROUND OF THE INVENTION
ISO/EEC 14496, commonly referred to as “MPEG-4,” is an international standard for coding of multimedia. Part 1 of the standard includes specifications for the description of a scene graph comprising one or more multimedia objects. Part 5 of the standard includes a software implementation of the specifications in the form of an MPEG-4 player. An MPEG-4 player parses a bitstream containing a scene description, constructs the scene graph, and renders the scene.
ISO/IEC 14496 specifies a system for the communication of interactive audio-visual scenes. This specification includes the following elements:
The coded representation of natural or synthetic, two-dimensional (2D) or three-dimensional (3D) objects that can be manifested audibly and/or visually (audio-visual objects);
The coded representation of the spatio-temporal positioning of audiovisual objects as well as their behavior in response to interaction;
The coded representation of information related to the management of data streams, including synchrorization, identification, description and association of stream content; and
A generic interface to the data stream delivery layer functionality.
The overall operation of a system communicating audiovisual scenes may be summarized as follows. At the sending terminal, the audio-visual scene information is compressed, supplemented with synchronization information, and passed to a delivery layer that multiplexes it into one or more coded binary streams that are transmitted or stored. At the receiving terminal, these streams are demultiplexed and decompressed. The audio-visual objects are composed according to the scene description and synchronization information, and are presented to the end user. The end user may have the option to interact with this presentation. Interaction information can be processed locally or transmitted back to the sending terminal. ISO/IEC 14496 defines the syntax and semantics of the bitstreams that convey such scene information, as well as the details of their decoding processes.
An audiovisual scene in MPEG-4 is composed from one or more individual objects or “nodes” arranged in an object tree, including primitive media objects that correspond to leaves in the tree and compound media objects that group primitive media objects together, encompassing sub-trees. For example, the primitive media objects corresponding to a visual representation of a talking person and his corresponding voice may be tied together to form a compound media object, containing both the aural and visual components of the talking person. Each node is of specific node-type, representing elements such as lines, squares, circles, video clips, audio clips, etc.
Scene description in MPEG-4 addresses the organization of audio-visual objects in a scene, in terms of both spatial and temporal attributes. This information enables the composition and rendering of individual audio-visual objects after the respective decoders have reconstructed the streaming data for them. The scene description is represented using a parametric approach, such as is defined by the Binary Format for Scenes (BIFS). The description consists of an encoded hierarchy/tree
100
of scene description nodes with attributes and other information, including event sources and targets, as illustrated in prior art FIG.
1
. Leaf nodes
102
of tree
100
correspond to elementary audio-visual data, whereas intermediate nodes
104
group this material to form compound audio-visual objects, and perform grouping, transformation, and other such operations on audio-visual objects. The scene description can evolve over time by using scene description updates. (Dashed lines indicate links to leaf nodes not shown).
In order to facilitate active user involvement with the presented audio-visual information, ISO/IEC 14496 provides support for user and object interactions. Interactivity mechanisms are integrated with the scene description information in the form of “routes,” which are linked event sources and targets, and “sensors,” which are special nodes that can trigger events based on specific conditions. These event sources and targets are part of scene description nodes, and thus allow close coupling of dynamic and interactive behavior with the specific scene at hand. Scene description node fields may be connected though routes, which cause a change in one node to affect the properties and behavior of another node. Constructing a scene involves parsing the scene hierarchy, node types, node fields values, and routes and linking them all in a scene graph.
Part 5 of the MPEG-4 standard provides reference software that implements different aspects of the MPEG-4 specification. One portion of the reference software is referred to as IM1, which includes several modules. The IM1 core module parses scene description bitstreams, constructs the memory structure of the scene graph, and prepares the scene graph for the Renderer module which traverses the scene graph and renders it on the terminal hardware (e.g., screen, speaker). IMI implements a two-plane approach in its handling of the scene graph. The two planes are referred to as the Scene Manipulation Plane and the Renderer Plane. The Scene Manipulation Plane parses the scene description stream, constructs the scene graph, manipulates it, and handles route activation without knowledge of the semantics of the node. The Renderer Plane traverses the scene graph and renders it without knowledge of the syntax of the scene description bitstream. The two planes are inplemented as two separate modules that communicate via the scene graph.
The IM1 player architecture is illustrated in FIG.
2
. The core, generally designated
200
, includes sub-modules that parse the scene description and construct the scene tree, collectively implementing the Scene Manipulation Plane. Modules
202
and
204
interact with the core module
200
through APIs. The Renderer
206
traverses the scene graph and renders the scene, implementing the Renderer Plane. Core
200
and Renderer
206
interface through a Scene Graph Interface
208
.
IM1 scene graph elements are represented several main classes including:
MediaObject
a base class for nodes.
NodeField
a base class for node fields.
Route
representing a route.
Specific node types are represented by classes derived from MediaObject. There are two layers of inheritance. For each node type, a class is derived from MediaObject implementing the specific node syntax. Each node can then be overloaded by specific implementations of MPEG players in the Renderer Plane to implement the rendering of the scene graph For example, the Appearance node is declared as:
class Appearance:public MediaObject
{
BIFS_DECLARE_NODE
exposedField SFMaterialNode material;
exposedField SFTextureNode texture;
exposedField SFNode<class TextuxeTransform>textureTransform;
};
BIFS_DECLARE_NODE is a macro that declares virtual functions, described below, that overloads the pure declarations in MediaObject.
The implementation of Appearance is as follows:
BIFS_IMPLEMENT_NODE_START(Appearance)
BIFS_FIELD(Appearance, material, 0, 0, 0, −1, 0, 0)
BIFS_FIELD(Appearance, texture, 1, 1, 1, −1, 0, 0)
BFS_FIELD(Appearance, textureTransform, 2, 2, 2, −1, 0, 0)
BIS_IMPLEMENT_NODE_END(Appearance)
BIFS_IMPLEMENT_NODE_START, BIFS_IMPLEMENT_NODE_END and BIFS_FIELD are macros that are used together to define the field table for each node. MediaObject uses these tables to access the fields of its derived objects in a generic way using virtual methods that are declared as “pure” in MediaObject and overloaded by each node. Example of these methods include:
int GetFieldCount ( );
// get the number of fields
in the node
const BifsFieldTable *GetFieldTable ( );
// get a pointer to the
field table
NodeField *GetField (int nIndex);
// get a pointer to the
NodeField object of a
specific field
This
Bayerl Raymond J.
Eitan, Pearl, Latzer & Cohen Zedek LLP
Optibase LTD
LandOfFree
PROTO implementation in MPEG-4 does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with PROTO implementation in MPEG-4, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and PROTO implementation in MPEG-4 will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3316787