Data processing: presentation processing of document – operator i – Presentation processing of document – Layout
Reexamination Certificate
1998-02-11
2004-10-26
Hong, Stephen S. (Department: 2178)
Data processing: presentation processing of document, operator i
Presentation processing of document
Layout
C715S252000, C718S100000, C718S102000, C345S215000
Reexamination Certificate
active
06810503
ABSTRACT:
FIELD OF THE INVENTION
This invention relates to computer programming, more particularly, to a method and apparatus for controlling the timing of the invocation of events within a computer runtime environment, and specifically, timing the invocation of multimedia events within a Web page runtime environment.
BACKGROUND OF THE INVENTION
The Internet and the World Wide Web are rapidly expanding, with businesses and individuals hosting their own Web pages. This has created a demand for richer Web page capabilities especially in the area of coordinated presentation and control of multimedia events, including being able to easily synchronize the execution of a plurality of multimedia events from a single point in time (i.e., playing a multimedia timeline). Because not all Web page owners are sophisticated computer users, the design and programming of Web pages must remain simple. Nor should synchronizing the execution of multimedia events to a timeline defined within in a Web page require complicated or lengthy user programs, or require the use of an external application which increases the cost of a solution. Instead designing and programming a Web page should be intuitive (i.e., “user-friendly”), while still providing sophisticated capabilities, such as prioritization of events which are scheduled for the same time and defining a window for execution of a time critical event.
Web pages are composed of various multimedia elements, controls, and applets as defined by the Hypertext Markup Language (HTML) for a given Web page. Multimedia can be characterized as some combination of visual media, audio media and time. Multimedia is an open environment, with timing being the common thread across all multimedia events. Multimedia experiences, such as the movement of graphics and the playing of sound, require coordinated execution of these events. For instance, the playing of a sound bite of a door slamming should be coordinated with the animation of the door closing so the sound is heard upon the visual closing of the door (and not before or after such time).
Providing synchronized multimedia experiences is complicated because timing control information is not inherent in the content of an HTML document. Past attempts at providing such synchronization and timing control of activities within a Web page have basically take on one of two forms: (1) external programs and (2) lengthy, complicated scripts or programs. These solutions generally are non-user-friendly, require additional expenses, and do not provide true synchronization of multiple events to a single timeline. Additionally, other approaches have not allowed synchronization between application-specific components (e.g., graphics and audio) and host-intrinsic components (e.g., HTML elements).
External multimedia control programs, such as Director, by Macromedia, can be expensive, and do not allow the direct creation and modification of a multimedia timeline by editing the HTML code. Rather, any changes and additions to the multimedia timeline must be made using the external program itself. Furthermore, the timing mechanism of some of these external programs are based on “frames” of time rather than directly on a time scale. A frame corresponds to a duration of time during which a set of defined activities are to be performed. Frames provide a method to sequentially perform sets of activities where there is some timing relationship based on frame rates and the time required for the sets of activities within the frame to be performed. However, individual events are not specified to be executed at a particular time (e.g., at time t=10.000 seconds), rather to execute within a frame (e.g., in frame
2
) which does not provide an intuitive indication of the exact time for executing the event.
A second approach has been to write lengthy code or scripts within a control loop of the computer program which evaluates the current time each time through the control loop. In which case, the control loop can tie up an entire processing thread which does not allow other applications or processes to proceed, or at least requires extra computer cycles. Every time the control loop is executed, a large number of statements must be evaluated to determine whether to execute the actual events. This approach is less than desirable, especially as the number of events to control increases beyond one or two. In this case, scripts become long and complicated, and often more complex than a typical Web author knows how to create by hand. The control loop requires a large amount of processing time to determine whether to execute an event, which is disadvantageous especially in the multimedia context where events must be synchronized to a timeline having a granularity on the order of milliseconds. Furthermore, to guarantee a desired performance, the number of events controlled by a control loop must remain small because the control loop must iterate at a rate at least as great as the fastest timing requirement. Moreover, in many implementations such as those using an HTML document, the code within the control loop is written as an interpreted script thus compounding its evaluation time in comparison with that which is implemented using a built-in service or compiled code. Therefore, as the number of events to control becomes large, the timing performance correspondingly degrades and the granularity to which events can be set to perform becomes coarser. Moreover, the timing performance varies significantly and unpredictably depending on the power, performance, and current operating load of the computer on which these scripts or sets of code are run.
A variant of this approach uses a software timer control to generate a control event on a regular basis and causing a control program to execute. The control program then evaluates the current time and initiates the execution of the desired multimedia event. For instance, on a Web page, the setTimeout command can be used. This HTML code variant is more desirable than that of a control loop as it decreases the amount of evaluation code required. However, using this approach, it is cumbersome at best to stop the execution of sets of active timelines, and, more importantly, there is no synchronization of two events to a single timeline. Instead, these events are independently scheduled when their respective setTimeout command is interpreted and therefore, they are not synchronized. Furthermore, when multiple events are scheduled to begin execution at the same instant of time, the order in which the events will be executed is undefined for the setTimeout command (and commonly in other timing control approaches) which means a Web page does not necessarily produce the same results every time it is run, either on the same or on a different platform. Furthermore, this approach has disadvantageous performance limitations because the scripts become long and complex similar to that of an approach previously described herein, and the timing performance is unpredictably sensitive to the performance of the machine on which these set of code or scripts are executed.
In addition to their performance limitations, the prior approaches are directed towards basic timing control. In other words, these approaches only provide a way of initiating the execution of an activity at some defined point in the future, which is only a subset of the identified need in the computer industry for controlling multimedia experiences and applications. Lacking from these prior approaches are robust capabilities for providing true prioritization among events scheduled to happen at the same time, and a drop threshold capability to ensure that time sensitive events whose execution windows have expired are not executed.
In non-Web environments, schedulers have been provided to be able to start a process at a specified future time (e.g., a UNIX “chron” job). However, these schedulers do not provide the capabilities required for synchronizing a plurality of multimedia events along a timeline. For instance, the timing granularity of such schedulers is usua
David Paul Christian
Hyman Michael Irvin
Kichline Evan Gomer
Vaddadi Phani Kumar
Hong Stephen S.
Huynh Cong-Lac
Leydig , Voit & Mayer, Ltd.
LandOfFree
Method and apparatus for controlling the timing of the... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for controlling the timing of the..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for controlling the timing of the... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3316215