Coded data generation or conversion – Digital code to digital code converters – To or from packed format
Reexamination Certificate
2002-08-27
2003-12-02
Jeanpierre, Paguy (Department: 2632)
Coded data generation or conversion
Digital code to digital code converters
To or from packed format
C341S050000
Reexamination Certificate
active
06657568
ABSTRACT:
TECHNICAL FIELD
This description relates to data packing for real-time streaming.
BACKGROUND
Real-time streaming is used, for example, in a trading market (e.g., a stock market) to broadcast data for every action that takes place in the market (e.g., trade volume, sell price) to a variety of customers. Often the data is sent in the form of changes to existing records. The market can produce data record changes every 250 ms. Although each of these data record changes may be small, when combined with others in the rapid succession in which they are being generated, the high aggregate volume of these small changes can consume all or a large portion of a network's bandwidth. Abstract Syntax Notation One (ASN.1) is a formal language for abstractly describing messages and it includes packed encoding techniques. Publications ISO 8825-2/ITU X.691 specify the packed encoding rules for ASN.1. The encoding techniques described in the publication align with byte boundaries when encoding a record.
SUMMARY
An objective is to generate a data packing solution more efficiently than the ISO standard to minimize the bandwidth requirements for high volume data and to increase the speed of delivery of real-time data to customers. In one aspect, there is a method comprising encoding a first update of data, encoding a second update of data and preparing for transmission the second update following the first update without regard to a boundary associated with a predefined number of bits. The method can include preparing for transmission the second update following the first update without adding one or more bits to align the first update or the second update with a byte boundary. In one embodiment, the predefined number can comprise a byte. In another embodiment at least one of the first and second updates is associated with financial market data. The method can also include streaming the first and second updates. The method can further comprise preparing for a first transmission a portion of the first and second updates that align with a transmission packet and preparing for a second transmission remaining portions of the first and second updates not prepared for the first transmission. The method can also include determining a minimum number of bits to encode a length identifier and encoding a numerical portion of the first update as a length identifier followed by a value. The method can further comprise transmitting the updates.
In another aspect, there is a method comprising encoding a first update associated with a first financial record, encoding a second update associated with a second financial record, buffering for transmission the second update following the first update without regard to a byte boundary and streaming the buffered first and second updates.
In yet another aspect there is a method comprising determining a minimum number of bits needed to represent a value without regard to a boundary associated with a predefined number of bits, generating an encoded entry corresponding to the value using the determined minimum number of bits and preparing for transmission the encoded entry without aligning the encoded entry with a boundary associated with a predefined number of bits. In one embodiment, the predefined number of bits comprises a byte. The method can include storing the encoded entry in a buffer and wherein transmitting further comprises transmitting at least a portion of contents of the buffer upon occurrence of an event, the portion based at least in part on alignment with a packet size. The method can also include storing a second encoded entry directly following any portion of the contents of the buffer not transmitted. In another embodiment, the event comprises i) passage of a predetermined amount of time or ii) storage of a predetermined amount of data within the buffer. The method can further comprise defining a set of data including one or more entries, and wherein the value is associated with a first entry. In another embodiment, the set of data comprises an XML file.
In another aspect, there is a method comprising i) receiving a stream of bits comprising a first portion associated with a first update and a second portion associated with a second update, the second portion following the first portion without regard to a boundary associated with a predefined number of bits and ii) decoding the stream of bits. In one embodiment, the predefined number comprises a byte. In another embodiment, the first portion is associated with a first record and the second portion is associated with a second record. The method can include receiving a first plurality of bits from the stream of bits, advancing a state machine in response to each bit and holding a current state of the state machine at the last bit in the first plurality if the last bit is not the last bit in the first portion. The method can also include receiving a second plurality of bits and advancing the state machine from the current state in response to the second plurality of bits. The method can further comprise advancing the state machine to an initial state at the end of the first portion.
In another aspect, there is a method comprising defining a range associated with a variable type, determining a first minimum number of bits to represent a maximum bit length to represent any value within the associated range, determining a second minimum number of bits to represent a specific value associated with the variable type and encoding the specific value other than any sign using no more bits than the sum of the first minimum number of bits and the second minimum number of bits. The method can include, if the variable type is signed, encoding the specific value using no more bits than the sum of the first minimum number of bits, the second minimum number of bits and a sign bit. In one embodiment, the variable type comprises a floating-point number, and the specific value comprises an integer portion and a decimal portion. The method can include determining a third minimum number of bits to represent the maximum value of a predefined precision, determining a fourth minimum number of bits to represent the integer portion of the specific value, determining a fifth minimum number of bits to represent the decimal portion of the specific value and determining the second minimum number of bits by summing the first, third, fourth and fifth minimum numbers of bits.
In another aspect, there is a method comprising defining a character set of a maximum number of characters, determining a minimum number of bits to represent the maximum number and encoding a string using the character set, each character in the string encoded using the determined minimum number of bits to represent a value of its order in the defined character set, the last character in the string followed by the determined minimum number of bits representing a value that is outside of the defined character set.
In another aspect, there is a system comprising a data structure to define data, and an encoder to encode a first update of data and a second update of data and to prepare for transmission the second update following the first update without regard to a boundary associated with a predefined number of bits. The system can include a decoder to receive the first and second updates, to determine where the first update ends and the second update begins and to decode the updates. The decoder can further comprise a state machine to return to an initial state at the determined end of the first set. The encoder can be further configured to buffer for transmission the second update following the first update without adding bits into the first encoded update to align the first update or the second update with the byte boundary. In one embodiment, the predefined number comprises a byte. In another embodiment, the first and second updates are associated with financial market data. The system can also include a transmitter to stream the first and second updates. The transmitter can be further configured to transmit in a first transmission a portion of the buffered f
Coelho Joe A.
Medvinsky Michail
Fish & Richardson P.C.
FMR Corp.
Jeanpierre Paguy
LandOfFree
Data packing for real-time streaming does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Data packing for real-time streaming, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Data packing for real-time streaming will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3121393