Data processing: database and file management or data structures – Database design – Data structure types
Reexamination Certificate
2001-06-15
2004-11-02
Mizrahi, Diane D. (Department: 2175)
Data processing: database and file management or data structures
Database design
Data structure types
C707S793000
Reexamination Certificate
active
06813623
ABSTRACT:
FIELD OF THE INVENTION
This invention relates to the field of transaction logging in the storage of data in database management systems (DBMS) such as relational database management systems.
BACKGROUND OF THE INVENTION
For reliable operation of a database management system, integrity and consistency are needed for the storage and updating of data for the database management system. To facilitate this requirement, one of the key features of a mature database management system is transaction logging. For each change to the database content, there is one or more log record that describes this change. Each log record is assigned a specific sequence number and log records are eventually transferred to permanent storage in same order as this sequence number. These log records must eventually be placed on permanent storage in order to ensure consistency and to provide a history of changes to the database. For relational database management systems (RDBMS) such as DB2, transaction logging is an integral process involved in making changes to the data contents of the database tables stored by the RDBMS. These changes include inserting, deleting or updating the database contents. For mature database management systems, transaction log records are first logged by database connection agents in a transaction log buffer in memory before being transferred to permanent storage such as a disk. The database manager generally has a dedicated thread of execution called the logger to manage the transferring of data from log buffer to permanent storage. The purpose of the transaction log buffer is to reduce the cost of small changes to the database contents by making the cost of logging of these small changes relatively inexpensive. Placing transaction log records into a log buffer is relatively simple when a single database application is involved since there is no contention for the use of the transaction log buffer by other application. The speed at which transaction logging takes place is dependent on the amount of data that the single application provides for transaction logging and the frequency that the application makes changes to the database contents. Transaction logging for a single database application can be handled in a simple serial fashion and there is no contention on the transaction log buffer. However; the situation becomes more complex for a database management system which can serve multiple database connections simultaneously. An efficient transaction logging mechanism is required that can handle the logging of data by multiple database applications to multiple database tables on permanent storage. With multiple database applications, contention for the transaction log buffer can inhibit transaction logging performance due to the serialized access to the transaction log buffer. For workloads which involve a high frequency of transactions that make changes to the database contents, transaction logging can become a performance bottleneck.
In prior art, changes made to the database by multiple database applications making use of the RDBMS involved granting exclusive access to the log buffer to one database connection agent (a database connection agent in this context is a task or process that resides in the RDBMS engine and performs various operations on behalf of a database application). The access to the log buffer is sequential, requiring other database connection agents to wait until the current database application's data had been copied into the log buffer. This serialized access can have a large negative impact to transaction logging throughput since only one database connection agent can copy transaction log records at any given time.
The following generally summarizes the steps taken in prior art when a transaction log record was copied into the transaction log buffer:
1. Obtain a latch (a latch in this context is simply a high speed mutually exclusive locking mechanism) which protects the transaction log buffer from having more than one database connection agent copying a transaction log record into it. Once the transaction log buffer is latched by one database connection agent, other database connection agents must wait until the latch is freed and the latch is obtained by the database connection agent.
2. Copy the transaction log record into the transaction log buffer starting at the current log buffer offset.
3. Update the transaction log buffer offset which marks the location that the next transaction log record should be copied to.
4. Release the latch which protects the transaction log buffer so that other database database connection agents can attempt to obtain the latch.
One problem in the prior art is that there is too much serialization in accessing the log buffer. It appears that there is no recognition in the prior art that the latch is only needed to determine the sequential order of the log records and the location in the log buffer for loading log record data. We have found that it is not necessary (at least for the purpose of determining the sequential order of log records) to hold the latch while copying log record data into the log buffer. If database connection agents are allowed to copy their log record data after releasing the latch (so multiple database connection agents can be copying concurrently), a problem is not knowing when database connection agents will have finished copying data into the log buffer.
We have found that the logger, which is used to transfer data records from the transaction log buffer to main storage needs to know when it can start transferring data from the log buffer to permanent storage. One aspect of this invention addresses this.
The prior art also had to contend with two competing goals when trying to improve transaction logging performance: one being application response time and the second being overall transaction logging throughput. This issue is related to the transferring of data from the log buffer to permanent storage. The performance of a storage device such as a disk usually varies with the amount of data for each transfer, and generally increases with the transfer size up to a certain level, depending on system characteristics, after which the performance gain becomes smaller for increasing transfer size. A storage device has a transfer size (or a range of transfer sizes) at which data transferring becomes efficient and levels off with larger transfer sizes. However, during the operation of the logger, there may be different amounts of data available in the logger buffer ready to be transferred to permanent storage. If the logger is implemented with logic to initiate the data transfer for all data available in the log buffer, the size of the transfer may be very large resulting in poor response time for some of the applications making use of the RDBMS. It is also possible that the amount of data to be transferred may be too small for good efficiency, which can result in poor system throughput. Taken to the extreme, if the logger writes each transaction record to storage one at a time, then each change to the database would have to endure the cost of writing a log record to permanent storage. While writing a specific log record to storage immediately benefits the individual database connection agent which made the request, the overall throughput is hindered, as will be appreciated by those skilled in the art. On the other hand, the overall throughput can benefit from waiting for a number of transaction records to be copied into the log buffer and then transferring them all to permanent storage at the same time. The longer the transaction logging task waits, the worse the potential response time to an individual request and the shorter the transaction logging task waits, the worse the overall throughput. One aspect of this invention addresses this issue by providing a preselected data transfer amount for use by the logger in transferring data from the log buffer to permanent storage.
Another potential problem with prior art is that the transaction logging task (performed by the logger) was responsible fo
Wilding Mark Francis
Zheng Roger Luo Quan
LandOfFree
Method and apparatus for chunk based transaction logging... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for chunk based transaction logging..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for chunk based transaction logging... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3281663