Dynamic buffer allocation

Details Dynamic buffer allocation Dynamic buffer allocation

2002-05-06

2004-11-16

Robinson, Greta (Department: 2177)

Data processing: database and file management or data structures

Database design

Data structure types

C707S793000, C707S793000, C707S793000

Reexamination Certificate

active

06820091

ABSTRACT:

BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates to a method for accelerating data access to buffer pools in the main storage of a computer. The invention also relates to a corresponding apparatus and a corresponding program product.
2. Description of the Related Art
In computer systems which run a plurality of application programs, large amounts of data have to be accessed within short time periods. Each of the application programs has assigned a segment of virtual storage which provides a larger address space than the real main or primary storage of the computer system. On an operating system level, paging operations are performed by which portions of program code and data are transferred between a slower mass storage, for example, a magnetic disk or tape storage, and the primary storage which permits a fast access to its contents. The operating system uses paging operations to keep available in the primary storage the active code and the data to be accessed and to reduce the transfer of data to or from the mass storage to a minimum (see, for example, Deitel, ‘Operating Systems’, Addison-Wesley Publishing Company, Reading, Mass., USA, 1990, pp. 230-241).
Within virtual storage each application has assigned buffer pools for keeping available a plurality of data files for being processed by that application. The buffer pools may occupy a large portion of the main or primary storage and are able to store large amounts of data sets or files. The data in a data set or file is organized in a plurality of records of different length and are assigned to the buffer pools which are of different size. Application programs access data sets to receive or transmit data records. Thus, the number of buffer pools and their capacity determines the amount of data in terms of data records which are contained in the main storage at a certain point of time and are available for a direct and fast access.
For each application there are usually fewer buffer pools available then data records. It is desirable to have in the buffer pools large amounts of that data which is often used by an application. For this purpose, an amount of data records of an application is initially read from the mass storage and transferred to the buffer pools which belong to this application in the main storage. If a data record to be accessed by an application is not among those contained in one of the buffer pools, the application has to initiate a read operation in the mass storage to transfer a number of data records, including the one to be accessed, into the main storage. The storage resources of the buffer pools are limited compared with the number of data records of the files used by the application. To allow such transfer it may thus be necessary to move in advance other data records from a buffer pool to the mass storage in order to make space for the data records to be accessed. Transfer operations to or from the mass storage are much slower than a read or write operation in the main storage. The former takes milliseconds while the latter takes only a few nanoseconds. The time ratio between both may in fact be 1:100 to 1:1000 depending on the type of mass storage used. Repeated transfers for making available required data records in the main storage may thus greatly reduce the throughput of the computer system. It is therefore a task to keep transfers of data records between the main storage and the mass storage to a minimum. The same applies to related index information which is used for the access to data sets or files.
A successful access to a data set or a file in a buffer pool may in particular depend on the assignment of the data sets or files to the buffer pools which available for an application. A useful distribution of the files over the available buffer pools may increase the hit ratio of data set accesses considerably. This may be achieved through manual tuning of the system by using statistics data on the data transfer operations performed during a certain time period. Such statistics data are either available or will be generated for the tuning process. Based on the statistics data the assignment of certain data files to certain buffer pools is forced or prevented to achieve a balanced distribution of the files over the buffer pools. The tuning process is time consuming and in particular difficult if a large number of data sets is used. The result of the tuning process is static and valid only for the conditions which exist during the time period considered. These conditions may change rapidly so that repeated tuning may be required. However, the reason for long program running times remain often unnoticed by the user and thus no tuning is performed.
SUMMARY OF THE INVENTION
It is an object of the invention to provide computer systems with an accelerated access to data used during the execution of a plurality of applications.
According to another object of the invention the availability of the data used during the execution of a process or application in the main storage is improved and the need of time-consuming data transfers between the mass storage and the main storage is reduced.
In particular, it is an object of the invention to automatically optimize the balance of the data set or file distribution over the buffer pools to increase the number of data records which are available for a direct and fast access by the application.
According to the invention, as defined in the claims, buffer pool usage statistics data are generated including a buffer pool identification, an identification of a data set or file stored therein and a number of buffer pool usage parameters. A threshold value is specified for each data set for scaling selected ones of the usage parameters. For each of the data sets or files a performance value is generated by comparing the selected ones of the usage parameters with the threshold value of the data set. The performance values are examined for the existence of a data set reassignment criterion and the data set is reassigned to another buffer pool if a reassignment criterion is found. The data set reassignment criterion is preferably represented by a negative performance value.
To determine a suitable buffer pool for receiving the data sets to be reassigned, a selection value for each of the buffer pools may be generated by comparing the pool size with selected ones of the usage parameters. The selection values are used to select a suitable buffer pool by using a selection criteria included in the selection value that indicates the suitability of a buffer pool to receive additional data sets. The selection criteria is preferably represented by the magnitude of the selection value.
The checking of the performance values and the data set reassignment are performed at restart time of an application program and may be repeated during the runtime of the application program after predetermined time intervals have expired. In this manner an automatic tuning of the data set-to-buffer assignment is performed at restart time of an application and repeatedly during the runtime of the application so that a change of the system conditions may be covered by automatic tuning update operations.


REFERENCES:
patent: 4509119 (1985-04-01), Gumaer et al.
patent: 5506986 (1996-04-01), Healy
patent: 5784698 (1998-07-01), Brady et al.
patent: 6064805 (2000-05-01), McCrory et al.
patent: 6553438 (2003-04-01), Coffman et al.

Affiliated with

Also associated with

Rating

Dynamic buffer allocation does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Dynamic buffer allocation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dynamic buffer allocation will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3362826