Electrical computers and digital processing systems: memory – Storage accessing and control – Control technique
Reexamination Certificate
1999-03-26
2004-04-06
Bragdon, Reginald G. (Department: 2188)
Electrical computers and digital processing systems: memory
Storage accessing and control
Control technique
C711S170000, C710S056000
Reexamination Certificate
active
06718445
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to buffer management in a computer file system and more particularly to a system and method for managing buffers in a hierarchical file system.
2. Description of the Prior Art
In computer systems, buffers are typically needed for Input/Output (I/O) operations. A computer operating system may consist of different components for managing I/O buffers, virtual storage and real storage. Buffer management systems often run on such operating systems that support virtual memory. The functions of a buffer management system are many and complex, one of which is the interactions of a buffer manager with the operating system components that manage virtual storage and physical memory.
In operating systems that support virtual memory, a page or segment of memory may be in one of several states:
(1) A virtual page may be in a pageable state. This means that the page contains valid data. If the operating system chooses to reuse the physical storage that the page occupies, it must be paged out to external storage. If the page is later referenced, then it must be paged in and given a physical storage page to use.
(2) A virtual page may be fixed in storage. I/O operations require that a buffer must reside in fixed storage. This means that the page contains valid data and may not be paged out at the operating system's discretion. A pageable buffer must be page fixed prior to the start of an I/O, using a costly operating system service. When an I/O completes, to make a buffer pageable again, or to “unfix” it, another costly operating system service is invoked. Page fix and page unfix operations tend to be relatively expensive, in terms of CPU time and in terms of system serialization which may prevent concurrent page fix/unfix operations in a multi-processor environment. Various techniques may be employed to minimize the use of these services.
(3) A virtual page may be unbacked or released (for the purpose of this disclosure, the two terms are interchangeable). This means that the virtual address of the page is valid, but the page has never been used or has been designated as no longer containing data that is of any use. A page in this state does not need to be written out to external storage, i.e. paged out, if the physical storage it occupies is needed for other use. Also if it has already been paged out, the slot it occupies on external storage may be reused and a reference to the virtual address will not result in an I/O operation to bring the page into physical memory, i.e. a page-in. Such a page will not occupy any storage if it has never been used, or if its prior location has been reclaimed. In all cases, however, the virtual address remains valid.
Pools of buffers are defined so that buffers may be shared among different files and different users. If different size buffers are needed for different purposes, then multiple pools may be used. Often there is a software component dedicated to the management of buffers—a buffer manager. A sophisticated system may consist of multiple buffer managers which operate on behalf of different data base subsystems. Different buffer management systems must compete for storage resources, in which case the underlying operating system acts as the supervisor.
Large buffer pools are used to help reduce the frequency of I/O, but large pools require large amounts of memory. Excessively large pools can result in excessive paging which degrades performance. The presence of independent buffer managers coexisting and competing under the same operating system image complicates matters since independent buffer managers are unaware of each other. Sometimes a system administrator or data base administrator is given the ability to act as the policeman, to tune the system by specifying the size of the various buffer pools.
To avoid the excessive use of the expensive page fix and unfix operations, a buffer manager may fix some buffers at initialization using user input or some internally derived value to control the amount of permanently fixed storage. The buffer manager will then fix the pages belonging to all the buffers in a specified address range during initialization. The fixed buffers use physical memory that is taken away from the system's pool of pageable storage. If the buffer manager takes too much, it may impact the rest of the system, causing excessive paging. Also, if the buffer manager fixes too many buffers, physical memory may go unused due to minimal use of buffers. If too few or no buffers are permanently fixed in storage, system performance may be degraded through increased use of the services to fix and unfix buffers and by increased page-ins when buffers are used.
Both the total buffer pool size and the partition of buffers into fixed and pageable buffers is usually static; that is, the partition cannot change without suspending service to users that require a buffer. Therefore, it can be seen that there is a need to provide a more dynamic non-disruptive way to modify the buffer pool size or to repartition the buffers between pageable, fixed and released states.
SUMMARY OF THE INVENTION
In order to address the shortcomings of the prior art described above, it is an object of the present invention to improve the performance and efficiency of a buffer management system in operating systems that supports virtual memory.
It is a further object of the invention to provide a buffer management system that provides a more dynamic, non-disruptive way to modify the buffer pool size.
It is still a further object of this invention to partition the buffer storage into pageable and non-pageable buffers, using a partition of unbacked buffers to accommodate the expansion or contraction of the buffer pool. It is an object of the invention to allow the user to define the size of of the buffer pool and the size of the partitions. More importantly, this invention allows the user to dynamically modify the size of the pool and partitions without causing any disruption of service to applications that require use of the buffer pool.
These and other objects and advantages are attained in accordance with the present invention which provides a buffer management system that includes a buffer. pool which is comprised of both fixed and virtual storage. The buffer pool is logically partitioned into three states, fixed, pageable and released, wherein each of the plurality of buffers in the buffer pool exists in one of these states. The system further comprises the buffer manager, which dynamically manages the buffers in the system by moving them between these three states based on a comparison between the actual usage of fixed and virtual storage compared with target usage values for the fixed and virtual storage.
The system of the present invention further includes a buffer index table, where each entry represents one buffer in the pool. The index table entries contain information about the state of the buffer and an index designating the next buffer in the list.
An advantage of the system of the present invention is that the buffer pool size can be changed dynamically as system conditions dictate, so there is no need for suspending service to users that require a buffer.
This, as well as other additional objects, features and advantages of the present invention will become apparent in the following detailed written description.
REFERENCES:
patent: 5675797 (1997-10-01), Chung et al.
patent: 5781801 (1998-07-01), Flanagan et al.
patent: 5813042 (1998-09-01), Campbell et al.
patent: 5838994 (1998-11-01), Valizadeh
patent: 5862409 (1999-01-01), Yoshimoto et al.
patent: 5867663 (1999-02-01), McClure et al.
Guo-Liang Wu et al., A Buffer Allocation Scheme for ATM Networks: Complete Sharing Based on Virtual Partition, Dec. 1995, vol. 3, No. 6.
Berger Jeffrey Allen
Kincaid W. James
Levish David
Lewis Cecilia Carranza
Bragdon Reginald G.
Kunzler & Associates
Vital Pierre M.
LandOfFree
System and method for performing dynamic buffer management... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for performing dynamic buffer management..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for performing dynamic buffer management... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3226930