Electrical computers and digital processing systems: memory – Storage accessing and control – Hierarchical memories
Reexamination Certificate
1998-07-31
2002-12-03
Kim, Matthew (Department: 2186)
Electrical computers and digital processing systems: memory
Storage accessing and control
Hierarchical memories
C711S134000, C711S136000, C711S141000
Reexamination Certificate
active
06490654
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to cache memories in computer systems. More specifically, the present invention relates to a cache memory replacement algorithm that determines which cache lines in a cache memory are eligible to be replaced when an associative set of the cache memory is full.
DESCRIPTION OF THE RELATED ART
In the art of computing, cache memories are used to store a portion of the memory contents of a main memory that are likely to be used soon. As used herein, the term “cache” will also be used to refer to a cache memory. Caches are typically smaller and faster than main memory, and are used to mask latencies involved in retrieving memory operands from main memory. In modern computer systems, cache access times are typically about 500% to 3000% faster than main memory access times.
An entry of a cache is known in the art as a cache line, and typically a cache line will store a small contiguous range of main memory contents, such as 32 or 64 bytes. While cache memories are not limited to CPUs, a primary application for cache memories is to store memory operands required by one or more central processing units (CPUs). Note that it is known in the art to provide multiple levels of caches. For example, a CPU may be provided with a level one (L1) cache on the same integrated circuit as the CPU, and a larger and slower level two (L2) cache in the same module as the CPU. In the discussion that follows, it will be assumed that memory operands are loaded into a cache from main memory. However, those skilled in the art will recognize that such operands may also be loaded from a higher level cache if the operands are present in the higher level cache.
Since cache memories are typically smaller than the main memories to which they are coupled, a strategy is required to determine which contents of the main memory are to be stored in the cache. This strategy usually comprises two components: a cache organization and a cache replacement algorithm. The replacement algorithm determines which cache line should be replaced when the cache (or an associative set of the cache, as described below) becomes full.
One of the simplest cache organizations is the direct-mapped cache organization. In a direct-mapped cache, a portion of the main memory address is used as an index, and the remainder of the main memory address (not including any bits of the main memory address that represent bytes within a cache line) is used as a tag. The number of bits used for the index corresponds to the size of the cache. For example, a direct-mapped cache having 64 cache lines will have a index comprising six bits. When a read operation occurs and the memory operand is not in the cache (i.e., the tag does not match), the memory operand is fetched from main memory and stored in the cache line corresponding to the index, and the tag is stored in a tag field associated with the cache line. Assuming the memory operand is still in the cache (i.e., the tags match) the next time a read operation occurs the memory operand will be retrieved from the cache. Incidently, the term “cache hit” is used in the art to refer to a memory access wherein the required memory operand is already in the cache, and the term “cache miss” is used in the art to refer to a memory access wherein the memory operand is not in the cache and must be loaded from main memory or a higher level cache.
The replacement algorithm used with a direct-mapped cache is trivial. For any given byte in the main memory, there is only one cache line in which the byte can be stored. Therefore, if the cache line is in use, the old contents of the cache line are simply overwritten with the new contents. The act of altering the contents of a cache line after the cache line has been loaded from memory is known in the art as “dirtying” the cache line. “Dirty” cache lines must be written back to main memory before the new contents can be stored in the cache line. If the old contents in the cache line are identical to the contents in main memory, the old contents may be overwritten without having to write back to main memory.
One problem associated with direct-mapped cache memories is that two often-used memory operands may need to be stored in the same cache line. Since the two memory operands will contend for the same cache line, much of the advantage provided by the cache will be lost as the two operands continuously replace each other.
Another cache organization is the associative cache organization. A fully-associative cache simply has a pool of cache lines, and a memory operand can be stored in any cache line. When a memory operand is stored in an associative cache, the address of the memory operand (excluding any bits representing the bytes stored within the cache line) is stored in a tag field associated with the cache line. Whenever a memory operation occurs, the tag fields associated with each cache line are searched to see if the memory operand is stored in the cache. One disadvantage of an associative cache is that all tag fields of all cache lines must be searched, and as the number of cache lines is increased, the time required to search all tag fields (and/or the complexity of the searching logic) also increases.
The set-associative cache organization is a hybrid of the direct-mapped and associative memory organizations. In a set-associative cache, an index portion of the memory address identifies a subset of the cache lines. As above, a tag field is associated with each cache line. However, only the tags of the subset of cache lines identified by the index need be associatively searched. For example, consider a cache having 256 entries organized into 64 subsets, with each subset having four cache lines. Such a memory will have an index comprising six bits. When a memory operation occurs, the index identifies one of the 64 subsets, and the tag fields associated with the four cache lines in the subset are searched to see if the memory operand is in the cache. The set-associative cache organization allows a cache to have many cache lines, while limiting the number of tag fields that must be searched. In addition, memory operands need not contend for the same cache line, as in a direct-mapped cache.
As used herein, the term “associative set” will be used to refer to all cache lines of a purely associative cache, and to a set of a set-associative cache. When an associative set is full and a new cache line must be stored in the associative set, an algorithm is required to determine which cache line can be replaced. Several such algorithms are known in the art. A “random” algorithm simply picks a cache line at random. While the implementation is simple, the random algorithm provides relatively poor results since there is no correspondence between the cache line contents selected for replacement and the probability that the selected contents will be needed soon.
A better algorithm is the first-in first-out (FIFO) algorithm. This algorithm treats the associative set as a circular queue wherein the cache line contents that have been in the associative set the longest are replaced. This algorithm provides better results than the random algorithm because the algorithm observes cache misses to create correspondence between the cache line selected for replacement and the probability that the cache line will be needed soon. The algorithm works well when all memory contents needed by the CPU are loaded into the cache and other cache misses do not cause the needed memory contents to be replaced. However, the algorithm does not recognize that if a cache line is repeatedly accessed by the CPU, it should not be replaced. The only factor considered is the length of time that the memory contents have been in the cache. The algorithm is slightly more complex to implement than the random algorithm. Typically a single counter associated with an associative set and is used to provide an index indicating which cache line is next in line for replacement, and the counter is incremented every time there is a cache miss and an operand is loaded from main m
Lyle Stephen B.
Voge Brendan A.
Wickeraad John A.
Hewlett--Packard Company
Kim Matthew
Peugh B. R.
Plettner David A.
LandOfFree
Method and apparatus for replacing cache lines in a cache... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for replacing cache lines in a cache..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for replacing cache lines in a cache... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2977112