Electrical computers and digital processing systems: processing – Processing control – Branching
Reexamination Certificate
2000-10-17
2002-04-16
Kim, Kenneth S. (Department: 2183)
Electrical computers and digital processing systems: processing
Processing control
Branching
C712S238000, C712S239000
Reexamination Certificate
active
06374348
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to the field of high performance computing systems, and methods for improving instruction execution. The invention is particularly useful for reducing branch instruction delays in highly pipelined processors.
BACKGROUND OF THE INVENTION
Many modern computing systems utilize a processor having a pipelined architecture to increase instruction throughput. In theory, pipelined processors can execute one instruction per machine cycle when an well-ordered, sequential instruction stream is being executed. This is accomplished even though the instruction itself may implicate or require a number of separate microinstructions to be effectuated. Pipelined processors operate by breaking up the execution of an instruction into several stages that each require one machine cycle to complete. For example, in a typical system, an instruction could require many machine cycles to complete (fetch, decode, ALU operations, etc.) Latency is reduced in pipelined processors by initiating the processing of a second instruction before the actual execution of the first instruction is completed. In the above example, in fact, multiple instructions can be in various stages of processing at any given time. Thus, the overall instruction execution latency of the system (which, in general, can be thought of as the delay between the time a sequence of instructions is initiated, and the time it is finished executing) can be significantly reduced.
The above architecture works well when program execution follows a sequential flow path. In other words, this model is premised on a sequential model of program execution, where each instruction in a program is usually the one immediately in memory following the one just executed. A critical requirement and feature of programs, however, is the ability to “branch” or re-direct program execution flow to another set of instructions; using branch instructions conditional transfer of control can be made to some other path in the executing program different from the current one. However, this path may or may not coincide with the next immediate set of instructions following the instruction that was just executed.
In general, prior art processors have a single address register for instructions that are to be executed, including a branch target address. The branch target address is an address indicating the destination address of the branch instruction. The branch instruction is executed quickly by the processor if the correct target address for the branch instruction is already stored in the address register. However, branch instructions can occur arbitrarily within any particular program, and it is not possible to predict with certainty ahead of time whether program flow will be re-directed. Various techniques are known in the art for guessing about the outcome of a branch instruction, so that, if flow is to be directed to another set of instructions, the correct target address can be pre-calculated, and a corresponding set of instructions can be prefetched and loaded in advance from memory to reduce memory access latencies. In general, since memory accesses are effectuated much slower than pipeline operations, execution can be delayed pending retrieval of the next instruction.
Sometimes, however, the guess about the branch outcome is incorrect, and this can cause a “bubble”, or a pipeline stall. A bubble or stall occurs, in general, when the pipeline contains instructions that do not represent the desired program flow (i.e., such as from an incorrectly predicted branch outcome). A significant time penalty is thus incurred from having to squash the erroneous instruction, flush the pipeline and re-load it with the correct instruction sequence. Depending on the size of the pipeline, this penalty can be quite large; to a significant degree, therefore, the desire for long pipeline designs (to increase effective instruction throughput) is counterbalanced by the stall penalty that occurs when such pipeline has to be flushed and re-loaded. Thus, significant effort has been expended in researching, designing and implementing intelligent mechanisms for reducing branch instruction latency.
To analyze branch instruction latency, it is helpful to think of a branch instruction as consisting of three operational steps:
(1) deciding the branch outcome
(2) calculating the branch target address (i.e., the location of the instruction that needs to be loaded)
(3) transferring control so that the correct instruction is executed next
In most systems, steps (1) and (2) must be resolved in this order by a branch instruction. Branch instructions also fall generally into two classes: conditional, and unconditional. When the branch is always taken it is referred to as an unconditional branch, and the above three operational steps are not required. A conditional branch is taken depending on the result of step (1) above. If the branch is not taken, the next sequential instruction is fetched and executed. If the branch is taken, the branch target address is calculated at step (2), and then control is transferred to such path at step (3). A good description of the state of the art in branch prediction can be found generally in section 4.3 of a textbook entitled Computer Architecture: A Quantitative Approach, 2
nd
edition, by Patterson and Hennessy, pages 262-278 are incorporated by reference herein.
In general, the number of penalty cycles associated with a branch instruction can be broken down into two categories: (1) fetch latency of the target instruction from decode of branch; this generally refers to the time required to fetch and place the target instruction of the branch into the pipeline after it has been identified; (2) latency of the branch condition generation; this refers generally to the process by which it is determined if the branch is actually taken or not-taken. Within a particular system it is usually more important to reduce category (1) penalties since they affect both conditional and unconditional branches, while the category (2) penalties are only associated with conditional branches. Moreover, category (2) penalties can be ameliorated to some extent by well-known techniques, including branch prediction. For example, in U.S. Pat. No. 5,742,804 to Yeh et. al., also incorporated by reference herein, a compiler inserts a “branch prediction instruction” sometime before an actual branch instruction. This prediction instruction also specifies the target address of the branch, to further save execution time. Instructions are pre-fetched in accordance with the hint provided by the prediction instruction, so that they will be ready for execution when control is transferred. The prediction itself on the branch outcome is made based on information acquired by the compiler at run time. There does not seem to be very optimal handling of mis-predictions in Yeh, however, and these “misses” can be costly from a branch penalty perspective. Accordingly, the approach shown there also appears to have senous litigation.
Looking more specifically at the breakdown of the category (1) time penalty within a particular pipelined computing system, it can be seen to consist of the following: reading the branch operand (0 to 1 cycles); calculating the branch target address (1-2 cycles); and accessing the instruction cache and putting the target instruction into the decode stage of the pipeline (1-2 cycles). Thus, in a worst case scenario, a branch instruction latency of 5 cycles can be incurred. In some types of programs where branch instructions are executed with some regularity (i.e., 20% of the time) it is apparent that the average branch instruction penalty can be quite high (an average of 1 cycle per instruction).
Various mechanisms have been proposed for minimizing the actual execution time latency for branch instructions. For instance, one approach used in the prior art is to compute the branch address while the branch instruction is decoded. This can reduce the average branch instruction cycle, but comes at the cost of an additional address
Irie Naohiko
Werner Tony Lee
Kim Kenneth S.
Loudermilk & Associates
LandOfFree
Prioritized pre-fetch/preload mechanism for loading and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Prioritized pre-fetch/preload mechanism for loading and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Prioritized pre-fetch/preload mechanism for loading and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2919989