Method for preventing buffer overflow attacks

Details Method for preventing buffer overflow attacks Method for preventing buffer overflow attacks

2000-03-02

2003-06-10

Gaffin, Jeffrey (Department: 2782)

Electrical computers and digital data processing systems: input/

Input/output data processing

Input/output data buffering

C710S040000, C711S170000, C712S228000

Reexamination Certificate

active

06578094

ABSTRACT:

BACKGROUND
1. Technical Field
The present invention relates generally to a method for preventing overflow of arrays/buffers that are dynamically allocated on a stack and, in particular, to a method that allows a procedure, which is passed a stack allocated array/buffer as an argument, to determine a “safe” upper bound on the size of the stack allocated array/buffer to thereby prevent back-chain and/or procedure return pointers from being overwritten due to overflow of the array/buffer.
2. Description of Related Art
Virtually all programs written in modern programming languages, such as C, C++ or Java, use a stack for maintaining a program stack while the program is running. A stack is a contiguous region of memory (comprising data items such as words, character, bits, etc.) that grows and shrinks based on the demands of the program. Typically, one end of this stack region (i.e., the bottom of the stack) is fixed at a predetermined address in memory, while the other end of the stack (i.e., the top of the stack) moves up/down to increase/decrease the amount of stack space available to the program. The program maintains what is known as a “stack pointer” that points the current position of the mutable end (i.e., the top of the stack). More specifically, the stack pointer is a either a register or memory word that contains the address of the top of the stack.
Programs execute by calling functions (or procedures or subroutines) which, in turn, may call other functions, etc. Each time a function is called, a block of memory called a “stack frame” is allocated on the stack to accommodate the state of the called function, i.e., for storing parameters, a return address and local variable, if any, associated with the called function. The running program typically maintains a pointer to the start of the region, known as the “frame pointer”.
Referring to
FIG. 1
, a diagram depicts a portion of a program stack in memory in which several stack frames are allocated. The stack in
FIG. 1
“grows downward” from a high memory address to a low memory address each time a stack frame is added in response to a procedure call. A Current Stack Frame is associated with the last called function. It is to be understood that the stack frame structures depicted in
FIG. 1
are, in general, exemplary of conventional structures that are well-known in the art. In general, a frame typically comprises data blocks for storing the following pieces of information:
a function return address: the instruction to be executed after the called function completes execution;
a previous (old) frame pointer: this is the frame pointer of the calling function;
parameters: the arguments that are passed by the calling function to the called function;
local variables: this is where the state specific to the function is stored; and
a frame pointer (FP) or local base (LB) pointer: this pointer points to a fixed location in the frame (e.g., in
FIG. 1
, the FP is used to identify the base (beginning) of the current stack frame). Because the FP points to a fixed location, it may be used to identify the address of the local variables.
By way of example in
FIG. 1
, conceptually, the first variable (Local Var
1
) for the current function is at address (Frame Pointer+offset
1
), and the second variable (Local Var
2
) is at (Frame Pointer+offset
2
). In addition, in the illustrative embodiment of
FIG. 1
, the Frame Pointer points to the address of the Old Frame Pointer of the previous stack frame associated with the calling function (although, in general, the FP plus some offset points to the old frame (the offset in
FIG. 1
has a value of 0)). This mechanism allows what is known as “following the back chain” to find the beginning of any previous stack frame on the stack.
A “calling sequence” refers to the instructions that are required to call a function and pass it parameters. During execution of the calling sequence, a stack frame is generated and “pushed” onto the stack for the called function. Calling sequences and stack frame organizations differ from operating system to operating system. In general, a typical calling sequence comprises the following steps. First, the calling function will push all arguments on the stack (i.e., pass the parameters to the called function). Next, the “return address” is pushed onto the stack and then and the program “jumps” to the new routine (called function). Next, the current FP pointer is pushed onto to the stack in the memory location after the “return address” (i.e., this saves the current FP as the “Old FP” so that it can be restored at procedure exit). The FP is set to the current stack address (i.e., the stack pointer is copied into FP to create the new (current) FP). Finally, the stack pointer is incremented to reserve the space needed to store the local variables of the called function.
A corresponding return sequence comprises the following steps. First, the stack pointer is set to the value of current FP. The next steps are to pop stack into frame pointer, pop stack, and then jump to top of stack value.
FIG. 1
illustrates a Current Stack Frame as comprising the memory locations from the Frame Pointer to the Stack Pointer (which points to the top of the stack). In addition, a Previous Stack Frame comprises the memory locations beginning from Old Frame Pointer
2
and ending at Return Address
1
. It is assumed that the function associated with the Previous Stack Frame is a “calling function” that calls the function (a “called function”) associated with the Current Stack Frame. It is further assumed that parameters Arg
1
and Arg
2
are the parameters that are passed to the called function from the calling function. Although Arg
1
, Arg
2
and Return Addressl are shown as being part of the Previous Stack Frame of the calling function for purposes of illustration, it is to be understood that other conventions consider such values as being elements of the Current Stack Frame.
Referring now to
FIG. 2
, a diagram illustrates a stack frame structure for the known AIX platform. AIX is an open operating system developed by International Business Machines (IBM) Corporation that is based on a version of UNIX. The AIX stack frame structure differs, for instance, from the stack frame structure of
FIG. 1
in that it dispenses with a frame pointer, basing everything off a Stack Pointer (in some special cases, however AIX will use a frame pointer). As shown in
FIG. 2
, in AIX, stacks grow downwards (i.e. the Stack Pointer decreases as new frames are created). Furthermore, in AIX, arrays typically grow upwards. For instance, as shown in
FIG. 2
, an Array Pointer is incremented as larger indexed elements of an array, Array Var[
1
] . . . Array Var[n], are accessed. The Stack Pointer points to the memory location at the top of a Current Frame associated with the procedure being executed. This memory location contains a Back Chain Pointer
2
which points to a memory location at the top of the previous stack frame of the calling procedure (which memory location contains a Back Chain Pointer
1
that points to the top of its calling procedure, and so on). The Back Chain Pointers provide the mechanism for “following the back chain.” The back chain pointer (not shown) of the first procedure in the program has a value of 0.
Furthermore, a Procedure Return Pointer
1
contains the address (in the calling procedure) to which the called current procedure must return after it completes execution. In particular, when the called procedure completes, the Stack Pointer is set to the top of the calling procedure by reading the Back Chain Pointer
2
. The Procedure Return Pointer
1
is then read and the program branches to this address.
The overflow of a stack allocated array (or buffer) can potentially lead to what is known as a “Stack Smashing Attack.” Buffer overflow occurs when incorrect programming allows a user of a program to write beyond the bounds of some array (buffer). This can occur, for example, when the programmer expects the user to input no more than N characte

Affiliated with

Also associated with

Rating

Method for preventing buffer overflow attacks does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Method for preventing buffer overflow attacks, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for preventing buffer overflow attacks will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3157991