Problem solve Get help with specific problems with your technologies, process and projects.

Stack overflows versus stack overruns

What is stack overflow? What is the fix?

A stack overflow is one of two things, which is what makes it a little confusing. In the first case, a true stack overflow is when you run out of stack space in the process. If you've ever had your application pause for a second or two and then just go away, that's a stack overflow due to infinite recursion. The stack in most OSes can only grow so far before it fills up. To avoid stack overflows, here are the basic rules:

  1. Be extra careful when doing recursion. Always verify completely the out case so you don't end up calling yourself infinitely.
  2. Be cognizant of how much space local variables take up. If you are putting 4K buffers on the stack, all over the place, you can easily run out of stack space as your code executes. Keep in mind that especially with Windows development, the messaging system can get quite reentrant so much more the stack is used than you expect. The general rule I use is if the data structure is more than 260 bytes, I probably need to look at allocating it from the heap instead of the stack. People often confuse a stack overrun with a stack overflow. A stack overrun is when you write more data to a variable on the stack than the variable original size. This will corrupt the variables or return address up the stack. Probably the best way to show this is with some pictures! Keep in mind that stack growns from high memory addresses to low memory addresses and all values are 4 bytes.

Here's the sample code:

void SomeFunc ( void ) { int iVar1 ; int iVar2 ; iVar1 = 0x33 ; iVar2 = 0x44 memcpy ( &iVar1 , 0 , 12 ) ; // Yes, this is a bug! }

If the instruction pointer is sitting on the memcpy instruction in SomeFunc, the stack looks like the following:

 (Stack Picture) (Stack Address) (Stack Values) +-------------------+ | Return Address | 0x0012FE1C 0x0040136B +-------------------+ | Saved Frame | 0x0012FE18 0x0012FF70 +-------------------+ | Local Variable 1 | 0x0012FE14 0x00000033 +-------------------+ | Local Variable 2 | 0x0012FE10 0x00000044 +-------------------+

After executing the memcpy, which is copying 12 bytes to a 4 byte location, the stack looks like the following:

 (Stack Picture) (Stack Address) (Stack Values) +-------------------+ | Return Address | 0x0012FE1C 0x00000000 +-------------------+ | Saved Frame | 0x0012FE18 0x00000000 +-------------------+ | Local Variable 1 | 0x0012FE14 0x00000000 +-------------------+ | Local Variable 2 | 0x0012FE10 0x00000044 +-------------------+

As you can see, the memcpy in the function doesn't do any error checking and blindly writes 12 bytes worth of zeros to a memory location that only holds four bytes. Consequently, it overwrote the saved frame and return address so when this function ends and tries to return to the caller, the CPU will pick the return address off the stack and put that into the instruction pointer on the CPU. Since that value is now zero, the program crashes.

Fixing stack overruns used to be quite hard as there were no tools on the market to help. Compuware's BoundsChecker, which I helped architect and develop, looks for stack overruns automatically. Additionally, the new C++ compiled in VC.NET also looks for stack overruns automatically in debug builds with the new /RTCx switches.

For more information about the new switches, see my August 2001 MSDN Magazine Bugslayer column. Of course, even though you have the automatic tools does not excuse the programmer from looking for these bugs as they are developing!

Dig Deeper on Enterprise infrastructure management