Heap Overflow: Understanding Execution

TL;DR

A heap overflow happens when a program writes data beyond the allocated memory space on the heap. This can overwrite important data structures, leading to crashes or even allowing attackers to control the program’s execution. We’ll cover how this works and how to identify what code is being executed as a result.

What is a Heap Overflow?

The heap is a region of memory used for dynamic allocation – when your program needs memory during runtime (e.g., creating objects). A heap overflow occurs when you write more data into a heap buffer than it can hold. This overwrites adjacent memory, potentially corrupting critical program information.

Identifying What Code is Executed

1. Understand the Crash: When a heap overflow causes a crash, the error message or debugger will give you clues about where the problem occurred. Look for segmentation faults (segfaults) or access violations.
2. Debugging with GDB: The GNU Debugger (GDB) is your primary tool.
   • Run the program under GDB:

     gdb ./your_program

   • Set a breakpoint: Set a breakpoint near where you suspect the overflow happens.

     break main

     or

     break function_name

   • Run the program:

     run

   • Examine memory: Use commands like x/100xw &variable_name to view the contents of memory around your vulnerable variable. The ‘x’ means examine, ‘100’ is the number of words (4 bytes each on a 32-bit system), ‘w’ means word format, and ‘&variable_name’ is the address of the variable.
   • Backtrace: After a crash, use bt (backtrace) to see the call stack. This shows you the sequence of function calls that led to the crash. Look for unexpected functions or addresses in the backtrace – these are often indicators of overwritten return addresses or function pointers.
3. Heap Analysis Tools: Several tools can help analyze heap memory.
   • Valgrind (Memcheck):

     valgrind --leak-check=full ./your_program

     Memcheck detects many types of memory errors, including heap overflows.

   • AddressSanitizer (ASan): ASan is a fast memory error detector built into compilers like GCC and Clang. Compile with the -fsanitize=address flag:

     gcc -fsanitize=address your_program.c -o your_program

     . Run the program as normal; ASan will report errors if it detects a heap overflow.

4. Return Address Overwrite: A common outcome of a heap overflow is overwriting the return address on the stack. This allows an attacker to redirect execution to arbitrary code.
   • Examine the Stack: In GDB, use x/20xw &return_address (replace ‘return_address’ with the actual variable name or address) to see if the return address has been modified.
   • Identify Overwritten Address: The overwritten return address will point to the code that is now being executed. This could be shellcode injected by an attacker, a function within your program, or even system libraries.
5. Function Pointer Overwrite: If you have function pointers stored on the heap, an overflow can overwrite them.
   • Locate Function Pointers: Identify where function pointers are stored in memory.
   • Examine Function Pointer Values: Use GDB to check if the values of these function pointers have been changed after the overflow. An altered function pointer will point to a different function than intended, causing that function to be executed instead.

Example Scenario

Let’s say you have code like this:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main() {
  char *buffer = (char *)malloc(10);
  strcpy(buffer, "This is a very long string..."); // Potential overflow!
  printf("Buffer contents: %sn", buffer);
  free(buffer);
  return 0;
}

In this example, strcpy doesn’t check the size of the input string. If the string is longer than 10 bytes, it will overflow the buffer. Using GDB and a backtrace after the crash would likely show that execution has been redirected to an unexpected location due to the overwritten return address.

Mitigation

• Use Safe Functions: Avoid functions like strcpy, gets, and sprintf. Use safer alternatives like strncpy, fgets, and snprintf that allow you to specify the maximum buffer size.
• Bounds Checking: Always check the length of input data before copying it into a buffer.
• Compiler Protections: Enable compiler protections like stack canaries (-fstack-protector) and address space layout randomization (ASLR).