Tuesday, February 20, 2007

Debugger Breakpoints

An overview of breakpoints, as implemented in the ZeroBugs debugger for Linux.

Breakpoints are central to the ZeroBugs debugger engine layer. Breakpoints can be set by the user, or by the debugger for internal purposes (such as detecting the creation of new threads).

Physical vs. Logical

Breakpoints can be classified in several ways. One categorization distinguishes between "logical breakpoints" and "physical breakpoints". What this means is that not all the breakpoints that you have inserted in the program are physically there, but the debugger will support the illusion that they are; reality is the realm of physical breakpoints, and logic is derived off perception. So if you perceive a breakpoint as being inserted in the debugged process, it is logically there, even though, physically, the debuggee has not been affected.

Let's consider a couple of examples, to help bring the discussion out of the philosophical realm:

  1. The user inserts a breakpoint at the beginning of a function that is not loaded into memory yet, because it lives in a shared library that has not been mapped into the debugge's memory space (just yet). The debugger nicely shields the user from knowing such details, and may say: "OK. I don't know what the address in memory of function `abc' is; but I know that it is implemented inside the dynamic library libabc.so; I will keep this in mind, so that if I later detect that libabc.so is loded, I will insert the breakpoint. "

  2. Another case may be that the debugger has inserted a breakpoint at the beginning of the pthread_create() function, to internally keep track of newly created threads. The user wants to insert a breakpoint at the same address, and does not need to know that a physical breakpoint is already there. The debugger associates two logical actions with the same physical breakpoint: one that internally updates the list of debugged threads, and another one that initiates an interaction with the user.

The logical breakpoints are implemented as actions associated with physical breakpoints. Each physical breakpoint maintains a list of actions. An action may be temporary (or once-only), which means that it gets discarded after being executed once. Once-only actions are similar to UNIX System V signal handlers. Non-temporary actions are executed each time the physical breakpoint is hit -- similar to BSD signal handlers.

Algorithm for executing breakpoint actions

// Execute actions on given thread
void BreakPoint::execute_actions(Thread* thread)
// The list of actions associated with this
// breakpoint may change during
// the execution of actions, and thus the
// iterators may be invalidated:
// make a copy of the actions and cycle thru
// the copy, to be safe.
ActionList tmp(this->actions_);
ActionList::iterator i = tmp.begin();
for (size_t d = 0; i != tmp.end(); ++d) {
if (is_disabled(*i)) {
++i; continue;
// a temporary action returns false
if ((*i)->execute(thread, this)) {
else {
// remove it from the master list
ActionList::iterator j = actions_.begin();
advance(j, d); actions_.erase(j);
// remove it from tmp as well so that
// destruction is not delayed
i = tmp.erase(i);

Software vs. Hardware

The Intel 386/486/585/686 family of chips offers support for debugging, including breakpoints. The CPU has 6 debug registers: 4 for addresses, one for control, and one for status. Each of the first 4 can hold a memory address that causes a hardware fault when accessed.

In Intel's lingo, a "fault" is a hardware notification, or event, that happens when the CPU is about to access a memory address -- that is, before the access happens. An "exception" is a similar notification, only that it happens after the access has occurred.

Remember: Hardware breakpoints are faults, software breakpoints are exceptions.

The control register holds some flags that specify the type of access (read, read-write, execute) and some other bits; the status register is helpful for determining which breakpoint was hit, when handling a system fault.

Thanks to Operating System magic, the hardware breakpoints are multiplexed, so we can have as many as N times 4 hardware breakpoints per program, where N is the number of threads in the program.Hardware breakpoints have the advantage of being non-intrusive -- the debugged program is not modified. Another advantage is that they can be set to monitor data as well as code. A debugger may use a hardware breakpoint to detect that a memory location is being accessed.

Software breakpoints are implemented as a special opcode (INT 0x3) that is inserted in the code at location to be monitored.

Nicely enough, Intel has a dedicated opcode for breakpoints. Other CPUs (PowerPC, for example) do not have a special opcode; on those platforms software breakpoints are implemented by inserting an invalid code at the desired location.

Software breakpoints have the main drawbacks of being slow and intrusive. The debugged program has to be modified, and the debugger needs to memorize the original opcode at the modified location, so that the debuggee's code is restored when the breakpoint is removed. When a software breakpoint is hit, the instruction pointer needs to be decremented, and the original opcode restored. Then the debugged program has to be stepped out of the breakpoint. After the breakpoint is handled, the breakpoint opcode is reinserted.

On UNIX derivatives (such as Linux), a debugger does not manipulate the debugged program directly; rather, it uses the operating system as a middle man (via the ptrace or /proc API). This implies that every time the debugger reads or writes into the debuggee's memory space, a context switch from user mode to kernel mode happens.

Another disadvantage of soft breakpoints is that they can only monitor code. Software breakpoints cannot be used for watching data accesses.

What makes software breakpoints indispensable is that there's no limit to how many can be inserted. Hardware breakpoints are a very scarce resource (you can run out of the 4 of them quite fast); software breakpoints are intrusive and slower, but can be used abundantly.

The design decision in my debugger is to use software breakpoints for user-specified breakpoints, and prefer hardware breakpoints for internal purposes. Watchpoints (breakpoints that monitor data access) are implemented as hardware breakpoints.

An example of breakpoints maintained by the debugger internally is stepping over function calls. A breakpoint is inserted at the location where the function returns, and control is given to the debuggee to run at full speed until the breakpoint is hit. The breakpoint is removed once it is hit, and the hardware resource can then be reused.

As a rule of thumb, the debugger employs the hardware support for cases where breakpoints are expected to be released after relatively short amounts of time. If no hardware registers are avaialable, the debugger falls back to using a software breakpoint.

Global vs. Per-Thread

Another categorization of breakpoints is by the what threads they affect in a multi-threaded program. A global breakpoint causes the program to stop, regardless of what thread has hit it. Per-thread breakpoints will stop the program only when reached by a given thread. Because all threads share the same code segment, a software breakpoint is also a global breakpoint, since
any thread that reaches the break opcode will stop.

The operating system creates the illusion of each thread running on its own CPU, therefore a hardware breakpoint may be private to a given thread.

A bit in the debug control register of the 386 chip can be used to control the global/per-task behavior of hardware breakpoints.

A thread ID can be added to the data structure or class that represents a software breakpoint. When the breakpoint is hit, the thread ID in the structure may be compared against the ID of the current thread. The behavior of a per-thread breakpoint can be emulated this way.

The debugger uses emulated breakpoints when it needs a hardware breapoint and none of the 4 debug registers is available.

Consider the case where the debugger uses a breakpoint for quickly stepping over function calls. The debugged program must stop only if the breakpoint at the function's return address is hit by the same thread that was current when the user gave the "step over" command.

No comments: