Introduction
Building your very own debugger is a great way to understand workings of a commercially available debugger. In this article, the reader will be exposed to certain aspects of the OS and CPU opcode (x86-32-bit only). This article will show the workings of breakpoints and working of OutputDebugString (since we will be handling these two events only) used commonly while debugging, readers are urged to investigate conditional breakpoint and step wise execution (line by line) that are commonly supported by most debuggers. Run to cursor is similar to breakpoint.
Background
Before we start, reader will require basic knowledge of OS. Discussions related to OS is beyond the scope of this article, please feel free to refer other articles (or write to me) while reading this. Reader would be required to be exposed to commercially available debuggers (for this article: VS2010) and have debugged applications before using break points
Break Points
Breakpoint allows users to place a break in the flow of a program being debugged. The user may do this to evaluate certain conditions at that point in execution.
The debugger adds an instruction : int 3 (opcode : 0xcc) at the particular address (where break point is desired) in the process space of the executable being debugged. After this instruction is encountered:-
the EIP is moved to the interrupt service routine (in this case int 3),
The service routine will save the CPU registers (all Interrupt service routines must do this) ,signal the attached debugger, the program that called DebugActiveProcess(process ID of the exe being debugged) look up MSDN for this API.
The debugger will run a debug loop (mentioned in code as EnterDebugLoop() in file Debugger.cpp)
The signal from the service routine will trigger WaitForDebugEvent(&de, INFINITE), the debug loop (mentioned in the code as EnterDebugLoop) will loop through every debug signal encountered by WaitForDebugEvent. After processing the debug routine, the debugger will restore the instruction by replacing 0xcc (int 3) with the original instruction and return from the service routine by calling ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_CONTINUE). ( Before Placing the break point, the debugger must use ReadProcessMemory to get the original BYTE at that memory location).
when it returns from an interrupt service routine( using IRET) , EIP will point to the next byte to be executed, but we want it to point to the previous byte (the one we restored), this is done while handling the break point, use GetThreadContext to get value of ESP (this holds on the stack, the return address of EIP). Subtract one from the return address (get the return address by using ReadprocessMemory) and use WriteProcessMemory to update it, now EIP will be restored correctly (there are different ways to achieve this, VS2010 and other debuggers may use a different approach).
OutputDebugString
This API is used to display a string on the debug console, the user may use this to display certain state related information or trace.
when this is API is occured , OUTPUT_DEBUG_STRING_EVENT event is triggered.
An attached debugger will handle this event in the debug loop (mentioned in the code as EnterDebugLoop).
The event handling API will provide information of the string relative to the Debuggee's process space
use ReadProcessMemory to acquire the string (memory dump) from another process.
Using the code
The attached code must be referred to at all times while reading this article. The break point (opcode: 0xcc) is introduced by
BYTE p[]={0xcc}; //0xcc=int 3 // Any source code blocks look like this
::WriteProcessMemory(pi.hProcess,(void*)address_to_set_breakpoint, p, sizeof(p), &d);
Read more: Codeproject