Debugger tricks: API call logging, the quick’n’dirty way (part 1)

One common task that you may be faced with while debugging a problem is to log information about calls to a function or function(s). While if you want to know about a function in your program that you have source code to, you could often just add some sort of debug print and rebuild the program, sometimes this isn’t practical. For example, you might not always be able to reproduce a problem and so it might not viable to have to restart with a debug-ified build because you might blow away your repro. Or, more importantly, you might need to log calls to functions that you don’t have source code to (or aren’t building as part of your program, or otherwise don’t want to modify).

For example, you might want to log calls to various Windows APIs in order to gain information about a problem that you are troubleshooting. Now, depending on what you’re doing, you might be able to do this by adding debug prints before and after every single call to the particular API. However, this is often less than convenient, and if you aren’t the immediate caller of the function you want to log, then you’re not going to be able to take that route anyway.

There are a number of API spy/API logging packages out there (and the Debugging Tools for Windows distribution even ships with one, called Logger, though it tends to be fairly fragile – personally, I’ve had it crash out on me more often than I’ve had it actually work). Although you might be able to use one of those, a big limitation of “shrink-wrapped” logging tools is that they won’t know how to properly log calls to custom functions, or functions that are otherwise not known to the logging tool. The better logging tools out there are user-extensible to a certain extent, in that they typically provide some sort of scripting- or programmming- language that allows the user (i.e. you) to describe function parameters and calling conventions, so that they can be logged.

However, it can often be difficult (or even impossible) to describe many types of functions to these tools – such as functions that contain pointers to structures that contain pointers to other structures, or other such non-trivial constructs. As a result, for many circumstances, I tend to recommend to not use so-called “shrink-wrapped” API logging tools in situations where I want to log calls to functions.

In the event that it’s not a feasible solution to implement debug prints in source code, though, it would appear on the surface that this leaves one without a usable solution for logging calls. Not so, in fact – it turns out that with some careful use of so-called “conditional breakpoints”, you can often use the debugger (e.g. WinDbg/ntsd/cdb/kd, which is what I shall be referring to for the rest of this article) to provide this sort of call logging. Using the debugger has many advantages; for instance, you can do this sort of API logging “on the fly”, and in situations where you can attach the debugger after the process has started, you don’t even need to start the program specially in order to log it. Even better, however, is that the debugger has extensive support for displaying data in meaningful forms to the user.

If you think about it, displaying data to the user is one of the prinicpal functions of the debugger, in fact. It’s also one of the major reasons why the debugger is highly extensible via extensions, such that complicated data structures can be displayed and interpreted in a meaningful fashion. By using the debugger to perform your API logging, you can take advantage of the rich functionality for displaying data that is already baked into the debugger (and its extensions, and even any custom extensions of your own that you have written) to double as a call logging facility.

Even better, because the debugger can read and display many data types in a meaningful fashion based off of symbol files (if you have private symbols, such as for programs you compile or provide), for data types that don’t have specific debugger extensions for displaying them (like !handle, !error (for error codes), !devobj, and soforth), you can often utilize the debugger’s ability to format data based off of type information in symbols. This is typically done via the dt command, and often provides a workable display for most custom data types without having to do any sort of complicated “training” like you might have to do with a logging program. (Some data structures, such as trees and lists may need some more intelligence than what is provided in dt for displaying all parts of the data structure. This is typically true for “container” data types, although even for those types, you can still often use dt to display actual members within the container in a meaningful fashion.) Utilizing the information contained within symbols files (via the debugger) for API logging also frees you from having to ensure that your logging program’s definitions for all of your structures and other types are in synch with the program you are debugging, as the debugger automagically receives the correct definitions based on symbols (and if you are using a symbol server that includes indexed versions of your own internal symbols, the debugger will even be able to find the symbols on its own).

Another plus to this approach is that, provided you are reasonably familiar with the debugger, you probably won’t have to learn a new description language like you might if you were using an API logging program. This is because you’re probably already familiar with many of the commands the debugger makes available for displaying data, from every-day debugger usage. (Even if you aren’t all that familiar with the debugger, there is extensive documentation that ships with the debugger by default which describes how to format and display data via various debugger commands. Additionally, there are many examples describing how to use most of the important or useful debugger commands out there on the Internet.)

Okay, enough about why you might want to consider using the debugger to perform call logging. Next time, a quick look and walkthrough describing how you can do this (it’s really quite simple, as alluded to previously), along with some caveats and gotchas that you might want to watch out for along the way.

One Response to “Debugger tricks: API call logging, the quick’n’dirty way (part 1)”

  1. […] Nynaeve Adventures in Windows debugging and reverse engineering. « Debugger tricks: API call logging, the quick’n’dirty way (part 1) […]