{"id":10,"date":"2006-07-12T07:00:14","date_gmt":"2006-07-12T12:00:14","guid":{"rendered":"http:\/\/www.nynaeve.net\/?p=10"},"modified":"2019-12-13T17:43:36","modified_gmt":"2019-12-13T22:43:36","slug":"x64-debugging-intro-part-2","status":"publish","type":"post","link":"http:\/\/www.nynaeve.net\/?p=10","title":{"rendered":"Introduction to x64 debugging, part 2"},"content":{"rendered":"<p>Last time, I talked about some of the basic differences you&#8217;ll see when switching to an x64 system if you are doing debugging using the Debugging Tools for Windows package.\u00c2\u00a0 In this installment, I&#8217;ll run through some of the other differences with debugging that you&#8217;ll likely run into &#8211; in particular,\u00c2\u00a0how changes to the x64 calling convention will make your life much easier when debugging.<\/p>\n<p>Although the x64 architecture is in many respects very similar to x86, many of the conventions of x86-Win32 that you might be familiar with have changed.\u00c2\u00a0 Microsoft took the opportunity to &#8220;clean house&#8221; with many aspects of Win64, since for native x64 programs, there is no concern of backwards binary compatibility.<\/p>\n<p>One of the major changes that you will quickly discover is that the calling conventions that x86 used (__fastcall, __cdecl, __stdcall) are not applicable to x64.\u00c2\u00a0 Instead of many different calling conventions, x64 unifies everything into a single calling conention that all functions use.\u00c2\u00a0 You can <a title=\"MSDN documentation on x64 calling convention\" href=\"http:\/\/msdn2.microsoft.com\/en-us\/library\/7kcdt6fy.aspx\">read the full details<\/a> of the new calling convention on MSDN, but I&#8217;ll give you\u00c2\u00a0the executive summary as it applies to debugging programs here.<\/p>\n<ul>\n<li>\u00c2\u00a0The first four arguments of a function are passed as registers; rcx, rdx, r8, and r9 respectively.\u00c2\u00a0 Subsequent arguments are passed on the stack.<\/li>\n<li>The caller allocates the space on the stack for parameter passing, like for __stdcall on x86.\u00c2\u00a0 However, the caller must allocat<em>e at least<\/em> 32 bytes of stack space for the callee to\u00c2\u00a0use a &#8220;register home space&#8221; the first four parameters (or scratch space).\u00c2\u00a0 This must be done even if the callee has no arguments or less than four arguments.<\/li>\n<li>The caller always cleans the stack of arguments passed (like __cdecl on x86) if necessary.<\/li>\n<li>Stack unwinding and exception handling are significantly different on x64; more details on that later.\u00c2\u00a0 The new stack unwinding model is data-driven rather than code-driven (like on x86).<\/li>\n<li>Except for dynamic stack adjustments (like _alloca), all stack space must be allocated in the prologue.\u00c2\u00a0 Effectively, for most functions, the stack pointer will remain constant throughout the execution process.<\/li>\n<li>The rax register is used for return values.\u00c2\u00a0 For return values larger than 64 bits, a hidden pointer argument is used.\u00c2\u00a0 There is no more spillover into a second register for large return values (like edx:eax, on x86).<\/li>\n<li>The rax, rcx, rdx, r8, r9, r10, r11 registers are volatile, all other registers must be preserved.\u00c2\u00a0 For floating point usage, the xmm0, xmm1, xmml2, xmm3, xmm4, xmm5 registers are volatile, and the other registers must be preserved.<\/li>\n<li>For floating point arguments, the xmm0 through xmm3 registers are used for the first four arguments, after which stack spillover is performed.<\/li>\n<li>The instructions permitted in function prologues and epilogues are highly restricted to a very small subset of the instruction set to\u00c2\u00a0facilitate unwinding operations.<\/li>\n<\/ul>\n<p>The main takeaways here from a debugging pespective are thus:<\/p>\n<ul>\n<li>Even though a register calling convention like __fastcall is used, the register arguments are often spilled to the &#8220;home area&#8221; and so are typically visible in call stacks, especially in debug builds.<\/li>\n<li>Due to the nature of parameter passing on x64, the &#8220;push&#8221; instruction is seldom used for setting up arguments.\u00c2\u00a0 Instead, the compiler allocates all space up front (like for local variables on x86) and uses the &#8220;mov&#8221; instruction to write stack parameters onto the stack for function calls.\u00c2\u00a0 This also means that you typically will not see an &#8220;add rsp&#8221; (or equivalent) after each function call, despite the fact that the caller cleans the stack space.<\/li>\n<li>The first stack arguments (argument 5, etc) will appear at [rsp+28h] instead of [rsp+08h], because of the mandatory register home area.\u00c2\u00a0 This is a departure from how __fastcall worked on x86, where the first stack argument would be at [esp+04h].<\/li>\n<li>Because of the data driven unwind semantics, you will see <em>perfect stack unwinding<\/em> even without symbols.\u00c2\u00a0 This means that even if you don&#8217;t have any symbols at all for a third party binary, you should always get\u00c2\u00a0a complete stack trace all the way back to the thread start routine.\u00c2\u00a0 As a side effect, this means that the stack traces captured by PageHeap or handle traces will be much more reliable than on x86, where they tended break at the first function that did not use ebp (because those stack traces never used symbols).<\/li>\n<li>Because of the restrictions on the prologue and epilogue instruction usage, it is very easy to recognize where the actual important function code begins and the boilerplate prologue\/epilogue code ends.<\/li>\n<\/ul>\n<p>If you&#8217;ve been debugging on x86 for a long time, then you are probably pretty excited about the features of the new calling convention.\u00c2\u00a0 Because of the perfect unwind semantics and constant stack pointer throughout function execution model, debugging code that you don&#8217;t have symbols for (and using the built-in heap and handle verification utilities) is <em>much<\/em> more reliable than x86.\u00c2\u00a0 Additionally, compiler generated\u00c2\u00a0code is usually easier to understand, because you don&#8217;t have to manually track the value of the stack pointer changing throughout the function call like you often did on x86 functions compiled with frame pointer omission (FPO) optimizations.<\/p>\n<p>\u00c2\u00a0There are some exceptions to the rules I laid out above for the x64 calling convention.\u00c2\u00a0 For functions that do not call any other functions (called &#8220;leaf&#8221; functions), it is permissible to utilize custom calling conventions so long as the stack pointer (rsp) is not modified.\u00c2\u00a0 If the stack pointer is modified then regular calling convention semantics are required.<\/p>\n<p>Next time, I&#8217;ll go into more detail on how exception handling and unwinding is different on x64 from the perspective of what the changes mean to you if you are debugging programs, and how you can access some of the metadata associated with unwinding\/exception handling and use it to your advantage within the debugger.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Last time, I talked about some of the basic differences you&#8217;ll see when switching to an x64 system if you are doing debugging using the Debugging Tools for Windows package.\u00c2\u00a0 In this installment, I&#8217;ll run through some of the other differences with debugging that you&#8217;ll likely run into &#8211; in particular,\u00c2\u00a0how changes to the x64 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[2,5],"tags":[],"_links":{"self":[{"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=\/wp\/v2\/posts\/10"}],"collection":[{"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10"}],"version-history":[{"count":1,"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=\/wp\/v2\/posts\/10\/revisions"}],"predecessor-version":[{"id":700,"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=\/wp\/v2\/posts\/10\/revisions\/700"}],"wp:attachment":[{"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.nynaeve.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}