Nynaeve

The system call dispatcher on x86 NT has undergone several revisions over the years.

Until recently, the primary method used to make system calls was the int 2e instruction (software interrupt, vector 0x2e). This is a fairly quick way to enter CPL 0 (kernel mode), and it is backwards compatible with all 32-bit capable x86 processors.

With Windows XP, the mainstream mechanism used to do system calls changed; From this point forward, the operating system selects a more optimized kernel transition mechanism based on your processor type. Pentium II and later processors will instead use the sysenter instruction, which is a more efficient mechanism of switching to CPL 0 (kernel mode), as it dispenses with some needless (in this case) overhead of usual interrupt dispatching.

How is this switch accomplished? Well, starting with Windows XP, the system service call stubs do not hardcode a particular instruction (say, int 2e) anymore. Instead, they indirect through a field in the KUSER_SHARED_DATA block (“SystemCall”). The meaning of this field changed in Windows XP SP2 and Windows Server 2003 SP1; in prior versions, the SystemCall field held the actual code used to make the system call (and was filled in at runtime with the proper values). In XP SP2 and Srv03 SP1, in the interests of reducing system attack surface, the KUSER_SHARED_DATA region was marked non-executable, and SystemCall becomes a pointer to a stub residing in NTDLL (with the pointer value being adjusted at runtime based on the processor type, to refer to an appropriate system call stub).

What this means for you today is that on modern systems, you can expect to see a sequence like so for system calls:

0:001> u ntdll!NtClose
ntdll!ZwClose:
7c821138 b81b000000       mov     eax,0x1b
7c82113d ba0003fe7f       mov     edx,0x7ffe0300
7c821142 ff12             call    dword ptr [edx]
7c821144 c20400           ret     0x4
7c821147 90               nop

0x7ffe0300 is +0x300 bytes into KUSER_SHARED_DATA. Looking at the structure definition, we can see that this is “SystemCall”:

0:001> dt ntdll!_KUSER_SHARED_DATA
   +0x000 TickCountLowDeprecated : Uint4B
   +0x004 TickCountMultiplier : Uint4B
   +0x008 InterruptTime    : _KSYSTEM_TIME
   [...]
   +0x300 SystemCall       : Uint4B
   +0x304 SystemCallReturn : Uint4B
   +0x308 SystemCallPad    : [3] Uint8B
   [...]

Since my system is Srv03 SP1, SystemCall is a pointer to a stub in NTDLL.

0:001> u poi(0x7ffe0300)
ntdll!KiFastSystemCall:
7c82ed50 8bd4             mov     edx,esp
7c82ed52 0f34             sysenter
ntdll!KiFastSystemCallRet:
7c82ed54 c3               ret

On my system, the system call dispatcher is using sysenter. You can look at the old int 2e dispatcher if you wish, as it is still supported for compatibility with older processors:

0:001> u ntdll!KiIntsystemCall
ntdll!KiIntSystemCall:
7c82ed60 8d542408         lea     edx,[esp+0x8]
7c82ed64 cd2e             int     2e
7c82ed66 c3               ret

The actual calling convention used by the system call dispatcher is thus:

• eax contains the system call ordinal.
• edx points to either the argument array of the system call on the stack (for int 2e), or the return address plus argument array (for sysenter).

For most of the time, though, you’ll probably not be dealing directly with the system call dispatching mechanism itself. If you are, however, now you know how it works.

This entry was posted on Wednesday, August 23rd, 2006 at 9:47 pm and is filed under Debugging, Programming, Reverse Engineering, Windows. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.