Comments on: Fast kernel debugging for VMware, part 1: Overview

By: Skywing

Skywing — Wed, 10 Oct 2007 17:16:12 +0000

Yeah, that’ll do it. The code for finding the kernel base is kind of fragile and uses the module name for comparisons, which falls flat on its face if you boot with /KERNEL=.

By: Steve

Steve — Wed, 10 Oct 2007 17:16:00 +0000

I have it setup with a checked version of windows 2003 (partially checked build), and I renamed the kernel components to halmacpi.chk, ntkrnlmp.chk and ntkrpamp.chk.

Is there a way I could recompile the driver on the vm side to look for specific variables instead of ntoskrnl.exe?? Maybe make a version that looks for a registry entry?

Thanks!

By: Steve

Steve — Wed, 10 Oct 2007 16:04:16 +0000

Ahah! Yes, I’m using a partially checked build right now!

By: Skywing

Skywing — Wed, 10 Oct 2007 15:43:37 +0000

If you just hit “reset” in the VMware console, the vmware-vmx process should be reused and the DLL won’t need to be reinjected. (Attempting to inject the DLL multiple times into the vmware-vmx process is harmless and will have no effect.)

vmxpatch.dll is designed to be able to handle driver reconnects across its lifetime (e.g. across a reboot); this worked in my tests.

These are most of the error statuses that can be returned from kdvmware’s DriverEntry along with the net start error message:

STATUS_PORT_DISCONNECTED (“The handle is invalid.”) – couldn’t connect to vmxpatch.dll
STATUS_NO_MEMORY (“Not enough storage space is available to process this command.”) – couldn’t allocate a block of contiguous physical memory
STATUS_NOT_FOUND (“Element not found.”) – couldn’t find (either ntoskrnl.exe OR ntkrnlpa.exe) OR kdcom.dll in the loaded module list.
STATUS_INSUFFICIENT_RESOURCES (“Insufficient system resources exist to complete the requested service.”) – memory allocation failure
STATUS_ACCESS_VIOLATION (“Invalid access to memory location.”) – MmMapLockedPagesSpecifyCache failed for for creating a locked view with alternate protection for kdcom.dll patching
STATUS_PROCEDURE_NOT_FOUND (“The specified procedure could not be found.”) – couldn’t locate a required kdcom.dll export

If you’re getting “element not found”, then that is most likely a failure to find either kdcom.dll or ntoskrnl.exe in the loaded module list. Are you starting the system with /KERNEL= or something of that sort?

There are also a number of debug prints in vmxpatch.dll which will be enabled if a debugger is attached to vmware-vmx.exe. However, if you’re getting STATUS_NOT_FOUND, I suspect that kdvmware.sys will have been aborting before it tries to talk to vmxpatch.dll.

By: Steve

Steve — Wed, 10 Oct 2007 14:54:12 +0000

I’ve been using this for a couple of days, and it works great in most cases. I have one case where I’m getting problems:

If I reboot my vm, I cannot re-establish connection with it afterwards. I cannot start the kdvmware driver, I get an error: Element not found.

I suspect I have to re-inject the dll in some way on the host side, but I guess I need to remove it first, and I havent’ gotten around that.

In other words I’ve tried re-injecting the dll after the reboot, and restarting the kdvmware service without any success. Any pointers?

By: Soren Dreijer

Soren Dreijer — Fri, 05 Oct 2007 22:19:04 +0000

I’ve always been really annoyed by the CPU spinning that’s going on in the VM when you’re broken into the debugger. It’s particularly annoying if you’re trying to compile something at the same time…

Can’t wait to try this out.

By: Nynaeve » Blog Archive » Fast kernel debugging for VMware, part 2: KD Transport Module Interface

Fri, 05 Oct 2007 22:06:21 +0000

[...] Nynaeve Adventures in Windows debugging and reverse engineering. « Fast kernel debugging for VMware, part 1: Overview [...]

By: Skywing

Skywing — Fri, 05 Oct 2007 22:00:45 +0000

I certainly agree that it’d be great if the virtualization vendors did this natively. With source level integration, a lot of the annoying parts of VMKD relating to things like loading the hook DLL into the user mode VM process (and having to manually find the right PID associated with the VM you want), poking around in the VM process for undocumented functions, and soforth could easily be eliminated. (If anyone from VMware / Virtual Server happens to read this and is so inclined, feel free to drop me a mail if you’re interested in doing that, btw!)

I’m also working on trying to get the DbgEng people to open up a nice pluggable interface to allow native VM debugging transports to more easily integrate with the debugger as well (as it is, VMKD internally has to basically reimplement the entire KDCOM framing protocol, for reasons that I’ll go into in other posts). For a couple of reasons, direct integration with DbgEng would also be much better than how VMKD currently operates (among other things, the speed at which dump files are written could be vastly improved even over the performance improvements that VMKD currently brings over serial port debugging).

No response from the debugger folks yet, though, on that front.

By: Steve

Steve — Fri, 05 Oct 2007 19:17:17 +0000

This is an extremly cool idea!!!

And it works great too!!! I can’t beleive that this hasn’t been implemented by vmware themselves ;)

You should try to contact microsoft to see if you can get it work this way with their vms as well!

I haven’t been this happy with my debugger since 1394! Finally a responsive debugger!!!

Thanks dude!!