Discovering the Arguments Passed to Windows API Functions with Public Symbols
I've been talking at a bit of a higher level lately, and today I'm just in the mood to go into some of the deeper debugging aspect that you may run across when looking at Windows Vista application compatibility. (Well, perhaps not deep to some, but for most of us, we haven't spent much time in a low-level debugger lately so the skills have frequently atrophied somewhat.)
When I talk about debugging for application compatibility, I always emphasize this point: the application itself is the same - Windows is now different. So the entry and exit points from Windows take on an exaggerated importance in these debugging sessions. However, internally, we have a bit of an advantage: those entry points are more expressive. You see, we have private symbols to Windows, so the Locals window in WinDbg is quite helpful. Everyone else just gets public symbols. So, those entry points are a little bit more mysterious. Of course, that doesn't mean you are stuck, it just means you have to do a touch more work.
For this exercise, I'm just going to use an application that I know checks the version number in Windows, the MSDN library, and the Debugging Tools for Windows. We're going to get the same information internal folks get without requiring private symbols!
Going in, I know that the application has an error where it tells me that it's expecting an older version of the operating system (in this case, Windows XP). Let's dissect that entry point.
MSDN tells us that the function is implemented in Kernel32.dll, so let's set our breakpoint:
0:000> bp kernel32!GetVersionExW
Now, let's run it up to that breakpoint. While we're here, let's take a look at the function.
0:000> uf kernel32!getversionexw
kernel32!GetVersionExW:
769248d8 8bff mov edi,edi
769248da 55 push ebp
769248db 8bec mov ebp,esp
769248dd 56 push esi
769248de 8b7508 mov esi,dword ptr [ebp+8]
769248e1 8b06 mov eax,dword ptr [esi]
769248e3 57 push edi
769248e4 bf1c010000 mov edi,11Ch
769248e9 3bc7 cmp eax,edi
769248eb 740b je kernel32!GetVersionExW+0x2b (769248f8)
769248ed 3d14010000 cmp eax,114h
769248f2 0f858e8d0200 jne kernel32!GetVersionExW+0x1c (7694d686)
769248f8 56 push esi
769248f9 ff15f0139076 call dword ptr [kernel32!_imp__RtlGetVersion (769013f0)]
769248ff 85c0 test eax,eax
76924901 0f85868d0200 jne kernel32!GetVersionExW+0x23 (7694d68d)
76924907 393e cmp dword ptr [esi],edi
76924909 7409 je kernel32!GetVersionExW+0x3a (76924914)
7692490b 33c0 xor eax,eax
7692490d 40 inc eax
7692490e 5f pop edi
7692490f 5e pop esi
76924910 5d pop ebp
76924911 c20400 ret 4
76924914 a046ef9c76 mov al,byte ptr [kernel32!BaseRCNumber (769cef46)]
76924919 88861b010000 mov byte ptr [esi+11Bh],al
7692491f ebea jmp kernel32!GetVersionExW+0x45 (7692490b)
7694d686 6a7a push 7Ah
7694d688 e8eeeeffff call kernel32!SetLastError (7694c57b)
7694d68d 33c0 xor eax,eax
7694d68f e97a72fdff jmp kernel32!GetVersionExW+0x25 (7692490e)
We want to esablish the calling convention. If we were using _fastcall, we'd be passing arguments in registers (ECX and EDX), but we don't even use those registers here. We're calling ret 4 to unwind our own stack, so we can't be using a cdecl function (where the caller is responsible for unwinding the stack). We're looking at a stdcall function.
(While you're looking at it - the first instruction - mov edi,edi - is an instruction that does absolutely nothing. It's a 2-byte filler. Why? To enable hot patching. With two byes, we can add a short jump, from which we can long jump to a new implementation of the function.)
Now, we could set up our stack and begin looking at the arguments passed, but in this case, we don't really care much about what was passed. What's interesting is what we return. So, let's get out of this function and back into our main function and have a look.
First, let's look at our return value. Again, looking at MSDN, we see that it returns a BOOL, returning non-zero upon success. This appears in the EAX register. Let's see how we did.
0:000> r eax
eax=00000001
OK, so the function succeeded. However, you're probably far more interested in the LPOSVERSIONINFO argument that the function modified! How do we get at that?
Well, first we need to know where it sits. Because the arguments are passed on the stack, the stack pointer tells us where to look. We look at ESP+8 ESP+4, and we'll begin to find our arguments. And, fortunately, we can use the dt command to format it nicely for us.
0:000> dt OSVERSIONINFO esp+8
DWM_Compositing_Rendering_Dem!OSVERSIONINFO
+0x000 dwOSVersionInfoSize : 6
+0x004 dwMajorVersion : 0
+0x008 dwMinorVersion : 0x1771
+0x00c dwBuildNumber : 2
+0x010 dwPlatformId : 0x650053
+0x014 szCSDVersion : [128] "rvice Pack 1"
0:000> dt OSVERSIONINFO esp+4
DWM_Compositing_Rendering_Dem!OSVERSIONINFO
+0x000 dwOSVersionInfoSize : 0x114
+0x004 dwMajorVersion : 6
+0x008 dwMinorVersion : 0
+0x00c dwBuildNumber : 0x1771
+0x010 dwPlatformId : 2
+0x014 szCSDVersion : [128] "Service Pack 1"
So, this is how the function is coming back to us. And now we can see our first argument. Fortunately for us, this is our only argument, and our job is done.
(You may be scratching your head as to why szCSDVersion seems to be missing the letters Se - I'm doing the same thing. Actually, no more head scratching. I mixed up esp+4 outside with ebp+8 inside - fixed this.)
That wasn't so hard, was it? A little help from MSDN, and we are on our way.
Let's do one more, for good measure. This application happens to display a message box, so let's fast forward to that API call. MSDN tells us that this is found in user32.dll, so let's set our breakpoint.
0:000> bp user32!messageboxw
Now that we're in the function, this time we want to stay in it. The arguments here are interesting going in. Let's take a look at this function:
0:000> uf user32!messageboxw
USER32!MessageBoxW:
76d5d667 8bff mov edi,edi
76d5d669 55 push ebp
76d5d66a 8bec mov ebp,esp
76d5d66c 833da89cd67600 cmp dword ptr [USER32!gfEMIEnable (76d69ca8)],0
76d5d673 7424 je USER32!MessageBoxW+0x32 (76d5d699)
76d5d675 64a118000000 mov eax,dword ptr fs:[00000018h]
76d5d67b 6a00 push 0
76d5d67d ff7024 push dword ptr [eax+24h]
76d5d680 6824a3d676 push offset USER32!gdwEMIThreadID (76d6a324)
76d5d685 ff150412d076 call dword ptr [USER32!_imp__InterlockedCompareExchange (76d01204)]
76d5d68b 85c0 test eax,eax
76d5d68d 750a jne USER32!MessageBoxW+0x32 (76d5d699)
76d5d68f c70520a3d67601000000 mov dword ptr [USER32!gpReturnAddr (76d6a320)],1
76d5d699 6a00 push 0
76d5d69b ff7514 push dword ptr [ebp+14h]
76d5d69e ff7510 push dword ptr [ebp+10h]
76d5d6a1 ff750c push dword ptr [ebp+0Ch]
76d5d6a4 ff7508 push dword ptr [ebp+8]
76d5d6a7 e849ffffff call USER32!MessageBoxExW (76d5d5f5)
76d5d6ac 5d pop ebp
76d5d6ad c21000 ret 10h
The first few lines set up our stack frame. We can step through the first 3 lines to set it up ourselves, or we can just use the existing stack pointer (ESP). Since it saves us some time, let's do that.
Looking at MSDN, the first argument is the HWND of the window that owns the message box. This will be a 4-byte HWND value. If we had stepped through our stack frame setup, we'd be starting at EBP+8, but since we haven't called PUSH EBP yet, we're going to be starting at ESP+4. So, our first argument is:
0:000> dc esp+4
0012fdd0 00000000
OK, so it's passing a NULL HWND. On to argument #2: a pointer to the text to display in the message box. We'll just walk 4 more bytes on the stack to grab that pointer:
0:000> ddu esp+8
0012fdd4 00402160 "This application requires Windows XP"
And what are they using as the caption in argument 3?
0:000> ddu esp+c
0012fdd8 00402134 "Unsupported Version"
And our last argument tells us what buttons it's going to support:
0:000> dc esp+10
0012fddc 00000010
Ah - this one is a bit harder to crack, because it's expressed as a UINT instead of a handy #define'd flag. Fortunately, MSDN also tells us where the API is defined (winuser.h), so we can just head there and search for MB_ - they're typically all grouped together. So, let's translate. 10 is MB_ICONHAND (which also happens to be MB_ICONERROR and MB_ICONSTOP if you continue reading the header), and, since we have to have buttons, we must have MB_OK (0x0). So, we must be calling this as MB_ICONHAND | MB_OK.
So, with a little help from the SDK header files, and a lot of help from MSDN, we were able to point our debugger at Windows APIs with public symbols and gather just as much information about arguments as the private symbols would give you.
Updated 2/23/2008: I'm so used to typing EBP+8 inside a method that I typed ESP+8 outside. I corrected this.
