• Ntdebugging Blog

    Debug Fundamentals Exercise 3: Calling conventions

    • 19 Comments

     

    Today’s exercise will focus on x86 function calling conventions.  The calling convention of a function describes the following:

     

    ·         The order in which parameters are passed

    ·         Where parameters are placed (pushed on the stack or placed in registers)

    ·         Whether the caller or the callee is responsible for unwinding the stack on return

     

    While debugging, an understanding of calling conventions is helpful when you need to determine why certain values are placed in registers or on the stack before a function call.

     

    Standard x86 calling convention on Windows:

    Name

    Arguments

    Unwinds stack

    Win32 (Stdcall)

    pushed onto stack from right to left

    callee

    Native C++ (Thiscall)

    pushed onto stack from right to left, "this" pointer in ecx

    callee

    COM (Stdcall for C++)

    pushed onto stack from right to left, then "this" is pushed

    callee

    Fastcall

    arg1 in ecx, arg2 in edx, remaining args pushed onto stack from right to left

    callee

    Cdecl

    pushed onto stack from right to left

    caller

     

     

    Question:

    Below are calls to 5 functions.  Each function takes two DWORD parameters.  Based on the code that calls each function, identify the calling convention used.

     

    // Call to Function1

    01002ffe 8b08            mov     ecx,dword ptr [eax]

    01003000 53              push    ebx

    01003001 687c2c0001      push    offset 01002c7c

    01003006 50              push    eax

    01003007 ff11            call    dword ptr [ecx]

     

    // Call to Function2

    01002490 50              push    eax

    01002491 688c110001      push    offset 0100118c

    01002496 e82a020000      call    dbgex4!Function2 (010026c5)

    0100249b 59              pop     ecx

    0100249c 59              pop     ecx

     

    // Call to Function3

    0100248e 8bd0            mov     edx,eax

    01002490 8bcf            mov     ecx,edi

    01002492 e8aeffffff      call    dbgex4!Function3 (01002445)

     

    // Call to Function4

    00413586 8b450c          mov     eax,dword ptr [ebp+0Ch]

    00413589 50              push    eax

    0041358a 8b4d08          mov     ecx,dword ptr [ebp+8]

    0041358d 51              push    ecx

    0041358e 8b4dec          mov     ecx,dword ptr [ebp-14h]

    00413591 e86fdfffff      call    dbgex4!Function4 (00411505)

     

    // Call to Function5

    01003540 56              push    esi

    01003541 8d85d4f9ffff    lea     eax,[ebp-62Ch]

    01003547 50              push    eax

    01003548 ff1558100001    call    dbgex4!Function5 (01001058)]

     

     

    Bonus: describe the calling convention used for x64.

     

     


    [Update: our answer. Posted 12/18/2008]

     

    Function1 - COM (Stdcall for C++)

     

    Function2 - cdecl

     

    Function3 - fastcall

     

    Function4 - Native C++ (Thiscall)

     

    Function5 - Win32 (Stdcall)

     

     

    Bonus: describe the calling convention used for x64:  

    http://msdn.microsoft.com/en-us/library/ms794533.aspx

     

Page 1 of 1 (1 items)