Debug Fundamentals Exercise 1: Reverse engineer a function

Debug Fundamentals Exercise 1: Reverse engineer a function

  • Comments 38

 

Hello ntdebuggers!  We’ve seen a lot of interest in our Puzzlers, and we’ve also seen requests and interest in topics covering debugging fundamentals.  So we’ve decided to combine the two topics and post a series of “Fundamentals Exercises”.  These exercises will be designed more as learning experiences rather than simply puzzlers.  We hope you find them interesting and educational!

 

Feel free to post your responses here, but we won’t put them on the site until after we post the “official” responses, so as to avoid spoilers.

 

 

Examine the following code, registers, and stack values to determine the following:

 

1.       When the function “DoTheWork” returns, what is the return value from that function?

2.       Bonus: what is the mathematical operation that “DoTheWork” performs?

 

Hints:

1.       The bracket notation [] in the assembly means to treat the value in brackets as a memory address, and access the value at that address.

2.       32-bit integer return values are stored in eax

 

 

// Code

0:000> uf eip

demo2!DoTheWork:

0040101c 55              push    ebp

0040101d 8bec            mov     ebp,esp

0040101f 8b4d08          mov     ecx,dword ptr [ebp+8]

00401022 8bc1            mov     eax,ecx

00401024 49              dec     ecx

00401025 0fafc1          imul    eax,ecx

00401028 83f902          cmp     ecx,2

0040102b 7ff7            jg      demo2!DoTheWork+0x8 (00401024)

0040102d 5d              pop     ebp

0040102e c3              ret

 

// Current register state

0:000> r

eax=00000007 ebx=7ffd9000 ecx=ffffffff edx=00000007 esi=00001771 edi=00000000

eip=0040101c esp=0012fe9c ebp=0012feac iopl=0         nv up ei pl nz na po nc

cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202

demo2!DoTheWork:

0040101c 55              push    ebp

 

// Current stack values for this thread

0:000> dps esp

0012fe9c  00406717 demo2!main+0x27

0012fea0  00000007

0012fea4  82059a87

0012fea8  00000007

0012feac  0012ff88

0012feb0  004012b2 demo2!mainCRTStartup+0x170

0012feb4  00000002

0012feb8  00980e48

0012febc  00980e80

0012fec0  00000094

0012fec4  00000006

0012fec8  00000000

0012fecc  00001771

0012fed0  00000002

0012fed4  76726553

0012fed8  20656369

0012fedc  6b636150

0012fee0  00003120

0012fee4  00000000

0012fee8  00000000

0012feec  00000000

0012fef0  00000000

0012fef4  00000000

0012fef8  00000000

0012fefc  00000000

0012ff00  00000000

0012ff04  00000000

0012ff08  00000000

0012ff0c  00000000

0012ff10  00000000

0012ff14  00000000

0012ff18  00000000

 


[Update: our answer. Posted 11/19/2008]

Wow - what a great response from our readers on this exercise!  It is great to see the various approaches to reverse engineering this code.  As for the answer, the numerical result (stored in eax) is 5040, and the corresponding mathematical operation is a factorial.  So 7! is the result calculated, given that 7 was passed to the function.  Congratulations to all of you that got it right! 

 

Many of you posted some C code to give an idea of what the original source of DoTheWork() might have looked like.  The original source was actually written in assembly!  However, it was written to be called from C, and it uses ebp in the same way that compiled C code might.  This function wasn’t written with optimal performance in mind, but rather for learning about reverse engineering.

 

 

Leave a Comment
  • Please add 5 and 4 and type the answer here:
  • Post
  • 1. 5040

    2. factorial

    Please, next time a bit more complicated. ;)

  • <i>Here's hoping that this looks like it did in plain text...</i>

    <pre>

    My Answers:

    1.      5040 (7!)

    2.      Factorial N  (N!)

    /////////////////////////////////////////////////////////////////////////////

    //

    //  Detailed explanation(s)

    //

    /////////////////////////////////////////////////////////////////////////////

    /////////////////////////////////////////////////////////////////////////////

    //  Function:   demo2!DoTheWork:    (In assembly)

    /////////////////////////////////////////////////////////////////////////////

    //

    //  Save the Prior Frame Pointer to the stack

    //

    0040101c 55              push    ebp

    //

    //  Set the Frame pointer to the current Stack pointer

    //

    0040101d 8bec            mov     ebp,esp

    //

    //  Right at this point, the stack looks like:

    //  EBP = ESP

    //  

    //  EPB - N -- Local variables, if any (here there aren't)

    //  EBP     -- Old EBP

    //  EBP + 4 -- Return Address back to calling function

    //  EBP + 8 -- First function Arg

    //

    //

    //  Put Arg1 into ECX

    //

    0040101f 8b4d08          mov     ecx,dword ptr [ebp+8]

    //

    //  Copy ECX into EAX

    //

    00401022 8bc1            mov     eax,ecx

    //

    //  LOOP:   ECX--

    //

    00401024 49              dec     ecx

    //

    //  EAX = EAX * ECX

    //

    00401025 0fafc1          imul    eax,ecx

    //

    //  If EXC is greater than 2, goto LOOP:

    //

    00401028 83f902          cmp     ecx,2

    0040102b 7ff7            jg      demo2!DoTheWork+0x8 (00401024)

    //

    //  Else it wasn't, so replace the Old Frame Pointer

    //

    0040102d 5d              pop     ebp

    //

    //  Return back to the calling function.

    //  Whatever is in EAX is effectively returned.

    //

    0040102e c3              ret

    /////////////////////////////////////////////////////////////////////////////

    //  Function:   demo2!DoTheWork: (In C)

    /////////////////////////////////////////////////////////////////////////////

    int DoTheWork(int Number)

    {

       int WorkingValue = Number;

       int Factorial    = WorkingValue;

       do {

           WorkingValue--;

           Factorial *= WorkingValue;

       }while (WorkingValue > 2);

       return(Factorial);

    }

    //

    //  Since we know what the function does, all we need to do is find the

    //  argument to it.  We can just look at the stack...

    //

    0012fe9c  00406717 demo2!main+0x27              // Return address for DoTheWork

    0012fea0  00000007                              // Arg1 "7"

    0012fea4  82059a87

    </pre>

  • Hello,

    a source code which produces almost directly the same code is as follows:

    int DoThework(int value)

    {

      int c = value;

      do {

        value = value * --c;

      } while (c > 2);

      return value;

    }

    Thus, this function multiplies value * value - 1 * value - 2 * ... * 2, and then ends.

    Thus, for positive values (value > 0), this is the faculty (value!) function.

    For value <= 0, the function calculates value * (value-1), as the loop will be executed exactly once.

  • I think the result is Factorial(2).

  • Well it seems to be calculating factorial of

    82059a87 .. Am I right :) ?

  • Very good exercise.

    The return value will be 0x2. And the function is for calculating factorial of a number.

  • I believe returned value will be 5040, and the code implements a factorial function.

    I recognized the function as factorial, but when I was about to calculate the return value I first thought that dword ptr [ebp+8] means

    0012fea4  82059a87

    Then I realized that

    0040101c 55              push    ebp

    has not yet been executed, and it will but another 4bytes on stack.

    Anyway, I'm not a low-level debugging expert, but I enjoy these a lot.

    Thanks for the exercise. Hope there will be a post with the full explanation later on.

  • 1. eax holds 5040.

    2. Factorial!

Page 3 of 3 (38 items) 123