• Ntdebugging Blog

    It’s not my fault! – A case of remote code injection gone bad

    • 5 Comments

    Today we’ll examine a case where a crash is occurring in a Microsoft process, in core Windows code, but the culprit isn’t the crashing code.  In fact, the culprit isn’t even running in the process that crashed!  But before I get ahead of myself, let’s start by examining a crash dump that shows the problem...

     

    // The crashing process is the Windows Sidebar in Vista...

    0:000> |

    .  0    id: 1b3c        create  name: sidebar.exe

     

    // The exception record shows that we have hit an access violation in RtlInitUnicodeString while reading from 00080000

    0:001> .exr -1

    ExceptionAddress: 77837e8b (ntdll!RtlInitUnicodeString+0x0000001b)

       ExceptionCode: c0000005 (Access violation)

      ExceptionFlags: 00000000

    NumberParameters: 2

       Parameter[0]: 00000000

       Parameter[1]: 00080000

    Attempt to read from address 00080000

     

    // Here’s the register context at the time of failure.

    // Looks like we are trying to find the end of a unicode string

    0:001> r

    Last set context:

    eax=00000000 ebx=00000000 ecx=ffffffff edx=0088fa88 esi=00000000 edi=00080000

    eip=77837e8b esp=0088fa60 ebp=0088fabc iopl=0         nv up ei pl zr na pe nc

    cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00010246

    ntdll!RtlInitUnicodeString+0x1b:

    77837e8b 66f2af          repne scas word ptr es:[edi]

     

    // Here’s the call stack that the debugger has tried to assemble for us...

    0:001> kb

      *** Stack trace for last set context - .thread/.cxr resets it

    ChildEBP RetAddr  Args to Child             

    0088fad0 77154911 00080000 0088fb1c 7782e4b6 ntdll!RtlInitUnicodeString+0x1b

    0088fadc 7782e4b6 00080000 77030f7b 00000000 kernel32!BaseThreadInitThunk+0xe

    0088fb1c 7782e489 7713361f 00080000 00000000 ntdll!__RtlUserThreadStart+0x23

    0088fb34 00000000 7713361f 00080000 00000000 ntdll!_RtlUserThreadStart+0x1b

     

    // The address we are trying to read is freed memory...

    0:003> du 00080000

    00080000 "????????????????????????????????"

    00080040 "????????????????????????????????"

    00080080 "????????????????????????????????"

    000800c0 "????????????????????????????????"

    00080100 "????????????????????????????????"

     

     

    So we crash because RtlInitUnicodeString attempted to deference an invalid pointer (address 00080000).  Where did RtlInitUnicodeString get this value?  Let’s unassemble the function and see...

     

    0:012> uf ntdll!RtlInitUnicodeString

    ntdll!RtlInitUnicodeString:

    77567e70 57              push    edi

    77567e71 8b7c240c        mov     edi,dword ptr [esp+0Ch]  // edi comes from here

    77567e75 8b542408        mov     edx,dword ptr [esp+8]

    77567e79 c70200000000    mov     dword ptr [edx],0

    77567e7f 897a04          mov     dword ptr [edx+4],edi

    77567e82 0bff            or      edi,edi

    77567e84 7422            je      ntdll!RtlInitUnicodeString+0x38 (77567ea8)

    77567e86 83c9ff          or      ecx,0FFFFFFFFh

    77567e89 33c0            xor     eax,eax

    77567e8b 66f2af          repne scas word ptr es:[edi]  // We crash here

    <snip>

     

     

    We can see from the above assembly that edi came from esp+c, which is the second parameter to this function (MSDN tells us that this is the SourceString parameter).  So naturally we’ll want to examine the caller of RtlInitUnicodeString to see if it is at fault.  If we are to believe the call stack that the debugger gave us, then caller of RtlInitUnicodeString is BaseThreadInitThunk...but that doesn’t make sense.  BaseThreadInitThunk normally is the function that calls the thread’s start function.  Why would anyone try to make RtlInitUnicodeString the start function for a thread?  Let’s assume that the debugger isn’t showing us the real call stack for a moment, and look at the raw stack values...

     

    0:001> dds esp

    0088fa60  77837e70 ntdll!RtlInitUnicodeString

    0088fa64  7713312c kernel32!LoadLibraryExW+0x6f

    0088fa68  0088fa88

    0088fa6c  00080000

    0088fa70  06255e62

    0088fa74  00000000

    0088fa78  00000000

    0088fa7c  00080000

    0088fa80  00000000

    0088fa84  00000000

    0088fa88  00000000

    0088fa8c  00080000

    0088fa90  00000000

    0088fa94  00000000

    0088fa98  00000000

    0088fa9c  00000000

    0088faa0  00000000

    0088faa4  0088fa70

    0088faa8  0088f678

    0088faac  0088fb0c

    0088fab0  7711e289 kernel32!_except_handler4

    0088fab4  71be953e

    0088fab8  fffffffe

    0088fabc  0088fad0

    0088fac0  77133630 kernel32!LoadLibraryW+0x11

    0088fac4  00080000

    0088fac8  00000000

    0088facc  00000000

    0088fad0  0088fadc

    0088fad4  77154911 kernel32!BaseThreadInitThunk+0xe

    0088fad8  00080000

    0088fadc  0088fb1c

    0088fae0  7782e4b6 ntdll!__RtlUserThreadStart+0x23

    0088fae4  00080000

    0088fae8  77030f7b urlmon!g_StaticLiteralTreeCode <PERF> (urlmon+0xe0f7b)

    0088faec  00000000

    0088faf0  00000000

    0088faf4  00080000

    0088faf8  c0000005

    0088fafc  771af389 kernel32!UnhandledExceptionFilter

    0088fb00  771af389 kernel32!UnhandledExceptionFilter

    0088fb04  0088fae8

    0088fb08  0088f678

    0088fb0c  ffffffff

    0088fb10  777f9834 ntdll!_except_handler4

    0088fb14  000ecb9f

    0088fb18  00000000

    0088fb1c  0088fb34

    0088fb20  7782e489 ntdll!_RtlUserThreadStart+0x1b

    0088fb24  7713361f kernel32!LoadLibraryW

    0088fb28  00080000

    0088fb2c  00000000

    0088fb30  00000000

    0088fb34  00000000

    0088fb38  00000000

    0088fb3c  7713361f kernel32!LoadLibraryW

    0088fb40  00080000

    0088fb44  00000000

    <snip - all zeros to base of stack at 00890000>

     

    We can see from the information above that there is more going on here than the “kb” output would lead us to believe.  Notice that the second non-zero value on the stack (working from the bottom up) is the address of the LoadLibraryW function.  This is not a return address, it is the start of the function.  This is a clue that the start address of this thread, specified by some other thread that called CreateThread, is actually LoadLibraryW.   This is a parameter to _RtlUserThreadStart, which is the bottommost function on this thread’s call stack.  So we can examine the raw data above, and reconstruct the call stack like so...

     

    ntdll!RtlInitUnicodeString

    kernel32!LoadLibraryExW

    kernel32!LoadLibraryW

    kernel32!BaseThreadInitThunk

    ntdll!__RtlUserThreadStart

    ntdll!_RtlUserThreadStart

     

    Does this make sense?  Let’s think it through.  In Vista, it is typical to see the bottom 3 functions at the start of a thread.  Following this is the start address of the thread, which in this case is LoadLibraryW.  It also makes sense that LoadLibraryW would call LoadLibraryExW.  That leaves us with the question of whether LoadLibraryW actually calls RtlInitUnicodeString.  Let’s find out...

     

    0:011> uf kernel32!LoadLibraryExW

    <snip>

    kernel32!LoadLibraryExW+0x60:

    76f7311d ff7508          push    dword ptr [ebp+8]

    76f73120 8d45cc          lea     eax,[ebp-34h]

    76f73123 50              push    eax

    76f73124 8b3d6c10f576    mov     edi,dword ptr [kernel32!_imp__RtlInitUnicodeString (76f5106c)]

    76f7312a ffd7            call    edi

    76f7312c a14cd50177      mov     eax,dword ptr [kernel32!BasepExeLdrEntry (7701d54c)]

    76f73131 3bc3            cmp     eax,ebx

    76f73133 0f8498dafeff    je      kernel32!LoadLibraryExW+0x78 (76f60bd1)

    <snip>

     

    We can see from the above assembly that LoadLibraryExW does indeed call RtlInitUnicodeString, so our re-assembled call stack makes sense. 

     

    Now we know how we got to RtlInitUnicodeString, but where did the bad pointer value come from?  Note that the bad value, 00080000, is actually the very first non-zero value on the stack, right before the address of LoadLibraryExW.  When a call is made to CreateThread, not only is the thread start address specified, but also an optional parameter to start function is specified.  That is what we are seeing here.  Both the thread start address and the parameter end up as parameters to _RtlUserThreadStart.

     

    Let’s sum up what we know so far.   Some other thread is calling CreateThread with LoadLibraryW as the start function address, and 00080000 as the parameter.  This leads LoadLibraryExW to call RtlInitUnicodeString with 00080000 as the SourceString parameter, and when RtlInitUnicodeString attempts to read from this address we crash because 00080000 is the address of freed memory.  Incidentally, what should 00080000 point to?  It is being used as the parameter to LoadLibrary, so it should be a pointer to a null terminated string that specifies the name of the library file to load.

     

    Why would one want to start a thread with LoadLibraryW as the start address?  The most common reason is code injection.  Typically a thread outside of the target process will use the CreateRemoteThread function to invoke LoadLibrary against a particular DLL, thus loading their code into the target process.  Assuming that this is what is going on, we’ll need to move beyond the crash dump and debug a live system that is having the problem.  Fortunately we noticed that the crashing systems we observed all had a particular third party application installed.   We set up a test system with the third party application, and we were able to sometimes reproduce the crash.  Looks like possibly a timing-related crash, since we didn’t have a 100% repro.

     

    So let’s move forward with the debug of the live system.  Note that the memory addresses in the live debug notes will differ from the above.  Also, addresses and module names that could identify the application vendor have been changed.  First things first: setting a breakpoint on ntdll!NtCreateThreadEx and running through the code showed that the LoadLibraryW thread was starting without any in-process thread starting the thread.  It also revealed that the LoadLibraryW thread started without crashing this time, and the library that it was loading was a DLL that belonged to the third party app in question....

     

    0:003> du 00b90000

    00b90000 "C:\Program Files\AppVendor\App N"

    00b90040 "ame\applib.dll"

     

    We now know that the thread that is creating the LoadLibraryW thread isn’t in the sidebar.exe process, and we know that the third party application is definitely using LoadLibraryW to remotely inject their code into sidebar.exe.  Sometimes this works, and sometimes it crashes because the DLL name buffer points to freed memory.  Next step, let’s debug the third party process to confirm our suspicions, and hopefully figure out why the memory is freed sometimes.  We’ll set breakpoints on VirtualAllocEx, WriteProcessMemory, CreateRemoteThread, and VirtualFreeEx in the third party process...

     

    //

    // Third party app calls calls VirtualAllocEx against the sidebar.exe process

    //

    0:003> kb

    ChildEBP RetAddr Args to Child

    0200f8c0 4c9864db 000002b0 00000000 00000090 kernel32!VirtualAllocEx

    WARNING: Stack unwind information not available. Following frames may be wrong.

    0200f914 4c986af9 000aada0 0200f938 000adc20 AppName!Ordinal1+0x411c0

    0200f96c 4c986d1f 00000000 0200f9a0 00000003 AppName!Ordinal1+0x417de

     

    0:003> kP

    ChildEBP RetAddr

    0200f8c0 6c9884db kernel32!VirtualAllocEx(

    void * hProcess = 0x000002b0,

    void * lpAddress = 0x00000000,

    unsigned long dwSize = 0x90,

    unsigned long flAllocationType = 0x1000,

    unsigned long flProtect = 4)

     

    // PID 2836 == sidebar.exe

    0:003> !handle 0x000002b0 f

    Handle 2b0

    Type     Process

    Attributes     0

    GrantedAccess    0x10143a:

    Synch

    CreateThread,VMOp,VMRead,VMWrite,QueryInfo

    HandleCount     8

    PointerCount     122

    Name     <none>

    Object Specific Information

    Process Id 2836

    Parent Process 2944

    Base Priority 8

     

    // 00b90000 is the base address of the allocation in this run

    0:003> gu

    eax=00b90000 ebx=0200f9ac ecx=0200f89c edx=77249a94 esi=0200f904 edi=00000090

    eip=4c9844db esp=0200f8dc ebp=0200f914 iopl=0 nv up ei pl zr na pe nc

    cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246

     

    //

    // Now the app calls WriteProcessMemory writing the following string:

    // "C:\Program Files\AppVendor\App Name\applib.dll" to the new virtual alloc

    //

    0:003> g

    Breakpoint 5 hit

    eax=00000000 ebx=0200f9ac ecx=0200f904 edx=00b90000 esi=00000090 edi=00000090

    eip=77371cc6 esp=0200f8bc ebp=0200f8d8 iopl=0 nv up ei pl nz na pe nc

    cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206

    kernel32!WriteProcessMemory:

    77371cc6 8bff mov edi,edi

     

    0:003> kP

    ChildEBP RetAddr

    0200f8b8 6c9885ba kernel32!WriteProcessMemory(

    void * hProcess = 0x000002b0,

    void * lpBaseAddress = 0x00b90000,

    void * lpBuffer = 0x000aada0,

    unsigned long nSize = 0x90,

    unsigned long * lpNumberOfBytesWritten = 0x0200f8e0)

    0200f8d8 4c986895 AppName!Ordinal1+0x4129f

     

     

    0:003> du 0x000aada0

    000aada0 "C:\Program Files\AppVendor\App Name\applib.dll"

     

     

    // Within the sidebar.exe process, we can see that after WriteProcessMemory in the app returns,

    // The address has the expected string….

    0:003> du 00b90000

    00b90000 "C:\Program Files\AppVendor\App N"

    00b90040 "ame\applib.dll"

     

    //

    // The app then calls CreateRemoteThread against the sidebar.exe process,

    // using LoadLibraryW as the start address, and the address of the virtual alloc as the parameter.

    //

     

    0:003> g

    Breakpoint 0 hit

    eax=000002b0 ebx=77370000 ecx=7739361f edx=77249a94 esi=0200f9ac edi=00000000

    eip=773b46ef esp=0200f8a0 ebp=0200f8d0 iopl=0 ov up ei pl nz na po nc

    cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000a02

    kernel32!CreateRemoteThread:

    773b46ef 6858010000 push 158h

     

    0:003> kb

    ChildEBP RetAddr Args to Child

    0200f89c 6c988714 000002b0 00000000 00000000 kernel32!CreateRemoteThread

    0200f8d0 4c9688ea 0200f8f8 00000000 6cade78c AppName!Ordinal1+0x413f9

    0200f914 4c968af9 000aada0 0200f938 000adc20 AppName!Ordinal1+0x415cf

     

    0:003> kP L1

    ChildEBP RetAddr

    0200f89c 6c988714 kernel32!CreateRemoteThread(

    void * hProcess = 0x000002b0,

    struct _SECURITY_ATTRIBUTES * lpThreadAttributes = 0x00000000,

    unsigned long dwStackSize = 0,

    <function> * lpStartAddress = 0x7739361f,

    void * lpParameter = 0x00b90000,

    unsigned long dwCreationFlags = 0,

    unsigned long * lpThreadId = 0x00000000)

     

    0:003> ln 0x7739361f

    Exact matches:

    kernel32!LoadLibraryW (wchar_t *)

     

    //

    // The application then calls VirtualFreeEx on the virtual alloc.

    //

    0:003> kb

    ChildEBP RetAddr Args to Child

    0200f8cc 6c98851c 000002b0 00b90000 00000000 kernel32!VirtualFreeEx

    WARNING: Stack unwind information not available. Following frames may be wrong.

    0200f914 4c988af9 000aada0 0200f938 000adc20 AppName!Ordinal1+0x41201

    0200f96c 4c988d1f 00000000 0200f9a0 00000003 AppName!Ordinal1+0x417de

     

    // Within the sidebar.exe process, the thread for LoadLibrary starts...

    0:003> kb

    ChildEBP RetAddr Args to Child

    0412f814 7722e489 7739361f 00b90000 00000000 ntdll!__RtlUserThreadStart

    0412f82c 00000000 7739361f 00b90000 00000000 ntdll!_RtlUserThreadStart+0x1b

     

    // But the memory has already been freed by the third party application...

    0:003> du 00b90000

    00b90000 "????????????????????????????????"

    00b90040 "????????????????????????????????"

    00b90080 "????????????????????????????????"

    00b900c0 "????????????????????????????????"

    00b90100 "????????????????????????????????"

     

    //

    // ...and sidebar.exe crashes once the address is read

    //

    (b14.4d8): Access violation - code c0000005 (first chance)

    First chance exceptions are reported before any exception handling.

    This exception may be expected and handled.

     

    eax=00000000 ebx=00000000 ecx=ffffffff edx=0412f780 esi=00000000 edi=00b90000

    eip=77237e8b esp=0412f758 ebp=0412f7b4 iopl=0 nv up ei pl zr na pe nc

    cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010246

    ntdll!RtlInitUnicodeString+0x1b:

    77237e8b 66f2af repne scas word ptr es:[edi]

     


    We’ve now come full circle, back to the original crash, at the same instruction in RtlInitUnicodeString.  So now we know that the problem is that the third party vendor frees the memory that it allocated in sidebar.exe, before it is read.  Sometimes the call to VirtualFreeEx happens after the memory is read, and in that scenario the crash doesn’t occur.  But if VirtualFreeEx completes before RtlInitUnicodeString has a chance to read it, then sidebar.exe crashes. 

     

    This information was passed on to the third party vendor, and hopefully they’ll make a change to their code to avoid this problem.  There are multiple ways that this could be addressed, but one potential fix would be for the third party app to wait on the thread handle returned from CreateRemoteThread before freeing the memory.

     

    -          Matthew Justice

     

     

    Share this post :
  • Ntdebugging Blog

    Help! My Server is Shutting Down for No Apparent Reason

    • 3 Comments

    Hello - Rob here with the GES team, and I have this nugget to pass on to you. I recently worked an issue where a Windows server rebooted intermittently for no apparent reason. The Windows System Event log did not yield any clues, other than this Event ID 6008-

     

    Log Name:      System.evt

    Source:        EventLog

    Date:          25-8-2008 19:06:58

    Event ID:      6008

    Task Category: None

    Level:         Error

    Keywords:      Classic

    User:          N/A

    Computer:      A2A000001

    Description: The previous system shutdown at 6:54:04 PM on 8/25/2008 was unexpected.

     

    There were no other symptoms or patterns to which the unexpected shutdown could be related. The shutdown could occur anytime of the day. Eventually we attached a debugger to see if we could catch anything, but this wasn’t successful.  Next we looked at the manufacturer’s mechanism used to log errors and found this piece of information -

     

    An Unrecoverable System Error has occurred (Error code 0x0000002D, 0x00000000)

     

    Note - each vendor has their own way to handle error codes. We noticed a one to one relationship with the vendor error above and the Event ID 6008 messages in the Windows System Event log.  So we engaged the hardware vendor who determined this error indicated an error on the PCI bus. They also informed us that this kind of error asserts an NMI on the bus.

     

    To narrow down which component was causing the error, we set the NMICrashDump DWORD value under the following key in the registry:

     

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl

     

    This is described in detail in the article, “927069 How to generate a complete crash dump file or a kernel crash dump file by using an NMI on a Windows-based system”

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;927069

     

    This registry value causes the machine to bugcheck with a STOP 0x80 (NMI_HARDWARE_FAILURE) when Windows detects an NMI, thus producing a dump file, or, if a debugger is attached, it breaks into the debugger

     

    After setting this registry value we hooked up the debugger again and waited... after awhile we got lucky because the debugger intercepted a STOP 0x80!

     

    At that time, I ran “!pci 0x102 ff” to get an overview of the various PCI devices and their respective states. The !pci output showed the following output (VendorID and DeviceID have been removed):

     

    PCI Configuration Space (Segment:0000 Bus:00 Device:1e Function:00)

    Common Header:

        00: VendorID       <vendor>

        02: DeviceID       <device>

        04: Command        0147 IOSpaceEn MemSpaceEn BusInitiate PERREn SERREn

        06: Status         4010 CapList SERR

        08: RevisionID     d9

        09: ProgIF         01 Subtractive

        0a: SubClass       04 PCI-PCI Bridge

        0b: BaseClass      06 Bridge Device

        0c: CacheLineSize  0000

        0d: LatencyTimer   00

        0e: HeaderType     01

        0f: BIST           00

        10: BAR0           00000000

        14: BAR1           00000000

        18: PriBusNum      00

        19: SecBusNum      01

        1a: SubBusNum      01

        1b: SecLatencyTmr  20

        1c: IOBase         20

        1d: IOLimit        30

        1e: SecStatus      6280 FB2BCapable InitiatorAbort SERR DEVSELTiming:1

        20: MemBase        f7e0

        22: MemLimit       f7f0

        24: PrefMemBase    d801

        26: PrefMemLimit   dff1

        28: PrefBaseHi     00000000

        2c: PrefLimitHi    00000000

        30: IOBaseHi       0000

        32: IOLimitHi      0000

        34: CapPtr         50

        38: ROMBAR         00000000

        3c: IntLine        ff

        3d: IntPin         00

        3e: BridgeCtrl     000b PERRREnable SERREnable VGAEnable

     

    We couldn't have gone much further without the vendor's assistance. They informed us that the Status shows us SERR, which indicates a PCI System Error has occurred in this PCI-PCI Bridge. At this point I had enough conclusive data to pass my findings to the hardware vendor for full collaboration on the problem. They continued investigating the issue.

     

    It should be noted that a hardware problem is not the only reason for an Event ID 6008. A quick search in the Microsoft Knowledge Base illustrates other things that could cause the event id to appear in the Windows System log.

    Share this post :
  • Ntdebugging Blog

    PGES-Windows NT Debugging Blog Live Chat (March 17, 2009)

    • 1 Comments

    We’d like the thank everyone who attended the Windows NT Debugging Blog Live Chat two weeks ago. Here is the transcript from the session for those not able to attend the event. Hope to see everyone again!

     

    Cheers,

    Ron

     

    Chat Topic: PGES-Windows NT Debugging Blog Live Chat  
    Date: Tuesday, March 17, 2009

    Please note: Portions of this transcript have been edited for clairty.

     

    Daniel (Moderator):
    Hello everyone-- thanks for coming to our chat today. It's time to start! Before we do, though, I'd like to have our Experts introduce themselves...

    Introductions:


    Naresh (Expert):
    Hello :)

    Ron Stock (Expert):
    Hello my name is Ron Stock I have worked for Microsoft for 10 years and I've been a member of GES for nearly 5 years.

    David win (Expert):
    OEM Support engineer and I help OEM's create images for shipping computers.

    East (Expert):
    Hi, I am Joseph East an Senior EE with Microsoft Core Platforms Global Escalation Services

    Morales (Expert):
    Hi everyone, my name is Michael Morales I am a Sr. Escalation Engineer with Microsoft for 10-years.  I work on the Windows Platform dealing primarily with performance related problems.

    [MS]Dave (Expert):
    Hi everyone!  I've been with Microsoft for 8 years and have worked in several areas of support.  I'm now an Escalation Engineer with GES.

    Bob (Expert):
    Hello My name is Bob Golding and I have been in GES for 11 years

    Mr Ninja (Expert):
    I am the Debug Ninja.  I'm a Senior Escalation Engineer, I've been with Microsoft for 11 years and I've been focused on debugging the OS for most of that time.

    Naresh (Expert):
    Hello my name is Naresh Jivanji. I am a Sr. Escalation Engineer with Global Escalation Services here at MS for 11years. My entire time with MS has been with the Windows OS.

    grahamm (Expert):
    Hi this is Graham McIntyre. I've been with Microsoft for 9 years. I've held EE roles in Exchange and Windows, and currently working in Windows supporting OEM customers.

    Scott Olson (Expert):
    Hello all I am an Senior Escalation Engineer with the Microsoft Platforms Global Escalation Team.  My specialty is with debugging kernel mode issues but I also work with debugging usermode issues.

    Daniel (Moderator):
    Please remember: You must check the "ask the experts" box before posting your questions or the Experts will not receive your question.

    jaysenb (Expert):
    Hi, my name is Jaysen, I am an OEM Esclation Engineer and I have been in GES for 3 years.

    vganga (Expert):
    Hello, I am Venkatesh Ganga, Senior Escalation Engineer, I've been with Microsoft for 9 years working in Windows Performance Speciality in GES

    Ivan Berg - OEM EE (Expert):
    Hello, my name is Ivan. I am a relatively new OEM Escalation Engineer (EE).

    chrcarr (Expert):
    Hello I am Chris. I am a new EE who started this year with the Microsoft Platforms Global Escalation Team.


    Start of chat:

    Scott Olson (Expert):
    Q:
    is there a way to navigate through work item objects?
    A: The !exqueue extension displays a list of items currently queued in the ExWorkerQueue work queues.

    JeffDa  Principal EE (Expert):
    Q:
    What is the status of the Debugging Exam (71-660) and are there any supporting classes or learning materials planned?
    A: There are several ways you can get training for windows internals.

    A: Code Machine www.codemachine.com

    Solomon Seminars www.solsem.com

    Azius www.azius.com

    Windows Internals courses are now available via the Microsoft Courseware Library.

    50155A: Win Internals for IT

    author: SELA release date: 12/15/2008

    Course Number: 50155A-EN

    50154A: Win Internals for programmers

    author: SELA release date: 12/16/2008

    Course Number: 50154A-EN

    You can find a Certified Partner with Learning Specialty at http://www.microsoft.com/learning/cpls/reasons/default.mspx

    Software Assurance Vouchers can pay for the CPLS training

    Note the Exam was release in October

    Morales (Expert):
    Q:
    Is there any way to see which is the issue recovering the Win32_product WMI query??
    A: Can you clarify your question just a little?....How are you using the Win32_product class?

    Tate [MSFT] (Expert):
    Q:
    If I debug an interrupt stall, how can I increase the chances for the debugger to interrupt the CPU given that the serial com device has a lower DIRQL than this respective device? I had success with CrashOnCtrlScroll, but I don't know if an uniprocessor wi
    A: You could use the NMI button to invoke the debugger.

    Ron Stock (Expert):
    Q:
    What about Debugging MVP category
    A: Hello Martin - That sounds like a great idea. I wasn't aware we didn't have a debug catergory but I'll look into it. Are you currently a MVP?

    Bob (Expert):
    You can modify the I/O APIC to generate NMI on a device such as the keyboard
    Bob (Expert):
    Calin Are you looking to cause a stop when the machine is hung ?
    Bob (Expert):
    Calin. The com port has a high intrrupt level.  What level is the device ?


    Morales (Expert):
    Q:
    ok I am trying to execute select * from Win32_Product to get all my products installed, I tried troubleshoot using windbg but I do not have sucess results
    A: What happens when you issue the query?...Are you not getting any results at all?..or are you getting an error?...You could also try using the Win32Reg_AddRemovePrograms (if you have SMS in your environment).

    Tate [MSFT] (Expert):
    Q:
    A stop would be good on uni, but also a live debug break. It happens sometimes that the connection is lost/stuck.
    A: NMI can invoke the debugger if connected.  Also, look into using dpc/isr tracing, especially via Windows Performance Toolkit (WPT aka xperf) to track time in ISR DPC

    Ron Stock (Expert):
    Martin - We appreciate your MVP work. Do you think there would be high demand for Debug MVP work?

    Naresh (Expert):
    Q:
    I have a problem in my wireless driver causing hung on machine. CrashOnCtrlScroll did not get invoked. Is that possible? How can I troubleshoot and get dump?
    A: Did you setup the registry key for CrashOnCtrlScroll?

    Todd (Expert):
    Q:
    I have a problem in my wireless driver causing hung on machine. CrashOnCtrlScroll did not get invoked. Is that possible? How can I troubleshoot and get dump?
    A: Depends on the cause of the hang.  The watchdog timer should eventually bugcheck the system.  You could hook a live debugger to the system and break in while the system is in the hung state.

    [MS]Dave (Expert):
    Q:
    when will the WinDE start beta program?
    A: We discussed this in the Global Engineering Conference held in late February.  Instructions for getting involved in the beta program will be included in the video & slide deck for Day 2.  There will be plenty of opporunities to get involved once it starts.  You can check back here:  http://blogs.msdn.com/ntdebugging/archive/2009/03/05/global-engineering-conference.aspx for updates.

    *Naresh (Expert):
    A:
    http://www.codeplex.com/WinPlatTools/SourceControl/changeset/view/14600#256939

    Naresh (Expert):
    A:
    The codeplex.com link has a GUI app to set the machine for a memory dump.

    Ron Stock (Expert):
    Q:
    but at least that will give you a good number of people to work with and receive feedback from real cases and real needs. for me - I will get access to you and get valuable experience
    A: Thanks for the feedback

    Morales (Expert):
    Q:
    for Win32_Product I got System.Management.ManagementStatus.Failed in the class {System.Management.PropertyDataCollection}, looks like is an enumeration issue, I tried to restore the WMI repository but does not works :(
    A: Enable logging, repeat the error and view the logs: 

    vganga (Expert):
    Q:
    what could be the best approach in the cases in which I have just the error message? ba (break on access?) search the pattern and execute break on access?
    A: Rene, Is the error message appears in the GUI or is it something logged in a file? If the error appears on the GUI then i will take a userdump with the error message showing up and then look at the call stack to see who is throwing it.

    Scott Olson (Expert):
    How can I be a debug ninja?

    Tate [MSFT] (Expert):
    Q:
    Is there a windbg scripts repository on the net?
    A: Check out Dmitry's blog http://www.dumpanalysis.org/blog/index.php/category/windbg-scripts/

    Ivan Berg - OEM EE (Expert):
    Q: Is there a windbg scripts repository on the net?
    A:http://www.codeplex.com/powerdbg

    East (Expert):
    Q:
    can somebody support me regarding ntfs and checkdisk?
    A: What type of support are you looking for? If you have a specific question, please ask us

    Ron Stock (Expert):
    Q:
    how can I be a MVP for Debugging?
    A: Hello Rene - Great question:) We're not really the team who handles the MVP work however I'll send an email to the team who works with the MVPs.

    chrcarr (Expert):
    Q:
    Is there a windbg scripts repository on the net?
    A: I am not aware of any one in particular however there is an open-source project that the community can contribute to, both extensions or scripts. http://odbgext.codeplex.com/

    Scott Olson (Expert):
    Q:
    What is the status of the Debugging Exam (71-660) and are there any supporting classes or learning materials planned?
    A: The exam is active and there are several ways you can get training for windows internals.  The Windows Internals book is a good place to start.

    grahamm (Expert):
    Q:
    here it is: http://www.microsoft.com/learning/en/us/Exams/70-660.aspx. Thanks!
    A: Thanks for the link!

    Ntfs Doctor (Expert):
    Q:
    can somebody support me regarding ntfs and checkdisk?
    A: What is your question about CHKDSK? There are many types of indexes in NTFS.  To answer your question, I would need to see the CHKDSK information.With chkdsk errors, you have to be very specific.  Can you paste one of these into the chat? Laszlo, open a private chat with me.

    Scott Olson (Expert):
    Q:
    is there a repository of good breakpoints? memory access, memcpy, registry access?
    A: Breakpoints are really subjective to what you are debugging.  Procmon is a good way to get a call stack and can help with setting breakpoints in the areas you are looking for.

    vganga (Expert):
    Q:
    how can I know the command line arguments after enable ".childdbg 1" and "sxe cpr"?
    A: Rene, I will use !PEB to get the command line arguments

    grahamm (Expert):
    Q:
    I cannot find exam 71-660 in MS Learning. "There are no learning resources that meet the specified criteria". The number 71 implies that it is still in beta?
    A: Yes, the exam number changed to 70-660 when the test was released. Try using that since the exam is released now.

    Naresh (Expert):
    Q:
    which is the best approach to debug defered actions on MSI installers? (Why installation failed) :)?
    A: I would first start with the logging to find out where this fails.

    Naresh (Expert):
    Q:
    which is the best approach to debug defered actions on MSI installers? (Why installation failed) :)?
    A:
    223300             How to enable Windows Installer logging

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;223300

    Scott Olson (Expert):
    Q:
    How do you become a debug ninja?


    Morales (Expert):
    Q:
    which is the best approach to debug deferred actions on MSI installers? (Why installation failed) :)?
    A: Rene, check this link out (Thanks to Trey Nash) - http://blogs.msdn.com/danwhite/archive/2007/02/28/how-to-debug-msi-deferred-custom-actions-when-all-else-fails.aspx

    Morales (Expert):
    Q:
    which is the best approach to debug defered actions on MSI installers? (Why installation failed) :)?
    A: Additional Link: http://msdn.microsoft.com/en-us/library/aa368264(VS.85).aspx

    Mr Ninja (Expert):
    Q:
    what is the best approach to debug services at starting?
    A: You can add an entry for your application under the Image File Execution Options registry key and then add a Debugger value to launch your service under a debugger.  You can find more information on this at http://support.microsoft.com/default.aspx?scid=kb;en-us;824344

    Ivan Berg - OEM EE (Expert):
    Q:
    What's the best way to track desktop heap leaks inside a process? (to narrow it down to a DLL or preferably the actual call.)
    A: Use UMDH (Part of the Debugging Tools for Windows package) for standard heap leaks. For COM & other types of leak, you can try LeakDiag - http://technet.microsoft.com/en-us/library/aa997647.aspx and http://go.microsoft.com/fwlink/?LinkId=41401

    East (Expert):
    Q:
    are there still job openings with PGES?
    A: No, not at this time

    [MS]Dave (Expert):
    Q:
    are you guys going to keep us excited with more debugging fundamentals exercises?
    A: Yes, there will be more coming soon

    Ivan Berg - OEM EE (Expert):
    Q:
    What's the best way to track desktop heap leaks inside a process? (to narrow it down to a DLL or preferably the actual call.)
    A: As far desktop heap specifically - try http://blogs.msdn.com/ntdebugging/archive/2007/01/04/desktop-heap-overview.aspx which also links to the Desktop Heap Monitor Tool - http://www.microsoft.com/downloads/details.aspx?familyid=5CFC9B74-97AA-4510-B4B9-B2DC98C8ED8B&displaylang=en

    chrcarr (Expert):
    Q:
    what are the "debugging fundamentals exercises"?
    A: Yes. When I get some more time I plan to do a few more. Maybe someone else will beat me to a #5 in the mean time. Stay tuned.

    chrcarr (Expert):
    Q:
    what are the "debugging fundamentals exercises"?
    A: Oops I answered the wrong question. See the blog: http://blogs.msdn.com/ntdebugging/archive/tags/Fundamentals+Exercise/default.aspx

    [MS]Dave (Expert):
    Q:
    PGES hires international candidates?
    A: Yes, we have Escalation Engineers all over the world.  Each region handles their own hiring, so if you're interested in a particular region, check out the international jobs section of the Microsoft website.  Currently we are not hiring in North America.

    Mr Ninja (Expert):
    http://www.fengyuan.com/download.html

    Scott Olson (Expert):
    Q:
    can we expect more attention on SOS in future
    A: This extension was written by support engineers and included in the debugger package.  There should be updates in later versions of the debugger.  The team that wrote this is outside of our team.

    Naresh (Expert):
    Q:
    Field Engineer=Escalation Engineer?
    A: No. Field Engineers go onsite and help the customer with data gathering. Very frequently once the Field Engineer is onsite a EE works directly with him.

    East (Expert):
    Q:
    Field Engineer=Escalation Engineer?
    A: No PFE do more traveling and do not really focus on debugging and code reviewing as EE do.

    Scott Olson (Expert):
    Q:
    can we expect more attention on SOS in future
    A: There are also external extensions available, search for sosex.dll

    Morales (Expert):
    Q:
    which is the best approach to debug defered actions on MSI installers? (Why installation failed) :)?
    A:http://msdn.microsoft.com/en-us/library/aa368264(VS.85).aspx

    Ron Stock (Expert):
    Q:
    how can I debug memory issues in Windows Mobile CE Edition without having the source code?
    A: Hello Rene - The Windows CE team is a different team so unfortunately we have limited info on debugging Windows Mobile.

    Scott Olson (Expert):
    Q:
    where can we found information about SOSEX.DLL ?
    A: live.com lists it at http://www.stevestechspot.com/SOSEXANewDebuggingExtensionForManagedCode.aspx

    Ron Stock (Expert):
    Q:
    there is a new just released version here: http://www.stevestechspot.com/SOSEXV2NowAvailable.aspx
    A: Thanks for the info.

    Tate [MSFT] (Expert):
    RE: Desktop heap check out Matj's blog again at http://blogs.msdn.com/ntdebugging/archive/2007/01/04/desktop-heap-overview.aspx and the reference to User Interface Objects http://msdn.microsoft.com/en-us/library/ms725486.aspx (See Managing User Objects for interesting breakpoints to watch).

    [MS]Dave (Expert):
    Q:
    given a window handle how can I get the information which function is calling it? or the best approach?
    A: I'd recommend using Spy++ to see what window messages are being passed.

    East (Expert):
    Q:
    given a window handle how can I get the information which function is calling it? or the best approach?
    A: You can also check out - http://blogs.msdn.com/ntdebugging/archive/2007/06/15/hung-window-no-source-no-problem-part-2.aspx

    East (Expert):
    Q:
    is here somebody who debugs active directory issues?
    A: Yes all we need is specific questions and we can narrow down the problem

    Mr Ninja (Expert):
    Q:
    how can I troubleshoot handle leak issues for WPF?
    A: You should be able to troubleshoot any handle leak using !htrace.  There's a Channel 9 video that describes this at http://channel9.msdn.com/posts/jeff_dailey/Understanding-handle-leaks-and-how-to-use-htrace-to-find-them/

    vganga (Expert):
    Q:
    Just to make sure I'm not chasing desktop heap ghosts: If a process starts, allocates desktop heap and then terminates without freeing it, does that get cleaned up afterwards or is it just consumed?
    A: LuxAeterna, the desktop heap is part of the session space, desk heap allocaitons will be there as long as the session exists if the app doesnt free it

    Morales (Expert):
    Q:
    which tools do you recommend for a success debugging out of sysinternals and windbg?
    A: Windbg and Sysinternals tools are completely different tool sets.  You can combine their usage to help find the root cause of the problem.  Process Explorer and Process Monitor are two of the more popular tools in the Sysinternals suite.  Windbg will help you debug a particular problem, however, you need to understand what you are looking at and what commands will help you debug the problem when using WinDBG. 

    Scott Olson (Expert):
    Q:
    what is IOCTL code 0x2d5190 in Win7? I can't find it in Win7 WDK.
    A: We can't find it either.  It seems to match a MASS_STORAGE ioctl though.  There are ioctl decoding tools out on the Internet.

    vganga (Expert):
    Q:
    vganga: in that case how can I monitor (similar to process audtiing) what processes are loaded in that window station/session?
    A: Procmon\Process Explorer does show the process creations but i am not sure if they show the winstation or not, they should have session id information. but from the livekd we can determine the winstaion\desktop of the process. i am not sure if there  is a tool out there that shows this information

    Tate [MSFT] (Expert):
    Q:
    can somebody tell what can we do if a service stucks in starting state?
    A: Create a couple hang dumps in a row and see what it's doing.

    Mr Ninja (Expert):
    Q:
    best approach to debug services at starting?
    A: You can add an entry for your application under the Image File Execution Options registry key and then add a Debugger value to launch your service under a debugger.  You can find more information on this at http://support.microsoft.com/default.aspx?scid=kb;en-us;824344

    Daniel (Moderator):
    Well we're out of time for today's chat. Thank you very much to all of our guests who joined us today as well as to our Experts for answering so many great questions-- have a great day everyone!

    Ron Stock (Expert):
    Q:
    thank you!
    A: Your welcome:)

    Tate [MSFT] (Expert):
    Q:
    Thanks Experts
    A: HAPPY DEBUGGING!

    Tate [MSFT] (Expert):
    Q:
    congrats on the posts from ntdebugging - really nice
    A: Thank you!

    MSDN link -   http://msdn.microsoft.com/en-us/chats/dd578579.aspx

    Share this post :
  • Ntdebugging Blog

    Part 1: Tate Calhoun presents Xperf at the 2009 Microsoft GEC

    • 5 Comments

     

    image_thumb

    Xperf was created by the Windows Fundamentals team to analyze system and application performance. Tate walks through the use of the new tool and shows how to analyze the data for specific scenarios. The slide deck is available from the attachment link below.

    Share this post :

  • Ntdebugging Blog

    Debug Fundamentals #4 : Spoiler

    • 2 Comments

    We’re posting the answers to Debug Fundamentals #4 in this blog. Additionally we posted all of your answers that trickled into the original blog. We deferred the posting of your answers to prevent spoiling it for the others but you should find them up there now. Stay tuned for Debug Fundamentals #5!

     

     

    Part One – Debugging

     

    1.       What caused the access violation?

    Code tried to execute with an invalid EIP address.

    0:000> !address @eip

     ProcessParameters 00280f20 in range 00280000 00284000

     Environment 00280808 in range 00280000 00284000

        00ac5000 : 00ac5000 - 7513b000

                        Type     00000000

                        Protect  00000001 PAGE_NOACCESS

                        State    00010000 MEM_FREE

                        Usage    RegionUsageFree

     

    2.       Examine the registers at the time of the crash.  Is there anything interesting about the contents?

    There are ASCII values in 3 of the registers.

    0:000> .formats @eax

      Chars:   SEGP  (little endian would be PGES)

    0:000> .formats @eip

    Evaluate expression:

      Chars:   GAME

    0:000> .formats @ebp

      Chars:   OVER

     

    3.       How did the registers get into this state?

    The saved EIP and EBP values on the stack were corrupt and the boom() function executed a leave and ret instruction, loading the corrupt values into the registers.

    4.       Find the offending data structure which caused this state to occur. What are its contents?

    “GAMEOVER Happy debugging courtesy of Platforms GES!” backwards

    0:000> dc @esp-38 l34/4

    001bfdc8  45532180 6d732047 74666f72 20506c61  .!SEG smroftalP

    001bfdd8  79206f66 72746573 20636f75 67696e67  fo ysetruoc gnig

    001bfde8  65627567 70792064 20486170 4f564552  gubed yppaH REVO

    001bfdf8  47414d45                             EMAG


     

    5.       Are there security concerns with this access violation? Why?

    A malicious user could potentially manipulate EIP to execute arbitrary code and compromise the system.

    6.        Why is this class of crashes not seen in the wild much anymore?

    This exercise was explicitly compiled with the /GS- flag.  The Microsoft compiler appropriately enables buffer overrun security checks by default using the /GS compiler switch. This is a randomized “cookie” value placed on the stack during the function prolog right before the saved EIP and EBP. During the function epilog this value is asserted before executing the ret instruction. If it is not correct the program is terminated.

    More info : http://msdn.microsoft.com/en-us/library/8dbf701c.aspx

    Part Two – Reverse Engineering

     

    Examine the functions main(), snap(), crack(), pop(), and boom().

    1.       Describe with a sentence or two what each one is doing.  (There is no need to comment on every assembly instruction or re-write the code in C here unless you really feel you need to.)

    main() – Allocates a buffer on the stack and makes a call to snap() passing it the buffer.

    //

    // Allocate a buffer on the stack

    //

    int

    main()

    {

          char buf[sizeof(string) + 13];           

          return snap ((char *)buf);

    }

    snap() – Does a strcpy() like operation from a global buffer into the buffer parameter and passes it to crack().

    //

    // Copy static string to stack

    //

    int

    snap(char * buf)

    {

          for (int i = 0; string[i]; i++)

          {

                buf[i] = string[i];

          }

         

          return crack (buf);

    }

     

    crack() – Does a string reverse operation on the buffer parameter then counts the number of characters before finding a special delimiting character (0xC3) and passes the buffer and character count to pop().

    //

    // Reverse the string

    //

    int

    crack(char * buf)

    {

          char * s = buf;

          char * p = s;

         

          while (*p) p++;

          p--;

     

          while (p > s)

          {

                char t = *s;

                *s++ = *p;

                *p-- = t;

          }

     

          // strlen of the secret

          char * len = buf;

          unsigned i = 0;

               

          while (*len != '\xc3')

          {

                len++;

                i++;

          }

         

          i++;

               

          return pop (buf, i);

    }


     

    pop() – Does a memmove() like operation removing the character count parameter from the start of the string buffer and passes it to boom().

    //

    // Do a memory move on the string, hiding the secret

    //

    int pop(char * buf, unsigned size)

    {

          volatile char * a = buf;

          char * b = buf + size;

         

          while (*b)

                *a++ = *b++;

         

          *a = '\0';

         

          return boom (buf);     

    }

     

    boom() – Allocates a stack buffer and does a strcpy() operation from the parameter into it. Takes each char of the new stack buffer and swaps the nibbles of each character so that 0y1101 1001 is 0y1001 1101. Then returns ‘SEGP’; or PGES as a DWORD.

    //

    // Copy string to our local buf, decrypt and boom!!!

    //

    int

    boom(char * buf)

    {

          char new_buf[sizeof(string) - 0x14];

          register unsigned int i;     

         

          for (i = 0; buf[i]; i++)

          {

                new_buf[i+1] = buf[i];

          }

         

                while (*buf)

                *buf ++ = *buf << 4 | *buf >> 4;

     

          return (int)'SEGP';

    }


    Bonus:

     

    1.       If this access violation occurred while the program was running without a debugger present, would there be anything different about the crashing register state or the data structure which caused it? If so what is it and why?

    When a debugger is present, an unhandled exception is immediately trapped and the pristine state of the process can be examined. When a debugger is not present, an unhandled exception handler is executed in the context of the original exception before the JIT debug handling takes effect and a debugger is attached. Since the offending corruption was in stack memory which has been popped off the stack before the access violation occurred, the unhandled exception handler reused and overwrote the offending stack memory.

    More Info on debugging and unhandled exception filters:

    Debugging a custom unhandled exception filter  

    Don’t perform complicated tasks in your unhandled exception filter

     

    2.       The functions from “Part Two – Reverse Engineering” perform operations on a set of data. During this manipulation some data is lost. Find this data and decode its “secret” message.

                    “<DBGNINJA>”

     

    Share this post :
  • Ntdebugging Blog

    Part 2: Tate Calhoun presents Xperf at the 2009 Microsoft GEC

    • 4 Comments

    image_thumb

    Xperf was created by the Windows Fundamentals team to analyze system and application performance. Tate walks through the use of the new tool and shows how to analyze the data for specific scenario's. The slide deck is available from the attachment link below.

    Share this post :

  • Ntdebugging Blog

    ‘Microsoft Platforms Global Escalation Services Community’ Presentation at the 2009 Microsoft GEC

    • 1 Comments

    Principal Escalation Engineer, Jeff Dailey, presents  "Microsoft Platforms Global Escalation Services Community" at the 2009 GEC. Please enjoy the video and click on the link below to download the slide deck.

    Share this post :

  • Ntdebugging Blog

    Global Engineering Conference

    • 3 Comments

    The Global Escalation Services team at Microsoft hosted the 1st annual Global Engineering Conference on February 25th & 26th. This year’s theme of “Building the Engineering Community” was centered around our efforts to increase knowledge sharing, collaboration, and best practices throughout the engineering community.  We hope you enjoy these sessions and look forward to your attendance next year.

     

    The conference was held virtually via Live Meeting with Q&A sessions following every presentation.

    Each recorded presentation will be accompanied with a link to the corresponding slide deck.

     

    Day 1:

     

    T. Roy founder of www.codemachine.com

    T. Roy presented on instrumentation techniques for developers and engineers who instrument binaries to help resolve complex operating system problems.

    http://blogs.msdn.com/ntdebugging/archive/2009/03/31/t-roy-from-codemachine-presents-supporting-support-at-the-2009-microsoft-gec.aspx

     

    Mark Russinovich

    Mark discussed new features and tools in the Sysinternals Tools suite.

    No video

     

    Best Practices for engaging Microsoft Support.

    Members from the Escalation Services team share best practices when engaging Microsoft support, included in this video is a demo of the new MPSReports tool and a demo of using Hyper-V to package up problems that can be reproduced and send to Microsoft.

    No video

     

    Day 2:

     

    Jeff Dailey - Principal Escalation Engineer

    Jeff discusses the different locations where you can find great content from the Escalation Services team to help solve common and difficult problems.

    http://blogs.msdn.com/ntdebugging/archive/2009/03/24/microsoft-platforms-global-escalation-services-community-presentation-at-the-2009-microsoft-gec.aspx

     

    Tate Calhoun - Escalation Engineer

    Tate walks through the use of the new XPERF tool and how to analyze the data for specific scenario's.

    http://blogs.msdn.com/ntdebugging/archive/2009/03/31/part-1-tate-calhoun-presents-xperf-at-the-2009-microsoft-gec.aspx 

    http://blogs.msdn.com/ntdebugging/archive/2009/03/31/part-2-tate-calhoun-presents-xperf-at-the-2009-microsoft-gec.aspx

     

    Citrix - Escalation Engineers

    Citrix engineers Nicholas Vasile, Dmitry Vostokov, and Kapil Ramlal, share tools they have created to take advantage of the ETW tracing infrastructure in Windows, and debugging scripts, and best practices for engaging issues that require multi-vendor support.

    http://blogs.msdn.com/ntdebugging/archive/2009/08/26/video-citrix-engineers-nicholas-vasile-dmitry-vostokov-and-kapil-ramlal-at-the-gec-conferences.aspx

     

    Dennis Smeltzer - Executive Director TSANet

    Dennis discusses a proposed new TSANet offering which would allow Escalation engineers to contact each other to collaborate on issues that require multi-vendor support.

    http://blogs.msdn.com/ntdebugging/archive/2009/03/24/tsanet-presentation-at-the-2009-microsoft-gec.aspx

     

    Share this post :
  • Ntdebugging Blog

    T.Roy from CodeMachine presents 'Supporting Support' at the 2009 Microsoft GEC

    • 2 Comments

    image

    Developer and Technical support folks have to deal with crashes and hangs day in & day out. In many cases ONE crash dump is all they have to 'root cause' a problem, and often critical pieces of information required to nail down a problem are missing from that one crash dump. In this talk T.Roy from CodeMachine (http://www.codemachine.com/) discusses simple programming techniques to improve diagnosing problems in your code which will help support folks get more out of the crash dumps. This should enable them to determine root cause of an issue from a single crash dump, and should prevent having to ask the customer to reproduce the problem again to get another crash dump. The PDF slides for this presentation are available in the link below.

    Share this post :
  • Ntdebugging Blog

    TSANet Presentation at the 2009 Microsoft GEC

    • 2 Comments

     

    image 

    This year’s Global Escalation Conference theme of “Building the Engineering Community” was centered around our efforts to increase knowledge sharing, collaboration, and best practices throughout the engineering community. In this presentation Dennis Smeltzer, Executive Director of TSANet, discusses a proposed new TSANet offering which would allow Escalation engineers to contact each other to collaborate on issues that require multi-vendor support. Additionally Dennis covers the TSANet Community Model, Mission Critical Customer, Globalization, and several other topics. Please enjoy the video and click on the link below to download the slide deck.

     

    Share this post :
Page 1 of 2 (13 items) 12