Bryan Griffin's Blog

Did you know that you p/Invoke from PowerShell?

2009-02-19T21:48:00Z

Check out this walkthough by Lee Holmes, fellow Softie and author of Windows PowerShell Cookbook: for Windows, Exchange 2007, and MOM V3.

Listing firewall exceptions with PowerShell - yes we can

2009-02-19T06:02:00Z

On an internal PowerShell scripting DL the question was posed as to how one would get a list of applications with firewall exceptions. Here was the answer that I sent:

function getfwexceptions

    $fwmgr = (new-object -com hnetcfg.fwmgr).localpolicy.currentprofile

    $fwmgr.authorizedapplications

This function instantiates an instance of the HNetCfg.FwMgr COM object and gives you back something like this:

Name                 : Internet Explorer

ProcessImageFileName : C:\program files\internet explorer\iexplore.exe

IpVersion            : 2

Scope                : 0

RemoteAddresses      : *

Enabled              : True

Of course, with PowerShell you're passing around objects and can filter this output to your liking.

Happy scripting!

Netgear passphrase recovery

2009-02-18T10:14:00Z

The internal wireless on one of my laptops (an HP Pavillion dv9005us which appear to have a recall) decided to stop working sometime back. I had already sent this thing back to have the system board replaced once before because the LAN connection lost its MAC address. For the longest time I used a cabled connection but decided to buy a wireless card for it last weekend. I got home and was configuring it when I realized that I had forgotten the passphrase for my wireless network.

What was I to do?! Reset the router? I don't think so; I'm way too lazy for that. This was a Netgear router which has this button which lets you save the current configuration file. Is the password in there? You bet; it's there somewhere but I was guessing that it was encrypted somehow. Not! I opened the file with Notepad, looked around and ended up finding it in clear text.

Protect Ya Neck: Securing web applications with threat modeling

2008-12-17T20:22:00Z

Currently I work on building large scale web services that face the the public as part of what Microsoft calls software plus services (Software+Services). In this area of Internet facing applications all aspects of security are important, including intelligent attack mitigation at the software level. The identification of possible threats and vulnerabilities are found through an iterative process called threat modeling during the design phase of the development lifecycle. Here are some links that can help you build more secure web applications through the process of threat modeling.

Threat Modeling Web Applications - MSDN Patterns & Practices

Threat Modeling at the MSDN Security Development Center

Microsoft Application Threat Modeling Blog

Microsoft Application Consulting & Engineering Team Blog

Threat Modeling Articles at The Security Development Lifcycle Blog (MSDN)

Threat Modeling from Microsoft Press

Writing Secure Code, Second Edition from Microsoft Press

Sysinternals tools from the web? You betcha

2008-12-16T04:13:00Z

Did you know that you can access and even run Sysinternals tools right from the Internet?

http://live.sysinternals.com/

I tend to use them like this...

C:\utils>pushd \\live.sysinternals.com

Z:\>strings /accepteula c:\utils\stringtest.txt

Strings v2.4
Copyright (C) 1999-2007 Mark Russinovich
Sysinternals - www.sysinternals.com

Test String

Z:\>popd

C:\utils>

An updated system is a happy system (Stop 0x0000000A or IRQL_NOT_LESS_OR_EQUAL)

2008-12-12T06:44:00Z

Earlier this week one of my co-workers asked me to look at a virtual machine he had which was crashing frequently; and by frequently I mean around 40 times since 11/25. I already knew that the stop code was 0xA (IRQL_NOT_LESS_OR_EQUAL) and found out that it also had stopped with 0xD1 (DRIVER_IRQL_NOT_LESS_OR_EQUAL) a few times. The first dump that I looked at was corrupt but this box had plenty more crashes in it, but until then I decided to reinstall the VM Additions package and see if that was an easy fix. I checked back the next day and found that it had bugchecked 3 more times since. I took a quick look at the latest dump (pool corruption), here is some of the output.

kd> kv

ChildEBP RetAddr  Args to Child

87d97bac 81cd5b2a badb0d00 00000001 00000001 nt!_KiTrap0E+0x2ac (FPO: [0,0] TrapFrame @ 87d97bac) (CONV: cdecl)

87d97c68 81ce0117 000009c0 00000001 000039c0 nt!ExpAllocateBigPool+0x201 (CONV: stdcall)

87d97cc8 81db9017 00000001 000039b8 44506343 nt!ExAllocatePoolWithTag+0x112 (CONV: stdcall)

87d97d10 81db8ea0 87d97d2c 84d9deb0 81cf563c nt!PfSnBuildDumpFromTrace+0x5a (CONV: stdcall)

87d97d38 81db9332 84d9deb0 87d97d7c 81c6b602 nt!PfSnEndTrace+0xa2 (CONV: stdcall)

87d97d44 81c6b602 84d9deb0 00000000 847ed7d8 nt!PfSnEndTraceWorkerThreadRoutine+0xe (CONV: stdcall)

87d97d7c 81dafc1d 84d9deb0 87d9c680 00000000 nt!ExpWorkerThread+0xfd (CONV: stdcall)

87d97dc0 81c9a19e 81c6b505 80000001 00000000 nt!PspSystemThreadStartup+0x9d (CONV: stdcall)

00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

kd> .trap 87d97bac

ErrCode = 00000002

eax=00000003 ebx=000039c0 ecx=00000600 edx=00000001 esi=00000003 edi=9dd859c0

eip=81cd5b2a esp=87d97c20 ebp=87d97c68 iopl=0         nv up di pl nz ac pe nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010016

nt!ExpAllocateBigPool+0x201:

0008:81cd5b2a 895ffc          mov     dword ptr [edi-4],ebx ds:0023:9dd859bc=????????

kd> dd edi-4 l1

9dd859bc  ????????

kd> !pool 9dd859bc

Pool page 9dd859bc region is Paged pool

*9dd82000 : large page allocation, Tag is CcPD, size is 0x39c0 bytes

        Pooltag CcPD : Prefetcher trace dump, Binary : nt!ccpf

One of the things that was of interest is the pool tag as you can see in the output of !pool. Hmmm, Prefetcher. Since pool corruption is hard to catch without help so I used Driver Verifier to analyze some suspicious drivers including fileinfo.sys and fltmgr.sys. When I rebooted the machine it only took about 2 minutes for it to bugcheck and here is stack trace from the dump with driver verification turned on.

kd> kv

ChildEBP RetAddr  Args to Child

85617b58 81cd36d4 000000c4 000000c0 89e70e28 nt!KeBugCheckEx+0x1e (CONV: stdcall)

85617b70 81ec9e0d 92747008 85617b9c 81ec262b nt!VfBugCheckNoStackUsage+0x23 (FPO: [0,0,0]) (CONV: stdcall)

85617b7c 81ec262b 84aa5020 84ab0438 84aa5020 nt!VfBeforeCallDriver+0x88 (CONV: stdcall)

85617b9c 81c679d7 853cab29 89e70e28 84aa5020 nt!IovCallDriver+0x1fd (CONV: fastcall)

85617bb0 853cab29 92758b04 927e4960 853c1e00 nt!IofCallDriver+0x1b (CONV: fastcall)

85617be4 853ce827 84aa5020 84ab0438 92758b04 fltmgr!FltpQueryInformationFile+0xc3 (FPO: [Non-Fpo]) (CONV: stdcall)

85617c18 853cf910 84aa5020 927e4960 85617c44 fltmgr!FltpGetFileName+0x63 (FPO: [Non-Fpo]) (CONV: stdcall)

85617c28 853ccdb4 927e4960 00000000 927e4960 fltmgr!FltpGetOpenedFileName+0x18 (FPO: [Non-Fpo]) (CONV: stdcall)

85617c44 853cf990 927e4960 00000000 927e4960 fltmgr!FltpCallOpenedFileNameHandler+0x8a (FPO: [Non-Fpo]) (CONV: stdcall)

85617c60 853d0129 927e4960 00000000 92758b04 fltmgr!FltpGetNormalizedFileNameWorker+0x16 (FPO: [Non-Fpo]) (CONV: stdcall)

85617c78 853cd51f 927e4960 00000000 927e4960 fltmgr!FltpGetNormalizedFileName+0x19 (FPO: [Non-Fpo]) (CONV: stdcall)

85617c90 853b8f13 927e4960 84ab0438 00000000 fltmgr!FltpCreateFileNameInformation+0x81 (FPO: [Non-Fpo]) (CONV: stdcall)

85617cc0 853cda61 84f84d54 00000001 00000000 fltmgr!FltpGetFileNameInformation+0x31b (FPO: [Non-Fpo]) (CONV: stdcall)

85617cd4 80409378 84ab0438 84ab6890 927e4960 fltmgr!FltGetFileNameInformationUnsafe+0x59 (FPO: [Non-Fpo]) (CONV: stdcall)

85617d14 853d3b70 83823c40 84ab6890 00000228 fileinfo!FIStreamQueryWorker+0x76 (FPO: [Non-Fpo]) (CONV: stdcall)

85617d44 81c6b602 00000000 00000000 832c92d8 fltmgr!FltpProcessGenericWorkItem+0x32 (FPO: [Non-Fpo]) (CONV: stdcall)

85617d7c 81dafc1d 83823c40 8561c680 00000000 nt!ExpWorkerThread+0xfd (CONV: stdcall)

85617dc0 81c9a19e 81c6b505 00000001 00000000 nt!PspSystemThreadStartup+0x9d (CONV: stdcall)

00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

Alright, now we have a more interesting call stack and the debugger says that fileinfo!FIStreamQueryWorker is the culprit. FileInfo.sys is associated with Superfetch/Prefetch which corroborates the evidence in the very first dump I looked at where you can see the pool allocated with the CcPD tag. Let's check the dates on fltmgr and fileinfo to see if they're old.

kd> lmvm fileinfo

start    end        module name

80402000 80412000   fileinfo   (private pdb symbols)  c:\symbols\fileinfo.pdb\B768A0AE91484C23A64BA91BEF9C57AD1\fileinfo.pdb

    Loaded symbol image file: fileinfo.sys

    Image path: \SystemRoot\system32\drivers\fileinfo.sys

    Image name: fileinfo.sys

    Timestamp:        Thu Nov 02 01:36:47 2006 (4549AE1F)

    CheckSum:         0001C10F

    ImageSize:        00010000

    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4

kd> lmvm fltmgr

start    end        module name

853b1000 853e2000   fltmgr     (private pdb symbols)  c:\symbols\fltMgr.pdb\FA42D1F108C1425599238F576CBDDE831\fltMgr.pdb

    Loaded symbol image file: fltmgr.sys

    Image path: \SystemRoot\system32\drivers\fltmgr.sys

    Image name: fltmgr.sys

    Timestamp:        Thu Nov 02 01:30:58 2006 (4549ACC2)

    CheckSum:         00035410

    ImageSize:        00031000

    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4

After checking another x86 Vista box that was hanging around these file dates should be 1/20/2008 instead of 11/2/2006. Doh, this machine didn't have SP1! I should have checked all of this at first but I assumed that the machine was up-to-date. Apparently there were some funny apps on it that we didn't want to break so SP1 had not been installed. I stopped and disabled the Superfetch service and the machine has been happy ever since.

SERVICE_NAME: sysmain

        TYPE               : 20  WIN32_SHARE_PROCESS

        START_TYPE         : 4   DISABLED

        ERROR_CONTROL      : 0   IGNORE

        BINARY_PATH_NAME   : C:\Windows\system32\svchost.exe -k LocalSystemNetworkRestricted

        LOAD_ORDER_GROUP   :

        TAG                : 0

        DISPLAY_NAME       : Superfetch

        DEPENDENCIES       : rpcss

                           : fileinfo

        SERVICE_START_NAME : LocalSystem

If you notice under dependencies you will see the system driver fileinfo.sys.

The key takeaway here is to make sure that you have all appropriate updates installed on your system or you could run into some strange behavior.

Farewell OneCare

2008-11-20T04:30:00Z

Well, we've come to the end of the road with the OneCare product. It was a great product packed with neat features that made PC use carefree. Frankly, I'm sad to see it go but it's nice to see that Microsoft is going to offer a free solution. At any rate, I'll be off working on a few new technologies which are still part of the identity and security space.

http://www.microsoft.com/presspass/press/2008/nov08/11-18NoCostSecurityPR.mspx

Dang ol' runtime errors!

2008-11-04T18:57:00Z

A couple of months ago a friend of mine asked me to take a look at an IT job posting that they were going to go for. Being the good buddy that I am I followed the instructions to find the description but was confronted with a runtime error on each page. I had been working on something earlier that week and had script debugging turned on in IE. I guess it wasn't a big deal but I was drawn like a moth to a flame since this was happening on every single page on the site (at least all the ones I checked). It didn't help that I was asked if I wanted to debug the script in each case, what do you think; of course I do.

First get the stack trace and read from the bottom of the stack to the top. When looking at the stack you see that there are four calls in jscript, the last being OnScriptError. Looking at those previous three calls you see that each function is packing around the memory address 069ac110.

0:016> kb

ChildEBP RetAddr  Args to Child

0c1beefc 76980dde 7696b0b2 0005024a 00000001 ntdll!KiFastSystemCallRet

0c1bef00 7696b0b2 0005024a 00000001 00000000 USER32!NtUserWaitMessage+0xc

0c1bef34 7696bcda 000802dc 0005024a 00000001 USER32!DialogBox2+0x202

0c1bef5c 769bccdc 76960000 05d6b4b8 000105be USER32!InternalDialogBox+0xd0

0c1beffc 769bd25e 00000014 05daf898 00000000 USER32!SoftModalMessageBox+0x69f

0c1bf14c 769bd394 0c1bf158 00000028 000105be USER32!MessageBoxWorker+0x2c7

0c1bf1a4 769bd610 000105be 0c1bf280 00000000 USER32!MessageBoxTimeoutW+0x7f

0c1bf1c4 6d775985 000105be 0c1bf280 00000000 USER32!MessageBoxExW+0x1b

0c1bf1e4 769bd6ac 000105be 0c1bf280 00000000 IEFRAME!Detour_MessageBoxExW+0x2c

0c1bf200 6ba8ee6e 000105be 0c1bf280 00000000 USER32!MessageBoxW+0x45

0c1bf884 6ba0900e 0bbbcd08 0c1bf8f4 061889b8 mshtml!CScriptHolder::DoYouWishToDebug+0x20e

0c1bf8e8 6ee85320 05d13078 00000000 0c1bf954 mshtml!CScriptHolder::OnScriptError+0xa5

0c1bf8f8 6ee97f32 0bbbcd08 ffffffff 05f7b5a4 jscript!COleScript::OnScriptError+0x31

0c1bf954 6ee7fdd8 069ac110 0bbbcd08 86664004 jscript!COleScript::CreateScriptBody+0xe1

0c1bf9cc 6ee7fc46 069ac110 05f7b5a4 00000000 jscript!COleScript::ParseScriptTextCore+0x211

0c1bf9f8 6b8aca57 061889bc 069ac110 05f7b5a4 jscript!COleScript::ParseScriptText+0x2b

0c1bfa58 6b8ac888 05c7e970 05c1fb18 00000000 mshtml!CScriptCollection::ParseScriptText+0x240

0c1bfb14 6b99ece3 00000000 00000000 0688a660 mshtml!CScriptElement::CommitCode+0x263

0c1bfb3c 6b99ed2f 00000000 05c1fb18 0688a9a0 mshtml!CMarkup::CommitQueuedScripts+0x53

0c1bfb54 6b8cade8 00000000 05c1fb18 00000000 mshtml!CMarkup::CommitQueuedScriptsInline+0x3c

Dump whatever is at the suspect memory address and we find the error.

0:016> dU 069ac110

069ac110  "Synapse Website Usher Error: Dom"

069ac150  "ain not found...."

Since the runtime error was happening on each page I looked at the source of some of the pages and found that there were a few lines of HTML which inserted a JavaScript file used for analytics.

More than likely the average user would not have seen the errors because most folks have script debugging turned off, but I sent an email to the webmaster because it's always good to know about potential bugs.

By the way, this has apparently been fixed.

Snapfish and unresponsive browser - a quick analysis

2008-11-02T02:00:00Z

Last week I was trying to get prints of some photos I had from a good friend's wedding. I had success with Internet photo printing before at a different store so I decided to use the service offered by a nearby store for quick pickup. I go to the website and begin to upload the pictures and while navigating around my browser (IE7) became unresponsive and gave me the faded white screen for a while (at least a minute). NOOOO, you were working so well until just now! What was causing this behavior?

In cases where you have processor time spikes or hanging apps you can use the !runaway command to see what threads are taking the most time. In the debugger output below you can see that thread 3 is taking a considerable chunk of the CPU time compared to the other threads.

0:020> !runaway
 User Mode Time
  Thread       Time
   3:25c       0 days 0:00:21.044
   0:158c      0 days 0:00:02.012
   5:17fc      0 days 0:00:01.450
   6:e60       0 days 0:00:01.216
   4:10f0      0 days 0:00:00.046
  14:1470      0 days 0:00:00.015
  13:4b8       0 days 0:00:00.015
   8:14c8      0 days 0:00:00.015
   7:e74       0 days 0:00:00.015
   1:1630      0 days 0:00:00.015
  20:5bc       0 days 0:00:00.000
  19:17f0      0 days 0:00:00.000
  18:10c8      0 days 0:00:00.000
  17:c04       0 days 0:00:00.000
  16:12bc      0 days 0:00:00.000
  15:1674      0 days 0:00:00.000
  12:1248      0 days 0:00:00.000
  11:5f0       0 days 0:00:00.000
  10:ecc       0 days 0:00:00.000
   9:16b8      0 days 0:00:00.000
   2:14e4      0 days 0:00:00.000

We switch to thread 3 and get a stack backtrace with extended frame count so that the whole stack is present (without this you will not get the last 2 frames). Looking the call stack you can see that some component (1BA7) had made a HTTP request and is now waiting on a response. Ok, now this makes sense because it seems that the HTTP response is taking a longer than desired.

0:020> ~3s

eax=05f9a000 ebx=75f7c54c ecx=05f99fe8 edx=00001000 esi=0433c154 edi=00000000

eip=77579a94 esp=0433c10c ebp=0433c17c iopl=0         nv up ei ng nz ac pe cy

cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000297

ntdll!KiFastSystemCallRet:

77579a94 c3              ret

0:003> kn250

 # ChildEBP RetAddr

00 0433c108 77579254 ntdll!KiFastSystemCallRet

01 0433c10c 75f7c244 ntdll!ZwWaitForSingleObject+0xc

02 0433c17c 75f7c1b2 kernel32!WaitForSingleObjectEx+0xbe

03 0433c190 76fc165d kernel32!WaitForSingleObject+0x12

04 0433c1d0 76f88b0e wininet!CServerInfo::GetConnection_Fsm+0x34f

05 0433c1e8 76f83f49 wininet!CFsm_GetConnection::RunSM+0x61

06 0433c200 76f86441 wininet!CFsm::Run+0x39

07 0433c218 76f8873b wininet!DoFsm+0x25

08 0433c250 76f885d3 wininet!HTTP_REQUEST_HANDLE_OBJECT::OpenConnection_Fsm+0x1bb

09 0433c25c 76f83f49 wininet!CFsm_OpenConnection::RunSM+0x33

0a 0433c274 76f86441 wininet!CFsm::Run+0x39

0b 0433c28c 76f88569 wininet!DoFsm+0x25

0c 0433c29c 76f88500 wininet!HTTP_REQUEST_HANDLE_OBJECT::OpenConnection+0x2f

0d 0433c2c0 76f8847d wininet!HTTP_REQUEST_HANDLE_OBJECT::MakeConnection_Fsm+0x81

0e 0433c2cc 76f83f49 wininet!CFsm_MakeConnection::RunSM+0x37

0f 0433c2e4 76f86441 wininet!CFsm::Run+0x39

10 0433c2fc 76f89125 wininet!DoFsm+0x25

11 0433c33c 76f89082 wininet!HTTP_REQUEST_HANDLE_OBJECT::SendRequest_Fsm+0xc2

12 0433c348 76f83f49 wininet!CFsm_SendRequest::RunSM+0x37

13 0433c360 76f86441 wininet!CFsm::Run+0x39

14 0433c378 76f86940 wininet!DoFsm+0x25

15 0433c394 76f8401d wininet!HTTP_REQUEST_HANDLE_OBJECT::HttpSendRequest_Start+0x474

16 0433c3a8 76f83f49 wininet!CFsm_HttpSendRequest::RunSM+0xa0

17 0433c3c0 76f86441 wininet!CFsm::Run+0x39

18 0433c3d8 76faf7af wininet!DoFsm+0x25

19 0433c400 76f9d34e wininet!HttpWrapSendRequest+0x148

*** ERROR: Symbol file could not be found.  Defaulted to export symbols for 1BA7.tmp -

1a 0433c438 06e477c5 wininet!HttpSendRequestW+0x5e

WARNING: Stack unwind information not available. Following frames may be wrong.

1b 0433efa4 06e14502 1BA7!ffStart+0x23df5

1c 00000000 00000000 1BA7+0x4502

What is 1BA7? It's the Snapfish control which is on the website that I was using to upload the pictures.

0:003> lm vm 1BA7

start    end        module name

06e10000 06f18000   1BA7       (export symbols)       1BA7.tmp

    Loaded symbol image file: 1BA7.tmp

    Image path: C:\Users\bryangr\AppData\Local\Temp\Low\1BA7.tmp

    Image name: 1BA7.tmp

    Timestamp:        Thu Jul 03 11:56:02 2008 (486D20C2)

    CheckSum:         0010DF3F

    ImageSize:        00108000

    File version:     2.5.0.5

    Product version:  2.5.0.5

    File flags:       0 (Mask 17)

    File OS:          4 Unknown Win32

    File type:        2.0 Dll

    File date:        00000000.00000000

    Translations:     0409.04b0

    CompanyName:      Snapfish Inc.

    ProductName:      Snapfish File Upload ActiveX Control

    InternalName:     Snapfish File Upload ActiveX Control - Application maintained by Aaron Chan

    OriginalFilename: QuickUploader.iaa

    ProductVersion:   2, 5, 0, 5

    FileVersion:      2, 5, 0, 5

    FileDescription:  Snapfish File Upload ActiveX Control

    LegalCopyright:   Copyright (C) 2004-2005 Snapfish Inc.

    Comments:         Snapfish File Upload ActiveX Control

I g'd the debugee and after a minute or so it began to respond again so I continued to order my prints. So what can be done to correct issues like this?

Check for new versions of the control
Make sure you have current system patches
Uncheck "Automatically detect settings" under Tools-> Internet Options->Connections->LAN Settings

Have you ever needed to log diagnostic information from instrumented builds? Sure you have.

2008-10-28T19:44:00Z

There have been cases where I wanted to instrument my app and have the ability to log extensive diagnostic information to a file, starting a new file each day or after the file reached a certain size. When this is necessary I end up finding myself using TextWriterTraceListener and bootlegging some sort of rollover mechanism. Recently, it was brought to my attention that the Microsoft.VisualBasic.Logging namespace had a gem called FileLogTraceListener which I had never noticed before because I primarily develop in C#. I find that this class is magically delicious and I hope that you do too.

Enjoy!

Why all the hanging .exe's?

2008-10-25T23:14:00Z

I know that it's been quite a while since I have written a blog. I have been plenty busy up here in the Pac-Northwest. Since my last post I have moved into new condo and back to the Server & Tools group to work in my original group - OneCare. Now that I've covered that, I give you one of the issues that I ran into a little while back.

During some standard processes I had some automation going out and running this binary that had been working fine for a while. A couple of days after the job kicked off I was on a server investigating something totally unrelated when I noticed that the exe that was being called (from a network share) was hanging. After checking a couple of other servers I found that it was hanging on them as well so I needed to investigate. I proceeded to investigate by attaching a debugger on the offending executable. You can see that the process had been up for over 2 days - but why? This had worked so well before.

    1 Process Uptime: 2 days 6:09:16.000

Once I look at the stack backtrace I can see something immediately that is not right. The binary that I call from the network share is explicitly called without the GUI - so why are there windowing APIs on the call stack?

    3 0:000> kbn
    4 
    5    0  Id: 355ec.3488c Suspend: 1 Teb: 7ffdf000 Unfrozen
    6  # ChildEBP RetAddr  Args to Child              
    7 00 0012f758 7739d05f 77392c53 00000000 00000000 ntdll!KiFastSystemCallRet
    8 01 0012f790 7738f152 4411008c 00000000 00000001 user32!NtUserWaitMessage+0xc
    9 02 0012f7b8 773a16c2 77380000 00200838 00000000 user32!InternalDialogBox+0xd0
   10 03 0012fa78 773a0fa4 0012fbe0 ffffffff ffff0000 user32!SoftModalMessageBox+0x94b
   11 04 0012fbc8 7739df11 0012fbe0 00000000 7739de84 user32!MessageBoxWorker+0x2ba
   12 05 0012fc34 735df73f 0012fc4c 00000000 00000000 user32!MessageBoxIndirectA+0xb8
   13 06 0012fc74 735df5bb 0012fcb8 00000000 00000000 msvbvm60!AppCtlProc+0x2b
   14 07 0012fc9c 735df8b6 735df6e4 0012fcb8 0012fcb8 msvbvm60!APP::GetPropA+0x115
   15 08 0012fccc 735d3c78 00000000 00bc0528 0012fcf4 msvbvm60!_HrGetFileName+0x6f
   16 09 0012fd30 735d3cc6 00bf1e94 00bb1fa4 00000030 msvbvm60!CreateEmbeddedObject+0x149
   17 0a 0012fd50 7363c331 00168aa4 00000000 00000030 msvbvm60!CreateEmbeddedObject+0x197
   18 0b 0012fd6c 7363c613 00168aa4 00000000 00000030 msvbvm60!EpiGetInvokeArgs+0x40f
   19 0c 0012fed4 73577a5a 0014ee38 0040286c 00bb1e94 msvbvm60!CoerceArg+0x19b
   20 0d 0012fef8 73573959 0040286c 00000000 0040286c msvbvm60!LIBID30_CDAO+0xa
   21 0e 0012ff18 735736d2 00400000 00400000 00000000 msvbvm60!_CEDIT_vtbl::`vftable'+0x131
   22 0f 0012ff38 735735d8 00000000 00400000 00000000 msvbvm60!_CDRIVE_vtbl::`vftable'+0x4a
   23 *** ERROR: Module load completed but symbols could not be loaded for [CONFIDENTIAL]
   24 10 0012ffb8 00402736 0040286c 77e523e5 00000000 msvbvm60!_CDIR_vtbl::`vftable'+0x100
   25 WARNING: Stack unwind information not available. Following frames may be wrong.
   26 11 0012fff0 00000000 0040272c 00000000 78746341 [CONFIDENTIAL]+0x2736

Next I dump the stack with the parameters and since I have all the symbols I can see that there is a structure that is of interest which I suspect will have the text of the window that is hanging there and cannot be seen by logging in.

    1 0:000> kP
    2 <snip>
    3 0012fa78 773a0fa4 user32!SoftModalMessageBox(
    4             struct _MSGBOXDATA * lpmb = 0x0012fbe0)+0x94b
    5 </snip>

I run dt (display types) on the struct to dump our the contents so that I can get the address of the string that cantains the message text. (requires private symbols)

   34 0:000> dt _MSGBOXDATA 0x0012fbe0
   35 user32!_MSGBOXDATA
   36 <snip>
   37    +0x00c lpszText         : 0x002007e0  -> 0x52
   38 </snip>

Now that I have the address of the string I can dump that out with the SIE extention's !showstringw (double-byte words) command.

   40 0:000> !showstringw 0x002007e0
   41 Run-time error '70':Permission denied

What do have we here! A security problem? But wait didn't we already execute the application? We already have access to the filesystem so what's the problem? I look deeper in the stack at the initial windowing function which was probably called from the dll that encountered the problem. So I inspect the output from kP and see user32!MessageBoxIndirectA on the call stack which takes a struct. Upon examining the struct I see that the call was bubbled up via odbc32.dll.

    1 0:000> dt tagMSGBOXPARAMSA 0x0012fc4c
    2 odbc32!tagMSGBOXPARAMSA
    3 ...

It turns out that the root cause in this case was the result of datasource issues.

Quick 'n Dirty Patch Validation

2008-04-18T03:21:00Z

Since patching is currently on the brain I thought that I'd also include this little VBScript that I wrote a while back to do a quick 'n dirty patch validation. The script takes two arguments 1) a file with the servers to target (one per line) and 2) a file with the hotfix ID that is expected to be installed (accepts form KB12345 or 12345, one per line). After the script is complete you will have a semicolon delimited file that you can import into Excel and filter. Happy patching.

Example usage: C:\>cscript verifyqfe.vbs servers.txt hotfixes.txt

Const KEY_QUERY_VALUE = &H0001

Const HKEY_LOCAL_MACHINE = &H80000002

VerifyQfeInstalled

Sub VerifyQfeInstalled()

    Dim aHostArray, aQfeArray

    Dim bQfeFound

    Dim iIndex, iInnerIndex

    Dim oWmi, oFs, oHostList, oLog, oQfeList, oQfeItem, colQfeItems, oRegistry

    Dim sDate, sTime, sDateTime, sLog, sHostList, sQfeList, sHosts, sQfeContent, sServer, sQfeId, sQfeInfo, sHelpLink, sInstallDate

    sDate = Date()

    sDate = Replace(sDate, "/", "_")

    sTime = Time()

    sTime = Replace(sTime, ":", "_")

    sTime = Replace(sTime, " ", "_")

    sDateTime = sDate & "_" & sTime

    sLog = "VerifyQfeInstalled_" & sDateTime & ".log"

    Set oFs = CreateObject("Scripting.FileSystemObject")

    Set oLog = oFs.CreateTextFile(sLog)

    'oLog.WriteLine("Created file [" & sLog & "] and opened it for WRITING")

    Wscript.StdOut.WriteLine "Processing started."

    If Wscript.Arguments.Count < 2 Then

        Wscript.StdOut.WriteLine "ERROR: You must supply a host list file and QFE list file!"

        oLog.Close()

        oFs.DeleteFile sLog, True

        Set oLog = Nothing

        Set oFs = Nothing

        Wscript.Quit()

    Else

        sHostList = Trim(Wscript.Arguments(0))

        sQfeList = Trim(Wscript.Arguments(1))

    End If

    If (oFs.FileExists(sHostList)) Then

        'oLog.Write("Opening [" & sHostList & "] for READING...")

        Set oHostList = oFs.OpenTextFile(sHostList, 1, False, 0)

        sHosts = oHostList.ReadAll

        'oLog.Write("READ...")

        oHostList.Close()

        Set oHostList = Nothing

        'oLog.WriteLine("CLOSED")

        aHostArray = Split(sHosts, vbNewLine)

        'oLog.WriteLine("Read [" & UBound(aHostArray) + 1 & "] items.")

        'oLog.WriteBlankLines(1)

        If (oFs.FileExists(sQfeList)) Then

            'oLog.Write("Opening [" & sQfeList & "] for READING...")

            Set oQfeList = oFs.OpenTextFile(sQfeList, 1, False, 0)

            sQfeContent = oQfeList.ReadAll

            'oLog.Write("READ...")

            oQfeList.Close()

            Set oQfeList = Nothing

            'oLog.WriteLine("CLOSED")

            aQfeArray = Split(sQfeContent, vbNewLine)

            'oLog.WriteLine("Read [" & UBound(aQfeArray) + 1 & "] items.")

            'oLog.WriteBlankLines(1)

        Else

            Wscript.StdOut.WriteLine "ERROR: The file you specified for the QFE list does not exist!"

            oLog.Close()

            oFs.DeleteFile sLog, True

            Set oLog = Nothing

            Set oFs = Nothing

            Wscript.Quit()

        End If

    Else

        Wscript.StdOut.WriteLine "ERROR: The file you specified for the host list does not exist!"

        oLog.Close()

        oFs.DeleteFile sLog, True

        Set oLog = Nothing

        Set oFs = Nothing

        Wscript.Quit()

    End If

    For iIndex = 0 To UBound(aHostArray)

        sServer = Trim(aHostArray(iIndex))

        Wscript.StdOut.WriteLine "Checking " & sServer

        If (PingHost(sServer)) Then

            Set oWmi = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & sServer & "\root\cimv2")

            For iInnerIndex = 0 To UBound(aQfeArray)

                bQfeFound = False

                If (Left(Trim(aQfeArray(iInnerIndex)), 2) <> "KB") Then

                    sQfeId = "KB" & Trim(aQfeArray(iInnerIndex))

                Else

                    sQfeId = Trim(aQfeArray(iInnerIndex))

                End If

                sHelpLink = GetHelpLink(sServer, sQfeId)

                sInstallDate = GetSecondaryDate(sServer, sQfeId)

                If (sInstallDate <> "") Then

                    sInstallDate = MakeFriendlyDate(sInstallDate)

                End If

                Set colQfeItems = oWmi.ExecQuery("SELECT * FROM Win32_QuickFixEngineering WHERE HotFixID='" &sQfeId & "'")

                If (colQfeItems.Count <> 0) Then

                    For Each oQfeItem in colQfeItems

                        bQfeFound = True

                        sQfeInfo = UCase(oQfeItem.CSName) & ";"  & oQfeItem.Description & ";" & oQfeItem.HotFixID & ";" & oQfeItem.InstallDate & ";" & sInstallDate & ";" & oQfeItem.InstalledBy & ";" & bQfeFound & ";" & sHelpLink

                        oLog.WriteLine(sQfeInfo)

                    Next

                Else

                    sQfeInfo = UCase(sServer) & ";;" & sQfeId & ";;;;" & bQfeFound & ";"

                    oLog.WriteLine(sQfeInfo)

                End If

            Next

        Else

            Wscript.StdOut.WriteLine "**WARNING** " & sServer & " is not pingable!"

            'oLog.WriteLine("**WARNING** " & sServer & " is not pingable!")

        End If

    Next

    oLog.Close()

    Wscript.StdOut.WriteLine "Done processing."

    Wscript.StdOut.WriteLine "Log file is [" & sLog & "]."

    Set oQfeItem = Nothing

    Set colQfeItems = Nothing

    Set oWmi = Nothing

    Set oLog = Nothing

    Set oFs = Nothing

End Sub

Function PingHost(ByVal sServer)

    Dim oShell, oExec

    Dim sResults

    Dim bIsSuccess

    bIsSuccess = False

    Set oShell = CreateObject("WScript.Shell")

    Set oExec = oShell.Exec("ping.exe -n 2 -w 1000 " & sServer)

    sResults = LCase(oExec.StdOut.ReadAll)

    If InStr(sResults, "reply from") Then

        If InStr(sResults, "destination host unreachable") Then

            'WScript.Echo "ERROR: Destination Unreachable!"

        Else

            bIsSuccess = True

            'WScript.Echo "SUCCESS"

        End If

    Else

        Wscript.StdOut.WriteLine "ERROR: Host Unreachable!"

    End If

    Set oExec = Nothing

    Set oShell = Nothing

    PingHost = bIsSuccess

End Function

Function MakeFriendlyDate(ByVal sDate)

    Dim sYear, sMonth, sDay, sInstalledOn

    sYear = Left(sDate, 4)

    sMonth = Mid(sDate, 5, 2)

    sDay = Left(sDate, 2)

    sInstalledOn = sMonth & "/" & sDay & "/" & sYear

    MakeFriendlyDate = sInstalledOn

End Function

Function GetHelpLink(ByVal sServer, ByVal sQfeId)

    Dim oRegistry

    Dim sKeyRoot, sKey, sValue, sHelpLink

    Set oRegistry = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & sServer & "\root\default:StdRegProv")

    sKey = "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\" & sQfeId

    sValue = "HelpLink"

    oRegistry.GetStringValue HKEY_LOCAL_MACHINE, sKey, sValue, sHelpLink

    Set oRegistry = Nothing

    GetHelpLink = sHelpLink

End Function

Function GetSecondaryDate(ByVal sServer, ByVal sQfeId)

    Dim oRegistry

    Dim sKeyRoot, sKey, sValue, sInstallDate

    Set oRegistry = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & sServer & "\root\default:StdRegProv")

    sKey = "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\" & sQfeId

    sValue = "InstallDate"

    oRegistry.GetStringValue HKEY_LOCAL_MACHINE, sKey, sValue, sInstallDate

    Set oRegistry = Nothing

    GetSecondaryDate = sInstallDate

End Function

Doc, do you think you can patch 'em up?

2008-04-18T02:44:00Z

To some of you the second Tuesday of each month is just another day, but to the rest of us this day is known as "patch Tuesday". The second Tuesday of each month is when Microsoft releases security updates for it's products to the world. Our group previously had a patching process that worked like a charm for the majority of our boxes which were running W2K3 but now we have several WS08 RTM boxes in production as well. During the last patching cycle I had to rig up a less-than-elegant solution to patch those WS08 servers so I told myself that after the dust settled I'd look into a better way of handling it as well as consolidating the two processes.

After pondering how I would attack this issue I found that the Windows Update Agent API was a likely candidate to assist me. I fired up Visual Studio and set a reference to wuapi.dll then quickly threw together a proof-of-concept to show to the rest of my team. Below you will find the source necessary to query Windows Update, add packages to an install manifest, download said packages, and install the packages. Hopefully this will get you started if you are in fact looking for a custom Windows Update solution.

static void Main(string[] args)

    StreamWriter hLogFileHandle = null;

    hLogFileHandle = GetLogFileHandle(@"C:\sdmedic\logs\wuagent.log");

    WriteToLogFile(hLogFileHandle, "Installation started by: " + Environment.UserName + Environment.NewLine);

    WriteToLogFile(hLogFileHandle, DateTime.Now.ToString());

    UpdateSessionClass oUpdateSession = new UpdateSessionClass();

    UpdateSearcher oUpdateSearcher = (UpdateSearcher)oUpdateSession.CreateUpdateSearcher();

    ISearchResult oSearchResult = oUpdateSearcher.Search("IsInstalled=0 and Type='Software'");

    if (oSearchResult.Updates.Count > 0)

        Console.WriteLine("Found " + oSearchResult.Updates.Count + " updates.");

        WriteToLogFile(hLogFileHandle, "Found " + oSearchResult.Updates.Count + " updates.");

        for (int iIndex = 0; iIndex < oSearchResult.Updates.Count; iIndex++)

            //Console.WriteLine(oSearchResult.Updates[iIndex].Title);

    UpdateCollectionClass oNeededUpdates = new UpdateCollectionClass();

    for (int iIndex = 0; iIndex < oSearchResult.Updates.Count; iIndex++)

        IUpdate oUpdatePackage = oSearchResult.Updates[iIndex];

        oNeededUpdates.Add(oUpdatePackage);

        Console.WriteLine("Added " + oUpdatePackage.Title + " to download manifest.");

        WriteToLogFile(hLogFileHandle, "Added " + oUpdatePackage.Title + " to download manifest.");

    UpdateDownloader oDownloader = oUpdateSession.CreateUpdateDownloader();

    oDownloader.Updates = oNeededUpdates;

    Console.Write("Downloading updates...");

    WriteToLogFile(hLogFileHandle, "Downloading updates...");

    oDownloader.Download();

    Console.WriteLine("done");

    WriteToLogFile(hLogFileHandle, "done");

    Console.WriteLine("Verifying downloaded updates...");

    WriteToLogFile(hLogFileHandle, "Verifying downloaded updates...");

    for (int iIndex = 0; iIndex < oSearchResult.Updates.Count; iIndex++)

        IUpdate oUpdatePackage = oSearchResult.Updates[iIndex];

        if (oUpdatePackage.IsDownloaded)

            Console.Write("Verified " + oUpdatePackage.Title);

            WriteToLogFile(hLogFileHandle, "Verified " + oUpdatePackage.Title);

        Console.WriteLine("");

    UpdateCollectionClass oUpdateManifest = new UpdateCollectionClass();

    for (int iIndex = 0; iIndex < oSearchResult.Updates.Count; iIndex++)

        IUpdate oUpdatePackage = oSearchResult.Updates[iIndex];

        if (oUpdatePackage.IsDownloaded)

            oUpdateManifest.Add(oUpdatePackage);

            Console.WriteLine("Added " + oUpdatePackage.Title + " to install manifest.");

            WriteToLogFile(hLogFileHandle, "Added " + oUpdatePackage.Title + " to install manifest.");

    UpdateInstaller oInstaller = (UpdateInstaller)oUpdateSession.CreateUpdateInstaller();

    oInstaller.Updates = oUpdateManifest;

    Console.WriteLine("Installing...");

    WriteToLogFile(hLogFileHandle, "Installing...");

    IInstallationResult oInstallationResult = oInstaller.Install();

    Console.WriteLine("Result: " + oInstallationResult.ResultCode.ToString());

    WriteToLogFile(hLogFileHandle, "Result: " + oInstallationResult.ResultCode.ToString());

    Console.WriteLine("Reboot: " + oInstallationResult.RebootRequired.ToString());

    WriteToLogFile(hLogFileHandle, "Reboot: " + oInstallationResult.RebootRequired.ToString());

    for (int iIndex = 0; iIndex < oSearchResult.Updates.Count; iIndex++)

        Console.WriteLine(oUpdateManifest[iIndex].Title + ": " + oInstallationResult.GetUpdateResult(iIndex).ResultCode.ToString());

        WriteToLogFile(hLogFileHandle, oUpdateManifest[iIndex].Title + ": " + oInstallationResult.GetUpdateResult(iIndex).ResultCode.ToString());

        Console.WriteLine(oUpdateManifest[iIndex].Title + ": " + oInstallationResult.GetUpdateResult(iIndex).RebootRequired.ToString());

        WriteToLogFile(hLogFileHandle, oUpdateManifest[iIndex].Title + ": " + oInstallationResult.GetUpdateResult(iIndex).RebootRequired.ToString());

    if (oInstallationResult.RebootRequired)

        Console.WriteLine("Initiating system restart.");

        WriteToLogFile(hLogFileHandle, "Initiating system restart.");

        ExitWindows(ExitWindowsTypes.Reboot);

        CloseLogFileHandle(hLogFileHandle);

    CloseLogFileHandle(hLogFileHandle);

Don't let the doorknob hit you...

2008-02-26T23:54:00Z

The last few weeks have been a blur; in part because I took a mini-vacation, but the majority of my time has been spent trying to do my part to get a few of our flagship products out the door. It was a long, hard, and painful road to the release of Windows Server 2008 but I think that it was definetly worth it. Personally, I've been waiting specifically for IIS7 and it's more modular design, plus I like having the ability to extend sub-systems without having to be a magician (meaning you probably won't see me writing a C++ ISAPI fiter again).

In addition to WS08 and IIS7 I've also been looking forward to Visual Studio 2008. There are lots of cool things that come with VS08 that would make life so much easier for me like; the ability to target multiple versions of the .NET Framework, or automatic properties (part of the new C# compiler), or extension methods. Anyway, whatever product that you've been waiting for I hope you find the enhancements gratifying and most of all helpful in making your job a breeze!

F5 brought sexy back; at least to networking

2008-02-12T17:41:00Z

Over the years I‘ve dealt with a number of technologies and expensive devices that do funny things with network traffic. None of them really made me smile; you know what I mean if you’ve ever tried to read a book titled “Telecommunication Traffic Engineering with Multi-Protocol Label Switching”. The world of networking and telecommunications was in my opinion the driest ever after you get past the basics, and even that’s questionable for some.

To get to my point, I’ve done quite a bit of work in the world of web operations and in doing so I ran into these strange “application-aware network devices” called load balancers. At this point I had the googly eyes and was interested. How could it be; a network device that actually cares what’s flowing through it. Anyway, I’ve had the chance to work with a few major appliances: Cisco’s LocalDirector and Content Switching Module, Nortel’s Alteon 180e and F5’s Local Traffic Manager. Now at one point I was a big Cisco buff and enjoyed the mystique of operating in the IOS, I felt almost as elite as when I worked on UNIX, I carried my laptop and console cable like an exhibition of coolness. In the world of web ops you live in a world where precise application upgrades are crucial and downtime is sometimes not an option. In that world, hopping on a network device and issuing some cryptic commands to pull servers out of the web farm is not as elegant as one would like. Enter F5’s BIG-IP and its iControl SDK.

With iControl you can now use SOAP/XML calls to control your F5 device. This means that now you can do some integration and have self-aware applications that can add and remove servers from the farm based on performance. Now that’s sexy! The other heavy hitters came along with similar solutions but in my opinion F5 had a leg up. F5 also has some other appliances that I think are really interesting and seem to be best in class, like the 3DNS product (I guess now it’s called Global Traffic Manager) and the WebAccelerator.