thread degraded mode ... the sequel.

BrettSh — Sun, 19 Mar 2006 12:08:00 GMT

I've run in thread degraded mode for 2 or 3 weeks without a hitch before, this time I wasn't so lucky. After about 6 hours explorer spun up taking 50% CPU (given it's a HT machine, that usually means 1 thread spinning endlessly). Heuve! That just won't do. But we can merely degrade this thread too ... once we find it ...

So I ctrl-C in the debugger window that got started from part 1 ...

Alternatively: If you don't have debugger attached up to the process yet, you can do so, by going to the C:\debuggers directory (expained in part 1), typing "tlist" to find the Process ID (PID) of the explorer.exe process, and then run "ntsd -p <PID>".

At the debug prompt we type "!runaway", which gives you something like this:
    0:076> !runaway
     User Mode Time
    Thread       Time
    73:1acc      0 days 6:36:51.328
       1:ec0       0 days 1:39:27.203
    15:1ac       0 days 0:06:01.640
    12:f94       0 days 0:05:44.281
    ... deleted the other ~80 threads ...
The time column is cumulative CPU time the thread has used.

At this point you 'g' the debugger, wait for a short timed interval, then hit ctrl-C again to re-break into the debugger. I waited 30 seconds myself, and then re-run the !runaway command ...
    0:082> !runaway
     User Mode Time
    Thread       Time
    73:1acc      0 days 6:37:21.359
       1:ec0       0 days 1:39:28.406
    15:1ac       0 days 0:06:01.671
    12:f94       0 days 0:05:44.281
    ...
You can see from the blue that thread 73 is our culprit, as it's cumulative CPU time went up by nearly exactly 30 seconds. Note the culprit thread isn't guaranteed to be the top thread, but it was in my case.

So the "~f" command from the first blog affects the "current" thread in the debugger, which you can see above is thread 82, not exactly, well ... not at all what is desired. We want thread 73 to be frozen though, so here is how you freeze a specific thread:
    0:082> ~73f
    0:082> g
Now I'm back to normal, CPU settles down, go back to work.

Sometime later the next thing is I AV'd when I typed a search in the MSN Desktop Search box on the task bar see the bottom of my start bar ... since MSN DS was the source of the original issue, not sure why I'd have expected that to work (I can be really stupid sometimes, I'll blog more about that), one more thread to freeze and then 'g' the process.

The last thing that AV'd is <window>-E. Don't know why that AV'd, but don't care I can live without file explorer.

Though I am not sure I can live without desktop search ... I sense a reboot is in my future ...

Try explorer's thread degraded mode ...

BrettSh — Fri, 17 Mar 2006 04:29:00 GMT

On any given day that is past say 7 days of uptime, I have 100 - 300 windows open, not kidding, here is a shot of my current task bar ... there shouldn't be anything msft confidential there, at least that you'd actually be able to read more than 4 or 5 letters of ... I know by heart I have 41 rows, so that's 204 windows open there ... I turn off that "group similar taskbar buttons" "feature", and the buttons show up in order, so a given "job" usually has task buttons around each other (in fact the last 6 buttons there are for this blog post) ... so what does this have to do with explorer ...

The basic upshot of this computing lifestyle choice is that my heart skips a beat and then visceral pain sets in whenever explorer AVs (Access Violation) ... explorer is what controls the start bar, and when it restarts the task buttons will be in a random order ... for the *nix types, this is like your window manager core dumping ... it's awefulness.

Right so getting to thread degraded mode ... my own term, for when you simply freeze the AV'd thread in a process, and allow the process to continue on its merry way. You can do this because maybe the thread may not be doing something particularly useful, ergo it is "not a very serious AV", or maybe call it a "slight AV". The process often (sometimes?) continues to function.

How to use thread degraded mode:

First, you will have to prepare your machine for initiating thread degraded mode ...

You will need to get a user mode debugger (there may already be a ntsd.exe in your system32, which should work, but no one uses that anchient one) go get a good version, which for an x86 box installs from this exe (i think). Install it to C:\debuggers, everyone else around here seems to.

Navigate to this registry key (read more about it):
"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug"
Create registry string (REG_SZ) value with a name of "Debugger" and value of
"C:\debuggers\ntsd -p %ld -g -G -e %ld"
If it already has a value you may want to save it.

You only have to do those steps once, and now you are ready to run in thread degraded mode, if the need should arise.

The next time explorer (or any application) crashes / AVs on you, you will get an option to debug the process ... select "Debug" or "Yes" or whatever ... this will open a debugger attached to explorer ( you'll probably have to alt-tab to find this new debugger window, because the task bar will be frozen/unresponsive while you debug it ;).

The debugger will open with the AV'ing thread as the current thread, so use "~f<enter>" (that is a tilde), to freeze this thread. Then "g<enter>" will let the task bar come back to you (maybe). At this point you should be praying that the thread you froze isn't holding any crucial critical sections or locks, and that things will return to "normal" ... your mileage may vary ... greatly.

It will look like this:
    0:008> ~f
    0:008> g

After you 'g' it, it will start printing this kind of thing in the debugger ...
    System 0: 1 of 84 threads are frozen
    System 0: 1 of 84 threads were frozen
    System 0: 1 of 84 threads are frozen
That's just explorer letting you know it loves you for not letting go, and putting it on life support.

Oh the crash was in MSN Desktop Search, but I don't fault (intended ;) them because I'm running the first beta of the software released in Dec 2004, I've heard they've had an update since then.

Anyway, as of approximately 8:20 AM (PST) yesterday (wed) morning, I've been running in thread degraded mode ... as I finish this post I've got 4 frozen threads ... there were a few more threads with "issues" but I don't have time to blog about them right yet ...

183 : Comp::Impl::WinNT

thread degraded mode ... the sequel.

Try explorer's thread degraded mode ...