August, 2007

  • Never doubt thy debugger

    Find folder size (conditionally) with LogParser

    • 6 Comments

    I was reading this post from Rahul a few days ago, and wondered if something similar could have been done (hopefully with less efforts) with LogParser... Well, it turned our that LogParser accepts a "-FS" switch which stands for File System as the input type, and if you run "logparser -h examples" you'll have a few sample commands you can use as the basis of your scripts. And guess what? Here's one of them:

    Print the 10 largest files on the C: drive:
    LogParser "SELECT TOP 10 * FROM C:\*.* ORDER BY Size DESC" -i:FS

    I created a similar structure and put in there some sample images, just to have some files to count:

    folder tree

    So... here's the script:

    logparser "SELECT SUM(Size) FROM c:\temp\Rahul\*.* WHERE INDEX_OF(Path, '_wow') > 0" -i:FS

    And the output:

    logparser output

    5 images, 35.391 bytes each = 179.655 bytes... here we are! smile_regular

    Carlo


    Quote of the Day:
    It is a great thing to start life with a small number of really good books which are your very own.
    --Sir Arthur Conan Doyle

  • Never doubt thy debugger

    Start to play with LogParser

    • 4 Comments

    A few months ago (back at the beginning of March) at the annual offsite meeting my virtual team had in Lisbon, my colleague Doug held an hour session about LogParser and some of the cool features it has. Doug promised to blog about it with more details, so I don't want to steal him an argument (and he for sure knows more than me on the subject), but I started play with it to analyze some logs I had, and among the others I wrote a couple of scripts to automate some tasks I previously had to do manually, and just wanted to share with you (and post on my blog as a future reference).

    So, here's the deal: you may know from Tess, strong named assemblies should not be deployed in the /bin folder; and if you ever had to troubleshoot a managed memory leak, you probably know that if you have an assembly which is loaded multiple times in memory because you deployed it in the /bin folder of every application you have, it's a good idea to install it in the GAC to avoid wasting server resources. So far so good, but how to find those assemblies? Well, as you can guess, the first thing is to use WinDBG.

    Strong named assemblies in /bin

    First, let's see which assemblies are loaded in the Shared Domain:

     1: 0:000> !dumpdomain -stat
     2: Loaded Son of Strike data table version 5 from "C:\Windows\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
     3: Domain Num Assemblies Size Assemblies Name
     4: 0x793f05b8 1 2,138,112 System Domain
     5: 0x793f1a88 20 11,731,968 Shared Domain
     6: 0x0014af28 2 2,482,176 DefaultDomain
     7: 0x00184e88 87 14,065,152 /LM/W3SVC/3/Root/MyTestApp-1-128273920700311250
     8:  
     9: 0:000> !dumpdomain 0x793f1a88 
     10: Shared Domain: 0x793f1a88
     11: LowFrequencyHeap: 0x793f1aec
     12: HighFrequencyHeap: 0x793f1b44
     13: StubHeap: 0x793f1b9c
     14: Assembly: 0x15d44e48 [AjaxPro]
     15: ClassLoader: 0x15d6b568
     16: Module Name
     17: 0x15d792b8 c:\Windows\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyTestApp\b803c58e\4108c9e9\assembly\dl2\eb13c2c4\00ffc7aa_14a1c701\ajax.dll
     18:  
     19: Assembly: 0x15db5ca0 [System.EnterpriseServices]
     20: ClassLoader: 0x15d9a6a0
     21: Module Name
     22: 0x15da1990 c:\Windows\assembly\gac\system.enterpriseservices\1.0.5000.0__b03f5f7f11d50a3a\system.enterpriseservices.dll
     23: 0x15e149f0 c:\Windows\assembly\gac\system.enterpriseservices\1.0.5000.0__b03f5f7f11d50a3a\system.enterpriseservices.thunk.dll
     24:  
     25: Assembly: 0x13b7d0 [mscorlib, Version=1.0.5000.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]
     26: ClassLoader: 0x0013d158
     27: Module Name
     28: 0x79b7a000 c:\Windows\microsoft.net\framework\v1.1.4322\mscorlib.dll
     29:  
     30: Assembly: 0x15d84470 [INGLibrary]
     31: ClassLoader: 0x15d782c8
     32: Module Name
     33: 0x15d91eb0 c:\Windows\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyTestApp\b803c58e\4108c9e9\assembly\dl2\d6d79113\00ffc7aa_14a1c701\library.dll
    [...]

    Save the !dumpdomain output in a text file, and to the same with the !peb (process environment block) command:

     1: 0:000> !peb
     2: PEB at 7FFDF000
     3: InheritedAddressSpace: No
     4: ReadImageFileExecOptions: No
     5: BeingDebugged: No
     6: ImageBaseAddress: 00420000
     7: Ldr.Initialized: Yes
     8: Ldr.InInitializationOrderModuleList: 131f48 . 22b790
     9: Ldr.InLoadOrderModuleList: 131ec0 . 22b780
     10: Ldr.InMemoryOrderModuleList: 131ec8 . 22b788
     11: Base TimeStamp Module
     12: [...]
     13: C:\WINNT\system32\MSASN1.DLL
     14: C:\WINNT\Microsoft.NET\Framework\v1.1.4322\diasymreader.dll
     15: C:\WINNT\Microsoft.NET\Framework\v1.1.4322\mscorsn.dll
     16: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\root\804ee9b4\26f0df32\ruckcqbk.dll
     17: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyApp\5f12d534\212de01\t7eygstg.dll
     18: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyApp\5f12d534\212de01\assembly\dl2\86a3270b\005ce223_3a04c701\MyApp.dll
     19: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyApp\5f12d534\212de01\assembly\dl2\786b5a22\00028021_3a04c701\extractadgroups.dll
     20: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyApp\5f12d534\212de01\assembly\dl2\1271e8de\00a9138a_1dbac601\interop.activeds.dll
     21: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyApp\5f12d534\212de01\assembly\dl2\18ce4216\004ebb1c_3a04c701\mscexceptionhandler.dll
     22: c:\winnt\assembly\gac\system.web.services\1.1.5000.0__b03f5f7f11d50a3a\system.web.services.dll
     23: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyApp\5f12d534\212de01\assembly\dl2\e318c820\00028021_3a04c701\storeclass.dll
     24: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\MyApp\5f12d534\212de01\assembly\dl2\6a0627f3\00ba61a8_fb03c701\eventlogger.dll
     25: [...]
     26: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\datamenufunctions\2a40e000\d731a401\assembly\dl2\361c9551\00ba61a8_fb03c701\eventlogger.dll
     27: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\datamenufunctions\2a40e000\d731a401\assembly\dl2\22d24657\00a81d1f_3a04c701\msesconnect.dll
     28: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\datamenufunctions\2a40e000\d731a401\assembly\dl2\e44bca6d\004ebb1c_3a04c701\mscexceptionhandler.dll
     29: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\datamenufunctions\2a40e000\d731a401\assembly\dl2\e525577d\002fb122_3a04c701\menuclass.dll
     30: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\dataproductcounts\ba0b3f7a\19117a01\vvyh8t5h.dll
     31: [...]
     32: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\dataproductprofile\6abb3f96\ce8180e1\assembly\dl2\7515c31f\00d54e20_3a04c701\msesregistry.dll
     33: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\dataproductprofile\6abb3f96\ce8180e1\assembly\dl2\79d2b595\00d54e20_3a04c701\utils.dll
     34: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\dataproductprofile\6abb3f96\ce8180e1\assembly\dl2\20aefb4c\004ebb1c_3a04c701\mscexceptionhandler.dll
     35: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\dataproductprofile\6abb3f96\ce8180e1\assembly\dl2\5eaca388\00a81d1f_3a04c701\msesconnect.dll
     36: c:\winnt\microsoft.net\framework\v1.1.4322\temporary asp.net files\dataproductprofile\6abb3f96\ce8180e1\assembly\dl2\337945df\00028021_3a04c701\cr.dll
     37: [...]

    Now that I have those files, I usually have opened them in Excel and with some tweaking I would have filtered the records to only show the files coming from the "temporary asp.net files" folder in the former case, and used the "Highlight duplicates" command in the latter.

    Enter LogParser

    To automate this process a while ago I wrote a quite simple console application, with a few command line switches to extract those values and it worked well, but then I wanted to get my hands dirty with LogParser to experiment some commands other that the useful ones Doug shared with the team.

    So after some attempts to get exactly what I wanted, here are is the batch file for !dumpdomain (Note: I split the command line on multiple rows for reading convenience, but you must place it on a single row to work properly):

     1: REM "USAGE: logparser logparser_!dd.bat <input file name> <output file name>" 
     2:  
     3: logparser "SELECT EXTRACT_FILENAME(text) as Strong_Named_Assemblies_In_/bin 
     4: INTO '%1' FROM '%2' 
     5: WHERE INDEX_OF(text, 'temporary asp.net files') > 0" -i:TEXTLINE -o:NAT -RTP:-1 -fileMode:0

    Which outputs something like:

     1: Strong_Named_Assemblies_In_/bin 
     2: -------------------------------
     3: ajax.dll
     4: library.dll

    And here's the batch for !peb:

     1: REM "USAGE: logparser logparser_!peb.bat <input file name> <output file name>" 
     2: 
     3: logparser "SELECT EXTRACT_FILENAME(text) as Duplicated_Assemblies, COUNT(Duplicated_Assemblies) as Count
     4: INTO '%1' FROM '%2'
     5: WHERE INDEX_OF(text, 'temporary asp.net files') > 0
     6: GROUP BY Duplicated_Assemblies HAVING COUNT(Duplicated_Assemblies) > 1" -i:TEXTLINE -o:NAT -RTP:-1 -fileMode:0

    Which outputs something like:

     1: Duplicated_Assemblies Count 
     2: ----------------------- -----
     3: mscexceptionhandler.dll 3
     4: eventlogger.dll 2
     5: msesconnect.dll 2

    Conclusions

    I realize that maybe there are better ways to do the same, probably Roberto could do the same directly within WinDBG with one of his scripts, but... just take this for what really is: a useful exercise with LogParser and a nice way make dump analysis a bit easier smile_regular

    Carlo


    Quote of the Day:
    It takes a steady hand to carry a full cup.
    --Anonymous
  • Never doubt thy debugger

    Time to say... see you later! :-)

    • 0 Comments

    closed for holidays

     

    Carlo


    Quote of the Day:
    Sincerity is the highest compliment you can pay.
    --Ralph Waldo Emerson
  • Never doubt thy debugger

    Process on Vista: Administrator or user?

    • 0 Comments

    This is an easy (and maybe already known) one for this hot summer day: how can you know if a process on Vista is running as ad Administrator or as a user? Process Explorer will tell you:

    administrators owner

    administrators deny

     

    Carlo

  • Never doubt thy debugger

    Free Download Manager

    • 3 Comments

    As I did in a previous post, here is another nice tool I've been using for a few weeks now, which I find very helpful when it comes to large downloads such as the recently released Visual Studio 2008 beta 2 (but now I'm using it for every download larger than 5 Mb): Free Download Manager.

    I short, FDM allows you to resume your interrupted downloads (if the server supports it, of course), limit the downloading speed (handy if you're browsing the Internet while downloading in background), schedule your downloads and perform some actions when done (scan for viruses, suspend/shutdown the computer, close the connection etc...), find and use mirror servers, split the download into several section to increase download speed (they say up to 600%: I've not measured it, but for sure it's faster), html spider, download entire sites etc... (here are some screenshots).

    The idea is not new (GetRight for instance do almost the same), up to you decide what you like most smile_regular

    Carlo


    Quote of the Day:
    Nature has given to us the seeds of knowledge, not knowledge itself.
    --Lucius Annæus Seneca
  • Never doubt thy debugger

    I have an OutOfMemory exception in my dump! Am I leaking memory?

    • 2 Comments

    Well... the short answer is: not necessarily smile_regular

    I had this discussion a few days ago with a customer who called CSS for a crash they were having, and since he was also starting to learn some debugging and dump analysis basics, he was interested to have some deeper details about my findings in the crash dump they sent in.

    In particular, he thought the crash was due so an OutOfMemoryException because checking the exceptions with WinDBG, he saw something like the following:

    0:000> !dumpallexceptions
    Going to dump the .NET Exceptions found in the heap.
    Number of exceptions of this type:        1
    Exception 0cf500bc in MT 79b94ee4: System.ExecutionEngineException

    -----------------

    Number of exceptions of this type:        1
    Exception 0cf5007c in MT 79b94dac: System.StackOverflowException

    -----------------

    Number of exceptions of this type:        1
    Exception 0cf5003c in MT 79b94c74: System.OutOfMemoryException

    -----------------

    The point is that we are pre-loading a few exception objects on the heap when starting an AppDomain, and the reason is quite simple if you think about it for a moment. Let's pretend we don't have exception objects preloaded: if an application ever reaches a condition where needs to allocate some more memory but there is not enough memory available, then the CLR will have to throw an OutOfMemoryException; but if there is no more memory available on the system (I'm avoiding some details here, but you get the idea), where can we find the memory needed to allocate a new exception object (and all other objects involved in the background)? smile_thinking

    I think you already found the answer: those objects must be already on the heap, thus the CLR is pre-loading them. There is a similar reason behind the ExecutionEnginException and the StackOverflowException objects.

     

    Carlo

Page 1 of 1 (6 items)