Remix!! Using Powershell to parse ESE Transaction Logs ...

Scott Oseychik — Wed, 07 Nov 2007 18:03:00 GMT

Let me preface this post by saying this: I'm a tad lazy. However, the newest addition to our team, Brad Hughes, is not. Far from it. That being said, he took it upon himself to rewrite my "Rough & Tough" approach to parsing ESE logs in Powershell. Enjoy ...

1. Download & install Powershell

2. Download & install strings.exe; make sure strings.exe is in your path

3. Place all your transaction logs into a temp directory (i.e. D:\templogs)

4. Fire up Powershell

5. Run the following command:

strings.exe -q -n 16 D:\templogs\*.log | foreach-object { ($_.Split(":".ToCharArray(),3)[2]) }| group-object | select-object count,name | sort count | export-csv C:\temp\output.csv

What this is doing:

· Identifies all strings in the logs greater than 16 chars

· Removes the D:\templogs\E00xxxx.log: from the output

· Sorts the output

· Finds all duplicate records, and retains a count

· Sorts the final output (ending with the largest # of occurrences)

· Writes all the output to D:\templogs\output.csv

As before, the output will be sorted from the least number of repeating occurences to greatest, but now it's in a nifty csv format that you use Excel to do all sorts of fancy sorting.

Note: this post will probably be obsolote in the next 15 minutes, as Brad will likely re-write this in assembly next.

Update: you'll have to put the output.csv file into a different directory from the logs that you're trying to parse. Otherwise, you'll get into an endless loop where we try to parse the output.csv file as well.

strings.exe -q -n 16 D:\templogs\*.log | foreach-object { ($_.Split(":".ToCharArray(),3)[2]) }| group-object | select-object count,name | sort count | export-csv C:\temp\output.csv

“Rough and Tough” guide to identifying patterns in Transaction Logs

Scott Oseychik — Thu, 12 Jul 2007 21:05:00 GMT

Often times, either due to a misconfiguration/bug/solar eclipse or otherwise, customers call into Microsoft Product Support Services complaining that their Exchange server is churning out transaction logfiles at an alarming rate. For every instance of this symptom, there are at least a dozen reasons why this is happening. Regardless, there's never been a good way to parse the transaction logs and extract any useful patterns. In lieu of rolling up my sleeves and actually writing code to accomplish such a task, I've slapped together a bunch of utilities that will do the job. Ugly? Sure. Useful? You bet. Having used it against many customer issues, I can attest that this actually works, and works quite well.

1. Download the "Unix for Win32" utilities from http://downloads.sourceforge.net/unxutils/UnxUtils.zip?modtime=1172730504&big_mirror=0

2. Extract all files from the UnxUtils\usr\local\wbin subsirectory to C:\UNIX

3. Download strings.exe from http://www.microsoft.com/technet/sysinternals/Miscellaneous/Strings.mspx, and place strings.exe into C:\UNIX

4. Make a C:\TMP directory (Unix tools need a Win32 equivalent of /tmp)

5. Make a directory for all your transaction log files (i.e. D:\customers\test), and place all the logs in this dir

6. From a cmd prompt, navigate to your C:\UNIX dir

7. Run the following command:

What this is doing:

· Identifies all strings in the logs greater than 16 chars

· Removes the D:\customers\test\E00xxxx.log: from the output

· Sorts the output

· Finds all duplicate records, and retains a count

· Sorts the final output (ending with the largest # of occurrences)

· Writes all the output to c:\log-output.wri (use WordPad / write.exe to open; notepad.exe mangles the output)

If you're running this on Vista, you'll have to modify the output directory as follows (as it won't let you write directly to the root of the C: drive) ...

The output will be sorted from the least number of repeating occurences to greatest, so crack open that log-output.wri file, scroll to the bottom, and commence spelunking!

Required Legalese: While I don't advocate nor endorse the use of Open Source Software, I’m hard-pressed to find a Microsoft equivalent utility (or suite of tools) to accomplish the same task.

Scott Oseychik : transaction logs

Remix!! Using Powershell to parse ESE Transaction Logs ...

“Rough and Tough” guide to identifying patterns in Transaction Logs