Often times, either due to a misconfiguration/bug/solar eclipse or otherwise, customers call into Microsoft Product Support Services complaining that their Exchange server is churning out transaction logfiles at an alarming rate. For every instance of this symptom, there are at least a dozen reasons why this is happening. Regardless, there's never been a good way to parse the transaction logs and extract any useful patterns. In lieu of rolling up my sleeves and actually writing code to accomplish such a task, I've slapped together a bunch of utilities that will do the job. Ugly? Sure. Useful? You bet. Having used it against many customer issues, I can attest that this actually works, and works quite well.
1. Download the "Unix for Win32" utilities from http://downloads.sourceforge.net/unxutils/UnxUtils.zip?modtime=1172730504&big_mirror=0
2. Extract all files from the UnxUtils\usr\local\wbin subsirectory to C:\UNIX
3. Download strings.exe from http://live.sysinternals.com/strings.exe, and place strings.exe into C:\UNIX
4. Make a C:\TMP directory (Unix tools need a Win32 equivalent of /tmp)
5. Make a directory for all your transaction log files (i.e. D:\customers\test), and place all the logs in this dir
6. From a cmd prompt, navigate to your C:\UNIX dir
7. Run the following command:
strings -q -n 16 D:\customers\test\*.log | cut -f3 -d: | sort | uniq -c | sort | tee c:\log-output.wri
What this is doing:
· Identifies all strings in the logs greater than 16 chars
· Removes the D:\customers\test\E00xxxx.log: from the output
· Sorts the output
· Finds all duplicate records, and retains a count
· Sorts the final output (ending with the largest # of occurrences)
· Writes all the output to c:\log-output.wri (use WordPad / write.exe to open; notepad.exe mangles the output)
If you're running this on Windows 7 or above, you'll have to modify the output directory as follows (as it won't let you write directly to the root of the C: drive) ...
strings -q -n 16 D:\customers\test\*.log | cut -f3 -d: | sort | uniq -c | sort | tee c:\users\yourname\log-output.wri
The output will be sorted from the least number of repeating occurences to greatest, so crack open that log-output.wri file, scroll to the bottom, and commence spelunking!
Well, not bad, but unfortunately it gave me this result after running through around 20 logfiles:
<.. cut ..>
10 <p class=MsoNormal><span style='font-size
14 <p class=MsoNormal><o
14 <p class=MsoNormal><span style='color
(These logs were generated by E2K7)
E? Is this longer than 16 characters?
- Given that E12 transaction logs are only 1MB now, you'll improve your odds of success by sampling more logs.
- Try swapping the 16 with 8, and see if that yields better/more results.
This is a great string parser routine and I've used for other things. Thanks so much Scott. I know many of my peers refer to it as well, great impact.
As I use this as needed, I copy all of the unix exe's into the main unix folder. Once done this puppy runs quite well.
Thanks again Scott!
What should I be looking for? I'm trying to find out why we are having an excessive spike in log creation lately. Here's the last bunch of lines from the results:
318 00) Eastern Time (US & Canada)
354 Mon, 15 Sep 2008 10
358 10pt; COLOR
361 15 Sep 2008 15
404 Monday, September 15, 2008 10
426 from TC3705.domain.com ([172.22.227.140]) by TC3647.domain.com with Microsoft SMTPSVC(6.0.3790.1830);
513 Mon, 15 Sep 2008 11
542 10pt; FONT-FAMILY
813 #d4d0c8; BACKGROUND-COLOR
815 Produced By Microsoft Exchange V6.5
1677 #d4d0c8; BORDER-TOP
Can you re-run the same command, only this time instead of looking for strings 16 chars (or greater), let's go with 8. The command line will look like this:
strings -q -n 8 D:\customers\test\*.log | cut -f3 -d: | sort | uniq -c | sort | tee c:\log-output.wri
Does it work on Exchange 2007?
Yes, this technique will work on Exchange 2007.
I get 'cut' is not recognized as an internal or external command, operable program or batch file. What am I doing wrong?
It's not in your path. Place cut.exe in your path, and you should be good to go ...
Thanks for the guide. I've got an Exchange 2003 environment where "Junk E-mail Rule" is the highest repeating occurence. Any ideas why this would be?
In conjuction with analyzing the transaction logs, you can also use Exmon to [hopefully] pinpoint a specific user who may be thrashing the server; perhaps their Junk E-mail Rule is corrupted:
Many thanks for this.Worked well at the first shot!!!
Glad to hear it helped out, Silju!
1028 Microsoft Exchange Server
1470 <b><span style='font-weight
2959 [1/21/2010 16
This is what I'm seeing after analyzing 200MB of log files...any thoughts?
Based on the data, it seems someone is initiating a mail loop (or is merely blasting the same content repeatedly) ... note the WNED (I'm guessing it's short for PWNED).
My recommendation: use ExMon to identify the users with the highest amount of activity, then disable their mailboxes one-by-one until the transaction log growth stops.