Scott Oseychik

Microsoft | Exchange G&C Team (Public Folders, Modern Groups, Site Mailboxes, Shared Mailboxes, ... )

Remix!! Using Powershell to parse ESE Transaction Logs ...

Remix!! Using Powershell to parse ESE Transaction Logs ...

Rate This
  • Comments 22

Let me preface this post by saying this: I'm a tad lazy.  However, the newest addition to our team, Brad Hughes, is not.  Far from it.  That being said, he took it upon himself to rewrite my "Rough & Tough" approach to parsing ESE logs in Powershell.  Enjoy ...

 

1.    Download & install Powershell

2.  Download & install strings.exe; make sure strings.exe is in your path

3.    Place all your transaction logs into a temp directory (i.e. D:\templogs)

4.    Fire up Powershell

5.    Run the following command:

 

strings.exe -q -n 16 D:\templogs\*.log | foreach-object { ($_.Split(":".ToCharArray(),3)[2]) }| group-object | select-object count,name | sort count | export-csv C:\temp\output.csv

 

 

What this is doing:

 

·         Identifies all strings in the logs greater than 16 chars

·         Removes the D:\templogs\E00xxxx.log: from the output

·         Sorts the output

·         Finds all duplicate records, and retains a count

·         Sorts the final output (ending with the largest # of occurrences)

·         Writes all the output to D:\templogs\output.csv

 

As before, the output will be sorted from the least number of repeating occurences to greatest, but now it's in a nifty csv format that you use Excel to do all sorts of fancy sorting.

 

Note: this post will probably be obsolote in the next 15 minutes, as Brad will likely re-write this in assembly next.

 

Update: you'll have to put the output.csv file into a different directory from the logs that you're trying to parse.  Otherwise, you'll get into an endless loop where we try to parse the output.csv file as well.

 

strings.exe -q -n 16 D:\templogs\*.log | foreach-object { ($_.Split(":".ToCharArray(),3)[2]) }| group-object | select-object count,name | sort count | export-csv C:\temp\output.csv

Comments
  • *Anything* is possible with Powershell :-) ... Let us know what you come up with!

    Regards,

    Scott Oseychik

  • I can get the script to run, however I only get two lines of output:

    Count: 1

    Name:  \temp\strins.exs [-a] [-f offset] [-b bytes] [-n length] [-o] [-q] [-s] [-u] <file or directory>

    No other output, what am I missing?

  • Hi Dave,

    I believe you have a syntax error in your example; change 'strins' to 'strings', and you should be good to go.

    Regards,

    Scott Oseychik

  • Scott, I believe I Strings.exe spelled correctly, however my question is was this designed for Exchange 2007 logs, as we are running 2010.  My memory fails me as to when MS moved from 512kb logs to 1024?  Do I need to put any values into the -b or -n fields?  Thanks for the assistance and quick response.  

  • Hi Dave,

    I was inferring the syntax error based on your original comment.  Also the size of the logfile shouldn't be an issue here (we still use this approach against Exchange 20130).

    I guess my next recommendation would be to give the "tried & true" method a spin:

    blogs.msdn.com/.../rough-and-tough-guide-to-identifying-patterns-in-ese-transaction-log-files.aspx

    This will rule out anything .NET and/or PowerShell related.

    Hope this helps!

    Scott Oseychik

  • what do asterisks in a transaction log indicate? encrypted data perhaps?

  • Hi Zeke,

    Simply means that there were asterisks contained in the transaction logs.  Any encrypted info will be just that: encrypted (not obfuscated/redacted via asterisks).

    In my experience, I've often found asterisks as part of a meeting request (or a meeting request acceptance) that was converted to plain text.

    Hope this helps,

    Scott Oseychik

Page 2 of 2 (22 items) 12
Leave a Comment
  • Please add 3 and 8 and type the answer here:
  • Post