IIS 7.0 provides the FREB method to get verbose requests handling information for all contents file and status code. If we have identified the exact content type and status code for investigation, checking FREB information will not be a challenge.
However, if the status code is general or the content type is not certain when we want to narrow down a problem, we have to enable FREB for all contents and a range of status code, on a busy server, the FREB results may contain quite a lot of xml files, such as 5000 or more in a short time. In this situation, if we want to group all client IP addresses and know which one used specific ASP.NET Session ID, or how various clients have been written customized module parameters, it will be difficult because the complicated XML files are not like CSV formatted files which can be fully parsed by log parser or imported to SQL tables for analysis easily.
In this situation, we can use the Powershell to quickly get information we want. Powershell is good at String Regular Expression, using select-string to get matched strings from file, format-table –groupby to groupby key words we wanted, or use Powershell programming to proceed data further on the filtered output.
Below are some simple scenarios on how we can use Powershell to retrieve data:
1. Find out all FREB files which Headervalue which contains IP addresses:
Command:
select-string "headervalue"">\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}" *.xml
Result:
fr2007423.xml:1821: <Data Name="HeaderValue">10.11.244.93:1262</Data>
fr2007424.xml:1363: <Data Name="HeaderValue">10.11.216.87:1817</Data>
fr2007425.xml:1821: <Data Name="HeaderValue">10.11.244.93:1264</Data>
fr2007700.xml:1545: <Data Name="HeaderValue">10.11.218.141:1190</Data>
2. Group FREB xml files by Application Pools:
Command:
select-string "appPoolId=" *.xml |format-table filename -groupby line
Result:
Line: appPoolId="DefaultAppPool"
Filename
--------
fr2007423.xml
fr2007424.xml
Line: appPoolId="TestAppPool"
Filename
--------
fr2007425.xml
fr2007700.xml
3. Group FREB xml files by “set-cookie” behavior, and find out which client IP received set-cookie requirement from server side from the group result.
When FREB number is limited, this requirement can be done manually:
a. Run this command first to get the grouped FREB file list under different set-cookie values:
select-string "set-cookie" *.xml |format-table filename -groupby line
Result:
Line: Set-Cookie: ASP.NET_SessionId=nqt543za1jrlsz55whtgllbf; path=/
Filename
--------
fr2007792.xml
Line: Set-Cookie: ASP.NET_SessionId=ah3ezs3wbd23gtfzksbhi2qg; path=/
Filename
--------
fr2007848.xml
b. And then check the two XML files one by one to find out the client IP.
Now if there are plenty of XML files, if we want to quickly go through the result and find interesting data, it’s better to use an automation method. Here I used Powershell programming:
1. Create C# program, add reference to the Powershell assembly:
C:\Program Files\Reference Assemblies\Microsoft\WindowsPowerShell\v1.0\System.Management.Automation.dll
2. Copy below code into the program.cs:
=========================
using System;
using System.Collections;
using System.Collections.ObjectModel;
using System.Text;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
using System.Text.RegularExpressions;
using System.Runtime.InteropServices;
using System.Configuration;
namespace DigFreb
{
class Program
{
static void Main(string[] args)
{
RunspaceInvoke invoker = new RunspaceInvoke();
string[] input = {@ConfigurationSettings.AppSettings["groupbykey"].ToString()};
string[] input2 = {@ConfigurationSettings.AppSettings["inlinekey"].ToString()};
string path = ConfigurationSettings.AppSettings["path"].ToString(); ;
IList errors;
string scriptBlock =
"(select-string $input -path " + path + "*.* " + "| ft filename -groupby line |out-string -stream)";
string matchPattern = "fr[0-9]*.xml";
Regex RE = new Regex(matchPattern, RegexOptions.Singleline);
string temp;
foreach (PSObject thisResult in
invoker.Invoke(scriptBlock, input, out errors))
{
temp = thisResult.ToString ();
if (temp.Length != 0)
{
Console.WriteLine("{0}", temp);
}
if (RE.IsMatch (temp) )
{ //Get data from subitems
// Create and open a runspace
Runspace runspace = RunspaceFactory.CreateRunspace();
runspace.Open();
// Create an empty pipeline
Pipeline pipeline = runspace.CreatePipeline();
Command dirCommand = new Command("select-string");
dirCommand.Parameters.Add("path",
path +temp);
dirCommand.Parameters.Add("pattern",input2[0] );
// Add the command to the pipeline
pipeline.Commands.Add(dirCommand);
Collection<PSObject> results = pipeline.Invoke();
foreach (PSObject tempResult in results)
{
Console.WriteLine("*****Found*****");
Console.WriteLine(tempResult.ToString());
Console.WriteLine("");
}
}
}
foreach (object thisError in errors)
{
Console.WriteLine("Error: {0}", thisError);
}
}
}
}
=======================================
3. Add app.config to allow you set different query strings in powershell out of the application:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<add key="groupbykey" value="set-cookie" />
<add key="inlinekey" value="headervalue">\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}" />
<add key="path" value="h:\temp\freb\" />
</appSettings>
<!--
< < less than
> greater than
& & ampersand
' ' apostrophe
" " quotation mark
-->
</configuration>
Compile the project as digfreb.exe, then you can use the .EXE easily to do below jobs once a time:
1. Group FREBs by a groupbykey
2. Go through grouped XML files and output interesting data by using the inlinekey.
From example:
Digfreb.exe >1.txt
Open the 1.txt, the result is as below, we can get the grouped information and how sub-items contain the interesting data we need:
Line: Set-Cookie: ASP.NET_SessionId=n43goc45yisyl3551ifuvi55; path=/
Filename
--------
fr2297517.xml
*****Found*****
H:\temp\freb\fr2007517.xml :1363: <Data Name="HeaderValue">10.18.220.232:1535</Data>
Line: Set-Cookie: ASP.NET_SessionId=mkecgo55giwapv55rtyphtna; path=/
Filename
--------
fr2297687.xml
*****Found*****
H:\temp\freb\fr2007687.xml :1545: <Data Name="HeaderValue">10.18.218.141:1189</Data>
Line: Set-Cookie: IIS40P=b7dd6f18899b677f8ea9b8dc3a41c3b02bb1542fa1ba9dae7b59
3f1406bd431b;Path=/
Filename
--------
fr2297701.xml
*****Found*****
H:\temp\freb\fr2007701.xml :1821: <Data Name="HeaderValue">10.18.218.141:1190</Data>
H:\temp\freb\fr007833.xml :1820: <Data Name="HeaderValue">10.18.220.55:1540</Data>
Regards,
Freist Li
