You may have recently heard about the new GC Notification feature that got introduced in .NET Framework 3.5 SP1 and CLR4.0. To get a better understanding of the feature and PI you may look at:  http://msdn.microsoft.com/en-us/library/system.gcnotificationstatus.aspx  and http://msdn.microsoft.com/en-us/library/system.gc.aspx.

At first look it sounded very interesting because I thought it might be used to take a web server out of the load balancer farm when it needs to perform a full GC Collect that might introduce latency for users. Most of load balancers like ARR http://www.iis.net/expand/ApplicationRequestRouting can remove temporarily a server from the web farm pool under certain conditions. One of these conditions can from a page that can be pooled at a regular basis. And if this pages does not contain some specific text data the server is taken out until the page says the server is operational. In ARR this is referred as the “Health Test Page” http://technet.microsoft.com/fr-fr/library/dd443513(WS.10).aspx.

I asked myself if “could we use this new GC notification feature to take a web server out of the farm using this mechanism when GC collections occurs”?

There as already some geeks (http://geekswithblogs.net/sdorman/archive/2008/11/07/clr-4.0-garbage-collection-changes.aspx ) out there that tried to tackle this feature but I could find one on ASP.NET.

The following article is my personal testing and research on using this feature in a web server environment so if you are interested in my findings read on!

 

Please note that full GC notification is available for both workstation and server GC but concurrent GC needs to be disabled to get this feature enabled.

 

In web server scenarios we are usually and by default using server GC. That being said there are certain cases such as web hosting where workstation GC can help. I am not going to cover more in this article when to use Workstation or Server GC but Workstation GC with concurrent GC enabled is mainly for UI apps; many non UI desktop apps (like services and what not) should be using wks GC with concurrent GC disabled.

As a result for my testing I used both Server and workstation GC and disabled concurrent GC. There are two ways to do that the first way is to modify the %windir%\Microsoft.NET\Framework{bitness}\{frameworkversion}\Aspnet.config  aspnet.config file  (that is global to the server) and the other way on IIS7.5 (Windows 2008 R2 and Windows 7 client) is to use the CLRConfigFile attribute that allows to externalize for a specific application pool.

 

Sample Aspnet.conf file

<?xml version="1.0" encoding="UTF-8" ?>

<configuration>

    <runtime>

        <legacyUnhandledExceptionPolicy enabled="false" />

        <legacyImpersonationPolicy enabled="true"/>

        <alwaysFlowImpersonationPolicy enabled="false"/>

        <SymbolReadingPolicy enabled="1" />

<!--

            <gcServer enabled="true" />

-->

            <gcServer enabled="false" />

<!--

            <gcTrimCommitOnLowMemory enabled="true" />

-->

            <gcConcurrent enabled="false" />

 

    </runtime>

</configuration>

 

Sample IIS7 - Add the following in applicationHost.config file

<applicationPools>

            <add name=”WebNot”          CLRConfigFile=”c:\myConfig\CLRConfigFile.txt” />

<applicationPools>

 

The CLRConfigFile.txt has then exactly the same syntax as the Aspnet.config file above.

There are multiple ways to find what GC you are running in a process.  My favorite is using Windbg and the Son Of a Strike extension (SoS). Windbg can be as part of the Windows Debugging tools at: http://www.microsoft.com/whdc/devtools/debugging/default.mspx .

 

!Threads -special

ThreadCount: 11

UnstartedThread: 0

BackgroundThread: 10

PendingThread: 0

DeadThread: 0

Hosted Runtime: no

                                      PreEmptive   GC Alloc           Lock

       ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception

   8    1 1ee8 0162f618      8220 Enabled  0237ba8c:0237d004 0160cca8     0 Ukn

  22    2  38c 0163b288      b220 Enabled  00000000:00000000 0160cca8     0 MTA (Finalizer)

  24    3 1c04 01663de0    80a220 Enabled  00000000:00000000 0160cca8     0 MTA (Threadpool Completion Port)

  25    4  808 01666bd0      1220 Enabled  00000000:00000000 0160cca8     0 Ukn

  17    5  6fc 016879f8   880a220 Enabled  063af204:063b0fe8 0160cca8     0 MTA (Threadpool Completion Port)

  26    6 14d8 0169b590      b020 Enabled  0636d014:0636efe8 01667278     0 MTA

  27    7 1b08 016bae30   180b220 Enabled  0241b068:0241d008 0160cca8     0 MTA (Threadpool Worker)

   9    8 1d9c 016c4948       220 Enabled  063ad084:063af000 0160cca8     0 Ukn

  30    9  7ec 0f882bc8   180b220 Enabled  023f4998:023f4fe8 0160cca8     0 MTA (Threadpool Worker)

   3    a 19a8 016b36c8       220 Enabled  023f71b8:023f8fe8 0160cca8     0 Ukn

  28    b 11d8 016cb040       220 Enabled  063b105c:063b2fe8 0160cca8     0 Ukn

 

       OSID     Special thread type

   17    6fc    IOCompletion

   19   1adc    DbgHelper

   20    668    GC

   21   15c4    GC

   22    38c    Finalizer

   24   1c04    Timer

   25    808    ADUnloadHelper

   27   1b08    ThreadpoolWorker

   30    7ec    ThreadpoolWorker

 

0:032> ~20s

eax=01c9fdb8 ebx=00000000 ecx=6791f128 edx=00000000 esi=00000694 edi=00000000

eip=772c64f4 esp=01c9fd00 ebp=01c9fd6c iopl=0         nv up ei pl zr na pe nc

cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246

ntdll!KiFastSystemCallRet:

772c64f4 c3              ret

0:020> k

ChildEBP RetAddr 

01c9fcfc 772c5e6c ntdll!KiFastSystemCallRet

01c9fd00 7568179c ntdll!NtWaitForSingleObject+0xc

01c9fd6c 75cef003 KERNELBASE!WaitForSingleObjectEx+0x98

01c9fd84 673ec661 kernel32!WaitForSingleObjectExImplementation+0x75

01c9fdc8 673ec597 mscorwks!PEImage::LoadImage+0x1af

01c9fe18 673ec5b6 mscorwks!CLREvent::WaitEx+0x117

01c9fe2c 675333b4 mscorwks!CLREvent::Wait+0x17

01c9fe58 6747cfce mscorwks!SVR::gc_heap::gc_thread_function+0x30

01c9fe70 75cf1174 mscorwks!SVR::gc_heap::gc_thread_stub+0x92

01c9fe7c 772db3f5 kernel32!BaseThreadInitThunk+0xe

01c9febc 772db3c8 ntdll!__RtlUserThreadStart+0x70

01c9fed4 00000000 ntdll!_RtlUserThreadStart+0x1b

 

Or the simple way:

.shell -ci "~*kb99" FINDstr gc_heap

mscorwks!SVR::gc_heap::gc_thread_function+0x30

Or

mscorwks!WKS::gc_heap::full_gc_wait+0x27

 

In fact my applicationHost.config file is defined like the bellow on my machine:

 

        <applicationPools>

            <add name="WebNot" CLRConfigFile="C:\inetpub\CLRConfigFile.txt">

                <processModel identityType="LocalSystem" />

            </add>

            <applicationPoolDefaults>

 

This sample app needs to be run in my scenario as LocalSystem because I needed to log to the Windows Event Logs. Make sure you don’t do that in production scenarios (front end applications should run with the least privileges) and use instead the IIS7.5 virtual user for application pool identities. Better tracing can be integrated in production applications by using the using the ETW API (http://learn.iis.net/page.aspx/245/how-to-capture-aspnet-page-trace-events-in-iis7-tracing/ ,  http://blogs.iis.net/chrisad/archive/2007/04/10/tracing-asp-net-provider-on-windows-vista.aspx ). As this is for demo/test purpose this was fine for me in that situation.

 

·         How does this sample code work?

The sample provided bellow is a simple “port” of the console application provided in the MSN article to ASP.NET: http://msdn.microsoft.com/en-us/library/system.gcnotificationstatus.aspx

 

The Global.asax Application_Start procedure will be called when the application starts (usually at the first calling call of default.aspx). It creates a separate thread listening to GC notifications. When a full GC notification arrives it triggers: OnFullGCApproachNotify(). We clear all allocated data; we take at this point the server off the farm by setting Application.Set("ServerCurrentState", false) and take the time. This can then be accessed by the default.aspx page to give the server status to the load balancer. We then call GC.Collect(). When GC.Collect is done it triggers OnFullGCCompleteEndNotify(). At this point, we take the time again make the server available to take new requests and log an application WebNot event that tell us when we completed a GC collection and how long it took.

 

 

Global.asax

<%@ Application Language="C#" %>

 

<script runat="server">

 

 

    // Variable for continual checking in the

    // While loop in the WaitForFullGCProc method.

    static bool checkForNotify = false;

    public static bool serverAvailable = false;

 

    // Variable for ending the example.

    static bool finalExit = false;

    static System.Diagnostics.EventLog appLog;

 

    static DateTime startTime, stopTime;

    static TimeSpan duration;

 

 

    protected void Application_Start(object sender, EventArgs e)

    {

 

        Object o = new Object();

 

        lock (o)

        {

            Application.Set("ServerCurrentState", false);

 

            try

            {

                appLog = new System.Diagnostics.EventLog();

                appLog.Source = "WebNot";

                appLog.WriteEntry("Starting load test.");

               

                GC.RegisterForFullGCNotification(10, 10);

                // Start a thread using WaitForFullGCProc.

                System.Threading.Thread thWaitForFullGC = new System.Threading.Thread(new System.Threading.ThreadStart(WaitForFullGCProc));

                thWaitForFullGC.Start();

 

                Application.Set("ServerCurrentState", true);

                serverAvailable = true;

                checkForNotify = true;

 

                Application.Set("LoadObject", new System.Collections.Generic.List<byte[]>());

 

            }

            catch (OutOfMemoryException)

            {

                Application.Set("ServerCurrentState", false);

                serverAvailable = false;

            }

            catch (InvalidOperationException invalidOp)

            {

                throw new Exception("GC Notifications are not supported while concurrent GC is enabled.\n"

                    + invalidOp.Message);

            }

        }

    }

 

 

    public void WaitForFullGCProc()

    {

        while (true)

        {

            // CheckForNotify is set to true and false in Main.

            while (checkForNotify)

            {

                // Check for a notification of an approaching collection.

                GCNotificationStatus s = GC.WaitForFullGCApproach();

                if (s == GCNotificationStatus.Succeeded)

                {

                    //Console.WriteLine("GC Notifiction raised.");

                    OnFullGCApproachNotify();

                }

                else if (s == GCNotificationStatus.Canceled)

                {

                    break;

                }

                else

                {

                    // This can occur if a timeout period

                    // is specified for WaitForFullGCApproach(Timeout)

                    // or WaitForFullGCComplete(Timeout) 

                    // and the time out period has elapsed.

                    break;

                }

 

                // Check for a notification of a completed collection.

                s = GC.WaitForFullGCComplete();

                if (s == GCNotificationStatus.Succeeded)

                {

                    //Console.WriteLine("GC Notifiction raised.");

                    OnFullGCCompleteEndNotify();

                }

                else if (s == GCNotificationStatus.Canceled)

                {

                    break;

                }

                else

                {

                    // Could be a time out.

                    break;

                }

            }

 

            System.Threading.Thread.Sleep(500);

            // FinalExit is set to true right before 

            // the main thread cancelled notification.

            if (finalExit)

            {

                break;

            }

        }

    }

 

    public void OnFullGCApproachNotify()

    {

 

        //make the server unavailable for the load balancer to process requests

        serverAvailable = false;

        Application.Set("ServerCurrentState", false);

 

 

        // Clear the allocated memory

        Object applicationObject = Application.Get("LoadObject");

        System.Collections.Generic.List<byte[]> load = (System.Collections.Generic.List<byte[]>)applicationObject;

        load.Clear();

       

        // This is a good time to induce a GC collection

        // because the runtime will induce a full GC soon.

        // To be very careful, you can check precede with a

        // check of the GC.GCCollectionCount to make sure

        // a full GC did not already occur since last notified.

 

        startTime = DateTime.Now;

        GC.Collect();

       

    }

 

 

    public void OnFullGCCompleteEndNotify()

    {

        // Method that informs the request queuing server

        // that this server is ready to accept requests again.

        serverAvailable = true;

        Application.Set("ServerCurrentState", true);

       

        stopTime = DateTime.Now;

        duration = stopTime - startTime;

       

        appLog.WriteEntry("Server available - duration:" + duration.Milliseconds.ToString());

    }

 

 

    protected void Session_Start(object sender, EventArgs e)

    {

 

    }

 

    protected void Application_BeginRequest(object sender, EventArgs e)

    {

 

    }

 

    protected void Application_AuthenticateRequest(object sender, EventArgs e)

    {

 

    }

 

    protected void Application_Error(object sender, EventArgs e)

    {

 

    }

 

    protected void Session_End(object sender, EventArgs e)

    {

 

    }

 

    protected void Application_End(object sender, EventArgs e)

    {

        checkForNotify = false;

        finalExit = true;

 

        GC.CancelFullGCNotification();

 

        appLog.WriteEntry("Stopping load test.");

        appLog.Dispose();       

    }

      

</script>

 

 

Default.aspx

<%@ Page Language="C#" %>

 

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

 

<script runat="server">

 

</script>

 

<html xmlns="http://www.w3.org/1999/xhtml">

<head runat="server">

    <title>GC Notification Sample Test Code</title>

</head>

<body>

    <form id="form1" runat="server">

<%

    Object applicationObject = Application.Get("ServerCurrentState");

    bool bServerCurrentState = (bool)applicationObject;

    ServerCurrentState.Text = bServerCurrentState.ToString();

 

    if (Request.QueryString.Get("AddLoad") == "true")

    {

        applicationObject = Application.Get("LoadObject");

        System.Collections.Generic.List<byte[]> load = (System.Collections.Generic.List<byte[]>)applicationObject;

        try

        {

            load.Add(new byte[1000]);

        }

        catch (OutOfMemoryException)

        {

            System.Diagnostics.EventLog appLog =

                new System.Diagnostics.EventLog();

            appLog.Source = "WebNot";

            appLog.WriteEntry("Cant allocate OutOfMemoryException.", System.Diagnostics.EventLogEntryType.Error);

            appLog.Dispose();

            LabelOOM.Text = "OutOfMemoryException can't continue";           

        }

       

        LabelAddLoad.Text = true.ToString();

    }

   

    

%>

    <div>

    <asp:Label runat="server" Text="Server Available:"></asp:Label><asp:Label runat="server" ID="ServerCurrentState" Text=""></asp:Label>

    </div>

    <div>

    <asp:Label ID="LabelAddLoadText" runat="server" Text="Added Memory Load:"></asp:Label><asp:Label runat="server" ID="LabelAddLoad" Text="False"></asp:Label>

    </div>

    <div>

    <asp:Label ID="LabelOOM" runat="server" Text=""></asp:Label>

    </div>

    </form>

</body>

</html>

 

 

In summary:

·         Call the default page to get a the server status (this is the health page a load balancer would call)

o   http://localhost/Default.aspx

·         Call the above page and add memory pressure by adding 1000 bytes to the shared application list.

o   http://localhost/Default.aspx?AddLoad=true

Sample output when calling: http://localhost/Default.aspx?AddLoad=true

Server Available:True

Added Memory Load:True

 

·         Check for WebNot Start/Stop and "Server available - duration:"Y messages. This last message means that the server is now back online to take new requests and it was taken offline for Y milliseconds.

 

Simple test use case:

I ran this over 30 minutes using VS Team Test with 3 users calling the default.aspx page without any think time and allocating data at every call by adding the AddLoad query string variable. You may find bellow the sample code. I also added a verification rule that would trigger me a load test error if the server goes off line.

 

AddLoad.vb

'------------------------------------------------------------------------------

' <auto-generated>

'     This code was generated by a tool.

'     Runtime Version:2.0.50727.4927

'

'     Changes to this file may cause incorrect behavior and will be lost if

'     the code is regenerated.

' </auto-generated>

'------------------------------------------------------------------------------

 

Option Strict Off

Option Explicit On

 

Imports Microsoft.VisualStudio.TestTools.WebTesting

Imports Microsoft.VisualStudio.TestTools.WebTesting.Rules

Imports System

Imports System.Collections.Generic

Imports System.Text

 

Namespace NotSample

 

    Public Class LoadCoded

        Inherits ThreadedWebTest

 

        Public Sub New()

            MyBase.New()

            Me.PreAuthenticate = True

        End Sub

 

        Public Overrides Sub Run()

            Dim request1 As WebTestRequest = New WebTestRequest("http://localhost/default.aspx")

 

            request1.Timeout = 60

            request1.QueryStringParameters.Add("AddLoad", "true")

            If (Me.Context.ValidationLevel >= Microsoft.VisualStudio.TestTools.WebTesting.ValidationLevel.High) Then

                Dim validationRule1 As ValidationRuleFindText = New ValidationRuleFindText

                validationRule1.FindText = """ServerCurrentState"">True"

                validationRule1.IgnoreCase = False

                validationRule1.UseRegularExpression = False

                validationRule1.PassIfTextFound = True

                AddHandler request1.ValidateResponse, AddressOf validationRule1.Validate

            End If

            MyBase.Send(request1)

            request1 = Nothing

        End Sub

    End Class

End Namespace

 

 

Results:

·         Using Workstation GC

logparser "select TimeWritten, Message  from application where SourceName like '%WebNot%' and TimeWritten >= TO_TIMESTAMP( '2009-12-11 11:18:45', 'yyyy-MM-dd hh:mm:ss' ) and TimeWritten <= TO_TIMESTAMP( '2009-12-11 11:49:00', 'yyyy-MM-dd hh:mm:ss' )" -o:DATAGRID

 

2009-12-11 11:18:45  Starting load test.

2009-12-11 11:18:46  Server available - duration:5

2009-12-11 11:18:48  Server available - duration:4

2009-12-11 11:18:52  Server available - duration:2

2009-12-11 11:18:54  Server available - duration:4

2009-12-11 11:18:55  Server available - duration:4

2009-12-11 11:18:58  Server available - duration:4

2009-12-11 11:19:00  Server available - duration:4

2009-12-11 11:19:02  Server available - duration:4

2009-12-11 11:19:05  Server available - duration:4

2009-12-11 11:19:07  Server available - duration:5

2009-12-11 11:19:09  Server available - duration:4

2009-12-11 11:19:11  Server available - duration:4

2009-12-11 11:19:14  Server available - duration:4

2009-12-11 11:19:16  Server available - duration:4

2009-12-11 11:19:18  Server available - duration:4

2009-12-11 11:19:20  Server available - duration:4

2009-12-11 11:19:22  Server available - duration:3

2009-12-11 11:19:23  Server available - duration:4

2009-12-11 11:19:25  Server available - duration:4

2009-12-11 11:19:27  Server available - duration:4

2009-12-11 11:19:29  Server available - duration:4

2009-12-11 11:19:32  Server available - duration:4

2009-12-11 11:19:34  Server available - duration:5

2009-12-11 11:19:36  Server available - duration:3

2009-12-11 11:19:38  Server available - duration:3

2009-12-11 11:19:40  Server available - duration:3

2009-12-11 11:19:43  Server available - duration:3

2009-12-11 11:19:45  Server available - duration:3

2009-12-11 11:19:47  Server available - duration:3

2009-12-11 11:19:49  Server available - duration:3

2009-12-11 11:19:51  Server available - duration:3

2009-12-11 11:19:53  Server available - duration:4

2009-12-11 11:19:55  Server available - duration:4

2009-12-11 11:19:57  Server available - duration:4

2009-12-11 11:19:59  Server available - duration:4

2009-12-11 11:20:01  Server available - duration:3

2009-12-11 11:20:04  Server available - duration:4

2009-12-11 11:20:07  Server available - duration:3

2009-12-11 11:20:09  Server available - duration:4

2009-12-11 11:20:11  Server available - duration:3

2009-12-11 11:20:13  Server available - duration:3

2009-12-11 11:20:15  Server available - duration:4

2009-12-11 11:20:18  Server available - duration:4

2009-12-11 11:20:19  Server available - duration:4

2009-12-11 11:20:21  Server available - duration:4

2009-12-11 11:20:23  Server available - duration:4

2009-12-11 11:20:26  Server available - duration:4

2009-12-11 11:20:28  Server available - duration:4

2009-12-11 11:20:30  Server available - duration:4

2009-12-11 11:20:32  Server available - duration:3

2009-12-11 11:20:34  Server available - duration:4

2009-12-11 11:20:36  Server available - duration:4

2009-12-11 11:20:38  Server available - duration:4

2009-12-11 11:20:40  Server available - duration:4

2009-12-11 11:20:43  Server available - duration:4

2009-12-11 11:20:45  Server available - duration:4

2009-12-11 11:20:47  Server available - duration:3

2009-12-11 11:20:49  Server available - duration:3

2009-12-11 11:20:51  Server available - duration:5

2009-12-11 11:20:53  Server available - duration:3

2009-12-11 11:20:55  Server available - duration:4

2009-12-11 11:20:57  Server available - duration:4

2009-12-11 11:21:00  Server available - duration:3

2009-12-11 11:21:02  Server available - duration:4

2009-12-11 11:21:05  Server available - duration:4

2009-12-11 11:21:07  Server available - duration:3

2009-12-11 11:21:09  Server available - duration:3

2009-12-11 11:21:11  Server available - duration:4

2009-12-11 11:21:13  Server available - duration:4

2009-12-11 11:21:16  Server available - duration:4

2009-12-11 11:21:18  Server available - duration:4

2009-12-11 11:21:20  Server available - duration:4

2009-12-11 11:21:23  Server available - duration:6

2009-12-11 11:21:25  Server available - duration:3

2009-12-11 11:21:27  Server available - duration:4

2009-12-11 11:21:30  Server available - duration:4

2009-12-11 11:21:33  Server available - duration:4

2009-12-11 11:21:35  Server available - duration:4

2009-12-11 11:21:37  Server available - duration:4

2009-12-11 11:21:40  Server available - duration:4

2009-12-11 11:21:44  Server available - duration:4

2009-12-11 11:21:46  Server available - duration:3

2009-12-11 11:21:49  Server available - duration:4

2009-12-11 11:21:51  Server available - duration:4

2009-12-11 11:21:54  Server available - duration:3

2009-12-11 11:21:56  Server available - duration:4

2009-12-11 11:21:59  Server available - duration:4

2009-12-11 11:22:02  Server available - duration:5

2009-12-11 11:22:05  Server available - duration:3

2009-12-11 11:22:07  Server available - duration:4

2009-12-11 11:22:10  Server available - duration:4

2009-12-11 11:22:12  Server available - duration:3

2009-12-11 11:22:14  Server available - duration:3

2009-12-11 11:22:16  Server available - duration:4

2009-12-11 11:22:18  Server available - duration:4

2009-12-11 11:22:20  Server available - duration:4

2009-12-11 11:22:24  Server available - duration:4

2009-12-11 11:22:26  Server available - duration:4

2009-12-11 11:22:28  Server available - duration:3

2009-12-11 11:22:29  Server available - duration:3

2009-12-11 11:22:32  Server available - duration:4

2009-12-11 11:22:34  Server available - duration:4

2009-12-11 11:22:36  Server available - duration:4

2009-12-11 11:22:38  Server available - duration:3

2009-12-11 11:22:40  Server available - duration:3

2009-12-11 11:22:42  Server available - duration:4

2009-12-11 11:22:45  Server available - duration:21

2009-12-11 11:22:47  Server available - duration:4

2009-12-11 11:22:49  Server available - duration:4

2009-12-11 11:22:52  Server available - duration:4

2009-12-11 11:22:54  Server available - duration:4

2009-12-11 11:22:56  Server available - duration:4

2009-12-11 11:22:58  Server available - duration:4

2009-12-11 11:23:00  Server available - duration:3

2009-12-11 11:23:02  Server available - duration:4

2009-12-11 11:23:04  Server available - duration:4

2009-12-11 11:23:06  Server available - duration:4

2009-12-11 11:23:08  Server available - duration:4

2009-12-11 11:23:10  Server available - duration:4

2009-12-11 11:23:12  Server available - duration:4

2009-12-11 11:23:14  Server available - duration:4

2009-12-11 11:23:16  Server available - duration:4

2009-12-11 11:23:18  Server available - duration:4

2009-12-11 11:23:20  Server available - duration:3

2009-12-11 11:23:22  Server available - duration:4

2009-12-11 11:23:24  Server available - duration:4

2009-12-11 11:23:26  Server available - duration:4

2009-12-11 11:23:28  Server available - duration:4

2009-12-11 11:23:31  Server available - duration:4

2009-12-11 11:23:34  Server available - duration:4

2009-12-11 11:23:35  Server available - duration:5

2009-12-11 11:23:38  Server available - duration:4

2009-12-11 11:23:40  Server available - duration:4

2009-12-11 11:23:42  Server available - duration:4

2009-12-11 11:23:45  Server available - duration:4

2009-12-11 11:23:47  Server available - duration:4

2009-12-11 11:23:49  Server available - duration:3

2009-12-11 11:23:51  Server available - duration:4

2009-12-11 11:23:53  Server available - duration:4

2009-12-11 11:23:55  Server available - duration:4

2009-12-11 11:23:58  Server available - duration:4

2009-12-11 11:24:00  Server available - duration:60

2009-12-11 11:24:02  Server available - duration:4

2009-12-11 11:24:05  Server available - duration:4

2009-12-11 11:24:07  Server available - duration:3

2009-12-11 11:24:09  Server available - duration:4

2009-12-11 11:24:12  Server available - duration:4

2009-12-11 11:24:14  Server available - duration:4

2009-12-11 11:24:16  Server available - duration:5

2009-12-11 11:24:18  Server available - duration:4

2009-12-11 11:24:20  Server available - duration:5

2009-12-11 11:24:23  Server available - duration:4

2009-12-11 11:24:25  Server available - duration:3

2009-12-11 11:24:27  Server available - duration:3

2009-12-11 11:24:30  Server available - duration:4

2009-12-11 11:24:32  Server available - duration:3

2009-12-11 11:24:35  Server available - duration:4

2009-12-11 11:24:38  Server available - duration:3

2009-12-11 11:24:40  Server available - duration:4

2009-12-11 11:24:42  Server available - duration:3

2009-12-11 11:24:44  Server available - duration:4

2009-12-11 11:24:46  Server available - duration:4

2009-12-11 11:24:49  Server available - duration:4

2009-12-11 11:24:51  Server available - duration:4

2009-12-11 11:24:54  Server available - duration:3

2009-12-11 11:24:56  Server available - duration:4

2009-12-11 11:24:58  Server available - duration:4

2009-12-11 11:25:01  Server available - duration:4

2009-12-11 11:25:03  Server available - duration:4

2009-12-11 11:25:06  Server available - duration:3

2009-12-11 11:25:08  Server available - duration:4

2009-12-11 11:25:10  Server available - duration:3

2009-12-11 11:25:12  Server available - duration:4

2009-12-11 11:25:13  Server available - duration:4

2009-12-11 11:25:16  Server available - duration:4

2009-12-11 11:25:18  Server available - duration:4

2009-12-11 11:25:21  Server available - duration:3

2009-12-11 11:25:24  Server available - duration:7

2009-12-11 11:25:26  Server available - duration:5

2009-12-11 11:25:29  Server available - duration:4

2009-12-11 11:25:31  Server available - duration:4

2009-12-11 11:25:33  Server available - duration:4

2009-12-11 11:25:36  Server available - duration:4

2009-12-11 11:25:38  Server available - duration:3

2009-12-11 11:25:40  Server available - duration:3

2009-12-11 11:25:43  Server available - duration:4

2009-12-11 11:25:45  Server available - duration:4

2009-12-11 11:25:47  Server available - duration:4

2009-12-11 11:25:49  Server available - duration:4

2009-12-11 11:25:51  Server available - duration:3

2009-12-11 11:25:54  Server available - duration:5

2009-12-11 11:25:56  Server available - duration:4

2009-12-11 11:25:58  Server available - duration:3

2009-12-11 11:26:01  Server available - duration:3

2009-12-11 11:26:04  Server available - duration:4

2009-12-11 11:26:07  Server available - duration:7

2009-12-11 11:26:09  Server available - duration:3

2009-12-11 11:26:11  Server available - duration:4

2009-12-11 11:26:13  Server available - duration:3

2009-12-11 11:26:15  Server available - duration:3

2009-12-11 11:26:17  Server available - duration:4

2009-12-11 11:26:19  Server available - duration:4

2009-12-11 11:26:21  Server available - duration:3

2009-12-11 11:26:24  Server available - duration:4

2009-12-11 11:26:27  Server available - duration:4

2009-12-11 11:26:29  Server available - duration:3

2009-12-11 11:26:32  Server available - duration:4

2009-12-11 11:26:34  Server available - duration:4

2009-12-11 11:26:36  Server available - duration:3

2009-12-11 11:26:39  Server available - duration:3

2009-12-11 11:26:41  Server available - duration:4

2009-12-11 11:26:44  Server available - duration:4

2009-12-11 11:26:46  Server available - duration:4

2009-12-11 11:26:49  Server available - duration:3

2009-12-11 11:26:51  Server available - duration:4

2009-12-11 11:26:53  Server available - duration:4

2009-12-11 11:26:55  Server available - duration:4

2009-12-11 11:26:58  Server available - duration:3

2009-12-11 11:27:01  Server available - duration:4

2009-12-11 11:27:05  Server available - duration:4

2009-12-11 11:27:07  Server available - duration:4

2009-12-11 11:27:09  Server available - duration:5

2009-12-11 11:27:11  Server available - duration:5

2009-12-11 11:27:13  Server available - duration:4

2009-12-11 11:27:16  Server available - duration:4

2009-12-11 11:27:18  Server available - duration:3

2009-12-11 11:27:20  Server available - duration:4

2009-12-11 11:27:24  Server available - duration:5

2009-12-11 11:27:26  Server available - duration:17

2009-12-11 11:27:28  Server available - duration:4

2009-12-11 11:27:31  Server available - duration:4

2009-12-11 11:27:34  Server available - duration:4

2009-12-11 11:27:36  Server available - duration:3

2009-12-11 11:27:39  Server available - duration:3

2009-12-11 11:27:42  Server available - duration:4

2009-12-11 11:27:45  Server available - duration:4

2009-12-11 11:27:48  Server available - duration:4

2009-12-11 11:27:51  Server available - duration:4

2009-12-11 11:27:54  Server available - duration:4

2009-12-11 11:27:57  Server available - duration:4

2009-12-11 11:27:59  Server available - duration:4

2009-12-11 11:28:02  Server available - duration:3

2009-12-11 11:28:05  Server available - duration:4

2009-12-11 11:28:08  Server available - duration:4

2009-12-11 11:28:10  Server available - duration:3

2009-12-11 11:28:12  Server available - duration:3

2009-12-11 11:28:14  Server available - duration:4

2009-12-11 11:28:16  Server available - duration:3

2009-12-11 11:28:18  Server available - duration:4

2009-12-11 11:28:21  Server available - duration:3

2009-12-11 11:28:23  Server available - duration:5

2009-12-11 11:28:25  Server available - duration:4

2009-12-11 11:28:27  Server available - duration:4

2009-12-11 11:28:29  Server available - duration:3

2009-12-11 11:28:31  Server available - duration:4

2009-12-11 11:28:33  Server available - duration:4

2009-12-11 11:28:36  Server available - duration:3

2009-12-11 11:28:38  Server available - duration:4

2009-12-11 11:28:40  Server available - duration:4

2009-12-11 11:28:43  Server available - duration:5

2009-12-11 11:28:45  Server available - duration:3

2009-12-11 11:28:47  Server available - duration:3

2009-12-11 11:28:49  Server available - duration:5

2009-12-11 11:28:51  Server available - duration:3

2009-12-11 11:28:53  Server available - duration:4

2009-12-11 11:28:55  Server available - duration:3

2009-12-11 11:28:57  Server available - duration:3

2009-12-11 11:29:00  Server available - duration:4

2009-12-11 11:29:01  Server available - duration:4

2009-12-11 11:29:04  Server available - duration:4

2009-12-11 11:29:06  Server available - duration:3

2009-12-11 11:29:08  Server available - duration:4

2009-12-11 11:29:10  Server available - duration:3

2009-12-11 11:29:13  Server available - duration:4

2009-12-11 11:29:15  Server available - duration:4

2009-12-11 11:29:18  Server available - duration:4

2009-12-11 11:29:20  Server available - duration:4

2009-12-11 11:29:21  Server available - duration:4

2009-12-11 11:29:23  Server available - duration:4

2009-12-11 11:29:25  Server available - duration:4

2009-12-11 11:29:27  Server available - duration:4

2009-12-11 11:29:30  Server available - duration:4

2009-12-11 11:29:32  Server available - duration:4

2009-12-11 11:29:34  Server available - duration:3

2009-12-11 11:29:37  Server available - duration:3

2009-12-11 11:29:39  Server available - duration:4

2009-12-11 11:29:41  Server available - duration:4

2009-12-11 11:29:44  Server available - duration:4

2009-12-11 11:29:46  Server available - duration:4

2009-12-11 11:29:48  Server available - duration:4

2009-12-11 11:29:50  Server available - duration:4

2009-12-11 11:29:52  Server available - duration:4

2009-12-11 11:29:54  Server available - duration:4

2009-12-11 11:29:56  Server available - duration:4

2009-12-11 11:29:58  Server available - duration:3

2009-12-11 11:30:00  Server available - duration:5

2009-12-11 11:30:04  Server available - duration:4

2009-12-11 11:30:05  Server available - duration:4

2009-12-11 11:30:07  Server available - duration:4

2009-12-11 11:30:10  Server available - duration:3

2009-12-11 11:30:12  Server available - duration:3

2009-12-11 11:30:14  Server available - duration:4

2009-12-11 11:30:16  Server available - duration:4

2009-12-11 11:30:19  Server available - duration:4

2009-12-11 11:30:21  Server available - duration:4

2009-12-11 11:30:24  Server available - duration:4

2009-12-11 11:30:26  Server available - duration:3

2009-12-11 11:30:28  Server available - duration:4

2009-12-11 11:30:31  Server available - duration:4

2009-12-11 11:30:33  Server available - duration:5

2009-12-11 11:30:36  Server available - duration:4

2009-12-11 11:30:38  Server available - duration:4

2009-12-11 11:30:41  Server available - duration:6

2009-12-11 11:30:45  Server available - duration:4

2009-12-11 11:30:48  Server available - duration:4

2009-12-11 11:30:50  Server available - duration:3

2009-12-11 11:30:52  Server available - duration:4

2009-12-11 11:30:55  Server available - duration:4

2009-12-11 11:30:57  Server available - duration:5

2009-12-11 11:31:00  Server available - duration:2

2009-12-11 11:31:02  Server available - duration:2

2009-12-11 11:31:04  Server available - duration:4

2009-12-11 11:31:07  Server available - duration:4

2009-12-11 11:31:09  Server available - duration:4

2009-12-11 11:31:12  Server available - duration:5

2009-12-11 11:31:14  Server available - duration:4

2009-12-11 11:31:15  Server available - duration:4

2009-12-11 11:31:18  Server available - duration:5

2009-12-11 11:31:20  Server available - duration:4

2009-12-11 11:31:23  Server available - duration:4

2009-12-11 11:31:26  Server available - duration:4

2009-12-11 11:31:28  Server available - duration:5

2009-12-11 11:31:31  Server available - duration:5

2009-12-11 11:31:33  Server available - duration:4

2009-12-11 11:31:35  Server available - duration:4

2009-12-11 11:31:38  Server available - duration:4

2009-12-11 11:31:40  Server available - duration:4

2009-12-11 11:31:42  Server available - duration:4

2009-12-11 11:31:44  Server available - duration:3

2009-12-11 11:31:46  Server available - duration:5

2009-12-11 11:31:48  Server available - duration:4

2009-12-11 11:31:51  Server available - duration:4

2009-12-11 11:31:53  Server available - duration:3

2009-12-11 11:31:56  Server available - duration:3

2009-12-11 11:31:58  Server available - duration:6

2009-12-11 11:32:00  Server available - duration:4

2009-12-11 11:32:04  Server available - duration:4

2009-12-11 11:32:05  Server available - duration:5

2009-12-11 11:32:08  Server available - duration:4

2009-12-11 11:32:10  Server available - duration:6

2009-12-11 11:32:12  Server available - duration:5

2009-12-11 11:32:15  Server available - duration:4

2009-12-11 11:32:17  Server available - duration:5

2009-12-11 11:32:20  Server available - duration:4

2009-12-11 11:32:22  Server available - duration:4

2009-12-11 11:32:26  Server available - duration:4

2009-12-11 11:32:29  Server available - duration:4

2009-12-11 11:32:31  Server available - duration:4

2009-12-11 11:32:33  Server available - duration:5

2009-12-11 11:32:36  Server available - duration:4

2009-12-11 11:32:38  Server available - duration:4

2009-12-11 11:32:41  Server available - duration:6

2009-12-11 11:32:43  Server available - duration:5

2009-12-11 11:32:45  Server available - duration:6

2009-12-11 11:32:48  Server available - duration:4

2009-12-11 11:32:50  Server available - duration:5

2009-12-11 11:32:53  Server available - duration:4

2009-12-11 11:32:55  Server available - duration:5

2009-12-11 11:32:57  Server available - duration:5

2009-12-11 11:33:00  Server available - duration:4

2009-12-11 11:33:04  Server available - duration:4

2009-12-11 11:33:05  Server available - duration:5

2009-12-11 11:33:08  Server available - duration:3

2009-12-11 11:33:10  Server available - duration:4

2009-12-11 11:33:13  Server available - duration:4

2009-12-11 11:33:15  Server available - duration:5

2009-12-11 11:33:18  Server available - duration:6

2009-12-11 11:33:20  Server available - duration:4

2009-12-11 11:33:24  Server available - duration:3

2009-12-11 11:33:25  Server available - duration:4

2009-12-11 11:33:28  Server available - duration:4

2009-12-11 11:33:30  Server available - duration:6

2009-12-11 11:33:33  Server available - duration:4

2009-12-11 11:33:35  Server available - duration:5

2009-12-11 11:33:38  Server available - duration:4

2009-12-11 11:33:40  Server available - duration:4

2009-12-11 11:33:43  Server available - duration:4

2009-12-11 11:33:45  Server available - duration:5

2009-12-11 11:33:47  Server available - duration:4

2009-12-11 11:33:49  Server available - duration:4

2009-12-11 11:33:52  Server available - duration:4

2009-12-11 11:33:55  Server available - duration:5

2009-12-11 11:33:57  Server available - duration:4

2009-12-11 11:34:00  Server available - duration:5

2009-12-11 11:34:02  Server available - duration:4

2009-12-11 11:34:05  Server available - duration:4

2009-12-11 11:34:07  Server available - duration:4

2009-12-11 11:34:10  Server available - duration:4

2009-12-11 11:34:11  Server available - duration:4

2009-12-11 11:34:14  Server available - duration:4

2009-12-11 11:34:16  Server available - duration:4

2009-12-11 11:34:19  Server available - duration:6

2009-12-11 11:34:21  Server available - duration:6

2009-12-11 11:34:23  Server available - duration:3

2009-12-11 11:34:25  Server available - duration:2

2009-12-11 11:34:27  Server available - duration:3

2009-12-11 11:34:28  Server available - duration:3

2009-12-11 11:34:30  Server available - duration:2

2009-12-11 11:34:31  Server available - duration:2

2009-12-11 11:34:32  Server available - duration:3

2009-12-11 11:34:33  Server available - duration:3

2009-12-11 11:34:34  Server available - duration:3

2009-12-11 11:34:35  Server available - duration:2

2009-12-11 11:34:37  Server available - duration:2

2009-12-11 11:34:37  Server available - duration:2

2009-12-11 11:34:39  Server available - duration:3

2009-12-11 11:34:39  Server available - duration:2

2009-12-11 11:34:41  Server available - duration:3

2009-12-11 11:34:42  Server available - duration:2

2009-12-11 11:34:44  Server available - duration:5

2009-12-11 11:34:46  Server available - duration:2

2009-12-11 11:34:47  Server available - duration:3

2009-12-11 11:34:49  Server available - duration:2

2009-12-11 11:34:50  Server available - duration:2

2009-12-11 11:34:51  Server available - duration:3

2009-12-11 11:34:52  Server available - duration:3

2009-12-11 11:34:53  Server available - duration:3

2009-12-11 11:34:56  Server available - duration:3

2009-12-11 11:34:57  Server available - duration:2

2009-12-11 11:34:58  Server available - duration:3

2009-12-11 11:35:00  Server available - duration:7

2009-12-11 11:35:01  Server available - duration:2

2009-12-11 11:35:04  Server available - duration:2

2009-12-11 11:35:06  Server available - duration:3

2009-12-11 11:35:08  Server available - duration:3

2009-12-11 11:35:09  Server available - duration:3

2009-12-11 11:35:10  Server available - duration:3

2009-12-11 11:35:12  Server available - duration:2

2009-12-11 11:35:13  Server available - duration:4

2009-12-11 11:35:15  Server available - duration:3

2009-12-11 11:35:17  Server available - duration:2

2009-12-11 11:35:18  Server available - duration:3

2009-12-11 11:35:19  Server available - duration:5

2009-12-11 11:35:21  Server available - duration:2

2009-12-11 11:35:22  Server available - duration:3

2009-12-11 11:35:24  Server available - duration:5

2009-12-11 11:35:26  Server available - duration:2

2009-12-11 11:35:29  Server available - duration:3

2009-12-11 11:35:29  Server available - duration:2

2009-12-11 11:35:31  Server available - duration:3

2009-12-11 11:35:32  Server available - duration:3

2009-12-11 11:35:33  Server available - duration:2

2009-12-11 11:35:35  Server available - duration:2

2009-12-11 11:35:37  Server available - duration:5

2009-12-11 11:35:38  Server available - duration:3

2009-12-11 11:35:40  Server available - duration:2

2009-12-11 11:35:40  Server available - duration:3

2009-12-11 11:35:44  Server available - duration:3

2009-12-11 11:35:45  Server available - duration:3

2009-12-11 11:35:46  Server available - duration:2

2009-12-11 11:35:49  Server available - duration:3

2009-12-11 11:35:49  Server available - duration:3

2009-12-11 11:35:51  Server available - duration:4

2009-12-11 11:35:52  Server available - duration:2

2009-12-11 11:35:54  Server available - duration:3

2009-12-11 11:35:56  Server available - duration:3

2009-12-11 11:35:58  Server available - duration:3

2009-12-11 11:35:59  Server available - duration:2

2009-12-11 11:36:00  Server available - duration:3

2009-12-11 11:36:01  Server available - duration:3

2009-12-11 11:36:03  Server available - duration:2

2009-12-11 11:36:04  Server available - duration:4

2009-12-11 11:36:05  Server available - duration:2

2009-12-11 11:36:08  Server available - duration:3

2009-12-11 11:36:08  Server available - duration:3

2009-12-11 11:36:11  Server available - duration:5

2009-12-11 11:36:12  Server available - duration:2

2009-12-11 11:36:14  Server available - duration:3

2009-12-11 11:36:15  Server available - duration:3

2009-12-11 11:36:17  Server available - duration:3

2009-12-11 11:36:18  Server available - duration:2

2009-12-11 11:36:20  Server available - duration:3

2009-12-11 11:36:21  Server available - duration:4

2009-12-11 11:36:23  Server available - duration:2

2009-12-11 11:36:26  Server available - duration:2

2009-12-11 11:36:26  Server available - duration:2

2009-12-11 11:36:28  Server available - duration:2

2009-12-11 11:36:29  Server available - duration:3

2009-12-11 11:36:30  Server available - duration:2

2009-12-11 11:36:31  Server available - duration:3

2009-12-11 11:36:33  Server available - duration:3

2009-12-11 11:36:34  Server available - duration:2

2009-12-11 11:36:35  Server available - duration:2

2009-12-11 11:36:38  Server available - duration:3

2009-12-11 11:36:39  Server available - duration:3

2009-12-11 11:36:41  Server available - duration:2

2009-12-11 11:36:42  Server available - duration:3

2009-12-11 11:36:44  Server available - duration:3

2009-12-11 11:36:45  Server available - duration:3

2009-12-11 11:36:46  Server available - duration:3

2009-12-11 11:36:48  Server available - duration:3

2009-12-11 11:36:49  Server available - duration:2

2009-12-11 11:36:50  Server available - duration:4

2009-12-11 11:36:52  Server available - duration:3

2009-12-11 11:36:52  Server available - duration:3

2009-12-11 11:36:55  Server available - duration:3

2009-12-11 11:36:55  Server available - duration:3

2009-12-11 11:36:57  Server available - duration:3

2009-12-11 11:36:58  Server available - duration:2

2009-12-11 11:36:59  Server available - duration:2

2009-12-11 11:37:01  Server available - duration:3

2009-12-11 11:37:02  Server available - duration:2

2009-12-11 11:37:03  Server available - duration:3

2009-12-11 11:37:04  Server available - duration:3

2009-12-11 11:37:05  Server available - duration:2

2009-12-11 11:37:06  Server available - duration:3

2009-12-11 11:37:08  Server available - duration:2

2009-12-11 11:37:09  Server available - duration:2

2009-12-11 11:37:10  Server available - duration:2

2009-12-11 11:37:11  Server available - duration:2

2009-12-11 11:37:12  Server available - duration:3

2009-12-11 11:37:14  Server available - duration:3

2009-12-11 11:37:15  Server available - duration:2

2009-12-11 11:37:16  Server available - duration:2

2009-12-11 11:37:18  Server available - duration:3

2009-12-11 11:37:19  Server available - duration:3

2009-12-11 11:37:20  Server available - duration:2

2009-12-11 11:37:22  Server available - duration:5

2009-12-11 11:37:23  Server available - duration:2

2009-12-11 11:37:24  Server available - duration:2

2009-12-11 11:37:26  Server available - duration:3

2009-12-11 11:37:27  Server available - duration:3

2009-12-11 11:37:30  Server available - duration:3

2009-12-11 11:37:30  Server available - duration:3

2009-12-11 11:37:32  Server available - duration:2

2009-12-11 11:37:33  Server available - duration:3

2009-12-11 11:37:35  Server available - duration:2

2009-12-11 11:37:36  Server available - duration:3

2009-12-11 11:37:38  Server available - duration:4

2009-12-11 11:37:40  Server available - duration:2

2009-12-11 11:37:42  Server available - duration:5

2009-12-11 11:37:42  Server available - duration:3

2009-12-11 11:37:45  Server available - duration:6

2009-12-11 11:37:46  Server available - duration:3

2009-12-11 11:37:47  Server available - duration:2

2009-12-11 11:37:48  Server available - duration:2

2009-12-11 11:37:50  Server available - duration:3

2009-12-11 11:37:51  Server available - duration:2

2009-12-11 11:37:53  Server available - duration:3

2009-12-11 11:37:53  Server available - duration:4

2009-12-11 11:37:55  Server available - duration:3

2009-12-11 11:37:56  Server available - duration:3

2009-12-11 11:37:58  Server available - duration:4

2009-12-11 11:37:59  Server available - duration:3

2009-12-11 11:38:02  Server available - duration:3

2009-12-11 11:38:02  Server available - duration:2

2009-12-11 11:38:05  Server available - duration:2

2009-12-11 11:38:06  Server available - duration:2

2009-12-11 11:38:08  Server available - duration:3

2009-12-11 11:38:08  Server available - duration:3

2009-12-11 11:38:10  Server available - duration:3

2009-12-11 11:38:10  Server available - duration:2

2009-12-11 11:38:12  Server available - duration:2

2009-12-11 11:38:14  Server available - duration:3

2009-12-11 11:38:15  Server available - duration:3

2009-12-11 11:38:18  Server available - duration:3

2009-12-11 11:38:19  Server available - duration:2

2009-12-11 11:38:20  Server available - duration:2

2009-12-11 11:38:21  Server available - duration:4

2009-12-11 11:38:22  Server available - duration:3

2009-12-11 11:38:23  Server available - duration:2

2009-12-11 11:38:26  Server available - duration:3

2009-12-11 11:38:27  Server available - duration:3

2009-12-11 11:38:29  Server available - duration:3

2009-12-11 11:38:30  Server available - duration:3

2009-12-11 11:38:31  Server available - duration:2

2009-12-11 11:38:32  Server available - duration:2

2009-12-11 11:38:35  Server available - duration:3

2009-12-11 11:38:36  Server available - duration:3

2009-12-11 11:38:38  Server available - duration:3

2009-12-11 11:38:39  Server available - duration:2

2009-12-11 11:38:41  Server available - duration:3

2009-12-11 11:38:42  Server available - duration:2

2009-12-11 11:38:43  Server available - duration:3

2009-12-11 11:38:44  Server available - duration:3

2009-12-11 11:38:45  Server available - duration:4

2009-12-11 11:38:48  Server available - duration:4

2009-12-11 11:38:50  Server available - duration:2

2009-12-11 11:38:51  Server available - duration:2

2009-12-11 11:38:53  Server available - duration:6

2009-12-11 11:38:54  Server available - duration:2

2009-12-11 11:38:56  Server available - duration:3

2009-12-11 11:38:57  Server available - duration:4

2009-12-11 11:38:58  Server available - duration:2

2009-12-11 11:39:00  Server available - duration:2

2009-12-11 11:39:01  Server available - duration:3

2009-12-11 11:39:04  Server available - duration:3

2009-12-11 11:39:04  Server available - duration:3

2009-12-11 11:39:06  Server available - duration:2

2009-12-11 11:39:07  Server available - duration:4

2009-12-11 11:39:08  Server available - duration:5

2009-12-11 11:39:09  Server available - duration:4

2009-12-11 11:39:10  Server available - duration:4

2009-12-11 11:39:12  Server available - duration:4

2009-12-11 11:39:14  Server available - duration:2

2009-12-11 11:39:16  Server available - duration:3

2009-12-11 11:39:17  Server available - duration:3

2009-12-11 11:39:19  Server available - duration:4

2009-12-11 11:39:21  Server available - duration:2

2009-12-11 11:39:23  Server available - duration:4

2009-12-11 11:39:24  Server available - duration:4

2009-12-11 11:39:26  Server available - duration:4

2009-12-11 11:39:27  Server available - duration:3

2009-12-11 11:39:28  Server available - duration:2

2009-12-11 11:39:29  Server available - duration:3

2009-12-11 11:39:30  Server available - duration:7

2009-12-11 11:39:32  Server available - duration:2

2009-12-11 11:39:34  Server available - duration:2

2009-12-11 11:39:35  Server available - duration:2

2009-12-11 11:39:37  Server available - duration:3

2009-12-11 11:39:38  Server available - duration:3

2009-12-11 11:39:40  Server available - duration:3

2009-12-11 11:39:41  Server available - duration:3

2009-12-11 11:39:43  Server available - duration:2

2009-12-11 11:39:44  Server available - duration:3

2009-12-11 11:39:45  Server available - duration:3

2009-12-11 11:39:46  Server available - duration:3

2009-12-11 11:39:47  Server available - duration:4

2009-12-11 11:39:48  Server available - duration:4

2009-12-11 11:39:49  Server available - duration:3

2009-12-11 11:39:52  Server available - duration:4

2009-12-11 11:39:53  Server available - duration:2

2009-12-11 11:39:54  Server available - duration:2

2009-12-11 11:39:57  Server available - duration:3

2009-12-11 11:39:57  Server available - duration:4

2009-12-11 11:39:59  Server available - duration:2

2009-12-11 11:40:01  Server available - duration:3

2009-12-11 11:40:02  Server available - duration:3

2009-12-11 11:40:05  Server available - duration:3

2009-12-11 11:40:05  Server available - duration:2

2009-12-11 11:40:07  Server available - duration:2

2009-12-11 11:40:08  Server available - duration:4

2009-12-11 11:40:11  Server available - duration:3

2009-12-11 11:40:11  Server available - duration:2

2009-12-11 11:40:13  Server available - duration:2

2009-12-11 11:40:14  Server available - duration:3

2009-12-11 11:40:15  Server available - duration:3

2009-12-11 11:40:16  Server available - duration:4

2009-12-11 11:40:17  Server available - duration:3

2009-12-11 11:40:20  Server available - duration:3

2009-12-11 11:40:21  Server available - duration:2

2009-12-11 11:40:22  Server available - duration:2

2009-12-11 11:40:23  Server available - duration:2

2009-12-11 11:40:25  Server available - duration:3

2009-12-11 11:40:26  Server available - duration:4

2009-12-11 11:40:28  Server available - duration:5

2009-12-11 11:40:30  Server available - duration:2

2009-12-11 11:40:32  Server available - duration:3

2009-12-11 11:40:33  Server available - duration:4

2009-12-11 11:40:34  Server available - duration:2

2009-12-11 11:40:36  Server available - duration:2

2009-12-11 11:40:37  Server available - duration:2

2009-12-11 11:40:39  Server available - duration:2

2009-12-11 11:40:40  Server available - duration:2

2009-12-11 11:40:42  Server available - duration:2

2009-12-11 11:40:43  Server available - duration:2

2009-12-11 11:40:45  Server available - duration:2

2009-12-11 11:40:46  Server available - duration:3

2009-12-11 11:40:48  Server available - duration:3

2009-12-11 11:40:49  Server available - duration:4

2009-12-11 11:40:50  Server available - duration:6

2009-12-11 11:40:51  Server available - duration:3

2009-12-11 11:40:54  Server available - duration:3

2009-12-11 11:40:54  Server available - duration:4

2009-12-11 11:40:56  Server available - duration:2

2009-12-11 11:40:58  Server available - duration:2

2009-12-11 11:40:59  Server available - duration:5

2009-12-11 11:41:00  Server available - duration:3

2009-12-11 11:41:02  Server available - duration:3

2009-12-11 11:41:03  Server available - duration:3

2009-12-11 11:41:04  Server available - duration:2

2009-12-11 11:41:06  Server available - duration:2

2009-12-11 11:41:06  Server available - duration:2

2009-12-11 11:41:08  Server available - duration:3

2009-12-11 11:41:08  Server available - duration:3

2009-12-11 11:41:10  Server available - duration:4

2009-12-11 11:41:11  Server available - duration:3

2009-12-11 11:41:14  Server available - duration:2

2009-12-11 11:41:15  Server available - duration:3

2009-12-11 11:41:17  Server available - duration:8

2009-12-11 11:41:18  Server available - duration:3

2009-12-11 11:41:21  Server available - duration:3

2009-12-11 11:41:22  Server available - duration:3

2009-12-11 11:41:23  Server available - duration:2

2009-12-11 11:41:24  Server available - duration:3

2009-12-11 11:41:26  Server available - duration:2

2009-12-11 11:41:28  Server available - duration:2

2009-12-11 11:41:29  Server available - duration:3

2009-12-11 11:41:30  Server available - duration:2

2009-12-11 11:41:31  Server available - duration:2

2009-12-11 11:41:32  Server available - duration:2

2009-12-11 11:41:33  Server available - duration:3

2009-12-11 11:41:34  Server available - duration:3

2009-12-11 11:41:36  Server available - duration:3

2009-12-11 11:41:38  Server available - duration:2

2009-12-11 11:41:39  Server available - duration:3

2009-12-11 11:41:40  Server available - duration:2

2009-12-11 11:41:42  Server available - duration:3

2009-12-11 11:41:43  Server available - duration:3

2009-12-11 11:41:44  Server available - duration:2

2009-12-11 11:41:45  Server available - duration:2

2009-12-11 11:41:48  Server available - duration:2

2009-12-11 11:41:48  Server available - duration:2

2009-12-11 11:41:51  Server available - duration:2

2009-12-11 11:41:52  Server available - duration:2

2009-12-11 11:41:54  Server available - duration:3

2009-12-11 11:41:55  Server available - duration:2

2009-12-11 11:41:56  Server available - duration:2

2009-12-11 11:41:57  Server available - duration:3

2009-12-11 11:41:58  Server available - duration:2

2009-12-11 11:42:00  Server available - duration:3

2009-12-11 11:42:01  Server available - duration:2

2009-12-11 11:42:03  Server available - duration:5

2009-12-11 11:42:04  Server available - duration:2

2009-12-11 11:42:07  Server available - duration:2

2009-12-11 11:42:08  Server available - duration:5

2009-12-11 11:42:09  Server available - duration:3

2009-12-11 11:42:12  Server available - duration:3

2009-12-11 11:42:12  Server available - duration:2

2009-12-11 11:42:14  Server available - duration:2

2009-12-11 11:42:14  Server available - duration:2

2009-12-11 11:42:16  Server available - duration:2

2009-12-11 11:42:16  Server available - duration:3

2009-12-11 11:42:19  Server available - duration:3

2009-12-11 11:42:20  Server available - duration:2

2009-12-11 11:42:21  Server available - duration:3

2009-12-11 11:42:23  Server available - duration:3

2009-12-11 11:42:24  Server available - duration:3

2009-12-11 11:42:25  Server available - duration:2

2009-12-11 11:42:26  Server available - duration:2

2009-12-11 11:42:28  Server available - duration:3

2009-12-11 11:42:29  Server available - duration:3

2009-12-11 11:42:31  Server available - duration:4

2009-12-11 11:42:32  Server available - duration:3

2009-12-11 11:42:34  Server available - duration:2

2009-12-11 11:42:34  Server available - duration:3

2009-12-11 11:42:36  Server available - duration:4

2009-12-11 11:42:36  Server available - duration:3

2009-12-11 11:42:38  Server available - duration:2

2009-12-11 11:42:38  Server available - duration:3

2009-12-11 11:42:39  Server available - duration:2

2009-12-11 11:42:42  Server available - duration:5

2009-12-11 11:42:42  Server available - duration:2

2009-12-11 11:42:44  Server available - duration:2

2009-12-11 11:42:46  Server available - duration:4

2009-12-11 11:42:47  Server available - duration:3

2009-12-11 11:42:49  Server available - duration:3

2009-12-11 11:42:49  Server available - duration:3

2009-12-11 11:42:50  Server available - duration:2

2009-12-11 11:42:52  Server available - duration:2

2009-12-11 11:42:54  Server available - duration:3

2009-12-11 11:42:56  Server available - duration:2

2009-12-11 11:42:58  Server available - duration:2

2009-12-11 11:42:58  Server available - duration:4

2009-12-11 11:43:00  Server available - duration:2

2009-12-11 11:43:01  Server available - duration:3

2009-12-11 11:43:02  Server available - duration:2

2009-12-11 11:43:04  Server available - duration:5

2009-12-11 11:43:05  Server available - duration:3

2009-12-11 11:43:08  Server available - duration:4

2009-12-11 11:43:09  Server available - duration:2

2009-12-11 11:43:11  Server available - duration:4

2009-12-11 11:43:12  Server available - duration:2

2009-12-11 11:43:15  Server available - duration:6

2009-12-11 11:43:15  Server available - duration:3

2009-12-11 11:43:18  Server available - duration:3

2009-12-11 11:43:18  Server available - duration:2

2009-12-11 11:43:20  Server available - duration:3

2009-12-11 11:43:22  Server available - duration:3

2009-12-11 11:43:23  Server available - duration:3

2009-12-11 11:43:26  Server available - duration:5

2009-12-11 11:43:27  Server available - duration:3

2009-12-11 11:43:28  Server available - duration:2

2009-12-11 11:43:29  Server available - duration:2

2009-12-11 11:43:32  Server available - duration:3

2009-12-11 11:43:33  Server available - duration:3

2009-12-11 11:43:35  Server available - duration:4

2009-12-11 11:43:37  Server available - duration:2

2009-12-11 11:43:39  Server available - duration:2

2009-12-11 11:43:41  Server available - duration:2

2009-12-11 11:43:43  Server available - duration:3

2009-12-11 11:43:44  Server available - duration:2

2009-12-11 11:43:46  Server available - duration:3

2009-12-11 11:43:47  Server available - duration:2

2009-12-11 11:43:48  Server available - duration:3

2009-12-11 11:43:49  Server available - duration:2

2009-12-11 11:43:50  Server available - duration:2

2009-12-11 11:43:52  Server available - duration:3

2009-12-11 11:43:53  Server available - duration:2

2009-12-11 11:43:55  Server available - duration:3

2009-12-11 11:43:56  Server available - duration:3

2009-12-11 11:43:58  Server available - duration:3

2009-12-11 11:43:59  Server available - duration:4

2009-12-11 11:44:00  Server available - duration:2

2009-12-11 11:44:02  Server available - duration:2

2009-12-11 11:44:02  Server available - duration:3

2009-12-11 11:44:03  Server available - duration:2

2009-12-11 11:44:05  Server available - duration:3

2009-12-11 11:44:05  Server available - duration:3

2009-12-11 11:44:08  Server available - duration:2

2009-12-11 11:44:09  Server available - duration:3

2009-12-11 11:44:10  Server available - duration:2

2009-12-11 11:44:11  Server available - duration:2

2009-12-11 11:44:13  Server available - duration:4

2009-12-11 11:44:14  Server available - duration:2

2009-12-11 11:44:17  Server available - duration:2

2009-12-11 11:44:17  Server available - duration:2

2009-12-11 11:44:18  Server available - duration:17

2009-12-11 11:44:21  Server available - duration:4

2009-12-11 11:44:21  Server available - duration:2

2009-12-11 11:44:24  Server available - duration:3

2009-12-11 11:44:24  Server available - duration:2

2009-12-11 11:44:27  Server available - duration:3

2009-12-11 11:44:27  Server available - duration:6

2009-12-11 11:44:29  Server available - duration:2

2009-12-11 11:44:30  Server available - duration:3

2009-12-11 11:44:31  Server available - duration:2

2009-12-11 11:44:32  Server available - duration:3

2009-12-11 11:44:33  Server available - duration:3

2009-12-11 11:44:35  Server available - duration:3

2009-12-11 11:44:36  Server available - duration:2

2009-12-11 11:44:38  Server available - duration:2

2009-12-11 11:44:39  Server available - duration:2

2009-12-11 11:44:41  Server available - duration:3

2009-12-11 11:44:42  Server available - duration:5

2009-12-11 11:44:43  Server available - duration:3

2009-12-11 11:44:45  Server available - duration:4

2009-12-11 11:44:46  Server available - duration:3

2009-12-11 11:44:47  Server available - duration:2

2009-12-11 11:44:49  Server available - duration:2

2009-12-11 11:44:49  Server available - duration:3

2009-12-11 11:44:51  Server available - duration:3

2009-12-11 11:44:51  Server available - duration:2

2009-12-11 11:44:53  Server available - duration:2

2009-12-11 11:44:55  Server available - duration:4

2009-12-11 11:44:56  Server available - duration:3

2009-12-11 11:44:58  Server available - duration:2

2009-12-11 11:44:58  Server available - duration:5

2009-12-11 11:44:59  Server available - duration:2

2009-12-11 11:45:02  Server available - duration:3

2009-12-11 11:45:03  Server available - duration:2

2009-12-11 11:45:04  Server available - duration:3

2009-12-11 11:45:05  Server available - duration:3

2009-12-11 11:45:08  Server available - duration:4

2009-12-11 11:45:09  Server available - duration:5

2009-12-11 11:45:11  Server available - duration:3

2009-12-11 11:45:11  Server available - duration:3

2009-12-11 11:45:13  Server available - duration:5

2009-12-11 11:45:14  Server available - duration:3

2009-12-11 11:45:16  Server available - duration:3

2009-12-11 11:45:17  Server available - duration:3

2009-12-11 11:45:19  Server available - duration:2

2009-12-11 11:45:19  Server available - duration:2

2009-12-11 11:45:21  Server available - duration:4

2009-12-11 11:45:21  Server available - duration:3

2009-12-11 11:45:24  Server available - duration:3

2009-12-11 11:45:24  Server available - duration:3

2009-12-11 11:45:25  Server available - duration:3

2009-12-11 11:45:28  Server available - duration:3

2009-12-11 11:45:28  Server available - duration:3

2009-12-11 11:45:30  Server available - duration:6

2009-12-11 11:45:32  Server available - duration:3

2009-12-11 11:45:33  Server available - duration:2

2009-12-11 11:45:34  Server available - duration:2

2009-12-11 11:45:36  Server available - duration:3

2009-12-11 11:45:36  Server available - duration:3

2009-12-11 11:45:39  Server available - duration:3

2009-12-11 11:45:39  Server available - duration:3

2009-12-11 11:45:41  Server available - duration:2

2009-12-11 11:45:43  Server available - duration:3

2009-12-11 11:45:45  Server available - duration:3

2009-12-11 11:45:46  Server available - duration:3

2009-12-11 11:45:47  Server available - duration:3

2009-12-11 11:45:49  Server available - duration:3

2009-12-11 11:45:50  Server available - duration:2

2009-12-11 11:45:51  Server available - duration:2

2009-12-11 11:45:52  Server available - duration:2

2009-12-11 11:45:55  Server available - duration:3

2009-12-11 11:45:56  Server available - duration:3

2009-12-11 11:45:57  Server available - duration:2

2009-12-11 11:46:00  Server available - duration:3

2009-12-11 11:46:01  Server available - duration:2

2009-12-11 11:46:03  Server available - duration:3

2009-12-11 11:46:05  Server available - duration:2

2009-12-11 11:46:07  Server available - duration:2

2009-12-11 11:46:08  Server available - duration:2

2009-12-11 11:46:10  Server available - duration:2

2009-12-11 11:46:11  Server available - duration:2

2009-12-11 11:46:12  Server available - duration:2

2009-12-11 11:46:13  Server available - duration:3

2009-12-11 11:46:16  Server available - duration:3

2009-12-11 11:46:16  Server available - duration:3

2009-12-11 11:46:18  Server available - duration:3

2009-12-11 11:46:19  Server available - duration:2

2009-12-11 11:46:21  Server available - duration:2

2009-12-11 11:46:21  Server available - duration:4

2009-12-11 11:46:23  Server available - duration:3

2009-12-11 11:46:23  Server available - duration:3

2009-12-11 11:46:26  Server available - duration:3

2009-12-11 11:46:26  Server available - duration:2

2009-12-11 11:46:28  Server available - duration:3

2009-12-11 11:46:28  Server available - duration:2

2009-12-11 11:46:30  Server available - duration:5

2009-12-11 11:46:32  Server available - duration:3

2009-12-11 11:46:34  Server available - duration:3

2009-12-11 11:46:35  Server available - duration:2

2009-12-11 11:46:37  Server available - duration:3

2009-12-11 11:46:37  Server available - duration:3

2009-12-11 11:46:39  Server available - duration:3

2009-12-11 11:46:39  Server available - duration:2

2009-12-11 11:46:40  Server available - duration:3

2009-12-11 11:46:43  Server available - duration:3

2009-12-11 11:46:44  Server available - duration:2

2009-12-11 11:46:45  Server available - duration:2

2009-12-11 11:46:46  Server available - duration:2

2009-12-11 11:46:48  Server available - duration:3

2009-12-11 11:46:49  Server available - duration:3

2009-12-11 11:46:51  Server available - duration:2

2009-12-11 11:46:51  Server available - duration:3

2009-12-11 11:46:53  Server available - duration:3

2009-12-11 11:46:54  Server available - duration:3

2009-12-11 11:46:55  Server available - duration:58

2009-12-11 11:46:56  Server available - duration:3

2009-12-11 11:46:57  Server available - duration:3

2009-12-11 11:46:59  Server available - duration:4

2009-12-11 11:46:59  Server available - duration:3

2009-12-11 11:47:02  Server available - duration:3

2009-12-11 11:47:03  Server available - duration:4

2009-12-11 11:47:04  Server available - duration:3

2009-12-11 11:47:05  Server available - duration:2

2009-12-11 11:47:07  Server available - duration:4

2009-12-11 11:47:09  Server available - duration:3

2009-12-11 11:47:10  Server available - duration:2

2009-12-11 11:47:13  Server available - duration:4

2009-12-11 11:47:14  Server available - duration:2

2009-12-11 11:47:16  Server available - duration:2

2009-12-11 11:47:18  Server available - duration:3

2009-12-11 11:47:19  Server available - duration:3

2009-12-11 11:47:20  Server available - duration:2

2009-12-11 11:47:22  Server available - duration:2

2009-12-11 11:47:22  Server available - duration:3

2009-12-11 11:47:25  Server available - duration:4

2009-12-11 11:47:26  Server available - duration:3

2009-12-11 11:47:28  Server available - duration:3

2009-12-11 11:47:28  Server available - duration:2

2009-12-11 11:47:29  Server available - duration:3

2009-12-11 11:47:30  Server available - duration:2

2009-12-11 11:47:32  Server available - duration:2

2009-12-11 11:47:34  Server available - duration:3

2009-12-11 11:47:35  Server available - duration:3

2009-12-11 11:47:37  Server available - duration:3

2009-12-11 11:47:38  Server available - duration:2

2009-12-11 11:47:39  Server available - duration:2

2009-12-11 11:47:41  Server available - duration:3

2009-12-11 11:47:43  Server available - duration:3

2009-12-11 11:47:44  Server available - duration:2

2009-12-11 11:47:46  Server available - duration:2

2009-12-11 11:47:46  Server available - duration:2

2009-12-11 11:47:49  Server available - duration:3

2009-12-11 11:47:49  Server available - duration:2

2009-12-11 11:47:51  Server available - duration:2

2009-12-11 11:47:52  Server available - duration:3

2009-12-11 11:47:53  Server available - duration:2

2009-12-11 11:47:54  Server available - duration:2

2009-12-11 11:47:56  Server available - duration:3

2009-12-11 11:47:57  Server available - duration:3

2009-12-11 11:47:58  Server available - duration:2

2009-12-11 11:48:00  Server available - duration:3

2009-12-11 11:48:01  Server available - duration:3

2009-12-11 11:48:03  Server available - duration:2

2009-12-11 11:48:04  Server available - duration:2

2009-12-11 11:48:06  Server available - duration:3

2009-12-11 11:48:07  Server available - duration:2

2009-12-11 11:48:09  Server available - duration:3

2009-12-11 11:48:09  Server available - duration:2

2009-12-11 11:48:12  Server available - duration:3

2009-12-11 11:48:12  Server available - duration:3

2009-12-11 11:48:14  Server available - duration:2

2009-12-11 11:48:15  Server available - duration:2

2009-12-11 11:48:17  Server available - duration:3

2009-12-11 11:48:17  Server available - duration:3

2009-12-11 11:48:19  Server available - duration:3

2009-12-11 11:48:20  Server available - duration:3

2009-12-11 11:48:21  Server available - duration:2

2009-12-11 11:48:21  Server available - duration:3

2009-12-11 11:48:23  Server available - duration:2

2009-12-11 11:48:24  Server available - duration:4

2009-12-11 11:48:25  Server available - duration:2

2009-12-11 11:48:27  Server available - duration:3

2009-12-11 11:48:27  Server available - duration:3

2009-12-11 11:48:30  Server available - duration:2

2009-12-11 11:48:31  Server available - duration:4

2009-12-11 11:48:32  Server available - duration:2

2009-12-11 11:48:33  Server available - duration:2

2009-12-11 11:48:34  Server available - duration:2

2009-12-11 11:48:35  Server available - duration:4

2009-12-11 11:48:37  Server available - duration:2

2009-12-11 11:48:38  Server available - duration:3

2009-12-11 11:48:40  Server available - duration:2

2009-12-11 11:48:41  Server available - duration:2

2009-12-11 11:48:42  Server available - duration:2

2009-12-11 11:48:43  Server available - duration:4

2009-12-11 11:48:45  Server available - duration:3

2009-12-11 11:48:47  Server available - duration:3

2009-12-11 11:48:49  Server available - duration:3

2009-12-11 11:48:50  Server available - duration:2

2009-12-11 11:48:52  Server available - duration:3

2009-12-11 11:48:52  Server available - duration:2

2009-12-11 11:48:55  Server available - duration:3

2009-12-11 11:48:56  Server available - duration:2

2009-12-11 11:48:57  Server available - duration:3

2009-12-11 11:49:00  Stopping load test.

 

Average GCCollectTime in milliseconds

logparser "select AVG(TO_INT(TRIM(EXTRACT_SUFFIX(Message,0,':'))))  as AVGGCCollectTime from application where SourceName like '%WebNot%' and TimeWritten >= TO_TIMESTAMP( '2009-12-11 11:18:45', 'yyyy-MM-dd hh:mm:ss' ) and TimeWritten <= TO_TIMESTAMP( '2009-12-11 11:49:00', 'yyyy-MM-dd hh:mm:ss' ) and Message like '%duration:%'" -o:DATAGRID

3

 

·         Using Server GC

 

logparser "select TimeWritten, Message  from application where SourceName like '%WebNot%' and TimeWritten >= TO_TIMESTAMP( '2009-12-11 10:25:31', 'yyyy-MM-dd hh:mm:ss' ) and TimeWritten <= TO_TIMESTAMP( '2009-12-11 10:56:31', 'yyyy-MM-dd hh:mm:ss' )" -o:DATAGRID

 

2009-12-11 10:25:31  Starting load test.

2009-12-11 10:25:33  Server available - duration:4

2009-12-11 10:25:36  Server available - duration:4

2009-12-11 10:25:44  Server available - duration:4

2009-12-11 10:25:54  Server available - duration:5

2009-12-11 10:26:09  Server available - duration:5

2009-12-11 10:26:24  Server available - duration:6

2009-12-11 10:26:40  Server available - duration:5

2009-12-11 10:26:49  Server available - duration:5

2009-12-11 10:27:04  Server available - duration:6

2009-12-11 10:27:12  Server available - duration:5

2009-12-11 10:27:20  Server available - duration:5

2009-12-11 10:27:29  Server available - duration:4

2009-12-11 10:27:45  Server available - duration:5

2009-12-11 10:27:55  Server available - duration:5

2009-12-11 10:28:13  Server available - duration:5

2009-12-11 10:28:31  Server available - duration:5

2009-12-11 10:28:44  Server available - duration:6

2009-12-11 10:28:59  Server available - duration:5

2009-12-11 10:29:16  Server available - duration:5

2009-12-11 10:29:32  Server available - duration:6

2009-12-11 10:29:48  Server available - duration:5

2009-12-11 10:30:04  Server available - duration:5

2009-12-11 10:30:20  Server available - duration:5

2009-12-11 10:30:35  Server available - duration:5

2009-12-11 10:30:51  Server available - duration:5

2009-12-11 10:31:06  Server available - duration:4

2009-12-11 10:31:21  Server available - duration:7

2009-12-11 10:31:35  Server available - duration:6

2009-12-11 10:31:48  Server available - duration:4

2009-12-11 10:31:56  Server available - duration:5

2009-12-11 10:32:04  Server available - duration:5

2009-12-11 10:32:18  Server available - duration:4

2009-12-11 10:32:32  Server available - duration:5

2009-12-11 10:32:47  Server available - duration:5

2009-12-11 10:33:04  Server available - duration:6

2009-12-11 10:33:12  Server available - duration:4

2009-12-11 10:33:19  Server available - duration:5

2009-12-11 10:33:27  Server available - duration:4

2009-12-11 10:33:39  Server available - duration:4

2009-12-11 10:33:55  Server available - duration:4

2009-12-11 10:34:09  Server available - duration:5

2009-12-11 10:34:24  Server available - duration:4

2009-12-11 10:34:38  Server available - duration:5

2009-12-11 10:34:51  Server available - duration:7

2009-12-11 10:35:06  Server available - duration:4

2009-12-11 10:35:20  Server available - duration:7

2009-12-11 10:35:29  Server available - duration:4

2009-12-11 10:35:44  Server available - duration:5

2009-12-11 10:35:58  Server available - duration:5

2009-12-11 10:36:13  Server available - duration:4

2009-12-11 10:36:28  Server available - duration:5

2009-12-11 10:36:42  Server available - duration:9

2009-12-11 10:36:57  Server available - duration:5

2009-12-11 10:37:12  Server available - duration:5

2009-12-11 10:37:26  Server available - duration:4

2009-12-11 10:37:41  Server available - duration:5

2009-12-11 10:37:56  Server available - duration:5

2009-12-11 10:38:03  Server available - duration:4

2009-12-11 10:38:18  Server available - duration:5

2009-12-11 10:38:33  Server available - duration:6

2009-12-11 10:38:41  Server available - duration:5

2009-12-11 10:38:54  Server available - duration:4

2009-12-11 10:39:02  Server available - duration:4

2009-12-11 10:39:15  Server available - duration:4

2009-12-11 10:39:23  Server available - duration:5

2009-12-11 10:39:38  Server available - duration:5

2009-12-11 10:39:52  Server available - duration:5

2009-12-11 10:40:07  Server available - duration:9

2009-12-11 10:40:21  Server available - duration:5

2009-12-11 10:40:36  Server available - duration:4

2009-12-11 10:40:51  Server available - duration:5

2009-12-11 10:40:59  Server available - duration:5

2009-12-11 10:41:14  Server available - duration:5

2009-12-11 10:41:29  Server available - duration:5

2009-12-11 10:41:43  Server available - duration:4

2009-12-11 10:41:55  Server available - duration:4

2009-12-11 10:42:10  Server available - duration:4

2009-12-11 10:42:25  Server available - duration:4

2009-12-11 10:42:40  Server available - duration:5

2009-12-11 10:42:54  Server available - duration:5

2009-12-11 10:43:03  Server available - duration:5

2009-12-11 10:43:12  Server available - duration:5

2009-12-11 10:43:27  Server available - duration:5

2009-12-11 10:43:43  Server available - duration:5

2009-12-11 10:43:52  Server available - duration:6

2009-12-11 10:44:01  Server available - duration:4

2009-12-11 10:44:09  Server available - duration:5

2009-12-11 10:44:18  Server available - duration:5

2009-12-11 10:44:34  Server available - duration:4

2009-12-11 10:44:50  Server available - duration:4

2009-12-11 10:45:06  Server available - duration:5

2009-12-11 10:45:15  Server available - duration:5

2009-12-11 10:45:24  Server available - duration:4

2009-12-11 10:45:40  Server available - duration:5

2009-12-11 10:45:55  Server available - duration:4

2009-12-11 10:46:11  Server available - duration:5

2009-12-11 10:46:27  Server available - duration:7

2009-12-11 10:46:44  Server available - duration:4

2009-12-11 10:47:00  Server available - duration:5

2009-12-11 10:47:09  Server available - duration:23

2009-12-11 10:47:19  Server available - duration:4

2009-12-11 10:47:28  Server available - duration:4

2009-12-11 10:47:44  Server available - duration:5

2009-12-11 10:48:01  Server available - duration:7

2009-12-11 10:48:13  Server available - duration:19

2009-12-11 10:48:18  Server available - duration:3

2009-12-11 10:48:24  Server available - duration:3

2009-12-11 10:48:29  Server available - duration:3

2009-12-11 10:48:34  Server available - duration:3

2009-12-11 10:48:39  Server available - duration:4

2009-12-11 10:48:44  Server available - duration:3

2009-12-11 10:48:49  Server available - duration:2

2009-12-11 10:48:54  Server available - duration:3

2009-12-11 10:48:59  Server available - duration:3

2009-12-11 10:49:04  Server available - duration:3

2009-12-11 10:49:09  Server available - duration:3

2009-12-11 10:49:14  Server available - duration:3

2009-12-11 10:49:19  Server available - duration:3

2009-12-11 10:49:23  Server available - duration:3

2009-12-11 10:49:28  Server available - duration:3

2009-12-11 10:49:33  Server available - duration:3

2009-12-11 10:49:38  Server available - duration:3

2009-12-11 10:49:43  Server available - duration:9

2009-12-11 10:49:48  Server available - duration:3

2009-12-11 10:49:53  Server available - duration:4

2009-12-11 10:49:58  Server available - duration:3

2009-12-11 10:50:03  Server available - duration:3

2009-12-11 10:50:08  Server available - duration:4

2009-12-11 10:50:13  Server available - duration:2

2009-12-11 10:50:17  Server available - duration:3

2009-12-11 10:50:22  Server available - duration:5

2009-12-11 10:50:27  Server available - duration:3

2009-12-11 10:50:32  Server available - duration:2

2009-12-11 10:50:37  Server available - duration:3

2009-12-11 10:50:43  Server available - duration:3

2009-12-11 10:50:48  Server available - duration:2

2009-12-11 10:50:54  Server available - duration:2

2009-12-11 10:51:00  Server available - duration:3

2009-12-11 10:51:05  Server available - duration:3

2009-12-11 10:51:11  Server available - duration:3

2009-12-11 10:51:16  Server available - duration:3

2009-12-11 10:51:22  Server available - duration:2

2009-12-11 10:51:27  Server available - duration:3

2009-12-11 10:51:32  Server available - duration:4

2009-12-11 10:51:37  Server available - duration:4

2009-12-11 10:51:42  Server available - duration:3

2009-12-11 10:51:47  Server available - duration:4

2009-12-11 10:51:51  Server available - duration:3

2009-12-11 10:51:57  Server available - duration:3

2009-12-11 10:52:02  Server available - duration:2

2009-12-11 10:52:07  Server available - duration:3

2009-12-11 10:52:12  Server available - duration:4

2009-12-11 10:52:17  Server available - duration:2

2009-12-11 10:52:22  Server available - duration:4

2009-12-11 10:52:27  Server available - duration:3

2009-12-11 10:52:32  Server available - duration:3

2009-12-11 10:52:37  Server available - duration:2

2009-12-11 10:52:42  Server available - duration:3

2009-12-11 10:52:47  Server available - duration:3

2009-12-11 10:52:52  Server available - duration:3

2009-12-11 10:52:57  Server available - duration:3

2009-12-11 10:53:02  Server available - duration:3

2009-12-11 10:53:07  Server available - duration:4

2009-12-11 10:53:11  Server available - duration:3

2009-12-11 10:53:16  Server available - duration:3

2009-12-11 10:53:21  Server available - duration:3

2009-12-11 10:53:27  Server available - duration:2

2009-12-11 10:53:32  Server available - duration:3

2009-12-11 10:53:37  Server available - duration:4

2009-12-11 10:53:42  Server available - duration:2

2009-12-11 10:53:47  Server available - duration:4

2009-12-11 10:53:52  Server available - duration:3

2009-12-11 10:53:57  Server available - duration:3

2009-12-11 10:54:01  Server available - duration:3

2009-12-11 10:54:06  Server available - duration:3

2009-12-11 10:54:11  Server available - duration:3

2009-12-11 10:54:16  Server available - duration:2

2009-12-11 10:54:21  Server available - duration:3

2009-12-11 10:54:27  Server available - duration:5

2009-12-11 10:54:31  Server available - duration:6

2009-12-11 10:54:36  Server available - duration:3

2009-12-11 10:54:41  Server available - duration:5

2009-12-11 10:54:46  Server available - duration:3

2009-12-11 10:54:51  Server available - duration:4

2009-12-11 10:54:57  Server available - duration:4

2009-12-11 10:55:03  Server available - duration:3

2009-12-11 10:55:07  Server available - duration:3

2009-12-11 10:55:12  Server available - duration:2

2009-12-11 10:55:17  Server available - duration:2

2009-12-11 10:55:23  Server available - duration:2

2009-12-11 10:55:28  Server available - duration:2

2009-12-11 10:55:33  Server available - duration:3

2009-12-11 10:55:38  Server available - duration:3

2009-12-11 10:55:43  Server available - duration:3

2009-12-11 10:56:31  Stopping load test.

 

 

Average GCCollectTime in milliseconds

logparser "select AVG(TO_INT(TRIM(EXTRACT_SUFFIX(Message,0,':'))))  as AVGGCCollectTime from application where SourceName like '%WebNot%' and TimeWritten >= TO_TIMESTAMP( '2009-12-11 10:25:31', 'yyyy-MM-dd hh:mm:ss' ) and TimeWritten <= TO_TIMESTAMP( '2009-12-11 10:56:31', 'yyyy-MM-dd hh:mm:ss' ) and Message like '%duration:%'" -o:DATAGRID

4

 

 

 

Lessons learned

In my scenario, the verification rule never got hit. At least on a small test run (30 minutes) in my application GC.Collect time was in average of 3,5 milliseconds and there was no much difference in this scenario in terms of notifications between workstation and server GC. The application took itself offline 169 times on server GC, the main reason indeed is because in my scenario the full GC notification kicked in as en average about every 10 seconds.

As a result, in practice a load balancer would never have the time to take the server out of the farm. That being said this API allows you to control when to do GC.Collect. It is important to underline that it is usually not recommend inducing your own GC.Collect but under some specific scenarios it might be worth knowing. For example if memory fragmentation occurs and a lot of objects are allocated GC.Collect time may start to grow. An application could for example based on the last GC.Collect time-taken ask for worker process recycling. You may reuse this sample code to experiment your web application against server and workstation GC and control GC Collect depending on your workload… Voila!

If you are interested to read more about how the CLR GC works I highly recommend reading Maoni’s WebLog at:  http://blogs.msdn.com/maoni/.

 Many thanks as well for the great feedback I have had when testing this functionality and writing this post.