Welcome to MSDN Blogs Sign in | Join | Help

Examine .Net Memory Leaks

Writing programs using .Net is very productive. One reason is because much of memory management is “managed” for you. In C, C++ and other “native” languages, if you allocate memory, you’re responsible for freeing it.  There were stopgap measures, like destructors, SmartPointers and reference counting, which helped, but were still cumbersome.

 

Foxpro manages memory for you, and has used garbage collection for decades: Heartbeat: Garbage collection in VFP and .NET are similar

 

However, you can still have memory leaks in .Net.

 

Try this in VS 2008: (I think it will work in 2005 too, can somebody verify? Thanks)

 

1.       File->New->Project->VB Windows Console Application.

2.       Paste the sample code below.

3.       Project->Properties->Debug->Enable Unmanaged Code debugging

If you’re running 64 bit, you can force 32 bit targeting: Project->Properties->Compile->Advanced Compile Settings->Target CPU->x86 (see this for more about x64)

 

 

 

The sample code has a loop that creates and releases an instance of a class MyWatcher that uses the FileSystemWatcher class that reacts to events, such as files being created in a directory.

The class has a member (Dim MyLargeMemoryEater(100000) As String) which eats up 4 bytes (8 bytes on x64) per array element.  At the end of each loop, the garbage collector is called to release everything. The class has a Finalize method that will be called when the garbage collector collects.

 

When you run the code, you see the increase in memory use in each loop. The increase per iteration is just a little more than the amount of memory used by MyLargeMemoryEater . Also, the Finalizers don’t run until the application is shutting down.

 

If you uncomment the “UnSubscribe” line, then the finalizer fires (on a different thread) and you can see the leak is gone.

 

Now let’s examine the leak by looking at the heap.

 

Make it leak, then put a breakpoint on the “Done” line after the loop. At this point, we suspect there are 100 instances of MyWatcher in the managed heap.

 

Wouldn’t it be great to see them? Let’s use SOS:

 

Open the Immediate window: Debug Menu->Windows-> Immediate window

 

Type in the lines in red

 

!load sos.dll

extension C:\Windows\Microsoft.NET\Framework\v2.0.50727\sos.dll loaded

!dumpheap -type MyWatcher

PDB symbol for mscorwks.dll not loaded

 Address       MT     Size

02b7431c 000d313c       16    

 

<…>

 

02bdc3f0 000d313c       16    

02bdc550 000d313c       16    

total 100 objects

Statistics:

      MT    Count    TotalSize Class Name

000d313c      100         1600 ConsoleApplication1.MyWatcher

Total 100 objects

 

 

So now we know that there are 100 instances still around. Why were they not garbage collected? Because somebody has a reference to them. Choose one of the instances (02bdc550), and use the gcroot command:

 

!gcroot 02bdc550

Note: Roots found on stacks may be false positives. Run "!help gcroot" for

more info.

Error during command: Warning. Extension is using a callback which Visual Studio does not implement.

 

Scan Thread 6948 OSTHread 1b24

Scan Thread 536 OSTHread 218

Scan Thread 6448 OSTHread 1930

DOMAIN(00601068):HANDLE(AsyncPinned):b15b4:Root:02bdda44(System.Threading.OverlappedData)->

02bddf38(System.Threading.IOCompletionCallback)->

02bdcfc4(System.IO.FileSystemWatcher)->

02b8b14c(System.IO.FileSystemEventHandler)->

02bdc550(ConsoleApplication1.MyWatcher)

 

Now we see that the FileSystemWatcher is referencing us via an eventhandler.

 

(The “!dumpheap -stat” command is also very useful  to see what’s on the heap)

 

 

 

 

See also:

Collecting garbage at the wrong time

SOS Debugging Extension (SOS.dll)

http://www.codeproject.com/KB/dotnet/Memory_Leak_Detection.aspx

http://www.julmar.com/blog/mark/PermaLink,guid,643649fc-0467-4f0d-9a95-323ed7ce4298.aspx

Debugging a memory leak in managed code: Ping - SendAsync

 

 

<Code Sample>

 

Module Module1

    Friend g_cnt As Integer

    Sub Main()

 

        'uncomment these 2 lines to test the event watcher

        'Dim oFileWatcher = New MyWatcher

        'MsgBox("Wait in msgbox. FSW events still fire: copy a file into d:\")

 

        Dim oldPeak = 0L

 

        For i = 1 To 100

            Dim oWatcher = New MyWatcher

 

            '            oWatcher.UnSubscribe() ' uncomment this line to remove handler

 

            oWatcher = Nothing

            GC.Collect()    ' collect garbage

            GC.WaitForPendingFinalizers()   ' allow finalizers

            GC.Collect()    ' collect again for any objects that had finalizers

            Dim newpeak = Process.GetCurrentProcess.PeakWorkingSet64

            Debug.WriteLine("All released? " + i.ToString + " " + _

                    " WorkingSet =" + Process.GetCurrentProcess.WorkingSet64.ToString("n0") + _

                    " Peak=" + newpeak.ToString("n0") + _

                    " delta =" + (newpeak - oldPeak).ToString)

            oldPeak = newpeak

 

        Next

        GC.Collect()    ' collect garbage

        GC.WaitForPendingFinalizers()   ' allow finalizers

        GC.Collect()    ' collect again for any objects that had finalizers

        Debug.WriteLine("Done")

 

 

    End Sub

End Module

 

Class MyWatcher

    Dim MyLargeMemoryEater(100000) As String ' make the instance bigger to magnify issue: 4 bytes per array item on x86

    Dim fsw As IO.FileSystemWatcher

    Sub New()

        fsw = New IO.FileSystemWatcher

        fsw.Path = "d:\"

        fsw.Filter = "*.*"

        AddHandler fsw.Created, AddressOf OnWatcherFileCreated

 

        fsw.EnableRaisingEvents = True

 

    End Sub

    Sub UnSubscribe()

        RemoveHandler fsw.Created, AddressOf OnWatcherFileCreated

    End Sub

    Sub OnWatcherFileCreated(ByVal sender As Object, ByVal args As System.IO.FileSystemEventArgs)

        Debug.WriteLine((New StackTrace).GetFrames(0).GetMethod.Name + " " + args.FullPath)

    End Sub

    Protected Overrides Sub Finalize()  ' called when garbage collector collects on the GC Finalizer thread.

        MyBase.Finalize()

        Debug.WriteLine((New StackTrace).GetFrames(0).GetMethod.Name + g_cnt.ToString + " Thread= " + System.Threading.Thread.CurrentThread.ManagedThreadId.ToString)

    End Sub

 

 

End Class

 

</Code Sample>

Persist user form size and location settings per session

My prior post (Create your own Test Host using XAML to run your unit tests) shows how to create a form and present it to the user. The user can resize and reposition the form, even on a 2nd monitor.

 

When the user exits the form, we can persist or remember the form size and location, so the next time the user form starts up, it will be positioned/sized as before.

 

The happens to be a XAML form, but it could be a WinForm for this to work. (The My.Settings feature has been in Visual Studio for many years…)

 

Because FoxPro has its own database engine, Fox can persist data in a Fox table. The positions and sizes of design time windows, such as the Command Window, Project Window, Database Windows, are persisted in the FoxPro Resource file. The FoxResource class (unzip the xsource.zip file in tools\xsource and look in the EnvMgr folder) can help users persist user forms there too.

 

 

To add this feature to your VB project:

 

Go to Project->Properties->Settings, and add 2 settings:

 

  • Location, type=System.Drawing.Point, default  = 0,0
  • Size, type=System.Drawing.Size, default=1024,768

 

 

In the Load or New method:

 

        If System.Windows.Forms.Screen.AllScreens.Count = 1 AndAlso _

           My.Settings.Location.X > System.Windows.Forms.Screen.PrimaryScreen.WorkingArea.Width Then

            Me.Left = 0 ' if the saved .X is out of range (multimonitor, remote desktop)

        Else

            Me.Left = My.Settings.Location.X

        End If

 

        Me.Top = My.Settings.Location.Y

        Me.Width = My.Settings.Size.Width

        Me.Height = My.Settings.Size.Height

        AddHandler Me.Closed, AddressOf OnWindowClosed

 

    Sub OnWindowClosed(ByVal sender As Object, ByVal e As EventArgs)

        My.Settings.Location = New System.Drawing.Point(CInt(Me.Left), CInt(Me.Top))

        My.Settings.Size = New System.Drawing.Size(CInt(Me.Width), CInt(Me.Height))

        My.Settings.Save()

    End Sub

 

 

 

The data is stored in an XML file like:

"C:\Documents and Settings\Calvinh\Local Settings\Application Data\Microsoft\<appname>\1.0.0.0\user.config”

which looks like:

 

<?xml version="1.0" encoding="utf-8"?>

<configuration>

    <userSettings>

        <<appname>.My.MySettings>

            <setting name="Location" serializeAs="String">

                <value>4, 4</value>

            </setting>

            <setting name="Size" serializeAs="String">

                <value>985, 494</value>

            </setting>

        </<appname>.My.MySettings>

    </userSettings>

</configuration>

 

 

If you’re feeling brave, you can persist multiple strings using System.Collections.Specialized.StringCollection.

 

I store things like which items are selected and how many asserts are fired per test. Each string will have 3 comma separated items: TestClass,TestName, and AssertCnt

        For Each itm In m_TestMethods

            Dim theitem = itm

            Dim res = From a In My.Settings.AssertsPerTest _

                      Let strs = CStr(a).Split(","c) _

                      Where strs(0) = theitem.TestClass And strs(1) = theitem.TestName _

                      Let Assertcnt = CInt(strs(2))

            If res.Count > 0 Then

                itm.AssertCnt = res.First.Assertcnt

            End If

 

        Next

 

And just before My.Settings.Save is called, we’ll create the string collection to save.

 

        Dim ColAssertsPerTest = New Specialized.StringCollection

        For Each itm In From d In TestMethods Where d.AssertCnt > 0

            ColAssertsPerTest.Add(itm.TestClass + "," + itm.TestName + "," + itm.AssertCnt.ToString)

        Next

        My.Settings.AssertsPerTest = ColAssertsPerTest

 

Notice the use of Linq in the For Each expressions.

 

I also use Linq to find the total # of asserts fired:

 

            Dim nTotalAssertsExpected = Aggregate itm In m_SelectedTests Into Sum(itm.AssertCnt)

 

 

See also: another example of persisting settings: The VB version of the Blog Crawler

 

 

Create your own Test Host using XAML to run your unit tests

A few days ago, somebody came into my office and plopped down a box. It seemed very light. He said that it was a new PC. I thought hmmm…. The box seems empty…Why am I getting a new PC?.


Apparently an inventory was made and my current hardware was at the lower end of the list.

 

So I started up the Dell Optiplex 755 with 4 gigs of RAM, and it was running Vista 64. Super: I hadn’t done much with 64 bit code yet.

 

Sure enough, debugging 64 bit native applications shows that the CPU has 64 bit wide registers (and more registers!). If I attach to a 64 bit process, the Debug->Windows->Registers looks like:

 

RAX = 0000000000000000 RBX = 000000001DEB7818 RCX = 000000000F99A027

RDX = 0000000000000000 RSI = 0000000000000001 RDI = 00000000242E5454

R8  = 000000001FD3FCB8 R9  = 0000000000000000 R10 = 0000000000000000

R11 = 0000000000000206 R12 = 0000000000000000 R13 = 0000000000000000

R14 = 0000000000000000 R15 = 0000000000000000 RIP = 000007FEF0FD3D29

RSP = 000000001FD3FCF0 RBP = 0000000000000000 EFL = 00000246

 

If I attach to a 32 bit process:

 

EAX = 00000000 EBX = 006AC138 ECX = 79E74400 EDX = 00000000 ESI = 00DF6700

EDI = 00000000 EIP = 59B80265 ESP = 001DDEE8 EBP = 001DDEE8 EFL = 00000206

 

001DDEFC = 06CD35C8

 

 

(The debugger is pretty amazing!)

 

The sheer size of the registers means instead of a maximum 2^32 =4 gig virtual address space, we have 2^64

To calculate that, type this in the Fox command window:

?LOG10(2^64)

?2^64

Which shows 19.27, 1.844674E+19

 

That means 19.27 zeros: 18,000,000,000,000,000,000 which is 18 billion billion or 18 exabytes!

 

Coincidentally, we have some new 64 bit code that was recently created by “patching” the 32 bit version, and I wanted some test tools for it.

 

I have a VS Test Project with many (32 bit) tests. (Use Visual Studio Test framework to create tests for your code)

 

So I created my own Test Host in a day:

 

This Test Host:

·         Uses the existing 32 bit test source code as linked files: if you make changes to the tests, they will automatically be updated in both 32 and 64 bit versions

·         Can be built as 32 bit or 64 bit (Project->Properties->Compile->Advanced Compile Options->Target CPU->AnyCPU or x86), so it can run tests in either 32 or 64 bits.  When set as AnyCPU and running on 64 bit OS like Vista 64, then it will run as 64 bit.

·         Does not require test deployment: you can test in place. VSTestHost requires deployment.

·         Uses reflection on itself to get and run the TestInitialize, TestCleanup and TestMethods.

·         Can run selected tests in a loop for stress tests, like leak detection.

·         Uses XAML and Data Binding (see Create your own media browser: Display your pictures, music, movies in a XAML tooltip)

o   Notice how the enabling and disabling of buttons is done via data binding.

o   The data binding updates the status (Success,Failure/Pending) and its color in the ListView.

o   Shows the test methods in a ListView with click/sort column headers.

·         The tests run on a worker thread. The UI is thus responsive and not blocked.

·         A progress bar and stopwatch also show

·         Can be run without UI: another assembly can call it to run tests

·         Can determine if it’s running 64 or 32:  IntPtr.Size=8 bytes on 64, 4 bytes on 32.

 

 

I’m thinking about adding a new 32 bit test case (for running in the VS IDE) that will launch all the 64 bit tests, or a new one for each of the 32 bit test cases, but that’s lower priority.

 

This new test host has already helped me find several 64 bit issues.

 

Follow my prior post: Use Visual Studio Test framework to create tests for your code to create a project.  (If you don’t have Fox or Excel, you can either get them or remove the Fox/Excel code in the test)

 

If you add these lines to the test

        System.Diagnostics.Debug.WriteLine("Sizeof IntPtr = " + IntPtr.Size.ToString)

        System.Diagnostics.Debug.WriteLine(System.Diagnostics.Process.GetCurrentProcess.MainModule.FileName)

 

You’ll get the output:

Sizeof IntPtr = 4

C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\vstesthost.exe

 

Nothing unexpected. VSTestHost is a 32 bit EXE, which means only 32 bit code is run.

 

 

To make the test host more interesting, add a few more tests to the bottom of UnitTests1:

    Shared oRandom As New Random

 

    <TestMethod()> _

    Sub EmptyTest()

 

    End Sub

 

    <TestMethod()> _

    Sub AlwaysFail()

        Assert.IsTrue(False, "True is never false!")

    End Sub

    <TestMethod()> _

    Sub SometimesFail()

        If oRandom.NextDouble < 0.4 Then

            Assert.IsTrue(False, "True is never false!")

        End If

    End Sub

    <TestMethod()> _

    Sub RarelyFails()

        If oRandom.NextDouble < 0.02 Then

            Assert.IsTrue(False, "True is rarely false!")

        End If

    End Sub

 

 

 

 

 

To create the Test Host:

 

Right Click on the Solution from above in Solution explorer, choose Add->New Project. This time choose Windows->WPF Application. Put in a folder next to the folder of the existing test project. I called mine MyTestHost

 

Right Click on MyTestHost in solution explorer, choose Set As Startup Project. That way, hitting F5 will run this new project.

Delete the Window1.XAML in the solution explorer. We’ll create our own.

 

Delete MyTestHost->Application.xaml  (we have our own Sub Main)

 

Right Click on MyTestHost in solution explorer, choose Add New Item, Code->Class. Name it MyTestHost.vb

 

Right Click on MyTestHost in solution explorer, choose Add Existing Item, navigate to the UnitTest1.vb file in the other project. Make sure you add as a Lnked File. The “Add” button on the dialog has a little down arrow that allows you to choose Add As Link.

 

Note: the Fox code in the unit test TestMethod1 will fail in 64 because there is no 64 bit version of it.