Calvin Hsia's WebLog

How to interrupt your code

I received a question:

 

 

Simply, is there a way of interrupting a vfp sql query once it has started short of closing down the process ?

I am running some complex queries on very large datasets which can sometimes take many minutes to complete.

 

 

Typically, a program that runs on your computer has a main thread of execution.

On this thread, the processor is fetching instructions from memory and executing them. If this thread is busy, perhaps executing a query, or even just in a loop, the processor won’t check for user input.

 

Imagine the processor to be a person working in a factory. If she’s busy, then she doesn’t have time to look at her email. One way to fix this is to add an item to her workload to look at the email once in a while. If she’s in a loop (repetitive work), then adding an item in the loop would work.

 

Another way to fix this is to have another person (thread) in the factory. His job is to watch the email for any pending requests. He has the ability to interrupt her work.

 

Many applications are single threaded. That means there is only one person responsible for doing the work, listening for input, and updating the User Interface. Sometimes he’s very busy, so to the user, the application seems non-responsive.

 

As above, a solution is to create another thread (think of it as adding another person who can help do the work). This other thread can monitor the email.

 

The sample below creates a temporary table of many records. It then does a non-optimized Cartesian join of the table with itself, making the main thread busy.

 

The Set Escape command tells the main thread to check for the Escape key (looking for the boss at the door).

 

The On Escape command says what to do if the boss is there.

 

As an alternative, you can inject your own code into the work loop. The WasKeyPressed function, for example, checks for a keypress.

 

If work is interrupted, the worker may not have finished the current task, perhaps leaving the environment in an abnormal state.

 

See also other threading samples, which allow you to create multiple threads in VB and FoxPro

 

Use Visual Studio Test framework to create tests for your code

Sample program to create multiple threads

Create multiple threads from within your application

More Multithread capabilities: interthread synchronization, error checking

Webcrawl a blog to retrieve all entries locally: RSS on steroids

The VB version of the Blog Crawler

 

 

 

CLEAR ALL

CLEAR

MODIFY COMMAND PROGRAM() NOWAIT

 

CREATE CURSOR foo (name c(10))

INSERT INTO foo (name ) VALUES ("foo")

 

*Try with ESCAPE ON and OFF

SET ESCAPE ON

?"Escape=",SET("Escape")

ON ESCAPE do OnEscapeHandler

FOR i = 1 TO 12

      APPEND FROM DBF("foo")  && Double the size each iteration

ENDFOR

?"Reccount=",RECCOUNT()

?"query starting: try to stop it!"

ns=SECONDS()

TRY

      * Cartesian product unoptimized join

      SELECT * FROM foo, foo bar WHERE foo.name=bar.name INTO CURSOR results

     

*   Sample that calls a method that checks for key press

*     SELECT * FROM foo, foo bar WHERE foo.name=bar.name AND WasKeyPressed() INTO CURSOR results

     

CATCH TO ex

      ?"Caught",ex.Message,ex.Details,ex.UserValue

ENDTRY

 

?"Query done"

?SECONDS()-ns

 

RETURN

PROCEDURE OnEscapeHandler

      ?"Interrupted"

      THROW "interrupted from "+PROGRAM()

      RETURN

 

FUNCTION WasKeyPressed as Boolean

      IF INKEY()>0

            THROW "Key pressed from "+PROGRAM()

      ENDIF

      RETURN .T.

 

 

Examine .Net Memory Leaks

Writing programs using .Net is very productive. One reason is because much of memory management is “managed” for you. In C, C++ and other “native” languages, if you allocate memory, you’re responsible for freeing it.  There were stopgap measures, like destructors, SmartPointers and reference counting, which helped, but were still cumbersome.

 

Foxpro manages memory for you, and has used garbage collection for decades: Heartbeat: Garbage collection in VFP and .NET are similar

 

However, you can still have memory leaks in .Net.

 

Try this in VS 2008: (I think it will work in 2005 too, can somebody verify? Thanks)

 

1.       File->New->Project->VB Windows Console Application.

2.       Paste the sample code below.

3.       Project->Properties->Debug->Enable Unmanaged Code debugging

If you’re running 64 bit, you can force 32 bit targeting: Project->Properties->Compile->Advanced Compile Settings->Target CPU->x86 (see this for more about x64)

 

 

 

The sample code has a loop that creates and releases an instance of a class MyWatcher that uses the FileSystemWatcher class that reacts to events, such as files being created in a directory.

The class has a member (Dim MyLargeMemoryEater(100000) As String) which eats up 4 bytes (8 bytes on x64) per array element.  At the end of each loop, the garbage collector is called to release everything. The class has a Finalize method that will be called when the garbage collector collects.

 

When you run the code, you see the increase in memory use in each loop. The increase per iteration is just a little more than the amount of memory used by MyLargeMemoryEater . Also, the Finalizers don’t run until the application is shutting down.

 

If you uncomment the “UnSubscribe” line, then the finalizer fires (on a different thread) and you can see the leak is gone.

 

Now let’s examine the leak by looking at the heap.

 

Make it leak, then put a breakpoint on the “Done” line after the loop. At this point, we suspect there are 100 instances of MyWatcher in the managed heap.

 

Wouldn’t it be great to see them? Let’s use SOS:

 

Open the Immediate window: Debug Menu->Windows-> Immediate window

 

Type in the lines in red

 

!load sos.dll

extension C:\Windows\Microsoft.NET\Framework\v2.0.50727\sos.dll loaded

!dumpheap -type MyWatcher

PDB symbol for mscorwks.dll not loaded

 Address       MT     Size

02b7431c 000d313c       16    

 

<…>

 

02bdc3f0 000d313c       16    

02bdc550 000d313c       16    

total 100 objects

Statistics:

      MT    Count    TotalSize Class Name

000d313c      100         1600 ConsoleApplication1.MyWatcher

Total 100 objects

 

 

So now we know that there are 100 instances still around. Why were they not garbage collected? Because somebody has a reference to them. Choose one of the instances (02bdc550), and use the gcroot command:

 

!gcroot 02bdc550

Note: Roots found on stacks may be false positives. Run "!help gcroot" for

more info.

Error during command: Warning. Extension is using a callback which Visual Studio does not implement.

 

Scan Thread 6948 OSTHread 1b24

Scan Thread 536 OSTHread 218

Scan Thread 6448 OSTHread 1930

DOMAIN(00601068):HANDLE(AsyncPinned):b15b4:Root:02bdda44(System.Threading.OverlappedData)->

02bddf38(System.Threading.IOCompletionCallback)->

02bdcfc4(System.IO.FileSystemWatcher)->

02b8b14c(System.IO.FileSystemEventHandler)->

02bdc550(ConsoleApplication1.MyWatcher)

 

Now we see that the FileSystemWatcher is referencing us via an eventhandler.

 

(The “!dumpheap -stat” command is also very useful  to see what’s on the heap)

 

 

 

 

See also:

Collecting garbage at the wrong time

SOS Debugging Extension (SOS.dll)

http://www.codeproject.com/KB/dotnet/Memory_Leak_Detection.aspx

http://www.julmar.com/blog/mark/PermaLink,guid,643649fc-0467-4f0d-9a95-323ed7ce4298.aspx

Debugging a memory leak in managed code: Ping - SendAsync

 

 

<Code Sample>

 

Module Module1

    Friend g_cnt As Integer

    Sub Main()

 

        'uncomment these 2 lines to test the event watcher

        'Dim oFileWatcher = New MyWatcher

        'MsgBox("Wait in msgbox. FSW events still fire: copy a file into d:\")

 

        Dim oldPeak = 0L

 

        For i = 1 To 100

            Dim oWatcher = New MyWatcher

 

            '            oWatcher.UnSubscribe() ' uncomment this line to remove handler

 

            oWatcher = Nothing

            GC.Collect()    ' collect garbage

            GC.WaitForPendingFinalizers()   ' allow finalizers

            GC.Collect()    ' collect again for any objects that had finalizers

            Dim newpeak = Process.GetCurrentProcess.PeakWorkingSet64

            Debug.WriteLine("All released? " + i.ToString + " " + _

                    " WorkingSet =" + Process.GetCurrentProcess.WorkingSet64.ToString("n0") + _

                    " Peak=" + newpeak.ToString("n0") + _

                    " delta =" + (newpeak - oldPeak).ToString)

            oldPeak = newpeak

 

        Next

        GC.Collect()    ' collect garbage

        GC.WaitForPendingFinalizers()   ' allow finalizers

        GC.Collect()    ' collect again for any objects that had finalizers

        Debug.WriteLine("Done")

 

 

    End Sub

End Module

 

Class MyWatcher

    Dim MyLargeMemoryEater(100000) As String ' make the instance bigger to magnify issue: 4 bytes per array item on x86

    Dim fsw As IO.FileSystemWatcher

    Sub New()

        fsw = New IO.FileSystemWatcher

        fsw.Path = "d:\"

        fsw.Filter = "*.*"

        AddHandler fsw.Created, AddressOf OnWatcherFileCreated

 

        fsw.EnableRaisingEvents = True

 

    End Sub

    Sub UnSubscribe()

        RemoveHandler fsw.Created, AddressOf OnWatcherFileCreated

    End Sub

    Sub OnWatcherFileCreated(ByVal sender As Object, ByVal args As System.IO.FileSystemEventArgs)

        Debug.WriteLine((New StackTrace).GetFrames(0).GetMethod.Name + " " + args.FullPath)

    End Sub

    Protected Overrides Sub Finalize()  ' called when garbage collector collects on the GC Finalizer thread.

        MyBase.Finalize()

        Debug.WriteLine((New StackTrace).GetFrames(0).GetMethod.Name + g_cnt.ToString + " Thread= " + System.Threading.Thread.CurrentThread.ManagedThreadId.ToString)

    End Sub

 

 

End Class

 

</Code Sample>

Persist user form size and location settings per session

My prior post (Create your own Test Host using XAML to run your unit tests) shows how to create a form and present it to the user. The user can resize and reposition the form, even on a 2^nd monitor.

 

When the user exits the form, we can persist or remember the form size and location, so the next time the user form starts up, it will be positioned/sized as before.

 

The happens to be a XAML form, but it could be a WinForm for this to work. (The My.Settings feature has been in Visual Studio for many years…)

 

Because FoxPro has its own database engine, Fox can persist data in a Fox table. The positions and sizes of design time windows, such as the Command Window, Project Window, Database Windows, are persisted in the FoxPro Resource file. The FoxResource class (unzip the xsource.zip file in tools\xsource and look in the EnvMgr folder) can help users persist user forms there too.

 

 

To add this feature to your VB project:

 

Go to Project->Properties->Settings, and add 2 settings:

 

• Location, type=System.Drawing.Point, default  = 0,0
• Size, type=System.Drawing.Size, default=1024,768

 

 

In the Load or New method:

 

        If System.Windows.Forms.Screen.AllScreens.Count = 1 AndAlso _

           My.Settings.Location.X > System.Windows.Forms.Screen.PrimaryScreen.WorkingArea.Width Then

            Me.Left = 0 ' if the saved .X is out of range (multimonitor, remote desktop)

        Else

            Me.Left = My.Settings.Location.X

        End If

 

        Me.Top = My.Settings.Location.Y

        Me.Width = My.Settings.Size.Width

        Me.Height = My.Settings.Size.Height

        AddHandler Me.Closed, AddressOf OnWindowClosed

 

    Sub OnWindowClosed(ByVal sender As Object, ByVal e As EventArgs)

        My.Settings.Location = New System.Drawing.Point(CInt(Me.Left), CInt(Me.Top))

        My.Settings.Size = New System.Drawing.Size(CInt(Me.Width), CInt(Me.Height))

        My.Settings.Save()

    End Sub

 

 

 

The data is stored in an XML file like:

"C:\Documents and Settings\Calvinh\Local Settings\Application Data\Microsoft\<appname>\1.0.0.0\user.config”

which looks like:

 

<?xml version="1.0" encoding="utf-8"?>

<configuration>

    <userSettings>

        <<appname>.My.MySettings>

            <setting name="Location" serializeAs="String">

                <value>4, 4</value>

            </setting>

            <setting name="Size" serializeAs="String">

                <value>985, 494</value>

            </setting>

        </<appname>.My.MySettings>

    </userSettings>

</configuration>

 

 

If you’re feeling brave, you can persist multiple strings using System.Collections.Specialized.StringCollection.

 

I store things like which items are selected and how many asserts are fired per test. Each string will have 3 comma separated items: TestClass,TestName, and AssertCnt

        For Each itm In m_TestMethods

            Dim theitem = itm

            Dim res = From a In My.Settings.AssertsPerTest _

                      Let strs = CStr(a).Split(","c) _

                      Where strs(0) = theitem.TestClass And strs(1) = theitem.TestName _

                      Let Assertcnt = CInt(strs(2))

            If res.Count > 0 Then

                itm.AssertCnt = res.First.Assertcnt

            End If

 

        Next

 

And just before My.Settings.Save is called, we’ll create the string collection to save.

 

        Dim ColAssertsPerTest = New Specialized.StringCollection

        For Each itm In From d In TestMethods Where d.AssertCnt > 0

            ColAssertsPerTest.Add(itm.TestClass + "," + itm.TestName + "," + itm.AssertCnt.ToString)

        Next

        My.Settings.AssertsPerTest = ColAssertsPerTest

 

Notice the use of Linq in the For Each expressions.

 

I also use Linq to find the total # of asserts fired:

 

            Dim nTotalAssertsExpected = Aggregate itm In m_SelectedTests Into Sum(itm.AssertCnt)

 

 

See also: another example of persisting settings: The VB version of the Blog Crawler

 

 

Create your own Test Host using XAML to run your unit tests

A few days ago, somebody came into my office and plopped down a box. It seemed very light. He said that it was a new PC. I thought hmmm…. The box seems empty…Why am I getting a new PC?.