Calvin Hsia's WebLog : Testing

Persist user form size and location settings per session

Calvin_Hsia — Thu, 03 Apr 2008 02:57:00 GMT

My prior post (Create your own Test Host using XAML to run your unit tests) shows how to create a form and present it to the user. The user can resize and reposition the form, even on a 2^nd monitor.

When the user exits the form, we can persist or remember the form size and location, so the next time the user form starts up, it will be positioned/sized as before.

The happens to be a XAML form, but it could be a WinForm for this to work. (The My.Settings feature has been in Visual Studio for many years…)

Because FoxPro has its own database engine, Fox can persist data in a Fox table. The positions and sizes of design time windows, such as the Command Window, Project Window, Database Windows, are persisted in the FoxPro Resource file. The FoxResource class (unzip the xsource.zip file in tools\xsource and look in the EnvMgr folder) can help users persist user forms there too.

To add this feature to your VB project:

Go to Project->Properties->Settings, and add 2 settings:

Location, type=System.Drawing.Point, default = 0,0
Size, type=System.Drawing.Size, default=1024,768

In the Load or New method:

If System.Windows.Forms.Screen.AllScreens.Count = 1 AndAlso _

My.Settings.Location.X > System.Windows.Forms.Screen.PrimaryScreen.WorkingArea.Width Then

Me.Left = 0 ' if the saved .X is out of range (multimonitor, remote desktop)

Else

Me.Left = My.Settings.Location.X

End If

Me.Top = My.Settings.Location.Y

Me.Width = My.Settings.Size.Width

Me.Height = My.Settings.Size.Height

AddHandler Me.Closed, AddressOf OnWindowClosed

Sub OnWindowClosed(ByVal sender As Object, ByVal e As EventArgs)

My.Settings.Location = New System.Drawing.Point(CInt(Me.Left), CInt(Me.Top))

My.Settings.Size = New System.Drawing.Size(CInt(Me.Width), CInt(Me.Height))

My.Settings.Save()

End Sub

The data is stored in an XML file like:

"C:\Documents and Settings\Calvinh\Local Settings\Application Data\Microsoft\<appname>\1.0.0.0\user.config”

which looks like:

<?xml version="1.0" encoding="utf-8"?>

<<appname>.My.MySettings>

</setting>

</setting>

</<appname>.My.MySettings>

</userSettings>

</configuration>

If you’re feeling brave, you can persist multiple strings using System.Collections.Specialized.StringCollection.

I store things like which items are selected and how many asserts are fired per test. Each string will have 3 comma separated items: TestClass,TestName, and AssertCnt

For Each itm In m_TestMethods

Dim theitem = itm

Dim res = From a In My.Settings.AssertsPerTest _

Let strs = CStr(a).Split(","c) _

Where strs(0) = theitem.TestClass And strs(1) = theitem.TestName _

Let Assertcnt = CInt(strs(2))

If res.Count > 0 Then

itm.AssertCnt = res.First.Assertcnt

End If

And just before My.Settings.Save is called, we’ll create the string collection to save.

Dim ColAssertsPerTest = New Specialized.StringCollection

For Each itm In From d In TestMethods Where d.AssertCnt > 0

ColAssertsPerTest.Add(itm.TestClass + "," + itm.TestName + "," + itm.AssertCnt.ToString)

My.Settings.AssertsPerTest = ColAssertsPerTest

Notice the use of Linq in the For Each expressions.

I also use Linq to find the total # of asserts fired:

Dim nTotalAssertsExpected = Aggregate itm In m_SelectedTests Into Sum(itm.AssertCnt)

See also: another example of persisting settings: The VB version of the Blog Crawler

Create your own Test Host using XAML to run your unit tests

Calvin_Hsia — Thu, 27 Mar 2008 19:26:00 GMT

A few days ago, somebody came into my office and plopped down a box. It seemed very light. He said that it was a new PC. I thought hmmm…. The box seems empty…Why am I getting a new PC?.

Apparently an inventory was made and my current hardware was at the lower end of the list.

So I started up the Dell Optiplex 755 with 4 gigs of RAM, and it was running Vista 64. Super: I hadn’t done much with 64 bit code yet.

Sure enough, debugging 64 bit native applications shows that the CPU has 64 bit wide registers (and more registers!). If I attach to a 64 bit process, the Debug->Windows->Registers looks like:

RAX = 0000000000000000 RBX = 000000001DEB7818 RCX = 000000000F99A027

RDX = 0000000000000000 RSI = 0000000000000001 RDI = 00000000242E5454

R8 = 000000001FD3FCB8 R9 = 0000000000000000 R10 = 0000000000000000

R11 = 0000000000000206 R12 = 0000000000000000 R13 = 0000000000000000

R14 = 0000000000000000 R15 = 0000000000000000 RIP = 000007FEF0FD3D29

RSP = 000000001FD3FCF0 RBP = 0000000000000000 EFL = 00000246

If I attach to a 32 bit process:

EAX = 00000000 EBX = 006AC138 ECX = 79E74400 EDX = 00000000 ESI = 00DF6700

EDI = 00000000 EIP = 59B80265 ESP = 001DDEE8 EBP = 001DDEE8 EFL = 00000206

001DDEFC = 06CD35C8

(The debugger is pretty amazing!)

The sheer size of the registers means instead of a maximum 2^32 =4 gig virtual address space, we have 2^64

To calculate that, type this in the Fox command window:

?LOG10(2^64)

?2^64

Which shows 19.27, 1.844674E+19

That means 19.27 zeros: 18,000,000,000,000,000,000 which is 18 billion billion or 18 exabytes!

Coincidentally, we have some new 64 bit code that was recently created by “patching” the 32 bit version, and I wanted some test tools for it.

I have a VS Test Project with many (32 bit) tests. (Use Visual Studio Test framework to create tests for your code)

So I created my own Test Host in a day:

This Test Host:

· Uses the existing 32 bit test source code as linked files: if you make changes to the tests, they will automatically be updated in both 32 and 64 bit versions

· Can be built as 32 bit or 64 bit (Project->Properties->Compile->Advanced Compile Options->Target CPU->AnyCPU or x86), so it can run tests in either 32 or 64 bits. When set as AnyCPU and running on 64 bit OS like Vista 64, then it will run as 64 bit.

· Does not require test deployment: you can test in place. VSTestHost requires deployment.

· Uses reflection on itself to get and run the TestInitialize, TestCleanup and TestMethods.

· Can run selected tests in a loop for stress tests, like leak detection.

· Uses XAML and Data Binding (see Create your own media browser: Display your pictures, music, movies in a XAML tooltip)

o Notice how the enabling and disabling of buttons is done via data binding.

o The data binding updates the status (Success,Failure/Pending) and its color in the ListView.

o Shows the test methods in a ListView with click/sort column headers.

· The tests run on a worker thread. The UI is thus responsive and not blocked.

· A progress bar and stopwatch also show

· Can be run without UI: another assembly can call it to run tests

· Can determine if it’s running 64 or 32: IntPtr.Size=8 bytes on 64, 4 bytes on 32.

I’m thinking about adding a new 32 bit test case (for running in the VS IDE) that will launch all the 64 bit tests, or a new one for each of the 32 bit test cases, but that’s lower priority.

This new test host has already helped me find several 64 bit issues.

Follow my prior post: Use Visual Studio Test framework to create tests for your code to create a project. (If you don’t have Fox or Excel, you can either get them or remove the Fox/Excel code in the test)

If you add these lines to the test

System.Diagnostics.Debug.WriteLine("Sizeof IntPtr = " + IntPtr.Size.ToString)

System.Diagnostics.Debug.WriteLine(System.Diagnostics.Process.GetCurrentProcess.MainModule.FileName)

You’ll get the output:

Sizeof IntPtr = 4

C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\vstesthost.exe

Nothing unexpected. VSTestHost is a 32 bit EXE, which means only 32 bit code is run.

To make the test host more interesting, add a few more tests to the bottom of UnitTests1:

Shared oRandom As New Random

<TestMethod()> _

Sub EmptyTest()

End Sub

<TestMethod()> _

Sub AlwaysFail()

Assert.IsTrue(False, "True is never false!")

End Sub

<TestMethod()> _

Sub SometimesFail()

If oRandom.NextDouble < 0.4 Then

Assert.IsTrue(False, "True is never false!")

End If

End Sub

<TestMethod()> _

Sub RarelyFails()

If oRandom.NextDouble < 0.02 Then

Assert.IsTrue(False, "True is rarely false!")

End If

End Sub

To create the Test Host:

Right Click on the Solution from above in Solution explorer, choose Add->New Project. This time choose Windows->WPF Application. Put in a folder next to the folder of the existing test project. I called mine MyTestHost

Right Click on MyTestHost in solution explorer, choose Set As Startup Project. That way, hitting F5 will run this new project.

Delete the Window1.XAML in the solution explorer. We’ll create our own.

Delete MyTestHost->Application.xaml (we have our own Sub Main)

Right Click on MyTestHost in solution explorer, choose Add New Item, Code->Class. Name it MyTestHost.vb

Right Click on MyTestHost in solution explorer, choose Add Existing Item, navigate to the UnitTest1.vb file in the other project. Make sure you add as a Lnked File. The “Add” button on the dialog has a little down arrow that allows you to choose Add As Link.

Note: the Fox code in the unit test TestMethod1 will fail in 64 because there is no 64 bit version of it.

Add references to:

Microsoft.VisualStudio.QualityTools.UnitTestFramework

System.Windows.Forms

There are 3 sections of code to paste below: Window1.XAML, Window1.XAML.VB (the code behind file), and MyTestHost.vb

Hit F5, click on the button to run the tests.

You can force the Testmethod1 to fail simply by forcing Excel to close before it’s done.

Find the Executing function's name

Calvin_Hsia — Wed, 27 Feb 2008 11:12:00 GMT

Often I want to write the SAME code that will display the name of the currently executing method or function. That way I can just copy/paste the same code into multiple methods.

For example, in sub Form1_Load I could put this line:

System.Diagnostics.Debug.WriteLine("in Form1_Load")

In Button1_Click I’d have to out a different line:

System.Diagnostics.Debug.WriteLine("in Button1_Click")

In FoxPro, I can use the PROGRAM() function. How do you do it in .Net? Like this:

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

System.Diagnostics.Debug.WriteLine("subroutine name = " + (New StackTrace).GetFrames(0).GetMethod.Name)

End Sub

The output is:

subroutine name = Form1_Load

I use something similar to this code in my Unit Tests (Use Visual Studio Test framework to create tests for your code) so the test code can detect what the name of the test is.

In fact, you can even find out the calling assembly within a program, and do something different depending on the caller!

In fact, run this code:

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

Dim st = New StackTrace

Dim Methods = st.GetFrames

For Each meth In Methods

System.Diagnostics.Debug.WriteLine(meth.GetMethod.DeclaringType.FullName + ":" + meth.GetMethod.Name)

End Sub

If you set a breakpoint in this code, you’ll see that the output shows the method names:

This is the actual debugger output:

WindowsApplication1.exe!WindowsApplication1.Form1:Form1_Load

mscorlib.dll!System.EventHandler:Invoke

System.Windows.Forms.dll!System.Windows.Forms.Form:OnLoad

System.Windows.Forms.dll!System.Windows.Forms.Form:OnCreateControl

System.Windows.Forms.dll!System.Windows.Forms.Control:CreateControl

System.Windows.Forms.dll!System.Windows.Forms.Control:WmShowWindow

System.Windows.Forms.dll!System.Windows.Forms.Control:WndProc

System.Windows.Forms.dll!System.Windows.Forms.ScrollableControl:WndProc

System.Windows.Forms.dll!System.Windows.Forms.ContainerControl:WndProc

System.Windows.Forms.dll!System.Windows.Forms.Form:WmShowWindow

System.Windows.Forms.dll!System.Windows.Forms.Form:WndProc

System.Windows.Forms.dll!System.Windows.Forms.Control+ControlNativeWindow:OnMessage

System.Windows.Forms.dll!System.Windows.Forms.Control+ControlNativeWindow:WndProc

System.Windows.Forms.dll!System.Windows.Forms.NativeWindow:DebuggableCallback

System.Windows.Forms.dll!System.Windows.Forms.SafeNativeMethods:ShowWindow

System.Windows.Forms.dll!System.Windows.Forms.Control:SetVisibleCore

System.Windows.Forms.dll!System.Windows.Forms.Form:SetVisibleCore

System.Windows.Forms.dll!System.Windows.Forms.Control:set_Visible

System.Windows.Forms.dll!System.Windows.Forms.Application+ThreadContext:RunMessageLoopInner

System.Windows.Forms.dll!System.Windows.Forms.Application+ThreadContext:RunMessageLoop

System.Windows.Forms.dll!System.Windows.Forms.Application:Run

Microsoft.VisualBasic.dll!Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase:OnRun

Microsoft.VisualBasic.dll!Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase:DoApplicationModel

Microsoft.VisualBasic.dll!Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase:Run

WindowsApplication1.exe!WindowsApplication1.My.MyApplication:Main

mscorlib.dll!System.AppDomain:_nExecuteAssembly

mscorlib.dll!System.AppDomain:ExecuteAssembly

Microsoft.VisualStudio.HostingProcess.Utilities.dll!Microsoft.VisualStudio.HostingProcess.HostProc:RunUsersAssembly

mscorlib.dll!System.Threading.ThreadHelper:ThreadStart_Context

mscorlib.dll!System.Threading.ExecutionContext:Run

mscorlib.dll!System.Threading.ThreadHelper:ThreadStart

This is the actual stack from the Call Stack window (turn off Tools->Options->Debugging->Just My Code):

> WindowsApplication1.exe!WindowsApplication1.Form1.Form1_Load(Object sender = {WindowsApplication1.Form1}, System.EventArgs e = {System.EventArgs}) Line 9 Basic

[Native to Managed Transition]

[Managed to Native Transition]

System.Windows.Forms.dll!System.Windows.Forms.Form.OnLoad(System.EventArgs e) + 0x28b bytes

System.Windows.Forms.dll!System.Windows.Forms.Form.OnCreateControl() + 0x52 bytes

System.Windows.Forms.dll!System.Windows.Forms.Control.CreateControl(bool fIgnoreVisible) + 0x172 bytes

System.Windows.Forms.dll!System.Windows.Forms.Control.CreateControl() + 0x1b bytes

System.Windows.Forms.dll!System.Windows.Forms.Control.WmShowWindow(ref System.Windows.Forms.Message m) + 0x8e bytes

System.Windows.Forms.dll!System.Windows.Forms.Control.WndProc(ref System.Windows.Forms.Message m) + 0x6ed bytes

System.Windows.Forms.dll!System.Windows.Forms.ScrollableControl.WndProc(ref System.Windows.Forms.Message m) + 0x45 bytes

System.Windows.Forms.dll!System.Windows.Forms.ContainerControl.WndProc(ref System.Windows.Forms.Message m) + 0x11 bytes

System.Windows.Forms.dll!System.Windows.Forms.Form.WmShowWindow(ref System.Windows.Forms.Message m) + 0x3e bytes

System.Windows.Forms.dll!System.Windows.Forms.Form.WndProc(ref System.Windows.Forms.Message m) + 0x230 bytes

System.Windows.Forms.dll!System.Windows.Forms.Control.ControlNativeWindow.OnMessage(ref System.Windows.Forms.Message m) + 0xd bytes

System.Windows.Forms.dll!System.Windows.Forms.Control.ControlNativeWindow.WndProc(ref System.Windows.Forms.Message m) + 0x36 bytes

System.Windows.Forms.dll!System.Windows.Forms.NativeWindow.DebuggableCallback(System.IntPtr hWnd, int msg = 24, System.IntPtr wparam, System.IntPtr lparam) + 0x57 bytes

[Native to Managed Transition]

[Managed to Native Transition]

System.Windows.Forms.dll!System.Windows.Forms.Control.SetVisibleCore(bool value = true) + 0x12f bytes

System.Windows.Forms.dll!System.Windows.Forms.Form.SetVisibleCore(bool value = true) + 0xdc bytes

System.Windows.Forms.dll!System.Windows.Forms.Control.Visible.set(bool value) + 0xe bytes

System.Windows.Forms.dll!System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(int reason = -1, System.Windows.Forms.ApplicationContext context = {Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.WinFormsAppContext}) + 0xee bytes

System.Windows.Forms.dll!System.Windows.Forms.Application.ThreadContext.RunMessageLoop(int reason, System.Windows.Forms.ApplicationContext context) + 0x53 bytes

System.Windows.Forms.dll!System.Windows.Forms.Application.Run(System.Windows.Forms.ApplicationContext context) + 0x15 bytes

Microsoft.VisualBasic.dll!Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.OnRun() + 0xc0 bytes

Microsoft.VisualBasic.dll!Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.DoApplicationModel() + 0xe4 bytes

Microsoft.VisualBasic.dll!Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.Run(string[] commandLine) + 0x62 bytes

[Native to Managed Transition]

[Managed to Native Transition]

mscorlib.dll!System.AppDomain.ExecuteAssembly(string assemblyFile, System.Security.Policy.Evidence assemblySecurity, string[] args) + 0x39 bytes

Microsoft.VisualStudio.HostingProcess.Utilities.dll!Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly() + 0x2b bytes

mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state) + 0x3b bytes

mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) + 0x81 bytes

mscorlib.dll!System.Threading.ThreadHelper.ThreadStart() + 0x40 bytes

You can get similar results from Foxpro. Program(-1) will return the # of stack frames. SYS(16) returns the executing program, and Program(i) returns the name of the procedure at the stack level i.

res=func1()

?res

PROCEDURE func1

?PROGRAM()

RETURN func2()

PROCEDURE func2

?PROGRAM()

?PROGRAM(-1)

FOR i = 1 TO PROGRAM(-1)

?SYS(16)+ PROGRAM(i)

ENDFOR

RETURN 4

Use Visual Studio Test framework to create tests for your code

Calvin_Hsia — Mon, 25 Feb 2008 20:15:00 GMT

While writing software over a period of weeks or months, various components of the software get completed at various times. You’ve tested and you’re satisfied they work, and you move on to develop another feature. Or you might check in the source code and somebody else on your team might break your code accidentally with another checkin. This break might not be discovered for days or months.

Checking in test code for the feature alongside the feature code can help prevent such scenarios. These “unit” tests must be simple to run, easy to develop and maintain, and enable you to test your code well. Running the tests might take a minute, but give you the peace of mind that the change you made didn’t break other features. (Or if they did, they might be intentional feature changes, so the tests can be updated)

Software originates at the developer and passes through various stages, like through testing, building, customer installation, Discovering the bug closer to the developer and the time of development is very valuable: waiting til after the tester or customer gets the application is way too late!

Start VS 2008 (I believe it has to be VS Team Server edition or VS Test edition or something like that: See VS 2008 Team System Test Edition)

From the menus, choose File->New->Project Test Documents->Test Project. If you don’t see the option, you probably don’t have the right version of VS. You can even open your existing VS solution, right click on the solution at the top of the Solution Explorer, then choose Add->New->Project. This way, your test code can be in the same solution as your target code.

You can change the default test language (VB or C#) from Tools->Options->Test Tools->Test Project

From the menus, choose Test->Windows->Test List Editor. I like to have that window dockable: right click on the tab and choose Dockable, then dock it where you like.

Open UnitTest1.vb from the solution explorer and you’ll see this code:

<TestMethod()> Public Sub TestMethod1()

' TODO: Add test logic here

End Sub

Add some code that does something. In your first example, just use this sample code that creates an instance of Excel and my favorite VFP COM Object "t1.c1"

The code manipulates Excel via VB and indirectly via the Fox server. It validates that setting and retrieving Excel cell values works.

If you don’t have Fox, then you can remove the Fox parts

<TestMethod()> Public Sub TestMethod1()

Dim oFox = CreateObject("t1.c1") ' no references needed: use late binding

Dim y = oFox.MyEval("_vfp.servername")

System.Diagnostics.Debug.WriteLine("y = " + y)

Dim oExcel = CreateObject("excel.application") ' no references needed: use late binding

Dim oxlName = oExcel.name

System.Diagnostics.Debug.WriteLine("Excel name = " + oxlName + oExcel.version.ToString)

oExcel.visible = True

oExcel.workbooks.add()

oExcel.Cells(1, 1).value = "Hi there!"

System.Diagnostics.Debug.WriteLine("Excel cell=" + oExcel.Cells(1, 1).value)

Assert.AreEqual("Hi there!", oExcel.Cells(1, 1).value)

Const nRows As Integer = 30

Const nCols As Integer = 10

Dim nSum(nCols) As Integer ' the sum of the cols

For i = 2 To nRows

For j = 1 To nCols

Dim nVal = i * 100 + j ' "(" + i.ToString + "," + j.ToString + ")"

nSum(j - 1) += nVal

If i < nRows / 2 Then ' half the vals are filled directly, other half through Fox

oExcel.Cells(i, j).value = nVal

Else

oFox.mydocmd("p2.Cells(" + CStr(i) + "," + CStr(j) + ").value=p3", oExcel, nVal)

End If

System.Diagnostics.Debug.WriteLine(nVal)

Assert.AreEqual(nVal, CInt(oExcel.Cells(i, j).value))

For j = 1 To nCols

oExcel.Cells(nRows + 1, j).Value = _

"=Sum(" + Chr(j + 64) + "2:" + Chr(j + 64) + nRows.ToString + ")" '"=SUM(A2:A22)"

Assert.AreEqual(nSum(j - 1), CInt(oExcel.Cells(nRows + 1, j).value))

oExcel.ActiveWorkbook.Close(False)

oExcel.visible = False

oExcel.quit()

End Sub

Notice that the test code looks like normal code, except for the Assert.AreEqual calls. These Assert calls will throw an exception if the values aren’t equal, causing the test to fail.

You can remove the manual test from the project by deleting it from the solution. You can add more tests by just adding more Subs with the “TestMethod()” attribute.

You can run and debug/trace/set breakpoints in the test by hitting F5. Or you can run/debug the tests from the Test List Editor window, in which case you can still run other projects from within the same solution. For example, add the Test project to the same solution as the project under test and you can debug your target project while the tests are still running!

Try inserting this code before the line that closes the workbook:

For i = 2 To 40

For j = 1 To 30

Dim cval = "(" + i.ToString + "," + j.ToString + ")"

If i < 20 Then

cval = "VB " + cval

oExcel.Cells(i, j).value = cval

Else

cval = "Fox" + cval

oFox.mydocmd("p2.Cells(" + CStr(i) + "," + CStr(j) + ").value=p3", oExcel, cval)

End If

System.Diagnostics.Debug.WriteLine(cval)

Assert.AreEqual(cval, oExcel.Cells(i, j).value)

It sets the Excel spreadsheet values via calling Excel directly from the test using late binding,and via calling the Fox server to set the value. The C1 class of T1.C1 has a method MyDoCmd defined like so:

proc MyDoCmd(cCmd as string,p2 as Variant,p3 as Variant,p4 as Variant,p5 as Variant) ;

helpstring 'Execute a command|%2'

&cCmd

The “&” just means it takes the string command passed in and executes it. The parameters are called “p2”, “p3”, which are the passed in as the values oExcel and cval

After the test runs, the Test Results pane will show the results. Double-click on one to see the details. If it failed, the results even show you a call stack with hyperlinks of the failure, so you can just click on a stack entry to be taken to the line of code that failed. The failure could be either in the test code or the target code, but you can debug into the scenario to understand what’s going on.

You can make your test assembly be a Friend assembly (using InternalsVisibleTo) to your target project so that you can create instances of various test target classes in your test project.