Janne Mattila's blog : Application Development

Maximize the use of CPU with parallel extensions (+ some WPF stuff)

jannemattila — Tue, 26 Aug 2008 22:03:00 GMT

Since this is my 40th post to this blog I decided to go back to square one… or post one actually :-) I’m going to create Windows Presentation Foundation (WPF) application that solves the Knight’s Tour puzzle. I actually didn’t know about this puzzle before I bought book called Puzzles for Programmers and Pros. That book had interesting puzzle that lead in to this post. So here we go!

I said that I’m going to create WPF application for my UI. You might ask why not the “good old” Windows Forms application...? Well for these simple reasons:

1. I don’t like to write code to OnPaint / MainForm_Paint methods.
2. I wanted to define my user interface and then just say in code “hey knight go there” and it should just draw the UI with the knight in the correct position. But the defined UI must be also scalable.
3. WPF doesn’t have same barriers than Windows Forms does => It’s the face of future applications!

I don’t probably have to explain my reason #1 for you if you have experienced the same that I have :-) You’ll end up writing the UI code a lot and that’s not what you’re trying to do. You’re trying to solve puzzle and you are suddenly focusing for the UI code. That’s wrong approach. Therefore reason #2 goes hand-in-hand with #1.

So let’s look the the UI of the running application:

It’s the view of the classic chess board and I decided to take shortcut when creating the knight. I decided to use gray circle instead (or ellipse actually) :-) And for the layout management I just took the easy approach by using Grid and defining Columns and Rows. Here is the XAML for the UI:

<Window x:Class="Window1"
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   Title="Knight's Tour" MinHeight="100" MinWidth="100" 
     Width="300" Height="300" Loaded="Window_Loaded">
  <Grid x:Name="Board">
    
    <Grid.ColumnDefinitions>
      <ColumnDefinition />
      <ColumnDefinition />
      <ColumnDefinition />
      <ColumnDefinition />
      <ColumnDefinition />
      <ColumnDefinition />
      <ColumnDefinition />
      <ColumnDefinition />
    </Grid.ColumnDefinitions>
    <Grid.RowDefinitions>
      <RowDefinition />
      <RowDefinition />
      <RowDefinition />
      <RowDefinition />
      <RowDefinition />
      <RowDefinition />
      <RowDefinition />
      <RowDefinition />
    </Grid.RowDefinitions>

    <Ellipse x:Name="Knight" Fill="Gray" Panel.ZIndex="1"
         Grid.Column="{Binding Path=KnightX}" 
         Grid.Row="{Binding Path=KnightY}" />
    
    <Rectangle x:Name="A8" Fill="White" />
    <Rectangle x:Name="B8" Grid.Column="1" />
    <!-- Etc... -->
  </Grid>
</Window>

You probably noticed the interesting part of the XAML... and that’s the lines 30 and 31 where Binding is defined. It means that these values coming from the public properties of the DataContext. So let’s look at the code behind that XAML and let’s discuss the binding little bit more:

using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Windows;
using System.Windows.Media;
using System.Windows.Shapes;

public partial class Window1 : Window, INotifyPropertyChanged
{
  private int knightX = 0;
  public int KnightX
  {
    get { return knightX; }
    set
    {
      knightX = value;
      NotifyPropertyChanged("KnightX");
    }
  }

  private int knightY = 0;
  public int KnightY
  {
    get { return knightY; }
    set
    {
      knightY = value;
      NotifyPropertyChanged("KnightY");
    }
  }

  private void Window_Loaded(object sender, RoutedEventArgs e)
  {
    Knight.DataContext = this;
    // Now moving the Knight is easy!
    KnightX = 3;
    KnightY = 3;
  }

  public event PropertyChangedEventHandler PropertyChanged;
  private void NotifyPropertyChanged(String info)
  {
    if (PropertyChanged != null)
    {
      PropertyChanged(this, new PropertyChangedEventArgs(info));
    }
  }
  //...

You might already noticed that my window also implements INotyfyPropertyChanged interface. And my two properties actually call NotifyPropertyChanged method when they are changed. So what’s this all about? Well if you don’t do this your values will be updated for the first time and after that they don’t actually get “bubbled” up to the ellipse anymore... unless you implement the notify mechanisms yourself. This is quite important and you should probably read more information about it on MSDN.

For the actual solving part I just used classic “old recursion” to solve the puzzle. And this is the part where we finally are going to the title of my post...

Classic one worker thread approach gives “fairly easy to implement but sub-optimal” solution. And you might ask why? And to answer this question I’m going so show you picture of task manager:

This picture was taken when my solver was running in “full speed ahead” –mode. And guess what... I’m not impressed! I’m actually just using single CPU (see the third box where green line has reached the roof)!!! So my worker thread approach is far from optimal resource usage.

Okay... What can I do then? I could do multiple threads and handle them manually but that’s again writing a lot of code that doesn’t have anything to do with the actual solving!? So if I would chosen Windows Forms + manual handling of multiple threads I would have a lot of code and just small fraction of that would actually do work that I was originally planning to do.

This is where Parallel Extensions (download) comes into the game! It’s additional library (System.Threading.dll) sitting on top of .NET Framework 3.5 and it’s currently in CTP phase. But I still highly recommend you to check it out if you want easily get more horse power to your algorithms.

I analyzed my code and noticed part where I could do things differently:

// My code was this:
foreach (int location in startLocations)
{
  // Calculations here!
}

// And I changed it to this:
Parallel.ForEach<int>(startLocations, (location) =>
{
  // Calculations here!
});

So I changed code in line 2 to be the code at line 8. What was the result at the task manager then:

Well I believe that I managed to get better use of the available horse power :-) It shows of course in the results:
“foreach”: ~50 solved solutions in ~5 minutes
”Parallel.ForEach”: ~450 solved solutions in < 5 minutes

To summarize... I just changed 1 line of code and I was able to get unbelievable results from it! So if you’re doing something similar then I recommend checking out the parallel extensions first before doing “own custom solution” for that.

I originally thought that I would go little bit deeper into the details of my solver but this post ended up too long even without it so maybe I’ll pass this time... But I’ll include video clip that shows the UI of the application when it’s solving.

Anyways... Happy hacking!

J

P.S. I’m also interested in F# and I’m probably going to do something fun with that too.

CRM 4.0 (or SharePoint or custom application) and DebugView

jannemattila — Wed, 07 May 2008 10:04:03 GMT

Every now and then I’m find myself trying to solve same issues over and over again :-) That’s why I found myself (again) using DebugView as my debugging assistant at remote box. If you don’t know what DebugView is then you should definitely try it out. I’m going to give you few ideas how you could use it at your applications. You can download the DebugView from Technet.

Why do I use DebugView? Well I want to get debug messages from running system BUT... I don’t want to write to EventLog or to File since it’s totally unnecessary to write all the messages all the time. I just want messages when I’m ready to observe the system. And DebugView is handy tool for that. Here is list of steps how I normally implement that kind of approach:
1) Create log class that implements logging/tracing/debugging (this is pretty much just wrapper to few simple method calls) (I recommend making this as singleton)
2) Use you log class in your application/solution
3) Use DebugView to follow the trace messages written by your log class

In 1) I mention that I recommend using singleton approach at the log class. This is important especially at the CRM 4.0 where you need to add default trace listener so that you’ll get the trace messages. Here is the code example for that:

1
2
3

using System.Diagnostics;
// ...
Trace.Listeners.Add(new DefaultTraceListener());

After you have added the default listener you can just use this oneliner:

1	Trace.WriteLine("My Trace: " + message);

Now your trace can be seen at the DebugView:

(here is CRM 4.0 where I have menu item that points to custom ASPX page and I’m using “My Trace” to see what’s happening in there)

DebugView has many built-in functionalities like “Save As” etc. that can help you on your debug/trace efforts. So I recommend that you learn to play around with it.

I also had the magic word “SharePoint” in my title. That is for the simple reason that this same story applies to SharePoint too. And don’t forget... Since System.Diagnostics.Trace is implemented at the .NET Framework this story also applies to all other applications which are built on top of that.

You might also want to read my previous post about tracing: CRM 4.0, SharePoint and ASP.NET Trace.

Anyways... Happy hacking!

J

How to install Windows SharePoint Services 3.0 Tools to XP or Vista

jannemattila — Thu, 16 Aug 2007 06:42:24 GMT

I got email from Philip Edney with a hint that how you can install Windows SharePoint Services 3.0 Tools: Visual Studio 2005 Extensions (VSeWSS) for none-Windows 2003 server OS. Of course since that's not supported you'll have to do small registry hack (but that's what developers do :-):

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0]
"Sharepoint"="Installed"

I just tested it with Windows Vista and it works! Before the hack it gives me "This product can be installed if Windows SharePoint Services 3.0 has been installed first." and after the hack it went fine. I think there're a lot of people who would like to try/use this option even if they fall for unsupported mode.

But thanks to Phil to let me know about that possibility! Update: Phil mentioned that he actually got the information from Raju Sakthivel so the credits chain just goes on and on :-)

Anyways... Happy hacking!

J

Solving small puzzles (with just a few lines of code)

jannemattila — Sun, 07 Jan 2007 15:42:34 GMT

For my first blog post I decided to choose subject that is just really me: Solving small puzzles. It's one of my weird hobbies that requires a bit of understanding of the puzzle and of course some programming skills. I have used this same idea that I'm going to present here to many puzzles.

Background of the puzzle: Survo

Survo puzzle is invention of Seppo Mustonen. He invented Survo puzzles in 2006, so it's quite new. Idea of Survo puzzle is similar to Sudoku. You need to add numbers to the puzzle so that row and column sums are same as given values. Biggest difference to Sudoku is the number of solutions. In Survo puzzle there will be only one possible solution. Puzzles are created so that solution is unique. As opposite to the Sudoku that might not be the case. Also numbers that are used to solve the case isn't limited to 1 to 9. Board size isn't fixed to some predefined size like in Sudoku and it's typically something like 3x3, 4x3 10x2 or 6x6. All this will make sense when we look at the example. More background can be find from Survo puzzle homepages or from paper written by Seppo Mustonen. It also worth mentioning that they give degree of the difficulty for the puzzle. Really easy puzzles are something around 1 to 10 and the most challenging Survo puzzle is 17000 (which is also known as the beast).

Important note: All Survo puzzles used in this page are taken from the Survo puzzle homepages or from published places such as Helsingin yliopiston tiedelehti (=Science magazine of Helsinki University).

First example of Survo puzzle

Here is our first Survo puzzle (already inside my application :). For now let's concentrate only to the grid where is many empty slots and few given numbers. We can see that this puzzle is 4x3 and puzzle has 3 extra numbers given already. Idea is to fill those white slots with numbers 1 to 12 (which comes from 4x3). And numbers 3, 6 and 8 are already given to the grid. So there is still available 1, 2, 4, 5, 7, 9, 10, 11 and 12 (=9 numbers). Those numbers should be put to white slots so that each row and column sum matches the values given in the gray slots. Ex. On column 1 you could put 10-8-9 (8 was already given) which satisfies the rule 27. But in order to those be good selections you need to still be able to satisfy the row restrictions too: 30, 18 and 30.

Normally people use paper and pen to solve these kind of problems, but I don't like that :) I prefer creating program that does the dirty work for me. And this kind of problems are typically solved with Brute force / Game theory or whatever you want to call it. The idea is anyway to go through every possible configuration at the board, but use some kind of knowledge of the game to rule out bad configurations or selections away. And why do we want to do that? Well just because it would take a lot of time to go through all the possible combinations. In this example case it would be something like 9! = 362880 combinations. Since our example puzzle is really simple (degree of difficulty isn't know for this example) we don't need to create super Algorithm to solve that. But when board size grows and is something like 5x5 it's totally a different case: 25! = 1.55 * 10^25. That kind of puzzle takes a long time to solve if you go through every possible combination without any optimizations that would rule out unfit solutions. Next I'm going to tell you how I started solving this problem.

Creating the Survo Puzzle Solver

I started working on my Survo Puzzle Solver with Visual Studio 2005 and C#. As you can see from the UI of the application -> I'm not UI Designer :) I just created minimal set of buttons and grid that displays the current situation on the board. Then I created first version of my Algorithm. Here it is in Pseudocode:

<solver_code1>
1: Bool Solve 2: { 3: If ( board is full) 4: { 5: // Evaluate returns true if solution is found 6: // or false if solution isn't correct one. 7: return Evaluate(); 8: } 9: Foreach(location in board) 10: { 11: If (location is free) 12: { 13: Foreach(choice from available choices) 14: { 15: addChoiceToBoard(choice); 16: If (Solve()) 17: { 18: // We have solution for this puzzle! 19: } 20: removeChoiceFromBoard(choice); 21: } 22: } 23: } 24: }</solver_code1>

So you can easily see that our code uses Recursion. It means that it calls itself (line 16). Solver code 1 inserts available numbers to the board slots that are free and when it notices that board is full (line 3) it validates the solutions (line 7). If Evaluate method returns true our solution has been found:

This solver code 1 clearly solves the problem... right? It does, but not completely. This approach has a lot to improve. I call this phase 1: Code the functionality without any optimization. We have now solution that works, but it's simply isn't usable for a bit harder puzzles. It just takes too long time to get the solution. So now it's time to analyze our solution little bit and try to optimize it to be more robust. If you run that kind of code you would see that it would fill entire board and then validate the correctness of the solution (see image below which is taken during the solving process):

And you can see right away that it isn't correct because the first row doesn't pass the check: 1 + 6 + 2 + 4 = 30. If that part doesn't match then why have we continued with search under it? It's even more noticeable from treelike presentation (click images to see them original size)(Note: Now you know that I can't even draw :).

Full tree search

Optimized search tree

In the first tree every single node of the tree needs to be tested before we know the answer. Optimized search tree shows black nodes that indicates that this subtree can be ignored entirely. So red nodes are the ones that won't even be tested and we still know that the solution can be found from the rest of the nodes! So from this we can go to our next phase. Let's call it phase 2: Simple optimization.

We don't need to modify our code much in order to make simple optimization. Let's add check each time when the row is filled. This one is pretty simple to implement and improves performance quite a lot:

<solver_code2>
1: Bool Solve 2: { 3: If ( board is full) 4: { 5: // Evaluate returns true if solution is found 6: // or false if solution isn't correct one. 7: return Evaluate(); 8: } 9: Foreach(location in board) 10: { 11: If (location is free) 12: { 13: Foreach(choice from available choices) 14: { 15: addChoiceToBoard(choice); 16: If (location is end of the row AND17: EvaluateRow()) 18: {19: If (Solve()) 20: { 21: // We have solution for this puzzle! 22: } 23: } 24: removeChoiceFromBoard(choice); 25: } 26: } 27: } 28: }</solver_code2>

So before we go to deeper recursion, we'll check the conditions for it (lines 16 and 17). If EvaluateRow method returns true then current row is correctly filled. This improved performance but we haven't reached our goal yet. Since that optimization was easy to make we can take bit harder step next. Let's call it phase 3: return of the optimization! Idea is to check the conditions every time we insert item to the board:

<solver_code3>
1: Bool Solve 2: { 3: If ( board is full) 4: { 5: // Solution is found! 6: return true; 7: } 8: Foreach(location in board) 9: { 10: If (location is free) 11: { 12: Foreach(choice from available choices) 13: { 14: addChoiceToBoard(choice); 15: If (Evaluate()) 16: {17: If (Solve()) 18: { 19: // We have solution for this puzzle! 20: } 21: } 22: removeChoiceFromBoard(choice); 23: } 24: } 25: } 26: }</solver_code3>

Now Evaluate method checks that does the current board configuration satisfy the conditions set. So this kind of example could happen: We have row limit set to 20 and there is 12 + 1+ 3 already put in place in that row (=17). We know that there is still one place to fill and we know what numbers we have left. So if the smallest available number would be 5, then we couldn't satisfy the conditions since 12 + 1 + 3 + 5 isn't 20. So we can take back and ignore that subtree too. It optimizes the search a lot. With that Algorithm we can go solve puzzle of the degree of difficulty 8700 in two seconds!

Of course we need to go through the full search of the tree so that we can verify the uniqueness of the solution. But that's easy to implement. Just don't stop even if solution is found (line 6 on Solver code 3). Just modify it to store the solution somewhere (text file perhaps) and continue the search.

Why would I like to do this? Isn't this kind of solving a bit stupid?

Well yes and no :) Yes in sense, that you could have much more fun solving these by paper and pen. I'm not saying that you can't do that anymore. No in sense that you can use this kind of codebase to test your optimization skills. And Visual Studio is great tool in that! You can run performance/profiling tests and then see where the application spends most of it's time. This is actually really good exercise. You should try it!

Another good thing is that this Algorithm is pretty useful in even more challenging games like chess, tic-tac-toe, checkers, 4 in a row, etc. Creating the base for the solver is easy... extending it with really good Evaluate method is hard. For example in chess you need to take so many variable into count that it makes my head spin. But if you learn from easier games like this you could easily develop such skills. It's always nice to play games that have cool Artificial intelligence.

Wow.. I thought that I start with small first post, but it somehow ended like this. Maybe next time I'll write something shorter.

Anyways... happy hacking!

J

Updated: I finally had time to test my solver on the "beast" (degree of difficulty 17000 => most difficult ever published Survo puzzle).

And it even suprised me when it managed to solve it in almost 10 seconds! It also verified uniqueness of the solution in 20 seconds. I'll post the solution into here, but don't peek if you do want to solve it by yourself :)

Updated(2): I almost forgot... Jezz Santos found this really nice Sudoku article that you should definitely checkout: Microsoft Sudoku: Optimizing UMPC Applications for Touch and Ink on MSDN by Stephen Toub.