Nathan Brixius : C#

Writing high performance code

Nathan Brixius — Fri, 11 Dec 2009 04:29:00 GMT

During Scott Hanselman's podcast we briefly discussed managed code performance. Here are a few of my thoughts about writing high performance code. In the spirit of a cheesy business book (or motivational speaker), I will give away my main points in the form of controversial-sounding statements before proceeding to flog them to death:

Don't trust anyone.
Managed is better.

Some background: before I came to Microsoft I built solvers for both discrete and continuous optimization problems, primarily in C/C++, with a bit of Fortran thrown in. Back in 2002 when I was on the Project Server team we were faced with the decision of whether to rebuild our scheduling engine in C++ or C#. For the past year or so I have worked on the Solver Foundation team, which is pure C#. We build solvers that solve a range of large-scale optimization problems, and they are “industrial grade”: tens or hundreds of thousands of constraints are not uncommon.

Don't trust anyone. Rico Mariani says the first rule of performance is to measure. Listen to Rico. There are many reasons for this. The first is that nobody has ever written code that is doing exactly what you are trying to do. Otherwise (being a smart programmer) you should just call their code or (being a smart business person) you should just buy their code, try to do it better, or do something different entirely (perhaps mixed martial arts). So you should seek to understand how the code behaves, otherwise you will never find out - until it is too late.

The second is that you know your scenarios and priorities better than others do. You should already have an idea of how fast the code is supposed to be. Now sometimes the requirement truly is "make it as fast as possible". For numerical libraries this is often the case because many problems can be made to be arbitrarily complicated. For example, if you are modeling fluid dynamics you could use tighter meshes and get more accurate results. In most other cases (perhaps in a user interface) you have a hard number you need to hit. Of course, there are some simple guidelines that may apply in most situations. If you never go to the database, your server app will be fast. If you never allocate memory, your numeric code will fly. True or false? Well…measure.

The third is that even if someone else has measured code that seems to be similar to yours, their results may be misleading. For example, you depend on different collection classes, or you are running on a different processor. Or your input data is different in some subtle way. To take a very simple example, suppose you want to sort lists of 1 million integers, and somebody says, "hey, I measured different sorting algorithms on arrays of 1 million integers and quicksort is best." And they hand over the code and all of their results. Well, what do those arrays look like? Because if the data is mostly sorted then something like insertion sort may work better.

If you follow this advice you may end up with more questions: doesn't this take a lot of time and effort? How do I know when to stop measuring, or stop optimizing my code? If you start profiling at the very start of development, and build some simple skills, it's not too bad (and you learn a lot). If, for example, you are considering whether to use managed or unmanaged code for a project, prototype a couple of key sections in C++ and C# and measure. I did this on the Project Server team and that was the best way to settle the argument. These things often get way too philosophical – just try it and see. I did standard critical path scheduling in both C# and C++ and found that the C# version was only 10-15% slower. We went with C# and in the course of development we were able to find algorithmic improvements that resulted in a faster implementation that the original C++ codebase. Get good at writing simple programs that answer one specific performance question. For example: should I use a list or an array? How should I read this data set? Be a scientist and seek to answer specific questions by controlling the conditions and varying only one thing at a time. And pick the right thing based on your knowledge of the customer scenario.

Managed or unmanaged? My blunt opinion is: unless your computation is dominated by BLAS-style computations you are better off using C# because of productivity, stability, and maintainability concerns.

Here are a couple of “pros” for C#:

There are clear performance differences, but in many cases when you look at the overall system performance they can be washed away due to other factors: I/O, a webservice call, DB access, etc.
Many programmers (myself included) are much more productive writing C# code. This gives me more time to investigate possible algorithmic improvements, write good unit tests, etc. There are opportunity costs!
I think it is easier to make use of multiple cores. For example, the parallel extensions available for .Net 3.5 (and built into 4.0) are very easy to use.

There are also some clear downsides:

It is easy to get lazy, or to unintentionally use constructs that are not efficient. People start using List<> and Dictionary<> when a good ol’ array will do. It’s also easy to miss unnecessary memory allocations, because everything is garbage collected. So you need to be vigilant about profiling the code.
There is a lot of unnecessary array bounds checking that slows things down. In particular, with LAPACK-style routines with tight, nested loops.
As far as I know, the managed code compilers do not make use of SSE2 instructions.

Others have different opinions and I’m not saying they’re wrong - so long as they have measured. Otherwise they are bullshitting you.

Solver Foundation on Hanselminutes

Nathan Brixius — Wed, 09 Dec 2009 15:57:00 GMT

The latest edition of Scott Hanselman's Hanselminutes podcast features yours truly. We talked about Solver Foundation, numerical optimization, and good old-fashioned code optimization. Scott's a great guy and doing the podcast was a lot of fun.

Those of you new to Solver Foundation (or this blog) may be curious about what you can actually do with Solver Foundation. Here is the download location for the free Express version of Solver Foundation. I hope that by listening to the podcast you already know the basics, so here are a few code-oriented posts:

Here is a short description of “what’s new” in our 2.0 release.
Here is a C# sample that solves traveling salesman problems.
Here is a LINQ to SQL sample using Solver Foundation.
Here is an F# sample. The F# integration features a domain specific language for creating optimization problems. It also supports “units of measure”, meaning you can associate real-life units such as miles, gallons, dollars, or touchdowns to parameters and decisions.
Erwin did a really nice post on doing portfolio management using Solver Foundation and Excel. I like the visuals. In fact Erwin's site has a number of great samples and documents.

And I'd be remiss if I didn't mention that Solver Foundation ships with TONS of samples that span languages, scenarios, and solvers.

The last part of our conversation was about writing high-performance code. I'll share some thoughts about that later this week.

Controlling Simplex and IPM termination conditions

Nathan Brixius — Mon, 30 Nov 2009 21:17:00 GMT

On our MSDN message board there was a question about how to control when the SimplexSolver aborts. You can do this by providing an abort callback to the params object sent to the solver. For example, to terminate after 100ms:

    private static void PetrochemLinearModel() {
      SimplexSolver solver = new SimplexSolver();
      int sa, vz;
      solver.AddVariable("SA", out sa);
      solver.SetBounds(sa, 0, 9000);
      solver.AddVariable("VZ", out vz);
      solver.SetBounds(vz, 0, 6000);

      int goal;
      solver.AddRow("goal", out goal);
      solver.AddGoal(goal, 0, true);
      solver.SetCoefficient(goal, sa, 20);
      solver.SetCoefficient(goal, vz, 15);

      int demand1, demand2, demand3;
      solver.AddRow("demand1", out demand1);
      solver.SetCoefficient(demand1, sa, 0.3);
      solver.SetCoefficient(demand1, vz, 0.4);
      solver.SetLowerBound(demand1, 1126);

      solver.AddRow("demand2", out demand2);
      solver.SetCoefficient(demand2, sa, 0.4);
      solver.SetCoefficient(demand2, vz, 0.2);
      solver.SetLowerBound(demand2, 1500);

      solver.AddRow("demand3", out demand3);
      solver.SetCoefficient(demand3, sa, 0.2);
      solver.SetCoefficient(demand3, vz, 0.3);
      solver.SetLowerBound(demand3, 500);

      Stopwatch s = new Stopwatch();
      var param = new SimplexSolverParams(() => s.ElapsedMilliseconds > 100);
      s.Start();
      var solution = solver.Solve(param);
      Console.WriteLine(solution.LpResult);
      Console.WriteLine("goal = " + solution.GetValue(goal).ToDouble());
      Console.WriteLine("sa   = " + solution.GetValue(sa).ToDouble());
      Console.WriteLine("vz   = " + solution.GetValue(vz).ToDouble());
    }

If you are using the InteriorPointSolver, the callback can reference solver properties (including iteration count and duality gap) to do more sophisticated stuff. For example:

      var param = new InteriorPointSolverParams();
      param.IterationStartedCallback = () => {
        Console.WriteLine("{0}\t{1:f6} {2:e6}", solver.IterationCount, solver.Primal, solver.Gap.ToDouble());
        return false;
      };
      var solution = solver.Solve(param);

Using Solver Foundation's Unconstrained Nonlinear Solver

Nathan Brixius — Thu, 15 Oct 2009 22:48:00 GMT

Hey, did you know that Solver Foundation can solve unconstrained nonlinear optimization problems? Yep, it's true. Few people know about it because 1) we haven't talked about it too much and 2) it is not accessible from the SFS (which means you can't use OML or the Excel add-in to solve such problems). To use the solver you need to work with a solver object directly. The Samples directory contains a simple example, but since there has been a couple of requests I thought I would give a slighly more involved example.

The CompactQuasiNewtonSolver class solves unconstrained optimization problems: it finds the minimum (or maximum) of an unbound objective function f(x). x is an array of doubles, and f needs to be twice differentiable. The CQN solver uses the L-BFGS algorithm. Like other implementations, you need to provide a function that will serve up the function value and gradient on demand, given the variables x. To do that, set the GradientAndValueAtPoint property to a delegate that fills in the gradient, and returns the function value. You'll also need to specify a starting point.

The CompactQuasiNewtonSolver has a Solve method that takes a solver parameters object. It has a few interesting properties, including the solver tolerance and maximum iteration count. On construction you can also pass in an "abort" delegate. This function will be called periodically during the course of execution - if you return true then the solver will return. In my sample below, I abuse it to print out the progress of the solver as it proceeds. Note that I would have liked to have used solver.NumberOfIterations rather than keeping track of the number of calls myself. That's a bug, and it will get fixed in our 2.1 release.

The function can have as many variables as you like. In my example I am minimizing a multidimensional variant of the Rosenbrock function, a standard test problem. You can modify the code to use your own objective function by modifying the ValueAndGradient method. You can also learn more about the solver by consulting the Solver Programming Primer (page 24 or so).

using System;
using Microsoft.SolverFoundation.Solvers;

namespace Microsoft.SolverFoundation.Samples {
  /// <summary> Minimizes the generalized Rosenbrock function, a standard test case
  /// in nonlinear optimization. You can read more about the Rosenbrock
  /// function here:
  ///  http://en.wikipedia.org/wiki/Rosenbrock_function
  /// </summary>
  public class Rosenbrock {
    public static void Main(string[] args) {

      int n = 4; // make sure I'm even, see below.
      double[] x = new double[n];
      for (int i = 0; i < x.Length; i++) {
        x[i] = -1;
      }

      CompactQuasiNewtonSolver solver = new CompactQuasiNewtonSolver(n);
      int iterationCount = 0;
      CompactQuasiNewtonSolverParams param = new CompactQuasiNewtonSolverParams(() => {
        Console.WriteLine("{0} | {1:e6}", /*solver.NumberOfIterations*/ iterationCount++, solver.ToleranceDifference);
        return false;
      }
        );
      
      solver.SetStartingPoint(x);
      solver.GradientAndValueAtPoint = (GetGradientAndValueAtPoint)ValueAndGradient;
      solver.Minimize = true;

      var solutionQuality = solver.Solve(param);

      Console.WriteLine();
      Console.WriteLine("Solution quality = " + solutionQuality);
      Console.WriteLine("f(x) = " + solver.SolutionValue);
      solver.GetSolutionPoint(x);
      for (int i = 0; i < x.Length; i++) {
        Console.WriteLine("x[{0}] = {1}", i, x[i]);
      }
    }

    /// <summary>Get the function value and gradient for the specified point.
    /// </summary>
    /// <param name="x">The point at which the value and gradient should be computed.</param>
    /// <param name="gradient">The gradient vector, the same size as x.</param>
    /// <returns>The function value.</returns>
    private static double ValueAndGradient(double[] x, double[] gradient) {
      // The gradient and value are both computed inside the method for efficiency.
      // It's also possible to refactor value and gradient calculation into submethods.
      double value = 0;
      for (int j = 0; j < x.Length; j += 2) {
        double t1 = 1.0 - x[j];
        double t2 = 10.0 * (x[j + 1] - x[j] * x[j]);
        gradient[j + 1] = 20.0 * t2;
        gradient[j] = -2.0 * (x[j] * gradient[j + 1] + t1);
        value += t1 * t1 + t2 * t2;
      }
      return value;
    }
  }
}

Solver Foundation 2.0 Preview: Second Order Conic Programming

Nathan Brixius — Fri, 02 Oct 2009 07:32:00 GMT

A couple of weeks ago I wrote about one of our big features for Solver Foundation 2.0: stochastic programming. In this post I want to talk about a new solver for 2.0: an interior-point solver for Second Order Conic Programming (SOCP). Second Order Conic Programming is the problem of minimizing a linear function where there are linear terms constrained to be inside a second order cone. Also called conic quadratic programming, SOCP is important because it can be used to model many real-world robust optimization, engineering, and finance problems. You can even model certain types of chance constraints using SOCP. Here are a few favorite SOCP links:

SOCP on wikipedia.
Lobo, Vandenberghe, Boyd & Lebret's paper has several applications.
Alizadeh and Goldfarb's paper is also excellent.
Cornuejols and Tutuncu's book Optimization Methods in Finance has a couple of chapters on conic optimization.
If you want to get truly hardcore, Nemirovski and Vandenberghe have great notes posted online.

An SOCP can contain conventional linear constraints Ax >= b as well as second-order cone constraints. Some solvers define SOCP in terms of conic variables rather than conic constraints. Solver Foundation’s interior point solver uses conic constraints rather than conic variables because (IMHO) it leads to a more flexible, easy-to-program solver API. A second-order cone is a pointed cone where the norm of n - 1 variables t_i is less than the other variable t_0 (so that a cross-section is a circle): t_0 >= || t_i ||, t_0 >= 0. A second-order cone in three dimensions looks like an ice cream cone. A second-order conic constraint is similar, except that each t can be a linear term that may combine several variables. The first term t_0 is called the "primary row".

What this means is that if you know how to build linear programming problems using the ILinearModel API I described last time, then you pretty much know how to build SOCP problems. You need to build a row that represents the conic constraint using AddRow, then you need to call AddRow once for each term in the conic constraint. That's it.

Here's a short code sample: given an InteriorPointSolver, pass in a list of matrices F and a corresponding list of matrices g. We want to find a vector x that minimizes the sum of ||Fx + g|| over all entries in F and g. The steps are as follows:

Create an InteriorPointSolver and the lists F and g.
Create the descision variables x, and add the goal just as in my previous post.
Iterate through the matrices. We will create one conic inequality for each matrix. The primary row is an artificial variable t, which is an upper bound on the norm of Fx + g. The other rows are simply the rows of F, with the bounds coming from g.

/// <summary> Create a simple sum-of-norms problem from F and g.
/// </summary>
/// <remarks>Returns the variable vids so the solution vector x can be 
/// retrieved after Solve().</remarks>
private static int[] BuildSumOfNorms(InteriorPointSolver solver, 
  List<double[][]> F, List<double[]> g) {

  int[] x = new int[F[0][0].Length];
  for (int i = 0; i < x.Length; i++) {
    solver.AddVariable("x" + i, out x[i]);
  }

  // Add the goal.
  int goal;
  solver.AddRow("goal", out goal);
  solver.AddGoal(goal, 1, true);
  for (int k = 0; k < F.Count; k++) {
    int cone, t;
    // Add variable t_k, which is the upper bound on the kth norm.
    solver.AddVariable("t_" + k, out t);
    solver.SetCoefficient(goal, t, Rational.One);
    solver.SetBounds(t, Rational.Zero, Rational.PositiveInfinity);

    // Add a quadratic cone.
    solver.AddRow("cone_" + k, SecondOrderConeType.Quadratic, out cone);
    solver.SetBounds(cone, 0, Rational.PositiveInfinity);

    // Add the conic constraint
    int conicRow1;
    solver.AddRow("cr1_" + k, cone, SecondOrderConeRowType.PrimaryConic, 
      out conicRow1);
    solver.SetCoefficient(conicRow1, t, Rational.One);
    solver.SetLowerBound(conicRow1, Rational.Zero);

    for (int i = 0; i < F[k].Length; i++) {
      int conicRow2;
      solver.AddRow("cr2_" + k + "_" + i, cone, SecondOrderConeRowType.Conic, 
        out conicRow2);
      for (int j = 0; j < F[k][i].Length; j++) {
        if (F[k][i][j] != 0) {
          solver.SetCoefficient(conicRow2, x[j], F[k][i][j]);
        }
      }
      solver.SetLowerBound(conicRow2, -g[k][i]);
    }
  }

  return x;
}

Those are the basics. We will have SOCP analogues of the LinearModel APIs, as well as APIs that let you look at the details of the conic constraints. In Version 2.0 we will support creation and solution of SOCP only using the InteriorPointSolver API. We will not have SFS or OML support for this release.

Using the Solver Foundation Interior Point Solver

Nathan Brixius — Fri, 25 Sep 2009 00:24:00 GMT

I've had a few requests on this topic, so I thought I would give a simple example of how to write code against Solver Foundation's interior point solver. If you are new to Solver Foundation, be warned that this is something of an "advanced topic". The most complete, powerful Solver Foundation programming interface is Solver Foundation Services. The solverfoundation.com site has links to more information about the SFS, and I have worked through several samples on this blog: production planning, two-way data binding, traveling salesman, parameterized models, LINQ-to-SQL.

The InteriorPointSolver class is capable of solving linear, quadratic, and (in our upcoming 2.0 release) second order conic optimization problems. Let's take as an example the production planning problem I wrote about in a previous post. It is a simple linear programming problem with two decisions (the production for each refinery) and three constraints (the demand for each refined product). In our SFS sample we simply declared the decisions, goals, and constraints and expressed them algebraically using the SolverContext and Model. The InteriorPointSolver class represents linear models in a slightly different way: using variables and rows. Variables are the analogue of SFS Decisions, whereas rows can represent either goals or constraints. Both variables and rows can have upper and/or lower bounds. In a linear model, both goals and constraints consist of linear combinations of variables, such as "3*x + 4*y + 5*z". Whereas in the SFS we can express these terms very naturally, with the InteriorPointSolver, the terms are expressed one at a time by specifying the coefficient associated with each variable in the row. Here is the C# code:

  private static void PetrochemLinearModel() {
    InteriorPointSolver solver = new InteriorPointSolver();
    int sa, vz;
    solver.AddVariable("SA", out sa);
    solver.SetBounds(sa, 0, 9000);
    solver.AddVariable("VZ", out vz);
    solver.SetBounds(vz, 0, 6000);

    int goal;
    solver.AddRow("goal", out goal);
    solver.AddGoal(goal, 0, true);
    solver.SetCoefficient(goal, sa, 20);
    solver.SetCoefficient(goal, vz, 15);

    int demand1, demand2, demand3;
    solver.AddRow("demand1", out demand1);
    solver.SetCoefficient(demand1, sa, 0.3);
    solver.SetCoefficient(demand1, vz, 0.4);
    solver.SetLowerBound(demand1, 1126);

    solver.AddRow("demand2", out demand2);
    solver.SetCoefficient(demand2, sa, 0.4);
    solver.SetCoefficient(demand2, vz, 0.2);
    solver.SetLowerBound(demand2, 1500);

    solver.AddRow("demand3", out demand3);
    solver.SetCoefficient(demand3, sa, 0.2);
    solver.SetCoefficient(demand3, vz, 0.3);
    solver.SetLowerBound(demand3, 500);

    var param = new InteriorPointSolverParams();
    var solution = solver.Solve(param);
    Console.WriteLine("goal = " + solution.GetValue(goal).ToDouble());
    Console.WriteLine("sa   = " + solution.GetValue(sa).ToDouble());
    Console.WriteLine("vz   = " + solution.GetValue(vz).ToDouble());
  }

You'll also need to add "using Microsoft.SolverFoundation.Common;" and "using Microsoft.SolverFoundation.Solvers;" to the top of your program. Extending this code to solve a quadratic program is easy: there is an overload of SetCoefficient that takes two variables as input. In my next post I will talk about second order conic programming, and a few other improvements in the 2.0 version of the InteriorPointSolver.

Quadratic Assignment Problems: solution enumeration

Nathan Brixius — Mon, 27 Jul 2009 19:53:00 GMT

In the last few posts we have given code for computing the Gilmore Lawler bound for quadratic assignment problems, and described two different branching techniques. In this post we show how to enumerate solutions of a small QAP. The possible solutions to a QAP range over permutations over unassigned facilities and locations. If lower bounds do not increase after repeated branching, we may eventually assign nearly all the facilities to locations. If there are only 2 or 3 remaining, we may as well just enumerate all the possible solutions and pick the best. In practice, we hope that we will rarely have to do this, otherwise the branch-and-bound tree will be too large to process efficiently. Here is a procedure that enumerates over all solutions of a size 3 QAP. Iterating over permutations is a standard textbook problem, so maybe you can write cleaner code than I did here!

/// <summary>
/// Assume that all facilities have been assigned to locations,
/// except three. EnumerateSolutions enumerates all six remaining assignments
/// returns the best.
/// </summary>
private double EnumerateSolutions(Qap q, BranchAndBoundNode node, int[] pBest) {
double objBest;
int nPerm = perm3.GetLength(0);
int n = perm3.GetLength(1);
double[] obj = new double[nPerm];
int[] r = node.p.UnusedIndices().ToArray();
int[] c = node.pInv.UnusedIndices().ToArray();
int[] p = (int[])node.p.Clone();
Debug.Assert(r.Length == n, "Only call me for size " + n + " QAPs.");

// Try all the possible permutations.
for (int i = 0; i < nPerm; i++) {
    for (int j = 0; j < n; j++) {
      p[r[j]] = c[perm3[i, j]];
    }
    obj[i] = q.Evaluate(p);
}

// Get the best permutation and objective.
int best = obj.ArgMin(out objBest);
p.CopyTo(pBest, 0);
for (int j = 0; j < n; j++) {
    pBest[r[j]] = perm3[best, j];
}
return objBest;
}

If you combine the code from the following posts you will see that we almost have a working B&B solver:

In my next post I will fill in a couple of the gaps, and then I will re-post all of the code. Then we'll think about what we can actually do with a QAP solver!

Quadratic Assignment Problems: strong and weak branching

Nathan Brixius — Mon, 20 Jul 2009 05:19:00 GMT

Let's pick up where we left off last time and write a score-based Branch delegate. It will be used as the basis for both of our "real" branching functions.

private BranchingDecision BranchCore(BranchAndBoundNode node, double bound, double[][] S) {
RowColumnSums(S, node.Size);
double rowVal, colVal;
int rowBest = _rowSum.ArgMax(out rowVal);
int colBest = _colSum.ArgMax(out colVal);
if (rowVal > colVal) {
return new BranchingDecision(true, rowBest);
}
else {
return new BranchingDecision(false, colBest);
}
}

private void RowColumnSums(double[][] U, int size) {
if (_rowSum == null) {
    _rowSum = new double[_qap.Size];
    _colSum = new double[_qap.Size];
}
_rowSum.ConstantFill(Double.MinValue);
_colSum.ConstantFill(Double.MinValue);

for (int i = 0; i < size; i++) {
    for (int j = 0; j < size; j++) {
      _rowSum[i] += U[i][j];
      _colSum[j] += U[i][j];
    }
}
}

If we can create a reasonable score matrix and pass it into BranchCore, then we've got something. Two common techniques for QAP are:

Weak branching. Let S_ij be the reduced cost U_ij that we got from computing the lower bound. (U_ij is a lower bound on the amount that the bound on the subproblem will increase. So large U_ij means the subproblem will be a lot easier.)
Strong branching. Let S_ij be the bound for subproblem ij. The bigger the bound, the easier the subproblem.

Weak branching is trivial to implement for the Gilmore-Lawler (or GLB) bound. As I mentioned a long time ago, the reduced cost matrix essentially comes for free.

private BranchingDecision WeakBranch(BranchAndBoundNode node, double bound, double[][] U) {
return BranchCore(node, bound, U);
}

For strong branching we need to compute bounds for each subproblem. We overwrite the U matrix and pass it into BranchCore. Since the U matrix is used to fathom subproblems later on in the code, we subtract the parent bound from the bound of each subproblem.

private BranchingDecision StrongBranch(BranchAndBoundNode node, double bound, double[][] U) {
Qap r = new Qap(node.Size - 1);
double[][] Ur = MatrixUtilities.NewMatrix(node.Size - 1, node.Size - 1);
for (int i = 0; i < node.Size; i++) {
    for (int j = 0; j < node.Size; j++) {
      Qap.Reduce(node.Qap, r, i, j);
      U[i][j] = GLB.Bound(r, Ur) - bound;
    }
}
return BranchCore(node, bound, U);
}

Strong branching in general produces better guesses that weak branching. However, it comes at a computational cost. Therefore I want to introduce one last complication…er…feature. Since strong branching is smarter but more costly, we should only use it where it is most needed. I want to have a set of rules that governs when to use strong or weak branching. Branching decisions are most important at the top of the tree, or on nodes where the bound sucks. We can measure the suckiness of the bound for a node by computing the relative gap: the ratio of the gap between the bound and the incumbent solution, and the corresponding gap at the root. So our branching rule data structure will have Depth and Gap properties, as well as properties that store the Branch delegate information.

/// <summary>A description of a branching rule: when and how.
/// </summary>
public class BranchingRule {
/// <summary>The ID for this rule.
/// </summary>
public int Index { get; set; }
/// <summary>Maximum depth for which the rule applies.
/// </summary>
public int Depth { get; set; }
/// <summary>Maximum relative gap for which the rule applies.
/// </summary>
public double Gap { get; set; }
/// <summary>Strong or Weak.
/// </summary>
public BranchType Type { get; set; }
/// <summary>StrongBranch() or WeakBranch().
/// </summary>
internal Branch Rule { get; set; }

public BranchingRule(int index, double gap, int depth, BranchType type) {
    Index = index;
    Depth = depth;
    Gap = gap;
    Type = type;
}

public override string ToString() {
    return "Depth = " + Depth + ", Gap = " + Gap + ", Type = " + Type.ToString();
}
}

We will have an array of BranchingRule called _branchingRule. Given a node, we will select the first rule in the array whose depth is larger than the node depth, and whose gap is larger than the node gap.

private Branch SelectBranchingRule(BranchAndBoundNode node) {
double gap = _results.RelativeGap(node.LowerBound);
BranchingRule criterion = _branchingRules.First(entry => (node.Level <= entry.Depth) && (gap >= entry.Gap));
node.RuleIndex = criterion.Index;
return criterion.Rule;
}

I am cheating here because I have not defined the local variable _results. We'll get to that in a few posts. For now, just assume that RelativeGap is defined as I described above: (Objective - bound) / (Objective - RootBound).

At long last, that covers branching. There are many variations, but they are essentially variations of strong or weak branching. At this point we only have two more main topics to cover: what to do when you get to the bottom of the tree, and how do you capture and update results.

Quadratic assignment problems: branching

Nathan Brixius — Mon, 13 Jul 2009 02:24:00 GMT

(This is part of a long-running series on quadratic assignment problems in C#. Click here to catch up.)

Branching means to divide the search space represented by a node into subnodes. QAP is about making assignments of facilities to locations, so two possible approaches come to mind:

Branch into two subproblems: one where an assignment i -> j is allowed, and another where it is disallowed.
Pick a facility and try assigning it to all possible locations, creating N subproblems. (Or pick a location and assign all facilities to it…)

The second approach, called polytomic branching, turns out to be more effective. The problem with the first ("single assignment branching") is that the subproblems aren't that much easier than the parent problem. You'll have to branch a whole bunch of times before you are able to determine anything.

We still need to decide whether to fix a facility ("row branching") or a location ("column branching"). And after that we need to figure out which particular facility or location is best. At this point you may be wondering, "does it even matter?" The answer is that, yes, branching decisions are earth shatteringly important. Where's what I mean: if you make bad branching choices it could take a thousand times longer to solve the problem, or more! This is part of the reason why QAP (and branch-and-bound) are so interesting. Bad branching decisions lead to subproblems that are just about as hard as the original problem. It's kind of like this. Here is a data structure that represents the results of a polytomic branching decision.

/// <summary>A branching decision, the result of applying a branching rule.
/// </summary>
internal class BranchingDecision {
/// <summary>If true, enumerate over all unassigned rows.
/// </summary>
public bool IsRowBranch { get; set; }
/// <summary>The row or column index.
/// </summary>
public int Index { get; set; }
/// <summary>Create a new instance.
/// </summary>
public BranchingDecision(bool isRowBranch, int index) {
IsRowBranch = isRowBranch;
Index = index;
}
}

Ultimately we want to write methods that produce BranchingDecisions. The following delegate represents such a method (we will write two methods in the next blog post):

/// <summary>Makes a branching decision.
/// </summary>
internal delegate BranchingDecision Branch(BranchAndBoundNode node, double bound, double[][] U);

If you accept the claim that branching choices make a big difference, then it is clear that we want to branch so we get the easiest subproblems possible. That is, nodes that will be processed by the branch-and-bound algorithm as quickly as possible. We don't know how to measure that directly (why?), so we have to guess. A general framework for polytomic branching is to compute a score for each of the N^2 subproblems, and store it in a matrix. S_ij is the score for the subproblem created by fixing facility i to location j. Then compute the row and column sums of S, and branch on the row or the column with maximum sum. Making a branching choice reduces to computing scores on subproblems.

The next step will be to write methods that compute scores and return BranchingDecisions. We'll look at two common ways to do that next time. We're about two or three posts away from having a pretty decent branch-and-bound solver for QAP.

Solver Foundation LINQ to SQL example

Nathan Brixius — Tue, 02 Jun 2009 17:45:00 GMT

Erwin, a modeling consultant and top Solver Foundation user, encountered some problems trying to do two-way data binding using DataTable objects. There are more details on this discussion thread. Ross, a member of the Solver Foundation team, was kind enough to code up a workaround for Erwin's example. In addition to this CS file, you will need to create a new DBML file called SampleDataContext, as I described in a previous post.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Data;
using System.Data.OleDb;
using System.Data.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;
using System.IO;
 
namespace OML1
{
    class Test
    {
        static void Main(string[] args)
        {
            Test t = new Test();
            t.Solve();
        }
 
        // Holds the OML model
        string strModel = @"Model[
              Parameters[Sets,I],
              Parameters[Reals,p[I]],
 
              Decisions[Reals[0,Infinity],x[I]],
 
              Constraints[
                 Foreach[{i,I}, x[i]==p[i]]
              ]
           ]";
 
 
        //  SFS
        SolverContext context;
        SampleDataContext data;
 
 
        //  Constructor
        public Test()
        {
            context = SolverContext.GetContext();
            data = new SampleDataContext("Data Source=Sql_server_name;Initial Catalog=DataPartitionAllocation20_5;Integrated Security=True");
            context.DataSource = data;
        }
 
 
        // Solve the problem
        public void Solve()
        {
            context.LoadModel(FileFormat.OML, new StringReader(strModel));
 
            Parameter p = context.CurrentModel.Parameters.First(q => q.Name == "p");
            p.SetBinding(data.P, "value", new string[] { "index" });
 
            Decision x = context.CurrentModel.Decisions.First(d => d.Name == "x");
            x.SetBinding(data.X, "value", new string[] { "index" }); 
 
            Solution solution = context.Solve();
            Console.Write("{0}", solution.GetReport());
 
            context.PropagateDecisions();
 
        }
 
    }
 
}

Solver Foundation TSP Part II: Directives, Solver Plugins, Model Libraries

Nathan Brixius — Fri, 08 May 2009 06:47:00 GMT

Here is my as-promised "part 2" to my traveling salesman sample. My goal is not to try and do any "production level" modeling, but instead "point the way" by highlighting different aspects of Solver Foundation's architecture. This time I want to talk about directives, plug-in solvers, and model reusability. The Solver Foundation architecture provides the ability to plug-in other solvers to handle one or more problem types. As of version 1.1, the Gurobi solver is Solver Foundation's default MIP (mixed integer programming) solver. Gurobi is a high-performance solver which is capable of taking on industrial strength problems, so my piddling little TSP example is no match for it. Gurobi has a number of parameters that can be set to tune solver performance - you set them by means of Directives. First, let's modify our code to turn on logging. Change the context.Solve() call to:

      GurobiDirective gurobiDirective = new GurobiDirective();
      gurobiDirective.OutputFlag = true;
      Solution solution = context.Solve(gurobiDirective);

Each plug-in solver also defines its own directives - therefore the full power and flexibility of plug-in solvers are available to you as a programmer. In this case I have just flipped on the logging bit - here's what I see.

Optimize a model with 185 Rows, 380 Columns and 1199 NonZeroes
Presolve removed 1 rows and 185 columns
Presolve time: 0.30s
Presolved: 184 Rows, 195 Columns, 832 Nonzeros
Objective GCD is 1

Root relaxation: objective 2.783286e+03, 36 iterations, 0.02 seconds

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0  2783.2857    0   14          -  2783.2857     -      -    0s
     0     0  3006.1964    0   29          -  3006.1964     -      -    0s
     0     0  3006.1964    0   29          -  3006.1964     -      -    0s
     0     0  3006.1964    0   25          -  3006.1964     -      -    0s
     0     0  3006.1964    0   25          -  3006.1964     -      -    0s
H    0     0                       6029.0000  3006.1964  50.1%     -    0s
     0     0  3006.1964    0   25  6029.0000  3006.1964  50.1%     -    0s
H    0     2                       3805.0000  3006.1964  21.0%     -    0s
     0     2  3006.1964    0   25  3805.0000  3006.1964  21.0%     -    0s
*   75    54              29       3644.0000  3077.0000  15.6%   8.2    0s
*  129    80              27       3381.0000  3077.0000  8.99%   8.7    0s
*  158    62              30       3323.0000  3077.0000  7.40%   8.1    0s

Cutting planes:
  Learned: 3
  Gomory: 1
  MIR: 3

Explored 1231 nodes (5693 simplex iterations) in 0.69 seconds
Thread count was 2 (of 2 available processors)

Optimal solution found (tolerance 1.00e-04)
Best objective 3.3230000000e+03, best bound 3.3230000000e+03, gap 0.0000%

For large models it is sometimes useful to tweak the solver options for performance. For example, if I wanted to spend a slightly larger amount of time doing MIP heuristics, and I wanted to eliminate presolve, then I can set those options easily:

      gurobiDirective.Heuristics = 0.1;
      gurobiDirective.Presolve = PresolveLevel.None;

There is a full reference in the Gurobi Solver Programming Primer that ships with Solver Foundation. My point is that you do not get a "least common denominator" solution - you get all the bells and whistles.

Gurobi is just one example - there are a wide range of plug-ins available, covering most of the popular LP and MIP solvers - check out solverfoundation.com for the full list. To use a plug-in solver, you need to: 1) install the solver (duh), 2) change your exe.config or web.config to point to the solver. No code changes are required. You can find additional documentation on the default MIP directives and the third-party Solver configuration in the SFS Programming Primer.

Last thing - modern programming languages like C# are cool because they promote reusability and maintainability. Don't forget about that when you are writing SFS code. It's common for models in a particular vertical (say, finance or transportation) to have common "blocks" of goals or constraints that are re-used over and over again. It's pretty easy to build reusable model libraries using extension methods. For example, take the assignment constraints in my TSP example. I could factor them out:

  public static class ModelingExtensions {
    public static void AssignmentConstraintsNoDiag(this Model model, Set s, Decision assign) {
      model.AddConstraint("A1", Model.ForEach(s, i => Model.Sum(Model.ForEachWhere(s, j => assign[i, j], j => i != j)) == 1));
      model.AddConstraint("A2", Model.ForEach(s, j => Model.Sum(Model.ForEachWhere(s, i => assign[i, j], i => i != j)) == 1));
    }
  }

Now I can just say model.AssignmentConstraintsNoDiag(city, assign). I could add other variations inside of ModelingExtensions as well. I think this aspect of Solver Foundation has not gotten enough attention - maybe because it doesn't make for a cool demo!

Creating parameterized Solver Foundation models using LINQ to SQL

Nathan Brixius — Tue, 05 May 2009 23:29:00 GMT

On the Solver Foundation MSDN forum there was a question about how to read model data from a DB and use it within a Solver Foundation model. In this post I will extend my production planning sample to use LINQ to SQL. To follow along at home you will need to have a recent version of SQL Server installed locally, and some basic knowledge of how to create SQL tables. You should also have compiled and run the code from my previous post.

Step 1: Create and populate the DB

The first step is to create tables corresponding to the entities in my model. I created a very simple DB with three tables: Countries, Products, and Yields. The Yields table has foreign key constraints to the Countries and Products tables. Here is a diagram:

To populate the DB I just wrote a script that inserts my problem data, and ran itin SQL Management Studio. Here's the script (and forgive my SQL):

GO 

DELETE FROM Yields
DELETE FROM Products
DELETE FROM Countries
GO

INSERT INTO Countries (Id, Name, Limit, Cost)
VALUES (0, 'SA', 9000, 20)

INSERT INTO Countries (Id, Name, Limit, Cost)
VALUES (1, 'VZ', 6000, 15)

GO

INSERT INTO Products (Id, Name, Demand)
VALUES (0, 'Gas', 1900)

INSERT INTO Products (Id, Name, Demand)
VALUES (1, 'Jet Fuel', 1500)

INSERT INTO Products (Id, Name, Demand)
VALUES (2, 'Lubricant', 500)

GO

INSERT INTO Yields (CountryId, ProductId, Value)
VALUES (0, 0, 0.3)
INSERT INTO Yields (CountryId, ProductId, Value)
VALUES (1, 0, 0.4)

INSERT INTO Yields (CountryId, ProductId, Value)
VALUES (0, 1, 0.4)
INSERT INTO Yields (CountryId, ProductId, Value)
VALUES (1, 1, 0.2)

INSERT INTO Yields (CountryId, ProductId, Value)
VALUES (0, 2, 0.2)
INSERT INTO Yields (CountryId, ProductId, Value)
VALUES (1, 2, 0.3)

Step 2: Create Entity and DataContext classes in Visual Studio

Scott Guthrie's blog (and the MSDN docs) show you exactly how to do this:

Add a new "Linq to SQL file" to your project called Petrochem.dbml.
Bring up the Server Explorer window, connect to your database and drag the tables into the dbml window. Visual Studio will automatically create a Datacontext class (mine is called PetrochemDataContext) and an entity class for each table that you include.

Step 3: Modify Solver Foundation Services data binding code

This is in fact very easy because Solver Foundation Services was designed to work well with LINQ. Take the PetrochemDataBinding sample from last time, and change the SetBinding statements to work with the PetrochemDataContext class instead of a hardcoded DataSet. The code is almost identical:

  private static void PetrochemLinqDataBinding() {
      SolverContext context = SolverContext.GetContext();
      context.ClearModel();
      Model model = context.CreateModel();

      PetrochemDataContext db = new PetrochemDataContext();

      Set products = new Set(Domain.Any, "products");
      Set countries = new Set(Domain.Any, "countries");

      Parameter demand = new Parameter(Domain.Real, "demand", products);
      demand.SetBinding(db.Products, "Demand", "Id");

      Parameter yield = new Parameter(Domain.Real, "yield", products, countries);
      yield.SetBinding(db.Yields, "Value", "ProductId", "CountryId");

      Parameter limit = new Parameter(Domain.Real, "limit", countries);
      limit.SetBinding(db.Countries, "Limit", "Id");

      Parameter cost = new Parameter(Domain.Real, "cost", countries);
      cost.SetBinding(db.Countries, "Cost", "Id");

      model.AddParameters(demand, yield, limit, cost);

      Decision produce = new Decision(Domain.RealNonnegative, "produce", countries);
      model.AddDecision(produce);

      model.AddGoal("goal", GoalKind.Minimize, Model.Sum(Model.ForEach(countries, c => cost[c] * produce[c])));

      model.AddConstraint("Demand",
        Model.ForEach(products, p => Model.Sum(Model.ForEach(countries, c => yield[p, c] * produce[c])) >= demand[p])
        );

      model.AddConstraint("Production limit",
        Model.ForEach(countries, c => produce[c] <= limit[c])
        );

      Solution solution = context.Solve(new SimplexDirective());
      Report report = solution.GetReport();
      Console.WriteLine(report);
    }

That's all there is to it! Note that instead of passing the entire collection (e.g. db.Countries) you could easily use LINQ statements or stored procedures, or whatever you like.

Solving traveling salesman problems using Solver Foundation

Nathan Brixius — Mon, 27 Apr 2009 18:18:00 GMT

Here's an example that I walked through during yesterday's INFORMS session. Erwin has two blog postings about Solver Foundation and the traveling salesman problem, but I want to throw in my two cents because I want to emphasize a couple of points:

By combining C# and Solver Foundation Services it is possible to express complex models clearly and succinctly.
It is very easy to build powerful, reusable model libraries using C# and Solver Foundation Services.
Solver Foundation Services code can be used in many different application environments (ASP.Net, silverlight, DB, command line apps, WPF, …) with minimal changes.

The traveling salesman problem is a classical problem in computer science, and you should bow your head in shame if you don't know about it (and turn in your conference badge if you happen to be in Phoenix). A salesperson needs to make a tour of a number of cities. The restrictions are that she wants to visit each city once and only once, and she wants to minimize the distance travelled. This is perhaps the definitive example of an NP-hard problem.

TSP can be solved using mixed integer programming - optimizing a linear goal with linear constraints, where some of the decision variables are integer. In this first post I will show how to formulate and solve a TSP model using Solver Foundation Services. In my second post I will show how to use the Gurobi MIP solver using SFS. There are many different ways to model the TSP - here is a nice introduction. My goal is to provide a clear, complete example - not build a "production level" TSP model, so I am going to choose a model formulation that dates back to 1960! First, I need to establish a couple of building blocks that will help me construct the data for the model. We need to know the distances between each pair of cities. Typically we are provided the coordinates of the cities and need to derive the distances. So I will introduce a Coordinate class that contains properties for the (x, y) coordinates, and properties to convert to latitude and longitude. Finally, a method that computes the distance between points.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;
using SolverFoundation.Plugin.Gurobi;

namespace Microsoft.SolverFoundation.Samples.TravelingSalesman {
  class Program {
    // TSP coordinate.
    public class Coordinate {
      public int Name { get; set; }

      // X-coordinate (from TSPLIB)
      public double X { get; set; }

      // Y-coordinate (from TSPLIB)
      public double Y { get; set; }

      public Coordinate(int name, double x, double y) {
        Name = name;
        X = x;
        Y = y;
      }

      // Latitude in radians.
      public double Latitude {
        get { return Math.PI * (Math.Truncate(X) + 5 * (X - Math.Truncate(X)) / 3) / 180; }
      }

      // Longitude in radians.
      public double Longitude {
        get { return Math.PI * (Math.Truncate(Y) + 5 * (Y - Math.Truncate(Y)) / 3) / 180; }
      }

      // Geographic distance between two points (as an integer).
      public int Distance(Coordinate p) {
        double q1 = Math.Cos(Longitude - p.Longitude);
        double q2 = Math.Cos(Latitude - p.Latitude);
        double q3 = Math.Cos(Latitude + p.Latitude);
        // There may rounding difficulties her if the points are close together...just sayin'.
        return (int)(6378.388 * Math.Acos(0.5 * ((1 + q1) * q2 - (1 - q1) * q3)) + 1);
      }
    }

    // TSP city-city arc.
    // 
    public class Arc {
      public int City1 { get; set; }
      public int City2 { get; set; }
      public double Distance { get; set; }
    }

    // Burma14 from TSPLIB. Optimal tour = 3323.
    private static Coordinate[] data = new Coordinate[] {
      new Coordinate(0, 16.47, 96.10),
      new Coordinate(1, 16.47, 94.44),
      new Coordinate(2, 20.09, 92.54),
      new Coordinate(3, 22.39, 93.37),
      new Coordinate(4, 25.23, 97.24),
      new Coordinate(5, 22.00, 96.05),
      new Coordinate(6, 20.47, 97.02),
      new Coordinate(7, 17.20, 96.29),
      new Coordinate(8, 16.30, 97.38),
      new Coordinate(9, 14.05, 98.12),
      new Coordinate(10, 16.53, 97.38),
      new Coordinate(11, 21.52, 95.59),
      new Coordinate(12, 19.41, 97.13),
      new Coordinate(13, 20.09, 94.55)
    };

(The data for this 14-city problem comes from the TSPLIB library). If you've been following my blog you know that the building blocks of a Solver Foundation model are: sets, parameters, decisions, goals, and constraints. I am going to implement a simple formulation that is centered around the following (indexed) decisions:

Assign[i,j]: this is equal to 1 if the optimal tour contains a trip (or arc) from city i to city j.
Rank[i]: this is equal to the number of cities visited after arriving at city i.

We have one parameter in our model:

Distance[I,j]: the distance from city i to city j.

With that in mind, here's the model. Explanation of the goals and constraints follow.

    static void Main(string[] args) {
      SolverContext context = SolverContext.GetContext();
      Model model = context.CreateModel();

      // ------------
      // Parameters
      Set city = new Set(Domain.IntegerNonnegative, "city");
      Parameter dist = new Parameter(Domain.Real, "dist", city, city);
      var arcs = from p1 in data
                 from p2 in data
                 select new Arc { City1 = p1.Name, City2 = p2.Name, Distance = p1.Distance(p2) };
      dist.SetBinding(arcs, "Distance", "City1", "City2");
      model.AddParameters(dist);

      // ------------
      // Decisions
      Decision assign = new Decision(Domain.IntegerRange(0, 1), "assign", city, city);
      Decision rank = new Decision(Domain.RealNonnegative, "rank", city);
      model.AddDecisions(assign, rank);

      // ------------
      // Goal: minimize the length of the tour.
      Goal goal = model.AddGoal("TourLength", GoalKind.Minimize,
        Model.Sum(Model.ForEach(city, i => Model.ForEachWhere(city, j => dist[i, j] * assign[i, j], j => i != j))));

      // ------------
      // Enter and leave each city only once.
      int N = data.Length;
      model.AddConstraint("assign 1",
        Model.ForEach(city, i => Model.Sum(Model.ForEachWhere(city, j => assign[i, j],
          j => i != j)) == 1));
      model.AddConstraint("assign 2",
        Model.ForEach(city, j => Model.Sum(Model.ForEachWhere(city, i => assign[i, j], i => i != j)) == 1));
      model.AssignmentNoDiag(city, assign);

      // Forbid subtours (Miller, Tucker, Zemlin - 1960...)
      model.AddConstraint("no subtours",
        Model.ForEach(city,
          i => Model.ForEachWhere(city,
            j => rank[i] + 1 <= rank[j] + N * (1 - assign[i, j]),
            j => Model.And(i != j, i >= 1, j >= 1)
          )
        )
      );

      Solution solution = context.Solve();

      // Retrieve solution information.
      Console.WriteLine("Cost = {0}", goal.ToDouble());
      Console.WriteLine("Tour:");
      var tour = from p in assign.GetValues() where (double)p[0] > 0.9 select p[2];
      foreach (var i in tour.ToArray()) {
        Console.Write(i + " -> ");
      }
      Console.WriteLine();
    }
  }

In my humble opinion, the "Parameter data =" line is an awesome example of the power of LINQ data binding in Solver Foundation.  We generate the 2D matrix of distances using a single LINQ expression.  It would be incredibly easy to change the code to retrieve the coordinate data from a database (perhaps using a LINQ expression once again), a file, or even a user application.

The goal is straightforward: minimize the distance traveled.  This is a product of the selected arcs and the distance matrix.   We have two types of constraints:

Assignment constraints: these ensure that we enter and leave each city only once.

Subtour constraints: these ensure that we do not have any subtours.  In a four city problem {A, B, C, D}, for example, we cannot have two cycles (A, B), (C, D).  We need to have one tour that contains all the cities.

The assignment constraints are easy using the ForEach and ForEachWhere operations.  I use ForEachWhere because I want to disallow arcs that enter and leave the same city - that doesn't make sense.  The subtour constraint is a little more complicated.   It relates the "assign" and "rank" decisions.  The key fact is that if there is an arc from city i to city j, rank[i] + 1 == j.  Of course, if the (i, j) arc is not part of the optimal tour then all bets are off.   Last note: notice that I can mix parameters, decisions, and C# variables in my expressions.

Once we solve the model we want to print out the cost, and the optimal tour. Getting the cost is very easy using goal.ToDouble(). We can get the tour using either Assign or Rank. I have chosen to use Assign because it gives me another opportunity to use LINQ. When you call GetValues() on a decision, you get arrays that contain the value along with the indexes for each decision. In this case, the last entry in the array is the one we are interested in. There are other ways to conveniently query decsision results, I'll save that for another time.

The next post will show how we can use Solver Foundation's plug-in model to tune the behavior of the Gurobi MIP solver.

Modeling a production planning problem using Solver Foundation

Nathan Brixius — Fri, 24 Apr 2009 23:01:00 GMT

In this post I am going to present two complete C# programs for modeling and solving a simple production problem using Solver Foundation. In this example we have two refineries (located in Saudi Arabia and Venzuela) that produce three products: gasoline, jet fuel, and lubricant. The goal is to minimize production costs, which depend on location. There is demand for each product which must be met. Finally, each production site has a limited production capacity.

The code for the simple case is below. Creating a model amounts to adding the goals, decisions, and constraints using the appropriate Add method. The signature for each is similar: the first argument is the name and the second argument is a term. Terms are created by combining decisions or parameters using the usual operators, or by using static methods on the Model class (as we will see in our second example). Finally, notice the call to SolverContext.Solve(). I have supplied a Directive that specifies that the Simplex solver should be used.

using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;

namespace Microsoft.SolverFoundation.Samples.Petrochem {
  class Program {
    static void Main(string[] args) {
      PetrochemSimple();
    }

    private static void PetrochemSimple() {
      SolverContext context = SolverContext.GetContext();
      context.ClearModel();
      Model model = context.CreateModel();

      Decision sa = new Decision(Domain.RealRange(0, 9000), "SA");
      Decision vz = new Decision(Domain.RealRange(0, 6000), "VZ");
      model.AddDecisions(sa, vz);

      model.AddGoal("goal", GoalKind.Minimize, 20 * sa + 15 * vz);

      model.AddConstraint("demand1", 0.3 * sa + 0.4 * vz >= 1900);
      model.AddConstraint("demand2", 0.4 * sa + 0.2 * vz >= 1500);
      model.AddConstraint("demand3", 0.2 * sa + 0.3 * vz >= 500);

      Solution solution = context.Solve(new SimplexDirective());
      Report report = solution.GetReport();
      Console.WriteLine(report);
    }
  }

Now let's reimplement the same example using data binding. Data binding is a powerful mechanism for creating large, maintainable models. Notice that in my first example, the numeric data such as the yields, the demands, and capacities were expressed directly in the terms. The first step in using Solver Foundation data binding is to lift these values into Parameters. It is often useful to create indexed parameters using Sets. In this example, there are two clearly defined Sets: the set of countries, and the set of products. In my example below I create a DataSet which contains the data for each of my parameters. (The GetData() method is just an example, it's not pretty but it is needed to complete the example.) Then I create a series of indexed parameters. For each of them I call the SetBinding method to associate the data with the parameter. In addition to the data, SetBinding also requires the caller to indicate which property specifies the values for the parameter. If the parameter is indexed, I also need to specify the parameters of the index properties. Since I am working with DataTables, these are simply column names. Notice that I could swap in any other data source that is enumerable - in particular LINQ works really well with Solver Foundation parameters.

After the parameters are created, I define the decisions, goals, and constraints. Notice that there is only one decision - it is indexed. The Model.Sum and Model.Foreach operations allow me to define a series of constraints over one or more indexed sets in one single statement. This means that if I were to add more countries or products, my model definition would not change at all.

    private static void PetrochemDataBinding() {
      SolverContext context = SolverContext.GetContext();
      context.ClearModel(); 
      Model model = context.CreateModel();

      // Retrieve the problem data.
      DataSet data = GetData();

      Set products = new Set(Domain.Any, "products");
      Set countries = new Set(Domain.Any, "countries");
      
      Parameter demand = new Parameter(Domain.Real, "demand", products);
      demand.SetBinding(data.Tables["Demand"].AsEnumerable(), "Demand", "Product");

      Parameter yield = new Parameter(Domain.Real, "yield", products, countries);
      yield.SetBinding(data.Tables["Yield"].AsEnumerable(), "Yield", "Product", "Country");

      Parameter limit = new Parameter(Domain.Real, "limit", countries);
      limit.SetBinding(data.Tables["Limit"].AsEnumerable(), "Limit", "Country");

      Parameter cost = new Parameter(Domain.Real, "cost", countries);
      cost.SetBinding(data.Tables["Cost"].AsEnumerable(), "Cost", "Country");

      model.AddParameters(demand, yield, limit, cost);

      Decision produce = new Decision(Domain.RealNonnegative, "produce", countries);
      model.AddDecision(produce);

      model.AddGoal("goal", GoalKind.Minimize, Model.Sum(Model.ForEach(countries, c => cost[c] * produce[c])));

      model.AddConstraint("Demand", 
        Model.ForEach(products, p => Model.Sum(Model.ForEach(countries, c => yield[p, c] * produce[c])) >= demand[p])
        );

      model.AddConstraint("Production limit",
        Model.ForEach(countries, c => produce[c] <= limit[c])
        );

      Solution solution = context.Solve(new SimplexDirective());
      Report report = solution.GetReport();
      Console.WriteLine(report);
    }

    private static DataSet GetData() {
      string[] products = new string[] { "Gas", "Jet Fuel", "Lubricant" };
      string[] countries = new string[] { "SA", "VZ" };

      double[][] yield = new double[][] {
        new double[] { 0.3, 0.4 }, 
        new double[] { 0.4, 0.2 }, 
        new double[] { 0.2, 0.3 } 
      };
      double[] demand = new double[] { 1900, 1500, 500 };
      double[] limit = new double[] { 9000, 6000 };
      double[] cost = new double[] { 20, 15 };

      DataSet dataSet = new DataSet();
      #region Fill DataSet
      DataTable table = new DataTable("Yield");
      dataSet.Tables.Add(table);
      table.Columns.Add("Product", typeof(string));
      table.Columns.Add("Country", typeof(string));
      table.Columns.Add("Yield", typeof(double));

      for (int p = 0; p < products.Length; p++) {
        for (int c = 0; c < countries.Length; c++) {
          DataRow row = table.NewRow();
          row[0] = products[p];
          row[1] = countries[c];
          row[2] = yield[p][c];
          table.Rows.Add(row);
        }
      }

      table = new DataTable("Demand");
      dataSet.Tables.Add(table);
      table.Columns.Add("Product", typeof(string));
      table.Columns.Add("Demand", typeof(double));
      for (int p = 0; p < products.Length; p++) {
        DataRow row = table.NewRow();
        row[0] = products[p];
        row[1] = demand[p];
        table.Rows.Add(row);
      }

      table = new DataTable("Limit");
      dataSet.Tables.Add(table);
      table.Columns.Add("Country", typeof(string));
      table.Columns.Add("Limit", typeof(double));
      for (int c = 0; c < countries.Length; c++) {
        DataRow row = table.NewRow();
        row[0] = countries[c];
        row[1] = limit[c];
        table.Rows.Add(row);
      }

      table = new DataTable("Cost");
      dataSet.Tables.Add(table);
      table.Columns.Add("Country", typeof(string));
      table.Columns.Add("Cost", typeof(double));
      for (int c = 0; c < countries.Length; c++) {
        DataRow row = table.NewRow();
        row[0] = countries[c];
        row[1] = cost[c];
        table.Rows.Add(row);
      }
      #endregion

      return dataSet;
    }

Simple matrix, vector, permutation utilities

Nathan Brixius — Wed, 25 Mar 2009 07:24:00 GMT

I am going to be rolling out the rest of my branch-and-bound algorithm in the next few posts. To make that easier, in this post I introduce some common matrix, vector, and permutation methods. It turns out that for technical computing applications, C#'s extension methods (introduced in 3.0) are awesome. With vectors it's great because you retain the control of having direct array access, but you get the nice object-oriented notation.

I've left out comments and error-checking, and it's not super-optimized, but you get the idea. None of these methods end up being the bottleneck. Most of the methods are obvious, but I do want to comment on the permutation methods. The goal of QAP is to find an optimal assignment of facilities to locations, represented as a permutation. In the course of our branch-and-bound algorithm, we'll be making partial assignments - that is, only some of the entries in the permutation will be filled in. By convention, p[i] == -1 will mean that facility i is unassigned. An index where p[i] < 0 is called "unused". When we branch, we'll want to pick an unused facility (or location), and try assigning all unused locations (or facilities) to it. The use of C#'s iterator and yield concepts really help here.

That said, here's the code.

  public static class MatrixUtilities {
    #region Vector
    public static void ConstantFill(this T[] data, T val) {
      for (int i = 0; i < data.Length; i++) {
        data[i] = val;
      }
    }

    public static int ArgMin(this double[] data, out double best) {
      if (data.Length == 0) {
        best = Double.MinValue;
        return -1;
      }
      int iBest = 0;
      best = data[0];
      for (int i = 1; i < data.Length; i++) {
        if (best > data[i]) {
          best = data[i];
          iBest = i;
        }
      }
      return iBest;
    }

    public static int ArgMax(this double[] data, out double best) {
      if (data.Length == 0) {
        best = Double.MinValue;
        return -1;
      }
      int iBest = 0;
      best = data[0];
      for (int i = 1; i < data.Length; i++) {
        if (best < data[i]) {
          best = data[i];
          iBest = i;
        }
      }
      return iBest;
    }
    #endregion

    #region Permutation
    public static void Swap(this int[] p, int i, int j) {
      int temp = p[i];
      p[i] = p[j];
      p[j] = temp;
    }

    public static int FindUnused(this int[] data, int index) {
      foreach (int unused in data.UnusedIndices()) {
        if (index-- <= 0) {
          return unused;
        }
      }
      return -1;
    }

    public static IEnumerable UnusedIndices(this int[] data) {
      for (int i = 0; i < data.Length; i++) {
        if (data[i] < 0) {
          yield return i;
        }
      }
    }

    public static string PermutationToString(this int[] p, bool oneBased) {
      StringBuilder build = new StringBuilder(p.Length * 4);
      int width = ((int)Math.Log10(p.Length)) + 2;
      build.Append("[");
      for (int i = 0; i < p.Length; i++) {
        int index = oneBased ? p[i] + 1 : p[i];
        build.Append(index.ToString().PadLeft(width));
      }
      build.Append("]");
      return build.ToString();
    }
    #endregion

    #region Matrix

    public static string MatrixToString(this double[][] A) {
      if (A != null) {
        StringBuilder build = new StringBuilder(A.Length * A.Length * 3);
        for (int i = 0; i < A.Length; i++) {
          if (A[i] != null) {
            for (int j = 0; j < A[i].Length; j++) {
              build.AppendFormat("{0,4}", A[i][j]);
              build.Append(" ");
            }
            build.AppendLine();
          }
        }
        return build.ToString();
      }
      return null;
    }

    public static double[][] NewMatrix(int m, int n) {
      double[][] M = new double[m][];
      for (int i = 0; i < M.Length; i++) {
        M[i] = new double[n];
      }
      return M;
    }

    public static double[][] NewMatrix(int n) {
      return NewMatrix(n, n);
    }
    #endregion
  }