Nathan Brixius : INFORMS

Solver Foundation TSP Part II: Directives, Solver Plugins, Model Libraries

Nathan Brixius — Fri, 08 May 2009 06:47:00 GMT

Here is my as-promised "part 2" to my traveling salesman sample. My goal is not to try and do any "production level" modeling, but instead "point the way" by highlighting different aspects of Solver Foundation's architecture. This time I want to talk about directives, plug-in solvers, and model reusability. The Solver Foundation architecture provides the ability to plug-in other solvers to handle one or more problem types. As of version 1.1, the Gurobi solver is Solver Foundation's default MIP (mixed integer programming) solver. Gurobi is a high-performance solver which is capable of taking on industrial strength problems, so my piddling little TSP example is no match for it. Gurobi has a number of parameters that can be set to tune solver performance - you set them by means of Directives. First, let's modify our code to turn on logging. Change the context.Solve() call to:

      GurobiDirective gurobiDirective = new GurobiDirective();
      gurobiDirective.OutputFlag = true;
      Solution solution = context.Solve(gurobiDirective);

Each plug-in solver also defines its own directives - therefore the full power and flexibility of plug-in solvers are available to you as a programmer. In this case I have just flipped on the logging bit - here's what I see.

Optimize a model with 185 Rows, 380 Columns and 1199 NonZeroes
Presolve removed 1 rows and 185 columns
Presolve time: 0.30s
Presolved: 184 Rows, 195 Columns, 832 Nonzeros
Objective GCD is 1

Root relaxation: objective 2.783286e+03, 36 iterations, 0.02 seconds

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0  2783.2857    0   14          -  2783.2857     -      -    0s
     0     0  3006.1964    0   29          -  3006.1964     -      -    0s
     0     0  3006.1964    0   29          -  3006.1964     -      -    0s
     0     0  3006.1964    0   25          -  3006.1964     -      -    0s
     0     0  3006.1964    0   25          -  3006.1964     -      -    0s
H    0     0                       6029.0000  3006.1964  50.1%     -    0s
     0     0  3006.1964    0   25  6029.0000  3006.1964  50.1%     -    0s
H    0     2                       3805.0000  3006.1964  21.0%     -    0s
     0     2  3006.1964    0   25  3805.0000  3006.1964  21.0%     -    0s
*   75    54              29       3644.0000  3077.0000  15.6%   8.2    0s
*  129    80              27       3381.0000  3077.0000  8.99%   8.7    0s
*  158    62              30       3323.0000  3077.0000  7.40%   8.1    0s

Cutting planes:
  Learned: 3
  Gomory: 1
  MIR: 3

Explored 1231 nodes (5693 simplex iterations) in 0.69 seconds
Thread count was 2 (of 2 available processors)

Optimal solution found (tolerance 1.00e-04)
Best objective 3.3230000000e+03, best bound 3.3230000000e+03, gap 0.0000%

For large models it is sometimes useful to tweak the solver options for performance. For example, if I wanted to spend a slightly larger amount of time doing MIP heuristics, and I wanted to eliminate presolve, then I can set those options easily:

      gurobiDirective.Heuristics = 0.1;
      gurobiDirective.Presolve = PresolveLevel.None;

There is a full reference in the Gurobi Solver Programming Primer that ships with Solver Foundation. My point is that you do not get a "least common denominator" solution - you get all the bells and whistles.

Gurobi is just one example - there are a wide range of plug-ins available, covering most of the popular LP and MIP solvers - check out solverfoundation.com for the full list. To use a plug-in solver, you need to: 1) install the solver (duh), 2) change your exe.config or web.config to point to the solver. No code changes are required. You can find additional documentation on the default MIP directives and the third-party Solver configuration in the SFS Programming Primer.

Last thing - modern programming languages like C# are cool because they promote reusability and maintainability. Don't forget about that when you are writing SFS code. It's common for models in a particular vertical (say, finance or transportation) to have common "blocks" of goals or constraints that are re-used over and over again. It's pretty easy to build reusable model libraries using extension methods. For example, take the assignment constraints in my TSP example. I could factor them out:

  public static class ModelingExtensions {
    public static void AssignmentConstraintsNoDiag(this Model model, Set s, Decision assign) {
      model.AddConstraint("A1", Model.ForEach(s, i => Model.Sum(Model.ForEachWhere(s, j => assign[i, j], j => i != j)) == 1));
      model.AddConstraint("A2", Model.ForEach(s, j => Model.Sum(Model.ForEachWhere(s, i => assign[i, j], i => i != j)) == 1));
    }
  }

Now I can just say model.AssignmentConstraintsNoDiag(city, assign). I could add other variations inside of ModelingExtensions as well. I think this aspect of Solver Foundation has not gotten enough attention - maybe because it doesn't make for a cool demo!

Solving traveling salesman problems using Solver Foundation

Nathan Brixius — Mon, 27 Apr 2009 18:18:00 GMT

Here's an example that I walked through during yesterday's INFORMS session. Erwin has two blog postings about Solver Foundation and the traveling salesman problem, but I want to throw in my two cents because I want to emphasize a couple of points:

By combining C# and Solver Foundation Services it is possible to express complex models clearly and succinctly.
It is very easy to build powerful, reusable model libraries using C# and Solver Foundation Services.
Solver Foundation Services code can be used in many different application environments (ASP.Net, silverlight, DB, command line apps, WPF, …) with minimal changes.

The traveling salesman problem is a classical problem in computer science, and you should bow your head in shame if you don't know about it (and turn in your conference badge if you happen to be in Phoenix). A salesperson needs to make a tour of a number of cities. The restrictions are that she wants to visit each city once and only once, and she wants to minimize the distance travelled. This is perhaps the definitive example of an NP-hard problem.

TSP can be solved using mixed integer programming - optimizing a linear goal with linear constraints, where some of the decision variables are integer. In this first post I will show how to formulate and solve a TSP model using Solver Foundation Services. In my second post I will show how to use the Gurobi MIP solver using SFS. There are many different ways to model the TSP - here is a nice introduction. My goal is to provide a clear, complete example - not build a "production level" TSP model, so I am going to choose a model formulation that dates back to 1960! First, I need to establish a couple of building blocks that will help me construct the data for the model. We need to know the distances between each pair of cities. Typically we are provided the coordinates of the cities and need to derive the distances. So I will introduce a Coordinate class that contains properties for the (x, y) coordinates, and properties to convert to latitude and longitude. Finally, a method that computes the distance between points.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;
using SolverFoundation.Plugin.Gurobi;

namespace Microsoft.SolverFoundation.Samples.TravelingSalesman {
  class Program {
    // TSP coordinate.
    public class Coordinate {
      public int Name { get; set; }

      // X-coordinate (from TSPLIB)
      public double X { get; set; }

      // Y-coordinate (from TSPLIB)
      public double Y { get; set; }

      public Coordinate(int name, double x, double y) {
        Name = name;
        X = x;
        Y = y;
      }

      // Latitude in radians.
      public double Latitude {
        get { return Math.PI * (Math.Truncate(X) + 5 * (X - Math.Truncate(X)) / 3) / 180; }
      }

      // Longitude in radians.
      public double Longitude {
        get { return Math.PI * (Math.Truncate(Y) + 5 * (Y - Math.Truncate(Y)) / 3) / 180; }
      }

      // Geographic distance between two points (as an integer).
      public int Distance(Coordinate p) {
        double q1 = Math.Cos(Longitude - p.Longitude);
        double q2 = Math.Cos(Latitude - p.Latitude);
        double q3 = Math.Cos(Latitude + p.Latitude);
        // There may rounding difficulties her if the points are close together...just sayin'.
        return (int)(6378.388 * Math.Acos(0.5 * ((1 + q1) * q2 - (1 - q1) * q3)) + 1);
      }
    }

    // TSP city-city arc.
    // 
    public class Arc {
      public int City1 { get; set; }
      public int City2 { get; set; }
      public double Distance { get; set; }
    }

    // Burma14 from TSPLIB. Optimal tour = 3323.
    private static Coordinate[] data = new Coordinate[] {
      new Coordinate(0, 16.47, 96.10),
      new Coordinate(1, 16.47, 94.44),
      new Coordinate(2, 20.09, 92.54),
      new Coordinate(3, 22.39, 93.37),
      new Coordinate(4, 25.23, 97.24),
      new Coordinate(5, 22.00, 96.05),
      new Coordinate(6, 20.47, 97.02),
      new Coordinate(7, 17.20, 96.29),
      new Coordinate(8, 16.30, 97.38),
      new Coordinate(9, 14.05, 98.12),
      new Coordinate(10, 16.53, 97.38),
      new Coordinate(11, 21.52, 95.59),
      new Coordinate(12, 19.41, 97.13),
      new Coordinate(13, 20.09, 94.55)
    };

(The data for this 14-city problem comes from the TSPLIB library). If you've been following my blog you know that the building blocks of a Solver Foundation model are: sets, parameters, decisions, goals, and constraints. I am going to implement a simple formulation that is centered around the following (indexed) decisions:

Assign[i,j]: this is equal to 1 if the optimal tour contains a trip (or arc) from city i to city j.
Rank[i]: this is equal to the number of cities visited after arriving at city i.

We have one parameter in our model:

Distance[I,j]: the distance from city i to city j.

With that in mind, here's the model. Explanation of the goals and constraints follow.

    static void Main(string[] args) {
      SolverContext context = SolverContext.GetContext();
      Model model = context.CreateModel();

      // ------------
      // Parameters
      Set city = new Set(Domain.IntegerNonnegative, "city");
      Parameter dist = new Parameter(Domain.Real, "dist", city, city);
      var arcs = from p1 in data
                 from p2 in data
                 select new Arc { City1 = p1.Name, City2 = p2.Name, Distance = p1.Distance(p2) };
      dist.SetBinding(arcs, "Distance", "City1", "City2");
      model.AddParameters(dist);

      // ------------
      // Decisions
      Decision assign = new Decision(Domain.IntegerRange(0, 1), "assign", city, city);
      Decision rank = new Decision(Domain.RealNonnegative, "rank", city);
      model.AddDecisions(assign, rank);

      // ------------
      // Goal: minimize the length of the tour.
      Goal goal = model.AddGoal("TourLength", GoalKind.Minimize,
        Model.Sum(Model.ForEach(city, i => Model.ForEachWhere(city, j => dist[i, j] * assign[i, j], j => i != j))));

      // ------------
      // Enter and leave each city only once.
      int N = data.Length;
      model.AddConstraint("assign 1",
        Model.ForEach(city, i => Model.Sum(Model.ForEachWhere(city, j => assign[i, j],
          j => i != j)) == 1));
      model.AddConstraint("assign 2",
        Model.ForEach(city, j => Model.Sum(Model.ForEachWhere(city, i => assign[i, j], i => i != j)) == 1));
      model.AssignmentNoDiag(city, assign);

      // Forbid subtours (Miller, Tucker, Zemlin - 1960...)
      model.AddConstraint("no subtours",
        Model.ForEach(city,
          i => Model.ForEachWhere(city,
            j => rank[i] + 1 <= rank[j] + N * (1 - assign[i, j]),
            j => Model.And(i != j, i >= 1, j >= 1)
          )
        )
      );

      Solution solution = context.Solve();

      // Retrieve solution information.
      Console.WriteLine("Cost = {0}", goal.ToDouble());
      Console.WriteLine("Tour:");
      var tour = from p in assign.GetValues() where (double)p[0] > 0.9 select p[2];
      foreach (var i in tour.ToArray()) {
        Console.Write(i + " -> ");
      }
      Console.WriteLine();
    }
  }

In my humble opinion, the "Parameter data =" line is an awesome example of the power of LINQ data binding in Solver Foundation.  We generate the 2D matrix of distances using a single LINQ expression.  It would be incredibly easy to change the code to retrieve the coordinate data from a database (perhaps using a LINQ expression once again), a file, or even a user application.

The goal is straightforward: minimize the distance traveled.  This is a product of the selected arcs and the distance matrix.   We have two types of constraints:

Assignment constraints: these ensure that we enter and leave each city only once.

Subtour constraints: these ensure that we do not have any subtours.  In a four city problem {A, B, C, D}, for example, we cannot have two cycles (A, B), (C, D).  We need to have one tour that contains all the cities.

The assignment constraints are easy using the ForEach and ForEachWhere operations.  I use ForEachWhere because I want to disallow arcs that enter and leave the same city - that doesn't make sense.  The subtour constraint is a little more complicated.   It relates the "assign" and "rank" decisions.  The key fact is that if there is an arc from city i to city j, rank[i] + 1 == j.  Of course, if the (i, j) arc is not part of the optimal tour then all bets are off.   Last note: notice that I can mix parameters, decisions, and C# variables in my expressions.

Once we solve the model we want to print out the cost, and the optimal tour. Getting the cost is very easy using goal.ToDouble(). We can get the tour using either Assign or Rank. I have chosen to use Assign because it gives me another opportunity to use LINQ. When you call GetValues() on a decision, you get arrays that contain the value along with the indexes for each decision. In this case, the last entry in the array is the one we are interested in. There are other ways to conveniently query decsision results, I'll save that for another time.

The next post will show how we can use Solver Foundation's plug-in model to tune the behavior of the Gurobi MIP solver.

Modeling a production planning problem using Solver Foundation

Nathan Brixius — Fri, 24 Apr 2009 23:01:00 GMT

In this post I am going to present two complete C# programs for modeling and solving a simple production problem using Solver Foundation. In this example we have two refineries (located in Saudi Arabia and Venzuela) that produce three products: gasoline, jet fuel, and lubricant. The goal is to minimize production costs, which depend on location. There is demand for each product which must be met. Finally, each production site has a limited production capacity.

The code for the simple case is below. Creating a model amounts to adding the goals, decisions, and constraints using the appropriate Add method. The signature for each is similar: the first argument is the name and the second argument is a term. Terms are created by combining decisions or parameters using the usual operators, or by using static methods on the Model class (as we will see in our second example). Finally, notice the call to SolverContext.Solve(). I have supplied a Directive that specifies that the Simplex solver should be used.

using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;

namespace Microsoft.SolverFoundation.Samples.Petrochem {
  class Program {
    static void Main(string[] args) {
      PetrochemSimple();
    }

    private static void PetrochemSimple() {
      SolverContext context = SolverContext.GetContext();
      context.ClearModel();
      Model model = context.CreateModel();

      Decision sa = new Decision(Domain.RealRange(0, 9000), "SA");
      Decision vz = new Decision(Domain.RealRange(0, 6000), "VZ");
      model.AddDecisions(sa, vz);

      model.AddGoal("goal", GoalKind.Minimize, 20 * sa + 15 * vz);

      model.AddConstraint("demand1", 0.3 * sa + 0.4 * vz >= 1900);
      model.AddConstraint("demand2", 0.4 * sa + 0.2 * vz >= 1500);
      model.AddConstraint("demand3", 0.2 * sa + 0.3 * vz >= 500);

      Solution solution = context.Solve(new SimplexDirective());
      Report report = solution.GetReport();
      Console.WriteLine(report);
    }
  }

Now let's reimplement the same example using data binding. Data binding is a powerful mechanism for creating large, maintainable models. Notice that in my first example, the numeric data such as the yields, the demands, and capacities were expressed directly in the terms. The first step in using Solver Foundation data binding is to lift these values into Parameters. It is often useful to create indexed parameters using Sets. In this example, there are two clearly defined Sets: the set of countries, and the set of products. In my example below I create a DataSet which contains the data for each of my parameters. (The GetData() method is just an example, it's not pretty but it is needed to complete the example.) Then I create a series of indexed parameters. For each of them I call the SetBinding method to associate the data with the parameter. In addition to the data, SetBinding also requires the caller to indicate which property specifies the values for the parameter. If the parameter is indexed, I also need to specify the parameters of the index properties. Since I am working with DataTables, these are simply column names. Notice that I could swap in any other data source that is enumerable - in particular LINQ works really well with Solver Foundation parameters.

After the parameters are created, I define the decisions, goals, and constraints. Notice that there is only one decision - it is indexed. The Model.Sum and Model.Foreach operations allow me to define a series of constraints over one or more indexed sets in one single statement. This means that if I were to add more countries or products, my model definition would not change at all.

    private static void PetrochemDataBinding() {
      SolverContext context = SolverContext.GetContext();
      context.ClearModel(); 
      Model model = context.CreateModel();

      // Retrieve the problem data.
      DataSet data = GetData();

      Set products = new Set(Domain.Any, "products");
      Set countries = new Set(Domain.Any, "countries");
      
      Parameter demand = new Parameter(Domain.Real, "demand", products);
      demand.SetBinding(data.Tables["Demand"].AsEnumerable(), "Demand", "Product");

      Parameter yield = new Parameter(Domain.Real, "yield", products, countries);
      yield.SetBinding(data.Tables["Yield"].AsEnumerable(), "Yield", "Product", "Country");

      Parameter limit = new Parameter(Domain.Real, "limit", countries);
      limit.SetBinding(data.Tables["Limit"].AsEnumerable(), "Limit", "Country");

      Parameter cost = new Parameter(Domain.Real, "cost", countries);
      cost.SetBinding(data.Tables["Cost"].AsEnumerable(), "Cost", "Country");

      model.AddParameters(demand, yield, limit, cost);

      Decision produce = new Decision(Domain.RealNonnegative, "produce", countries);
      model.AddDecision(produce);

      model.AddGoal("goal", GoalKind.Minimize, Model.Sum(Model.ForEach(countries, c => cost[c] * produce[c])));

      model.AddConstraint("Demand", 
        Model.ForEach(products, p => Model.Sum(Model.ForEach(countries, c => yield[p, c] * produce[c])) >= demand[p])
        );

      model.AddConstraint("Production limit",
        Model.ForEach(countries, c => produce[c] <= limit[c])
        );

      Solution solution = context.Solve(new SimplexDirective());
      Report report = solution.GetReport();
      Console.WriteLine(report);
    }

    private static DataSet GetData() {
      string[] products = new string[] { "Gas", "Jet Fuel", "Lubricant" };
      string[] countries = new string[] { "SA", "VZ" };

      double[][] yield = new double[][] {
        new double[] { 0.3, 0.4 }, 
        new double[] { 0.4, 0.2 }, 
        new double[] { 0.2, 0.3 } 
      };
      double[] demand = new double[] { 1900, 1500, 500 };
      double[] limit = new double[] { 9000, 6000 };
      double[] cost = new double[] { 20, 15 };

      DataSet dataSet = new DataSet();
      #region Fill DataSet
      DataTable table = new DataTable("Yield");
      dataSet.Tables.Add(table);
      table.Columns.Add("Product", typeof(string));
      table.Columns.Add("Country", typeof(string));
      table.Columns.Add("Yield", typeof(double));

      for (int p = 0; p < products.Length; p++) {
        for (int c = 0; c < countries.Length; c++) {
          DataRow row = table.NewRow();
          row[0] = products[p];
          row[1] = countries[c];
          row[2] = yield[p][c];
          table.Rows.Add(row);
        }
      }

      table = new DataTable("Demand");
      dataSet.Tables.Add(table);
      table.Columns.Add("Product", typeof(string));
      table.Columns.Add("Demand", typeof(double));
      for (int p = 0; p < products.Length; p++) {
        DataRow row = table.NewRow();
        row[0] = products[p];
        row[1] = demand[p];
        table.Rows.Add(row);
      }

      table = new DataTable("Limit");
      dataSet.Tables.Add(table);
      table.Columns.Add("Country", typeof(string));
      table.Columns.Add("Limit", typeof(double));
      for (int c = 0; c < countries.Length; c++) {
        DataRow row = table.NewRow();
        row[0] = countries[c];
        row[1] = limit[c];
        table.Rows.Add(row);
      }

      table = new DataTable("Cost");
      dataSet.Tables.Add(table);
      table.Columns.Add("Country", typeof(string));
      table.Columns.Add("Cost", typeof(double));
      for (int c = 0; c < countries.Length; c++) {
        DataRow row = table.NewRow();
        row[0] = countries[c];
        row[1] = cost[c];
        table.Rows.Add(row);
      }
      #endregion

      return dataSet;
    }

Solver Foundation at INFORMS Practice Conference

Nathan Brixius — Thu, 23 Apr 2009 00:34:00 GMT

A few of us on the Solver Foundation team will be presenting at the INFORMS Practice Conference in Phoenix, Arizona. I'm looking forward to it! We have both a workshop and a session - I will also be participating in Gurobi's session to show off its integration with Solver Foundation. Here are the details - looking forward to seeing you there.

Technology Workshop
Sunday, April 26, 9AM - 12PM
Microsoft Solver Foundation is a pure, managed code runtime for mathematical programming, modeling, and optimization. Microsoft Solver Foundation provides solvers and services to a broad community of users: from Excel users and analysts to programmers working on business critical scheduling, configuration, risk management and planning solutions. It provides services for model validation, parallel solving and workload scheduling, model interchange, and declarative data binding via LINQ and other NETFx technologies. As an open framework designed for third party extensibility, it exposes facilities for users to plug-in their own solvers while still leveraging all of the modeling services and capabilities of Microsoft Solver Foundation.

Software Tutorials
Monday, April 27, 9:10AM - 10:00AM
Solver Foundation is a framework and managed code runtime for mathematical programming, modeling and optimization. This tutorial will focus on the technical overview of Solver Foundation, and how Third-party solver vendors, modelers and solution specialists can leverage Solver Foundation from all CLS-compliant languages and Microsoft Office.