Luca Bolognese's WebLog : C# Programming

My PDC session is online - “Future directions for C# and Visual Basic”

lucabol — Mon, 23 Nov 2009 21:17:44 GMT

In case you are training to play the part of a first generation Italian immigrant in a Broadway show or if you want to understand why I never short msft stock: http://microsoftpdc.com/Sessions/FT11

Luca at PDC 2009 next week

lucabol — Fri, 13 Nov 2009 18:49:18 GMT

I’ll be in Los Angeles next week for PDC 2009. My session is called “Future Directions for C# and Visual Basic” and it comes on Tuesday immediately after the first keynote.

I’m planning on spending the first half of the session talking about the biggest trends influencing C# and VB. The second part will be about more future looking features we are playing with these days.

My deck has just two slides, as always. It’s all about watching me typing code on stage, hoping that I make crazy errors …

There are several other interesting language related sessions at PDC. Here is a list of them (please notice that the times might change). Also feel free to drop by the languages booth and chat. My team and I will be there most of the time.

Day	Time	Title	Presenter	Room
Tues 11/17	11:00 - 12:00	Future Directions for C# and Visual Basic	Luca Bolognese	Hall F
Tues 11/17	11:00 - 12:00	Accelerated Windows Application Development with Microsoft Visual C++ 2010	Boris Jabes	408B
Tues 11/17	12:30 - 13:15	Dynamic Binding in C# 4	Mads Torgersen	Hall F
Tues 11/17	12:30 - 13:15	Using Dynamic Languages to Build Scriptable Applications	Dino Viehland	403AB
Tues 11/17	13:30 - 14:30	C++ Forever: Interactive Applications in the Age of Manycore	Rick Molloy	515B
Tues 11/17	16:30 - 17:30	Manycore and the Microsoft .NET Framework 4: A Match Made in Microsoft Visual Studio 2010	Stephen Toub	502A
Tues 11/17	16:30 - 17:30	Code Contracts and Pex: Power Charge Your Assertions and Unit Tests	Mike Barnett, Nikolai Tillmann	408A
Wed 11/18	11:30 - 12:30	Microsoft Perspectives on the Future of Programming	Panel	Petree Hall C
Wed 11/18	12:00 - 13:00	F# for Architects – Hitting the Sweet Spot	Chris Smith	VS and .NET Pavillion
Wed 11/18	13:00 - 13:45	Code Like the Wind with Microsoft Visual Basic 2010	Lucian Wischik	Petree Hall D
Wed 11/18	13:00 - 13:45	Future of Garbage Collection	Patrick Dussud	Petree Hall C
Wed 11/18	13:00 - 13:45	Microsoft Project Code Name “M”: The Data and Modeling Language	Don Box, Jeff Pinkston	408A
Thu 11/19	08:30 - 09:30	PLINQ: LINQ, but Faster!	Ed Essey, Igor Ostrovsky	515A
Thu 11/19	10:00 - 11:00	A Lap around Microsoft Visual Studio 2010 for the Visual Basic Developer	Lisa Feigenbaum	VS and .NET Pavillion
Thu 11/19	10:00 - 11:00	Axum: A .NET Language for Safe and Scalable Concurrency	Niklas Gustafsson	515A
Thu 11/19	11:30 - 12:30	F# for Parallel and Asynchronous Programming	Luke Hoban	515A
Thu 11/19	12:45 - 13:30	Microsoft Visual C# IDE Tips and Tricks	DJ Park	Petree Hall D
Thu 11/19	12:45 - 13:30	Microsoft Visual Basic IDE Tips and Tricks	Dustin Campbell	Petree Hall C

Becoming really rich with C#

lucabol — Wed, 23 Sep 2009 02:40:00 GMT

Or maybe not, please do not hold me responsible if you lose money following this system. Having said that, it is my opinion that there are very few concepts that are important in investing. Three big ones are value, diversification and momentum. This post is about the latter two and how to use C# to create a simple trading system that uses both.

Diversification is ‘not put all your eggs in one basket’ (contrary to ‘put all of them in one basket and watch that basket’). I don’t believe you can ‘watch’ very much in financial markets, so I tend to prefer diversification.

Momentum is a mysterious tendency of financial prices that have risen the most in the recent past, to continue outperforming in the close future. In essence, buying the top stocks/sectors/asset classes tends to outperform buying the bottom ones over horizons from three months to one year.

The idea then is to rank some assets (i.e. ETFs) by how fast they have risen in the past, go long the top ones and short the bottom ones. There are hundreds of variations of this basic strategy, we’ll add the rule that we won’t buy assets that are below their 200 days moving average or sell short assets that are above it.

I’m writing this code with VS 2010 Beta 2 (which hasn’t shipped yet). It should be trivial to modify it to run on B1 (or maybe it does run on it already). I attach the code and data files to this post.

struct Event {
    internal Event(DateTime date, double price) { Date = date; Price = price; }
    internal readonly DateTime Date;
    internal readonly double Price;
}

We’ll use this simple structure to load the closing price for a particular date. My use of internal is kind of bizarre. Actually the whole code might look strange. It is an interesting (maybe un-elegant) mix of object orientation and functional programming.

class Summary {
    internal Summary(string ticker, string name, string assetClass,
                    string assetSubClass, double? weekly, double? fourWeeks,
                    double? threeMonths, double? sixMonths, double? oneYear,
                    double? stdDev, double price, double? mav200) {
        Ticker = ticker;
        Name = name;
        AssetClass = assetClass;
        AssetSubClass = assetSubClass;
        // Abracadabra ...
        LRS = (fourWeeks + threeMonths + sixMonths + oneYear) / 4;
        Weekly = weekly;
        FourWeeks = fourWeeks;
        ThreeMonths = threeMonths;
        SixMonths = sixMonths;
        OneYear = oneYear;
        StdDev = stdDev;
        Mav200 = mav200;
        Price = price;
    }
    internal readonly string Ticker;
    internal readonly string Name;
    internal readonly string AssetClass;
    internal readonly string AssetSubClass;
    internal readonly double? LRS;
    internal readonly double? Weekly;
    internal readonly double? FourWeeks;
    internal readonly double? ThreeMonths;
    internal readonly double? SixMonths;
    internal readonly double? OneYear;
    internal readonly double? StdDev;
    internal readonly double? Mav200;
    internal double Price;

    internal static void Banner() {
        Console.Write("{0,-6}", "Ticker");
        Console.Write("{0,-50}", "Name");
        Console.Write("{0,-12}", "Asset Class");
        //Console.Write("{0,-30}\t", "Asset SubClass";
        Console.Write("{0,4}", "RS");
        Console.Write("{0,4}", "1Wk");
        Console.Write("{0,4}", "4Wk");
        Console.Write("{0,4}", "3Ms");
        Console.Write("{0,4}", "6Ms");
        Console.Write("{0,4}", "1Yr");
        Console.Write("{0,6}", "Vol");
        Console.WriteLine("{0,2}", "Mv");
        //Console.Write("{0,6}", "Pr");
        //Console.WriteLine("{0,6}", "M200");
    }

    internal void Print() {

        Console.Write("{0,-6}", Ticker);
        Console.Write("{0,-50}", new String(Name.Take(48).ToArray()));
        Console.Write("{0,-12}", new String(AssetClass.Take(10).ToArray()));
        //Console.Write("{0,-30}\t", new String(AssetSubClass.Take(28).ToArray()));
        Console.Write("{0,4:N0}", LRS * 100);
        Console.Write("{0,4:N0}", Weekly * 100);
        Console.Write("{0,4:N0}", FourWeeks * 100);
        Console.Write("{0,4:N0}", ThreeMonths * 100);
        Console.Write("{0,4:N0}", SixMonths * 100);
        Console.Write("{0,4:N0}", OneYear * 100);
        Console.Write("{0,6:N0}", StdDev * 100);
        if (Price <= Mav200)
            Console.WriteLine("{0,2}", "X");
        else
            Console.WriteLine();
        //Console.Write("{0,6:N2}", Price);
        //Console.WriteLine("{0,6:N2}", Mav200);
    }
}

The class Summary above is how I want to present my results. A few comments on the code. I use Nullable<T> because some of this values can be null (i.e. not enough history), but I still don’t want to worry about it. It ends up working rather neatly.

I also print the results out to Console, which is crazy. I really should be using WPF/Silverlight as the presentation layer. Also the {0,4:N0} notation might be unfamiliar to some of you, but this is how mad Console guys like myself avoid using real UI frameworks. Sometimes we print things in color too.

The real meat is in the following line:

LRS = (fourWeeks + threeMonths + sixMonths + oneYear) / 4;

That is our highway to richness. It’s a very elaborated quant formula, never before shown, that calculate a magick relative strength (aka momentum) factor as the average of the performance of four weeks, three months, six months and one year.

class TimeSeries {
    internal readonly string Ticker;
    readonly DateTime _start;
    readonly Dictionary<DateTime, double> _adjDictionary;
    readonly string _name;
    readonly string _assetClass;
    readonly string _assetSubClass;

    internal TimeSeries(string ticker, string name, string assetClass, string assetSubClass, 
                                                                IEnumerable<Event> events) {
        Ticker = ticker;
        _name = name;
        _assetClass = assetClass;
        _assetSubClass = assetSubClass;
        _start = events.Last().Date;
        _adjDictionary = events.ToDictionary(e => e.Date, e => e.Price);
    }

I then built myself a little TimeSeries class that represents a series of (date, price). I choose a dictionary to store it because of my assumption that I will be accessing it by date a lot. In retrospect, I was kind of right and kind of wrong. It doesn’t really matter much.

bool GetPrice(DateTime when, out double price, out double shift) {
    // To nullify the effect of hours/min/sec/millisec being different from 0
    when = new DateTime(when.Year, when.Month, when.Day);
    var found = false;
    shift = 1;
    double aPrice = 0;
    while (when >= _start && !found) {
        if (_adjDictionary.TryGetValue(when, out aPrice)) {
            found = true;
        }
        when = when.AddDays(-1);
        shift -= 1;
    }
    price = aPrice;
    return found;
}

A TimeSeries can give you back the price at a particular date. This looks bizarre and complex, but there is a reason for it. I might ask for a date that doesn’t have a price associated with it (i.e. holidays, week-ends). In such cases I want to return the previous price which could be N days in the past.

I also want to return how many days in the past I had to go, so that other calculations (i.e. Return) can modify their end date by the same amount. Also I might not find such a price at all, in which case I don’t want to throw an exception, but instead notify the caller. In retrospect, I should have used double? to signify ‘price not found’.

double? GetReturn(DateTime start, DateTime end) {
    var startPrice = 0.0;
    var endPrice = 0.0;
    var shift = 0.0;
    var foundEnd = GetPrice(end, out endPrice, out shift);
    var foundStart = GetPrice(start.AddDays(shift), out startPrice, out shift);
    if (!foundStart || !foundEnd)
        return null;
    else
        return endPrice / startPrice - 1;
}

We can now go and calculate the return between two dates. Also the TimeSeries object needs to perform a little more calculations.

    internal double? LastWeekReturn() {
        return GetReturn(DateTime.Now.AddDays(-7), DateTime.Now);
    }
    internal double? Last4WeeksReturn() {
        return GetReturn(DateTime.Now.AddDays(-28), DateTime.Now);
    }
    internal double? Last3MonthsReturn() {
        return GetReturn(DateTime.Now.AddMonths(-3), DateTime.Now);
    }
    internal double? Last6MonthsReturn() {
        return GetReturn(DateTime.Now.AddMonths(-6), DateTime.Now);
    }
    internal double? LastYearReturn() {
        return GetReturn(DateTime.Now.AddYears(-1), DateTime.Now);
    }
    internal double? StdDev() {
        var now = DateTime.Now;
        now = new DateTime(now.Year, now.Month, now.Day);
        var limit = now.AddYears(-3);
        var rets = new List<double>();
        while (now >= _start.AddDays(12) && now >= limit) {
            var ret = GetReturn(now.AddDays(-7), now);
            rets.Add(ret.Value);
            now = now.AddDays(-7);
        }
        var mean = rets.Average();
        var variance = rets.Select(r => Math.Pow(r - mean, 2)).Sum();
        var weeklyStdDev = Math.Sqrt(variance / rets.Count);
        return weeklyStdDev * Math.Sqrt(40);
    }
    internal double? MAV200() {
        return _adjDictionary
               .ToList()
               .OrderByDescending(k => k.Key)
               .Take(200)
               .Average(k => k.Value);
    }
    internal double TodayPrice() {
        var price = 0.0;
        var shift = 0.0;
        GetPrice(DateTime.Now, out price, out shift);
        return price;
    }
    internal Summary GetSummary() {
        return new Summary(Ticker, _name, _assetClass, _assetSubClass, 
                           LastWeekReturn(), Last4WeeksReturn(), Last3MonthsReturn(),
                           Last6MonthsReturn(), LastYearReturn(), StdDev(), TodayPrice(),
                           MAV200());
    }
}

Nothing particularly interesting in this code. Just a bunch of calculations. The MAV200 is the 200 days moving average of closing prices. It shows a more functional way of doing things. The StdDev function is instead very imperative.

We now can work on downloading the prices. This is how you construct the right URL:

static string CreateUrl(string ticker, DateTime start, DateTime end) {
    return @"http://ichart.finance.yahoo.com/table.csv?s=" + ticker + "&a="
            + (start.Month - 1).ToString() + "&b=" + start.Day.ToString() + "&c="
            + start.Year.ToString() + "&d=" + (end.Month - 1).ToString() + "&e="
            + end.Day.ToString() + "&f=" + end.Year.ToString() + "&g=d&ignore=.csv";
}

And let’s set how many concurrent connections we are going to use …

ServicePointManager.DefaultConnectionLimit = 10;

On my machine, setting this number too high causes errors to be returned. I’m not sure on which side of the connection the problem lies.

We can then load all the tickers we want to load from a file. One of the files has Leveraged ETFs, which I want to filter out because they tend to pop up always at the top.

var tickers =
    //File.ReadAllLines("ETFs.csv")
    //File.ReadAllLines("ETFTest.csv")
    File.ReadAllLines("AssetClasses.csv")
    .Skip(1)
    .Select(l => l.Split(new[] { ',' }))
    .Where(v => v[2] != "Leveraged")
    .Select(values => Tuple.Create(values[0], values[1], values[2], values[3]))
    .ToArray();
var len = tickers.Length;

var start = DateTime.Now.AddYears(-2);
var end = DateTime.Now;
var cevent = new CountdownEvent(len);
var summaries = new Summary[len];

And then load all of them, making sure to make an asynchronous call so not to keep the thread busy.

for(var i = 0; i < len; i++)  {
    var t = tickers[i];
    var url = CreateUrl(t.Item1, start, end);
    using (var webClient = new WebClient()) {
        webClient.DownloadStringCompleted +=
                          new DownloadStringCompletedEventHandler(downloadStringCompleted);
        webClient.DownloadStringAsync(new Uri(url), Tuple.Create(t, cevent, summaries, i));
    }
}

cevent.Wait();

Notice the use of a Countdown event to wait for all the thread to complete before printing out the results. Also notice the new Tuple<T> class used to package things to send around.

We can then print out the top and bottom 15%:

var top15perc =
        summaries
        .Where(s => s.LRS.HasValue)
        .OrderByDescending(s => s.LRS)
        .Take((int)(len * 0.15));
var bottom15perc =
        summaries
        .Where(s => s.LRS.HasValue)
        .OrderBy(s => s.LRS)
        .Take((int)(len * 0.15));

Console.WriteLine();
Summary.Banner();
Console.WriteLine("TOP 15%");
foreach(var s in top15perc)
    s.Print();

Console.WriteLine();
Console.WriteLine("Bottom 15%");
foreach (var s in bottom15perc)
    s.Print();

Here is what we do when a request comes back with data:

static void downloadStringCompleted(object sender, DownloadStringCompletedEventArgs e) {
    var bigTuple =
             (Tuple<Tuple<string, string, string, string>, CountdownEvent, Summary[], int>)
              e.UserState;
    var tuple = bigTuple.Item1;
    var cevent = bigTuple.Item2;
    var summaries = bigTuple.Item3;
    var i = bigTuple.Item4;
    var ticker = tuple.Item1;
    var name = tuple.Item2;
    var asset = tuple.Item3;
    var subAsset = tuple.Item4;

    if (e.Error == null) {
        var adjustedPrices =
                e.Result
                .Split(new[] { '\n' })
                .Skip(1)
                .Select(l => l.Split(new[] { ',' }))
                .Where(l => l.Length == 7)
                .Select(v => new Event(DateTime.Parse(v[0]), Double.Parse(v[6])));

        var timeSeries = new TimeSeries(ticker, name, asset, subAsset, adjustedPrices);
        summaries[i] = timeSeries.GetSummary();
        cevent.Signal();
        Console.Write("{0} ", ticker);
    }
    else {
        Console.WriteLine("[{0} ERROR] ", ticker);
        //Console.WriteLine(e.Error);
        summaries[i] = new Summary(ticker, name, "ERROR", "ERROR", 0, 0, 0, 0, 0, 0,0,0); 
        cevent.Signal();
    }
}

We first unpack the Tuple we sent out originally, we then extract the Date and Price, create a Summary object and store it in the summaries array. It’s important to remember to Signal to the cevent in the error case as well because we want to print out the results even if some downloading failed.

And here is what you get for your effort:

A version of the AsyncCache found its way into the Parallel Programming samples …

lucabol — Thu, 21 May 2009 20:28:13 GMT

Go here to download them. It is in \ParallelExtensionsExtras\CoordinationDataStructures. It has a slightly different design in that it returns Tasks. I’m trying to get Stephen to blog about it so that you can compare them.

I talk about C# and VB Co-Evolution on Channel9 (and some F# …)

lucabol — Thu, 14 May 2009 20:26:18 GMT

The title says it all. If you are interested, go here.

Luca at NDC in Oslo 17 – 19 June 2009

lucabol — Tue, 24 Mar 2009 19:46:27 GMT

I’ll be speaking about the future of C# and F#. Oslo brings back so many memories …

“one of the world’s most important conferences for IT developers and leaders”

Simulating INumeric with dynamic in C# 4.0

lucabol — Thu, 05 Feb 2009 19:47:26 GMT

When I wrote my Excel financial library I agonized over the decision of which numeric type to use to represent money. Logic would push me toward decimal, but common usage among financial library writers would push me toward double. I ended up picking double, but I regret having to make that choice in the first place.

Conceptually, I'd like my numeric functions to work for anything that supports the basic arithmetic operators (i.e. +, -, * ...). Unfortunately that is not possible in .NET at this point in time. In essence you have to write your code twice as below.

   static double SumDouble(double a, double b) { return a + b; }
   static decimal SumDecimal(decimal a, decimal b) {return a + b;}

Granted, this is not a good state of affairs. We often discussed how to make it work, but we couldn't find a solution that was both fast to run and cheap for us to implement. More often than not we speculated about having the numeric types implement a specific INumeric interface and add a generic constraint to the C#/VB languages to make it work. Hence the title of this post.

With we implemented dynamic in C# 4.0 it occurred to me that you can fake your way into writing your code just once. For sure, this solution doesn't have the same performance characteristics of 'writing your code twice', but at least it doesn't duplicate your code.

This is how it looks like:

    static dynamic Sum1(dynamic a, dynamic b) { return a + b; }

The call to the '+' operator is resolved at runtime, by the C# binder, hence a performance penalty is incurred. The penalty is less than you might think, given that the DLR caches things under the cover so that no v-table lookup is performed the second time around. The whole thing is explained in more detail here. But still, it is not as fast as a normal '+' operator over a primitive type. I'll let you enjoy micro performance testing this one :-)

A slight refinement is to make the code generic so that a caller doesn't see a signature with dynamic types as arguments.

   static dynamic Sum2<T1, T2>(T1 a, T2 b)
   {
       dynamic ad = a;
       dynamic bd = b;
       return ad + bd;
   }

I could make the return type generic as well, but that would force the caller to be explicit about the types, making the calling code much less readable. The other good thing about this signature is that you get a different call site with each combination of type arguments and, since they are separate, the binding caches should stay small. With the former signature there is only one call site and the cache could pile up to the point where the DLR decides to discard it.

Here is how the calling code looks like right now:

       Console.WriteLine(Sum2(2m, 4m));
       Console.WriteLine(Sum2(2.0, 4.0));
       Console.WriteLine(Sum2(new DateTime(2000,12,1), new TimeSpan(24,0,0)));

Yet another way to write this code is as follows:

   public static T Sum3<T>(T a, T b)
   {

       dynamic ad = a;
       dynamic bd = b;
       return ad + bd;
   }

This gets around the problem of showing a dynamic return value and give you some more compile time type checking. But it prevents summing not coercible types. The compiler doesn't let you get there. The last line below wont' compile:

       Console.WriteLine(Sum3(2m, 4m));
       Console.WriteLine(Sum3(2.0, 4.0));
       //Console.WriteLine(Sum3(new DateTime(2000,12,1), new TimeSpan(24,0,0)));

Also notice that in VB you could have done this a long time ago :-)

   Function Sum(Of T1, T2)(ByVal a As T1, ByVal b As T2)
       Dim aa As Object = a
       Dim bb As Object = b
       Return aa + bb
   End Function

In summary, by using dynamic you can write your numeric code just once, but you pay a performance price. You are the only one who can decide if the price is worth paying in your particular application. As often, the profiler is your friend.

Financial Functions for .NET released !

lucabol — Fri, 05 Dec 2008 00:40:00 GMT

Today I released the following library on CodeGallery. It is the result of three months of coding during my paternity leave in Italy. You can get it from here.

What is it?
This is a .NET library that provides the full set of financial functions from Excel. The main goal for the library is compatibility with Excel, by providing the same functions, with the same behaviour. Note though that this is not a wrapper over the Excel library; the functions have been re-implemented in managed code so that you do not need to have Excel installed to use this library.

Where I can find documentation on these functions?
Just open Excel and click on Formulas/Financial or go to this

I don't think one of the function is right. Excel produces the wrong results! Why don't you do it right?
My goal is to replicate Excel results (right and wrong). Feel free to contribute to the effort by coding what you think is the right solution and I'll add an ExcelCompliant flag to the function to conditionally invoke your code.

How do I use the library?
Just add Financial.dll to the references in your project. The functions are provided as static methods on a Financial class in the System.Numeric namespace
I see the library was implemented with F#. But I don’t want to redistribute F# along with my application. What should I do?
There are two versions of the library. One of them statically links the F# libraries so that there is no dependency on F#. However, this assembly larger, so if you have F# installed, you can use the FinancialNotStandalone.dll instead.

How do I run the tests?
Run FinancialTests.exe. You need Excel 12 for the tests to work because they use Excel to test that the results are correct. You don't need Excel 12 to use the library in your own application.

How do I compile the library?
You need to have F# September CTP installed (you can get it from here). There are two batch files (CreateLibraryStandalone.bat and CreateLibraryNotStandalon.bat). Run them to compile the dll. You might have to change the path to the F# compiler inside these files

How do I compile the tests?
Run CreateTests.bat

Have you tested this thing?
Yes, I do have 201,349 testcases running against it. You can easily raise that number significantly by adding new values to test in testdef.fs. If you have a multiproc machine the testcases will run faster as I parallelize their execution.

Have you run performance tests on it?
Not at all. The only thing I checked is that all the recursive functions are tail recursive. Feel free to let me know if they are slow.

Are there any functions that behave different from Excel?
Yes, there are two of them.
CoupDays
The Excel algorithm seems wrong in that it doesn't respect the following:
coupDays = coupDaysBS + coupDaysNC.
This equality should stand. By manually counting the days, I'm pretty confident that my algorithm is correct.My result differs from Excel by +/- one or two days when the date spans a leap year.
VDB
In the excel version of this algorithm the depreciation in the period (0,1) is not the same as the sum of the depreciations in periods (0,0.5) (0.5,1).
VDB(100,10,13,0,0.5,1,0) + VDB(100,10,13,0.5,1,1,0) <> VDB(100,10,13,0,1,1,0)
Notice that in Excel by using '1' (no_switch) instead of '0' as the last parameter everything works as expected. The last parameter should have no influence in the calculation given that in the first period there is no switch to sln depreciation.
Overall, I think my algorithm is correct, even if it disagrees with Excel when startperiod is fractional.

Can you list the functions with their testcases results?
Succeeded 1840/1840 for PV
Succeeded 2024/2024 for FV
Succeeded 2240/2240 for PMT
Succeeded 853/853 for NPER
Succeeded 5355/5355 for IPMT
Succeeded 5355/5355 for PPMT
Succeeded 208/208 for CUMIPMT
Succeeded 208/208 for CUMPRINC
Succeeded 624/624 for ISPMT
Succeeded 12/12 for FVSCHEDULE
Succeeded 9/9 for IRR
Succeeded 21/21 for NPV
Succeeded 147/147 for MIRR
Succeeded 18/18 for XIRR
Succeeded 396/396 for DB
Succeeded 24/24 for SLN
Succeeded 132/132 for SYD
Succeeded 456/456 for DDB
Succeeded 2544/2544 for VDB excluding fractional startdates
Succeeded 11520/11520 for AMORLINC
Succeeded 23040/23040 for AMORDEGRC
Succeeded 15/15 for COUPDAYS excluding leap years
Succeeded 915/915 for COUPDAYSBS
Succeeded 915/915 for COUPDAYSNC
Succeeded 915/915 for COUPNUM
Succeeded 915/915 for COUPPCD
Succeeded 915/915 for COUPNCD
Succeeded 360/360 for ACCRINTM
Succeeded 1920/1920 for ACCRINT
Succeeded 10980/10980 for PRICE
Succeeded 1940/1940 for PRICEMAT
Succeeded 2910/2910 for YIELDMAT
Succeeded 1395/1395 for YEARFRAC
Succeeded 2745/2745 for INTRATE
Succeeded 1290/1290 for RECEIVED
Succeeded 2745/2745 for DISC
Succeeded 3660/3660 for PRICEDISC
Succeeded 2745/2745 for YIELDDISC
Succeeded 48/48 for TBILLEQ
Succeeded 69/69 for TBILLYIELD
Succeeded 81/81 for TBILLPrice
Succeeded 12/12 for DOLLARDE
Succeeded 12/12 for DOLLARFR
Succeeded 12/12 for EFFECT
Succeeded 12/12 for NOMINAL
Succeeded 5490/5490 for DURATION
Succeeded 5490/5490 for MDURATION
Succeeded 19320/19320 for ODDFPRICE
Succeeded 30600/30600 for ODDLPRICE
Succeeded 45900/45900 for ODDLYIELD
Test Cases Succeeded 201349/201349

A C# library to write functional code - Part V - The Match operator

lucabol — Tue, 15 Jul 2008 12:46:00 GMT

Other posts in the series:

Part I - Background

Part II - Tuples

Part III - Records

Part IV - Type Unions

Part V - The Match operator

This is my last post of this series. It is about the match operator. To the untrained eyes this operator might look like a case statement. But they are different.

The match operator combines control flow and decomposition in a single construct. As it often happens, even if these two things are well known concepts, putting them together gives you a different perspective on your code.

I have to admit that the similarity with a case statement triggered all sorts of bad reactions in my OO trained mind. Phrases from wise gurus on the tone of "never use an 'if' statement" was echoing in my mind. After a while I got over it and enjoyed the power that this operator gives.

In this library we won't get close to the beauty of the match operator in functional languages (i.e. F#), especially the decomposition piece. There might be a way to do better than this in C#, but this was enough for my purpose of learning about functional programming, so I didn't investigate further. Learning was my real goal.

How to use it

There are two versions of this operator. Let's start from the more eye pleasing one.

Let's assume a discriminated union like the following:

public class Node : TypeUnion<int, string> {

    public Node(int i) : base(i) { }
    public Node(string s) : base(s) { }

    public int IntNode { get { return Type1; } }
    public string StringNode { get { return Type2; } }
}

I can then match against this union with the following code:

var no = new Node(35);

r = F.Match(no, 
                (int i)    => (i + 3).ToString(),
                (string s) => s
            );

Assert.AreEqual("38", r);

Note that the match operator behaves a bit like a case statement, but it also gives you the 'right' type on the right of the '=>' for you to write code against.

I have to admit I'm rather happy of this syntax, but it has one severe limitation. You need to specify all the types of the type union and they have to be in the same order. For example, in the previous code I cannot match against string first and int second. I consider this to be a big deal.

A different implementation of match that doesn't have that limitation is as follows:

no = new Node("35");
r = F.Match(no,
    n => n.Is<int>(),   n => (n.As<int>() + 3).ToString(),
    n => n.Is<string>(),n => n.As<string>());

Assert.AreEqual("35", r);

Rather less pleasing to the eyes, but more robust to use.

By using this more generic version, you can obviously match against all the constructs we described in this series. I.E. against Tuples:

var t = F.Tuple("msft", 10, "Nasdaq");

var r = F.Match(t,
    i => i.Item2 < 5 && i.Item3 == "OTC", 
         i => i.Item1 + " is low price OTC stock",
    i => i.Item3 == "OTC",
         i => i.Item1 + "is a normal OTC stock",
    i => i.Item3 == "Nasdaq", 
         i => i.Item1 + " is a Nasdaq stock");

Assert.AreEqual("msft is a Nasdaq stock", r);

Or against sequences:

var i1 = new int[] { 1, 2, 3, 4, 5, 6, 7 };

var r1 = F.Match(i1,
                s => s.SequenceEqual(new int[] { 1, 2}),    
                       s => s.Where(i => i < 4),
                s => s.First() == 2,                        
                       s => s.Where(i => i == 1),
                s => s.Last() == 6,                         
                       s => s.Select(i => i * i),
                s => true,                                  
                       s => s.Where(i => i < 7));

Assert.IsTrue(i1.Take(6).SequenceEqual(r1));

The match operator, as I defined it, is very flexible (probably too flexible as it doesn't use the type system to enforce much).

How it is implemented

Let's start from the special match against union types: the one that is beautiful but flawed. Its implementation looks just like this (I just show the two parameters version):

public static R Match<T1, T2, R>(
                           this TypeUnion<T1, T2> u,
                           Func<T1, R> f1, Func<T2, R> f2) {

    if (u.Is<T1>()) return f1(u.As<T1>());
    if (u.Is<T2>()) return f2(u.As<T2>());

    throw new Exception("No Match for this Union Type");
}

It is easy to see the reason for the limitations of this function. The same thing that gives you type inference (nice to look at) also gives you the problem with the ordering of lambdas. Also note the I originally intended to have these as extension methods. In practice I ended up liking more the "F.Match" syntax. De gustibus I assume ...

The more general version looks like this:

public static R Match<T, R>(this T t,
    Func<T, bool> match1, Func<T, R> func1,
    Func<T, bool> match2, Func<T, R> func2) {

    if (match1(t))
        return func1(t);
    if (match2(t))
        return func2(t);

    throw new Exception("Nothing matches");
}

Again, it is easy to see in the implementation that this is an extremely general thing: a glorified case statement really. This is the beauty of it (you can match against anything) and the ugliness of it (you can write anything as match1 of func1, even code that doesn't reference t at all).

How to use this series to learn functional programming

Here is how I did it (it worked for me). Take this library and force yourself to write a medium size program using just these constructs.

I mean it. No objects, just records. No inheritance, just discriminated unions. Tuples to return values. Match operators everywhere. No iteration statements (for, while, etc...), just recursion. Extensive use of sequence operators. After a while I noticed that all my functions had the same pattern: they just match against the input and produce an output, usually by calling other functions or recursively calling themselves.

Obviously, this is overcompensating. After a while you will realize the different trade offs that come with a functional vs OO style of programming. At that point you'll be able to get the best of the two. Or maybe you'll be royally confused ...

At that point I picked up F# and I'm now enjoying the fact that all these constructs are directly embedded in the language (and yes, you can use while statements too).

A C# library to write functional code - Part IV - Type Unions

lucabol — Fri, 06 Jun 2008 11:45:00 GMT

Other posts in the series:

Part I - Background

Part II - Tuples

Part III - Records

Part IV - Type Unions

Part V - The Match operator

I'm sorry for my prolonged absence in the middle of this series of posts. I'm on a long paternity leave in Italy (playing beach volley every day). It's hard to have the discipline ...

A bunch of you wrote telling me to finish this. So here I go: let's talk about type unions. First of all: they are not called like that. The correct name is discriminated unions. I have no idea why I call them differently, but I want to be consistent with my previous mistake.

For those of you with a C++ background (like myself) they are like unions, just better (or worse depending on your convictions). They let you define a type that can represent one of several different types. You can then use the 'match' operator (discussed in the next post) to pattern match against it.

I won't elaborate on the pros and cons of this style of programming versus using polymorphism. I just want to show you how I implemented this construct in C#. As always, my usual caveat: this is just 'educational code', use it at your own risk, no extensive or perf related test has been done on it. You can download the zip file and check my unit tests for yourself.

How type unions are used

In my world, you declare a type union like this:

public class Person { }
public class Dog { }

public class Friend : TypeUnion<Person, Dog> {

    public Friend(Person p) : base(p) { }
    public Friend(Dog d) : base(d) { }
}

You inherit a type union from the "TypeUnion" class and use generic parameters that correspond to the types that the union can represent.

You can then create a type union as:

var fr = new Friend(new Dog());

Test its type by:

Assert.IsTrue(fr.Is<Dog>());
Assert.IsFalse(fr.Is<Person>());

Cast it to one of the types they represent:

var d = fr.As<Dog>();

Or use it with the 'match' operator (fully explained in an upcoming post):

var r = F.Match(fr,
    f => f.Is<Dog>(), f => f.As<Dog>().ToString(),
    f => f.Is<Person>(), f => f.As<Person>().ToString());

Assert.AreEqual(r, new Dog().ToString());

Or the slightly more pleasing:

r = F.Match(fr,
            (Person p) => p.ToString(),
            (Dog d) => d.ToString());

Assert.AreEqual(r, new Dog().ToString());

You get the idea.

How they are implemented

Nothing really sophisticated going on here. Let's take as an example a type union that can represent two types. I have versions that go to 5 types in the zip file.

First of all a TypeUnion is a Record:

public class TypeUnion<T1, T2> : Record<T1, T2> {

It has overloaded constructors to create a type union of a particular type:

public TypeUnion(T1 t1)
    : base(t1, default(T2)) {

    UnionType = t1.GetType();
}
public TypeUnion(T2 t2)
    : base(default(T1), t2) {

    UnionType = t2.GetType();
}

UnionType is used to 'remember' which type it is:

protected Type UnionType;

It also has properties to return the objects of all the types that can be stored:

protected T1 Type1 { get { return state.Item1; } }
protected T2 Type2 { get { return state.Item2; } }

The 'Is' operator is simply implemented as:

public bool Is<K>() {

    return typeof(K).IsAssignableFrom(UnionType);
}

And the 'As' operator looks like so:

public K As<K>() {

    if (!Is<K>())
        throw new Exception(string.Format(
          "In a TypeUnion cannot cast from {0} to {1}",
          UnionType.Name, typeof(K).Name));

    if (typeof(T1) == UnionType)
        return (K)(object) Type1;
    if (typeof(T2) == UnionType)
        return (K)(object) Type2;

    throw new Exception("Shouldn't get here");
}

I leave as an exercise to the reader to understand what happens if T1 and T2 are the same type or inherit from the same type. I could have written code to handle this case in a more explicit manner, but didn't.

Also, by reviewing my code I found an obvious bug in my Is<K>/As<K> code. I fixed it and re-posted the zip file in the second post of this series.

Now back to the beach. Next post is on the 'match' operator.

A C# library to write functional code - Part III - Records

lucabol — Mon, 21 Apr 2008 20:34:00 GMT

Other posts in the series:

Part I - Background

Part II - Tuples

Part III - Records

Part IV - Type Unions

Part V - The Match operator

Now that we know what Tuples are, we can start talking about Record, as they use a derivative of Tuples under the cover. But first, what is a record?

Well, in C# parlance a Record is a sort of immutable value object. I talked at length about these fellows in this series of blog posts. In functional parlance, a Record is a Tuple that lets you access its items by name.

You code Records like this:

    public class Order: Record<string, int> {

        public Order(string item, int qty): base(item,qty) {}

        public string Item { get { return state.Item1;}}
        public int Quantity { get { return state.Item2; } }
    }

    public class Customer: Record<string, IEnumerable<Order>> {

        public Customer(string name, IEnumerable<Order> orders) : base(name, orders) { }

        public string Name { get { return state.Item1; } }
        public IEnumerable<Order> Orders { get { return state.Item2; } }
    }

You need to do three things:

Inherit from a generic Record class specifying the types of the properties as parameters
Add a constructor that calls back to the Record constructor
Add getters to retrieve the values of the properties (no setters as it is immutable)

This may seem like plenty of work, but in return you get structural equality. Coding that by hand every time you use a record would be a royal pain. You lose control of the base class, but that is often not a problem as in functional-land you more often use type unions than inheritance.

How is it implemented?

First of all it is an abstract class with as many generic parameters as properties that you need in your Record. Let's use two as an example.

    public abstract class Record<T1, T2> {

This abstract class has a field of type STuple:

        protected readonly STuple<T1, T2> state;

What is a STuple? Well it is exactly the same as the Tuple described in Part II, but coded as a struct instead of a class. The reason to use a struct is to not allocate an additional object on the stack. This allows this solution to be as 'performant' as simply having coded the fields on the class itself. Or at least I think so ...

The Record class also has a constructor that simply initialize the STuple:

        public Record(T1 t1, T2 t2) { state = F.STuple(t1, t2); }

where

        internal static STuple<T1, T2> STuple<T1, T2>(T1 t1, T2 t2) {

            return new STuple<T1, T2>(t1, t2);
        }

The Equals method is very much the same as the Tuple's one, just delegating to the same DSEqual function that checks equality for Tuples.

        public override bool Equals(object right) {

            Utils.CheckNull(right);

            if (object.ReferenceEquals(this, right))
                return true;

            if (this.GetType() != right.GetType())
                return false;

            var rightT = right as Record<T1, T2>;

            return F.DSEquals(this.state, rightT.state);
        }

That's it. Not too difficult as most of the implementation is based on the Tuple's code. Next post will hopefully be more interesting. It is about Type Unions.

A C# library to write functional code - Part II - Tuples

lucabol — Tue, 08 Apr 2008 23:51:00 GMT

Other posts in the series:

Part I - Background

Part II - Tuples

Part III - Records

Part IV - Type Unions

Part V - The Match operator

Tuples are a way for you not to name things. In Object Oriented languages you got to name everything. If you need to represent a bunch of data, you create a class for it.

There is a strange asymmetry in mainstream OO languages in that you can pass multiple parameters to a function, but you can return just one value. Granted, there are ways around it: you can use 'ref' in C# or return some sort of collection where things are stored. But by and large the model is: you pass many, you get one; if you need to return more than one, create a class to represent this 'bunch of data'. Tuples are a way for you not to create such a class.

Tuples are also much more than that. Once you have the language concept of 'a bunch of data without a name', you can create arrays of them, you can pass them as parameters, use them as local variables. Wherever you'd use a type, you can use a Tuple instead.

This is particularly appealing to me as I like to use classes almost exclusively to represent things that have a counterpart in the domain I'm modeling (i.e. Customer, Account). I don't like to create classes/structs just for the sake of temporarily put some data together.

You can create your own Tuple class in C#, but the syntax gets ugly. Syntax matter. Syntax helps you to think differently about your program. We have syntax for anonymous types, but given that they cannot escape the scope of a method, they cannot be used as full replacement for Tuples.

In any case, to my implementation. Here is how you create a Tuple:

            var t1 = F.Tuple(34, "bo", 2.3);

not too bad. In F# it is better: (34, "bo", 2.3). And you often don't need the parenthesis. But still, my C# version is ok.

You then need to access its elements:

            var n = t1.Item1;
            var s = t1.Item2;

In F# you usually access them by doing pattern matching, which gives a more intuitive syntax. But again, my C# syntax is not terrible.

Tuples need to have structural equality, which means that the following has to work:

        ArrayList mad1 = new ArrayList { new List<IEnumerable<string>> { new string[] { "bo" }, new string[] { "bo" } },
                               32, "bo", new int[] { 4, 5, 6 } };
        ArrayList mad2 = new ArrayList { new List<IEnumerable<string>> { new string[] { "bo" }, new string[] { "bo" } },
                               32, "bo", new int[] { 4, 5, 6 } };
        ArrayList mad3 = new ArrayList { new List<IEnumerable<string>> { new string[] { "bo" }, new string[] { "bo" } },
                               32, "bo", new int[] { 4, 5, 5 } };

        Assert.AreEqual(F.Tuple(mad1, mad2, mad1), F.Tuple(mad2, mad1, mad2));
        Assert.AreNotEqual(F.Tuple(mad1, mad2, mad1), F.Tuple(mad1, mad3, mad1));

You can use Tuples as return values, parameters, locals etc. Unfortunately, the syntax is ugly when Tuples are part of the signature of a function:

    public Tuple<string, IEnumerable<Tuple<string, ObservationHistory>>> Execute() {

    }

With the above information, you can be a user of Tuples. From this point on, I'll talk about some details of the implementation (I also attach the full code to this post as a zip file).

    public class Tuple<T1> {

        public Tuple(T1 t1) {

            Item1 = t1;
        }

        public readonly T1 Item1;

        #region Equals, GetHashCode, ==, !=
    }

    public class Tuple<T1, T2> : Tuple<T1> {

        public Tuple(T1 t1, T2 t2) : base(t1) { Item2 = t2; }

        public readonly T2 Item2;

        #region Equals, GetHashCode, ==, !=
    }

So, Tuples are classes, not structs. The reason for it is fully described in this series of posts. They also inherit from one another. There are pros and cons to that. The main pros are that I had to write less code and that you can pass a Tuple<int, string> when a function expects a Tuple<int, string, int>. The main drawback is that you can pass a Tuple<int, string> when a function expects a Tuple<int, string, int>. Also notice the use of public fields. These is a problem with frameworks that insist on properties (i.e. Data Binding). Also, I just got to 5 as arity goes. The day I need 6 items, I'll add another one. It is boilerplate code (that I'd still like not to write).

The Equals method is a bit convoluted:

    internal static class Utils {

        public static void CheckNull<T>(T t) {

            if (t == null)
                throw new ArgumentNullException();
        }
     }

       public override bool Equals(object right) {

            Utils.CheckNull(right);

            if (object.ReferenceEquals(this, right))
                return true;

            if (this.GetType() != right.GetType())
                return false;

            var rightT = right as Tuple<T1, T2, T3>;

            return base.Equals(rightT) && F.DSEquals(this.Item3, rightT.Item3);
        }

I always get complaints when I show Equals methods that throw if null is passed in, but I stand by my logic, that the presence of null for these categories of 'structurally equal' classes is symptom of an error and I want to be notified. Returning false doesn't do that.

        internal static bool DSEquals<T>(T left, T right) {

            if (left == null && right == null)
                return true;

            if (left == null && right != null)
                return false;

            var len = left as IEnumerable;
            var ren = right as IEnumerable;

            if (len == null && ren == null)
                return left.Equals(right);

            if (len == null && ren != null)
                return false;

            if (len != null && ren == null)
                return false;

            return SequenceEqual(len, ren);
        }

DSEquals check the content of the Tuple and forward to SequenceEqual in case one slot of the Tuple contains an IEnumerable.

        internal static bool SequenceEqual(IEnumerable en1, IEnumerable en2) {

            var enumerator = en2.GetEnumerator();
            foreach (var o in en1) {

                if (!enumerator.MoveNext())
                    return false;

                if (!DSEquals(o, enumerator.Current))
                    return false;
            }

SequenceEqual checks that the number of items in the enumerator is the same and recursively calls DSEqual to check structural equality for items at the same index in the two enumerators.

GetHashCode is trivial (and maybe trivially wrong, one of these days I'll learn everything about GetHashCode() ).

        public override int GetHashCode() {

            return base.GetHashCode() | Item3.GetHashCode();
        }

The equality operators are equally simple.

        public static bool operator ==(Tuple<T1, T2, T3> t1, Tuple<T1, T2, T3> t2) {

            Utils.CheckNull(t1);
            Utils.CheckNull(t2);

            return t1.Equals(t2);
        }
        public static bool operator !=(Tuple<T1, T2, T3> t1, Tuple<T1, T2, T3> t2) {

            return !(t1 == t2);
        }

And ToString() prints my favorite Tuple format.

        public override string ToString() {

            return string.Format("{0},{1}", base.ToString(), Item3.ToString());
        }

I'm sure you can find plenty of issues in this code. As always, it is not 'production ready', it is more 'Luca having fun doing it'. In any case, there are some testcases in the solution to check the extent of my testing.

In the next post we'll look at Records.

A C# library to write functional code - Part I - Background

lucabol — Tue, 01 Apr 2008 20:36:00 GMT

Other posts in the series:

Part I - Background

Part II - Tuples

Part III - Records

Part IV - Type Unions

Part V - The Match operator

In December (slow time in msft) I decided to understand what functional programming is all about. When I say 'understanding' I don't mean just paying lip service to the main concepts by knowingly mentioning them in casual conversations (i.e. "look at this memoization, man!" or "this lambda function is so hot!". I can already do that. I intellectually know what the thing is.

I wanted to *really* understand it. For me that means writing plenty of code. I had a medium size application in my mind that I've been wanting to write for quite some time (stock price, dividends, splits downloading and various return calculations), so I went ahead and wrote it. I also wanted to use C#. It would have been easier in F#, but I work on the C# team and love using our own product.

My early attempts were unpleasing. I would fall back to my OO background and my functional code slowly reverted to OO code. My way of thinking about it, even if starting with the best intentions, would go back to: what are the objects, what are their responsibilities and such.

I needed to force myself somehow; kind of overcompensate on the other side. I hit on the idea of pragmatically defining functional programming and try to limit myself to the subset of language constructs inside my definition. As a way to define it, I used Chapter 3 of "Expert F#". I know, I know, I could have read 1,000s of academic papers and come up with a meta-analysis of all of them that formally defines what 'functional programming' really is. But life is too short. I trusted Don.

The problem is, several of the language constructs in my small definition of functional programming don't exist in C#. So I went ahead and created them. I built a little library to represent them and forced myself to write code using just this library. It worked.

In this series of posts I will describe what's inside this library. I want to emphasize that I built it for educational purpose only, not for performance or production code. Caveat emptor.

My plan is to cover the following:

Tuples
Records
Type Unions
Match

Let's see if I can find the time to actually write these posts :-)

"LINQ to SQL Overview" video

lucabol — Tue, 04 Mar 2008 00:51:00 GMT

This is my presentation from TechEd Barcelona: The link has been fixed. It should work now. Let me know if it doesn't. http://www.microsoft.com/emea/msdn/spotlight/sessionh.aspx?videoid=716. The European locale brings out my Italian accent ...

On this page there are a bunch of interesting presentations (i.e. Anders on LINQ): http://www.microsoft.com/emea/msdn/spotlight/default.aspx

Creating an immutable value object in C# - Part V - Using a library

lucabol — Fri, 11 Jan 2008 21:36:00 GMT