Luca Bolognese's WebLog : Financial

Financial Functions for .NET released !

lucabol — Fri, 05 Dec 2008 00:40:00 GMT

Today I released the following library on CodeGallery. It is the result of three months of coding during my paternity leave in Italy. You can get it from here.

What is it?
This is a .NET library that provides the full set of financial functions from Excel. The main goal for the library is compatibility with Excel, by providing the same functions, with the same behaviour. Note though that this is not a wrapper over the Excel library; the functions have been re-implemented in managed code so that you do not need to have Excel installed to use this library.

Where I can find documentation on these functions?
Just open Excel and click on Formulas/Financial or go to this

I don't think one of the function is right. Excel produces the wrong results! Why don't you do it right?
My goal is to replicate Excel results (right and wrong). Feel free to contribute to the effort by coding what you think is the right solution and I'll add an ExcelCompliant flag to the function to conditionally invoke your code.

How do I use the library?
Just add Financial.dll to the references in your project. The functions are provided as static methods on a Financial class in the System.Numeric namespace
I see the library was implemented with F#. But I don’t want to redistribute F# along with my application. What should I do?
There are two versions of the library. One of them statically links the F# libraries so that there is no dependency on F#. However, this assembly larger, so if you have F# installed, you can use the FinancialNotStandalone.dll instead.

How do I run the tests?
Run FinancialTests.exe. You need Excel 12 for the tests to work because they use Excel to test that the results are correct. You don't need Excel 12 to use the library in your own application.

How do I compile the library?
You need to have F# September CTP installed (you can get it from here). There are two batch files (CreateLibraryStandalone.bat and CreateLibraryNotStandalon.bat). Run them to compile the dll. You might have to change the path to the F# compiler inside these files

How do I compile the tests?
Run CreateTests.bat

Have you tested this thing?
Yes, I do have 201,349 testcases running against it. You can easily raise that number significantly by adding new values to test in testdef.fs. If you have a multiproc machine the testcases will run faster as I parallelize their execution.

Have you run performance tests on it?
Not at all. The only thing I checked is that all the recursive functions are tail recursive. Feel free to let me know if they are slow.

Are there any functions that behave different from Excel?
Yes, there are two of them.
CoupDays
The Excel algorithm seems wrong in that it doesn't respect the following:
coupDays = coupDaysBS + coupDaysNC.
This equality should stand. By manually counting the days, I'm pretty confident that my algorithm is correct.My result differs from Excel by +/- one or two days when the date spans a leap year.
VDB
In the excel version of this algorithm the depreciation in the period (0,1) is not the same as the sum of the depreciations in periods (0,0.5) (0.5,1).
VDB(100,10,13,0,0.5,1,0) + VDB(100,10,13,0.5,1,1,0) <> VDB(100,10,13,0,1,1,0)
Notice that in Excel by using '1' (no_switch) instead of '0' as the last parameter everything works as expected. The last parameter should have no influence in the calculation given that in the first period there is no switch to sln depreciation.
Overall, I think my algorithm is correct, even if it disagrees with Excel when startperiod is fractional.

Can you list the functions with their testcases results?
Succeeded 1840/1840 for PV
Succeeded 2024/2024 for FV
Succeeded 2240/2240 for PMT
Succeeded 853/853 for NPER
Succeeded 5355/5355 for IPMT
Succeeded 5355/5355 for PPMT
Succeeded 208/208 for CUMIPMT
Succeeded 208/208 for CUMPRINC
Succeeded 624/624 for ISPMT
Succeeded 12/12 for FVSCHEDULE
Succeeded 9/9 for IRR
Succeeded 21/21 for NPV
Succeeded 147/147 for MIRR
Succeeded 18/18 for XIRR
Succeeded 396/396 for DB
Succeeded 24/24 for SLN
Succeeded 132/132 for SYD
Succeeded 456/456 for DDB
Succeeded 2544/2544 for VDB excluding fractional startdates
Succeeded 11520/11520 for AMORLINC
Succeeded 23040/23040 for AMORDEGRC
Succeeded 15/15 for COUPDAYS excluding leap years
Succeeded 915/915 for COUPDAYSBS
Succeeded 915/915 for COUPDAYSNC
Succeeded 915/915 for COUPNUM
Succeeded 915/915 for COUPPCD
Succeeded 915/915 for COUPNCD
Succeeded 360/360 for ACCRINTM
Succeeded 1920/1920 for ACCRINT
Succeeded 10980/10980 for PRICE
Succeeded 1940/1940 for PRICEMAT
Succeeded 2910/2910 for YIELDMAT
Succeeded 1395/1395 for YEARFRAC
Succeeded 2745/2745 for INTRATE
Succeeded 1290/1290 for RECEIVED
Succeeded 2745/2745 for DISC
Succeeded 3660/3660 for PRICEDISC
Succeeded 2745/2745 for YIELDDISC
Succeeded 48/48 for TBILLEQ
Succeeded 69/69 for TBILLYIELD
Succeeded 81/81 for TBILLPrice
Succeeded 12/12 for DOLLARDE
Succeeded 12/12 for DOLLARFR
Succeeded 12/12 for EFFECT
Succeeded 12/12 for NOMINAL
Succeeded 5490/5490 for DURATION
Succeeded 5490/5490 for MDURATION
Succeeded 19320/19320 for ODDFPRICE
Succeeded 30600/30600 for ODDLPRICE
Succeeded 45900/45900 for ODDLYIELD
Test Cases Succeeded 201349/201349

Downloading stock prices in F# - Part VI - Code posted

lucabol — Tue, 21 Oct 2008 01:45:00 GMT

Other parts:

An unnamed friend told me that I should stop posting small snippets of code and instead post entire solutions on CodeGallery. I did it for this one and here is the link.

Here is what's in the zip file:

BackTestF - main library to download stock prices

Common.fs - common things used in the rest of the project (i.e. data modeling and common funcs)
html.fs - functions to scrap html tables, rows and cells
loader.fs - this is where the main async downloading algorithms are implemented
persistence.fs - async saving and loading of stock prices to files
algorithms.fs - async calculations of compound yearly returns given tickers and dates
dotNetWrapper.fs - gives a .NET friendly interface to the whole library so that you can use it from C#/VB.NET

Tests - too few testcases running on xUnit (you need to download xUnit separately to run them
ReturnCalculator - simple console application to show usage of the library
Bob - rough winforms UI application that uses the library. An unnamed friend (Jonathan) promised me that he was going to create a UI for my little library. My requirements were very simple: "I want the best UI app of this century, one that fully takes advantage of the async nature of my code". Bob doesn't fully satisfy my requirements yet :)

The code in persistence.fs, algorithms.fs and especially dotnetwrapper.fs is pretty rough and uninteresting. This is why I don't blog about it. I reserve the right to do it if I get around to clean it up.

Downloading stock prices in F# - Part IV - Async loader for splits

lucabol — Sat, 20 Sep 2008 00:59:00 GMT

Other parts:

Downloading splits is a messy affair. The problem is that Yahoo doesn't give you a nice comma-delimitated stream to work with. You have to parse the Html yourself (and it can be on multiple pages). At the end of the post, the overall result is kind of neat, but to get there we need a lot of busywork.

First, let's define a function that constructs the correct URL to download splits from. Notice that you need to pass a page number to it.

let splitUrl ticker span page =
    "http://finance.yahoo.com/q/hp?s=" + ticker + "&a="
    + (span.Start.Month - 1).ToString() + "&b=" + span.Start.Day.ToString() + "&c=" 
    + span.Start.Year.ToString() + "&d=" + (span.End.Month - 1).ToString() + "&e="
    + span.End.Day.ToString() + "&f=" + span.End.Year.ToString() + "&g=v&z=66&y="
    + (66 * page).ToString();

The reason for this particular url format (i.e. 66 * page) is completely unknown to me. I also have the feeling that it might change in the future. Or maybe not given how many people rely on it.

I then describe the driver function for loading splits:

let rec loadWebSplitAsync ticker span page splits =
    let parseSplit text splits =
        List.append splits (parseSplits (scrapHtmlRows text)),
                                           not(containsDivsOrSplits (scrapHtmlCells text))
    async {     
        let url = splitUrl ticker span page
        let! text = loadWebStringAsync url
        let splits, beyondLastPage = parseSplit text splits
        if beyondLastPage then return splits else
                                 return! loadWebSplitAsync ticker span (page + 1) splits }

This is a bit convoluted (it is an Async recursive function). Let's go through it in some detail. First there is a nested function parseSplit. It takes an html string and a list of observations and returns a tuple of two elements. The first element is the same list of observations augmented with the splits found in the text. The second element is a boolean that is true if we have navigated beyond the last page for the splits.

The function to test that we are beyond the last page is the following:

let containsDivsOrSplits cells =
    cells |> Seq.exists
        (fun (x:string) -> Regex.IsMatch(x, @"\$.+Dividend", RegexOptions.Multiline)
                           || Regex.IsMatch(x, "Stock Split"))

This function just checks if the words "Stock Split" or "Dividend" are anywhere in the table. If they aren't, then we have finished processing the pages for this particular ticker and date span.

The function to extract the splits observations from the web page takes some cells (a seq<seq<string>>) as input and returns an observation list. It is reproduced below:

let parseSplits rows =
    let parseRow row =
        if row |> Seq.exists (fun (x:string) -> x.Contains("Stock Split"))
        then
            let dateS = Seq.hd row
            let splitS = Seq.nth 1 row
            let date = DateTime.Parse(dateS)
            let regex = Regex.Match(splitS,@"(\d+)\s+:\s+(\d+)\s+Stock Split",
                                                                   RegexOptions.Multiline)
            let newShares = shares (float (regex.Groups.Item(1).Value))
            let oldShares = shares (float (regex.Groups.Item(2).Value))
            Some({Date = date; Event = Split(newShares / oldShares)})
        else None
    rows |> Seq.choose parseRow |> Seq.to_list

It just take a bunch of rows and choose the ones that contain stock split information. For these, it parses the information out of the text and creates a Split Observation out of it. I think it is intuitive what the various Seq functions do in this case. Also note my overall addiction to the pipe operator ( |> ). In my opinion this is the third most important keyword in F# (after 'let' and 'match').

Let's now go back to the loadWebSplitAsync function and discuss the rest of it. In particular this part:

async {     
    let url = splitUrl ticker span page
    let! text = loadWebStringAsync url
    let splits, beyondLastPage = parseSplit text splits
    if beyondLastPage then return splits else
        return! loadWebSplitAsync ticker span (page + 1) splits }

First of all it is an Async function. You should expect some Async stuff to go on inside it. And indeed, after forming the URL in the first line, the very next line is a call to loadWebStringAsync. We discussed this one in the previous installment. It just asynchronously loads a string from an URL. Notice the bang after 'let'. This is your giveaway that async stuff is being performed.

The result of the async request is parsed to extract splits. Also, the beyondLastPage flag is set if we have finished our work. If we have, we return the split observation list; if we haven't, we do it again incrementing the page number to load the html text from.

Now that we have all the pieces in places, we can wrap up the split loading stuff inside this facade function:

let loadSplitsAsync ticker span = loadWebSplitAsync ticker span 0 []

And finally put together the results of this post and the previous one with the overall function-to-rule-them-all:

let loadTickerAsync ticker span =
    async {
        let prices = loadPricesAsync ticker span
        let divs =  loadDivsAsync ticker span
        let splits = loadSplitsAsync ticker span
        let! prices, divs, splits = Async.Parallel3 (prices, divs, splits)
        return prices |> List.append divs |> List.append splits
        }

All right, that was a lot of work to get to this simple thing. This is a good entry point to our price/divs/split loading framework. It has the right inputs and outputs: it takes a ticker and a date span and returns an Async of a list of observations. Our caller can decide when he wants to execute the returned Async object.

Notice that in the body of the function I call Async.Parallel. This is debatable. A more flexible solution is to return a tuple containing three Asyncs (prices, divs, splits) and let the caller decide how to put them together. I decided against this for simplicity reasons. This kind of trade-off is very common in Async programming: giving maximum flexibility to your caller against exposing something more understandable.

I have to admit I didn't enjoy much writing (and describing) all this boilerplate code. I'm sure it can be written in a better way. I might rewrite plenty of it if I discover bugs. I kind of like the end result though. loadTickerAsync has an overall structure I'm pretty happy with.

Next post, some algorithms with our observations ...

Book review: The Halo Effect

lucabol — Mon, 07 Jan 2008 22:04:22 GMT

When I read "Built to Last", "In search of excellent" and "Good to great" I immediately thought: "What a bunch of BS!!". But again, I think that about most business books.

The content of these books seemed particularly ludicrous to me. I found both logical inconsistencies and methodological flaws in their process. For example, picking successful companies after the fact is analogous to picking the winners of a lottery and claim that they have exceptional skills. Betting the farm on a single concept (i.e. Big Hairy Goal) is clearly the right thing when you are right, but most often than not you are wrong. What about all the companies that did that and failed? Also, Keeping to the Core worked really well for Coca Cola, but really badly for Kodak.

These are just examples. These books are packed full of such reasonably sounding absurdities. I thought that everybody would immediately spot this. I thought that the mediocre subsequent performance of the companies originally selected as excellent would obviously demonstrate the unsoundness of such conclusion. I was very wrong. These tomes are hugely successful. People seemed to disagree with me. They really like this stuff. Given that my work and my life were not overly impacted one way or the other, I left the matter to rest.

And then came "The Halo Effect". It was a pleasure for me to read, as it confirmed my own ideas (confirmation bias anyone?) and added many more.

The title is slightly misleading. A better title would have been: "Why the idea of finding common traits among successful companies is baloney". I know, not as catchy. The catchy halo effect, mentioned in the title, is just the biggest delusion that people have about the topic.

I found an experiment recounted in the book to be very illuminating. They divided a set of people in multiple groups and gave each group a problem to solve. They then told half of the groups that they did a very good job and the other half that they did a very poor job. The thing is, they did so at random. They then asked people to rate the dynamics of their group on several factors. People that were told that they did a good job reported to have had wonderful dynamics: open discussions, inspired leaders, high caliber people and such. The opposite was true for the other groups.

It didn't matter what the actual dynamics had really been: some groups were very confrontational, others were very amicable; some were very vocal, others were very quiet. In the end, people who were told that they did good recalled the experience in positive terms. People who were told that they did poorly, recalled the experience in negative terms. Remember, good and bad outcomes were assigned randomly. Also, the results have been confirmed in several similar studies.

It is not a good strategy that drives performance. It is perceived performance that makes people think that they had a good strategy. This is: "The halo effect". This simple concept has dramatic repercussions.

First of all, it invalidates the main process used in these studies: if you use newspapers articles and managers recollection to draw conclusions, you are just piling halos over halos. People would certainly give all sort of positive attributes to companies that succeeded. They do it exactly because they succeeded. And when these companies fail, they suddenly give all sort of negative attributes to them. The book is full of examples of companies that were rated as excellent when their stock price went up and terrible when their stock price went down. Nothing changed in these companies, they were still doing the same exact thing.

The book contains many other insights of why these 'studies' are deeply flawed. It calls them delusions. You are probably familiar with some of them if you have a scientific background: for example, Delusion of Correlation and Causality, Delusion of Single Explanation (factors are not independent), Delusion of Connecting the Winning Dots (ad hoc after the fact selection). Other delusions are specific to the analyzed domain: Delusion of Lasting Success, Delusion of Absolute Performance, Delusion of Organizational Physics.

In the end what should managers do? Instead of following these books's precepts, they should focus on two things: strategy and execution. Both of these things are probabilistic endeavors where the absolutely wrong measuring stick is the result that you obtain. I always found this to be a very common error people fall into at whatever level in an organization. The error is judging the soundness of a decision by its result. A decision should be judged by the soundness of the process to get to the decision and the information available at the time the decision was taken. This is the only thing that is consistent with the probabilistic nature of the world we live in. The book profiles three leaders who embody this probabilistic view.

In summary, please go and read this book. There are many more things in it than I described in this (albeit long and boring) post.

Bisection-based XIRR implementation in C#

lucabol — Tue, 18 Dec 2007 02:17:00 GMT

Here is a quick implementation of XIRR (using Excel nomenclature) written in C#.

Disclaimer: this is a super simple Bisection-based implementation. People tend to prefer the Newton method, but this is simpler and works for the app I'm writing. I decided to post it because I couldn't find one on the net when I looked for it. I attached testcases to show the extent of my testing.

It is called CalculateXIRR and it is invoked by passing a list of cash flows, a tolerance and a max number of iterations.

using System;
using System.Linq;
using Money = System.Decimal;
using Rate = System.Double;
using System.Collections.Generic;

public struct Pair<T, Z> {
    public Pair(T first, Z second) { First = first; Second = second; }
    public readonly T First;
    public readonly Z Second;
}
public class CashFlow {

    public CashFlow(Money amount, DateTime date) { Amount = amount; Date = date; }

    public readonly Money Amount;
    public readonly DateTime Date;
}

public struct AlgorithmResult<TKindOfResult, TValue> {

    public AlgorithmResult(TKindOfResult kind, TValue value) {

        Kind = kind;
        Value = value;
    }

    public readonly TKindOfResult Kind;
    public readonly TValue Value;
}

public enum ApproximateResultKind {
    ApproximateSolution,
    ExactSolution,
    NoSolutionWithinTolerance
}

public static class Algorithms {

    internal static Money CalculateXNPV(IEnumerable<CashFlow> cfs, Rate r) {

        if (r <= -1)
            r= -0.99999999; // Very funky ... Better check what an IRR <= -100% means

        return (from cf in cfs
                let startDate = cfs.OrderBy(cf1 => cf1.Date).First().Date
                select cf.Amount / (decimal) Math.Pow(1 + r, (cf.Date - startDate).Days / 365.0)).Sum(); 
    }

    internal static Pair<Rate, Rate> FindBrackets(Func<IEnumerable<CashFlow>, Rate, Money> func, IEnumerable<CashFlow> cfs) {

        // Abracadabra magic numbers ...
        const int maxIter = 100;
        const Rate bracketStep = 0.5;
        const Rate guess = 0.1;

        Rate leftBracket = guess - bracketStep;
        Rate rightBracket = guess + bracketStep;
        var iter = 0;

        while (func(cfs, leftBracket) * func(cfs, rightBracket) > 0 && iter++ < maxIter) {

            leftBracket -= bracketStep;
            rightBracket += bracketStep;
        }

        if (iter >= maxIter)
            return new Pair<double, double>(0, 0);

        return new Pair<Rate, Rate>(leftBracket, rightBracket);
    }
   
    // From "Applied Numerical Analyis" by Gerald
    internal static AlgorithmResult<ApproximateResultKind, Rate> Bisection(Func<Rate, Money> func, Pair<Rate, Rate> brackets, Rate tol, int maxIters) {

        int iter = 1;

        Money f3 = 0;
        Rate x3 = 0;
        Rate x1 = brackets.First;
        Rate x2 = brackets.Second;

        do {
            var f1 = func(x1);
            var f2 = func(x2);

            if (f1 == 0 && f2 == 0)
                return new AlgorithmResult<ApproximateResultKind, Rate>(ApproximateResultKind.NoSolutionWithinTolerance, x1);
            
            if (f1 * f2 > 0)
                throw new ArgumentException("x1 x2 values don't bracket a root");

            x3 = (x1 + x2) / 2;
            f3 = func(x3);

            if (f3 * f1 < 0)
                x2 = x3;
            else
                x1 = x3;

            iter++;

        } while (Math.Abs(x1 - x2)/2 > tol && f3 != 0 && iter < maxIters);

        if (f3 == 0)
            return new AlgorithmResult<ApproximateResultKind, Rate>(ApproximateResultKind.ExactSolution, x3);

        if (Math.Abs(x1 - x2) / 2 < tol)
            return new AlgorithmResult<ApproximateResultKind, Rate>(ApproximateResultKind.ApproximateSolution, x3);

        if (iter > maxIters)
            return new AlgorithmResult<ApproximateResultKind, Rate>(ApproximateResultKind.NoSolutionWithinTolerance, x3);

        throw new Exception("It should never get here");
    }

    public static AlgorithmResult<ApproximateResultKind, Rate> CalculateXIRR(IEnumerable<CashFlow> cfs, Rate tolerance, int maxIters) {

        var brackets = FindBrackets(CalculateXNPV, cfs);
        
        if (brackets.First == brackets.Second)
            return new AlgorithmResult<ApproximateResultKind, double>(ApproximateResultKind.NoSolutionWithinTolerance, brackets.First);

        return Bisection(r => CalculateXNPV(cfs,r), brackets, tolerance, maxIters);
    }
}

// TESTS

using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
using System;
using Rate = System.Double;

namespace TimeLineTest
{
    [TestClass()]
    public class AlgorithmsTest {

        IEnumerable<CashFlow> cfs = new CashFlow[] {
                new CashFlow(-10000, new DateTime(2008,1,1)),
                new CashFlow(2750, new DateTime(2008,3,1)),
                new CashFlow(4250, new DateTime(2008,10,30)),
                new CashFlow(3250, new DateTime(2009,2,15)),
                new CashFlow(2750, new DateTime(2009,4,1))
            };

        IEnumerable<CashFlow> bigcfs = new CashFlow[] {
                new CashFlow(-10, new DateTime(2000,1,1)),
                new CashFlow(10, new DateTime(2002,1,2)),
                new CashFlow(20, new DateTime(2003,1,3))
            };

        IEnumerable<CashFlow> negcfs = new CashFlow[] {
                new CashFlow(-10, new DateTime(2000,1,1)),
                new CashFlow(-1, new DateTime(2002,1,2)),
                new CashFlow(1, new DateTime(2003,1,3))
            };

        IEnumerable<CashFlow> samedaysamecfs = new CashFlow[] {
                new CashFlow(-10, new DateTime(2000,1,1)),
                new CashFlow(10, new DateTime(2000,1,1)),
            };

        IEnumerable<CashFlow> samedaydifferentcfs = new CashFlow[] {
                new CashFlow(-10, new DateTime(2000,1,1)),
                new CashFlow(100, new DateTime(2000,1,1)),
            };

        IEnumerable<CashFlow> bigratecfs = new CashFlow[] {
                new CashFlow(-10, new DateTime(2000,1,1)),
                new CashFlow(20, new DateTime(2000,5,30)),
            };

        IEnumerable<CashFlow> zeroRate = new CashFlow[] {
                new CashFlow(-10, new DateTime(2000,1,1)),
                new CashFlow(10, new DateTime(2003,1,1)),
                };

        IEnumerable<CashFlow> doubleNegative = new CashFlow[] {
                new CashFlow(-10000, new DateTime(2008,1,1)),
                new CashFlow(2750, new DateTime(2008,3,1)),
                new CashFlow(-4250, new DateTime(2008,10,30)),
                new CashFlow(3250, new DateTime(2009,2,15)),
                new CashFlow(2750, new DateTime(2009,4,1))
            };

        IEnumerable<CashFlow> badDoubleNegative = new CashFlow[] {
                new CashFlow(-10000, new DateTime(2008,1,1)),
                new CashFlow(2750, new DateTime(2008,3,1)),
                new CashFlow(-4250, new DateTime(2008,10,30)),
                new CashFlow(3250, new DateTime(2009,2,15)),
                new CashFlow(-2750, new DateTime(2009,4,1))
            };


        double r = 0.09;
        double tolerance = 0.0001;
        int maxIters = 100;

        private TestContext testContextInstance;

        public TestContext TestContext {
            get {
                return testContextInstance;
            }
            set {
                testContextInstance = value;
            }
        }

        [TestMethod()]
        public void CalculateXNPV() {

            Assert.AreEqual(2086.6476020315416570634272814M, Algorithms.CalculateXNPV(cfs, r));
            Assert.AreEqual(-10.148147600710372651326920258M, Algorithms.CalculateXNPV(negcfs, 0.5));
            Assert.AreEqual(4.9923725815954514810351876895M, Algorithms.CalculateXNPV(bigcfs, 0.3));
        }

        [TestMethod]
        public void FindBrackets() {

            var brackets = Algorithms.FindBrackets(Algorithms.CalculateXNPV, cfs);
            Assert.IsTrue(brackets.First < 0.3733 && brackets.Second > 0.3733);

            brackets = Algorithms.FindBrackets(Algorithms.CalculateXNPV, bigcfs);
            Assert.IsTrue(brackets.First < 0.5196 && brackets.Second > 0.5196);

            brackets = Algorithms.FindBrackets(Algorithms.CalculateXNPV, negcfs);
            Assert.IsTrue(brackets.First < -0.6059 && brackets.Second > -0.6059);
        }

        [TestMethod]
        public void XIRRTest() {

            var irr = Algorithms.CalculateXIRR(cfs, tolerance, maxIters);
            Assert.AreEqual(0.3733, irr.Value, 0.001);
            Assert.AreEqual(ApproximateResultKind.ApproximateSolution, irr.Kind);

            irr = Algorithms.CalculateXIRR(bigcfs, tolerance, maxIters);
            Assert.AreEqual(0.5196, irr.Value, 0.001);
            Assert.AreEqual(ApproximateResultKind.ApproximateSolution, irr.Kind);

            irr = Algorithms.CalculateXIRR(negcfs, tolerance, maxIters);
            Assert.AreEqual(-0.6059, irr.Value, 0.001);
            Assert.AreEqual(ApproximateResultKind.ApproximateSolution, irr.Kind);

            irr = Algorithms.CalculateXIRR(samedaysamecfs, tolerance, maxIters);
            Assert.AreEqual(ApproximateResultKind.NoSolutionWithinTolerance, irr.Kind);

            irr = Algorithms.CalculateXIRR(samedaydifferentcfs, tolerance, maxIters);
            Assert.AreEqual(ApproximateResultKind.NoSolutionWithinTolerance, irr.Kind);

            irr = Algorithms.CalculateXIRR(bigratecfs, tolerance, maxIters);
            Assert.AreEqual(4.40140, irr.Value, 0.001);
            Assert.AreEqual(ApproximateResultKind.ApproximateSolution, irr.Kind);

            irr = Algorithms.CalculateXIRR(zeroRate, tolerance, maxIters);
            Assert.AreEqual(0, irr.Value, 0.001);
            Assert.AreEqual(ApproximateResultKind.ApproximateSolution, irr.Kind);

            irr = Algorithms.CalculateXIRR(doubleNegative, tolerance, maxIters);
            Assert.AreEqual(-0.537055, irr.Value, 0.001);
            Assert.AreEqual(ApproximateResultKind.ApproximateSolution, irr.Kind);

            irr = Algorithms.CalculateXIRR(badDoubleNegative, tolerance, maxIters);
            Assert.AreEqual(ApproximateResultKind.NoSolutionWithinTolerance, irr.Kind);
        }
    }
}

Parsing QIF Quicken files in C#

lucabol — Fri, 31 Aug 2007 23:43:00 GMT

I'm slightly prouder of the structure of this code than the one in the previous blog post. You can simply inherit from QIFParserBase and override a couple of abstract methods to customize the behavior. Still, I just tested it on a couple of test QIF files. It is not production quality at all.

Notice that I don't even have Quicken. I'm producing these test file with FundManager, which I use for my investments. If your software generates QIF files differently, than you have to modify the code. It shouldn't be too hard.

It works with VS 2008 beta 2.

Retrieve prices, dividends and splits for a stock in C#

lucabol — Thu, 30 Aug 2007 23:25:00 GMT

I wrote this code very quickly and I'm kind of ashamed of it, but it gets the job done (I think). You need the HTML Agility Pack for the stock splits retrieving code. You can download it from here or you can simply comment out the code. I wrote it against Visual Studio 2008 beta 2, but it should be trivial to port it to previous versions.

You run it from a command window like this: "priceretriever msft 1/1/1990 2/3/2003". The last two parameters are optional and default to 1/1/1980 and today.

Enjoy.

A financial education (my ideas)

lucabol — Fri, 20 Apr 2007 21:55:39 GMT

When I came to US, I wanted to get an introduction on personal finance. I needed to know what to insure, when, how to buy an house and all this sort of basic things. At the time I read Personal Finance for Dummies. I then discovered that very many US folks don’t know about these things either and I found myself recommending this simple book to plenty of people. You can find it for a few bucks in used book-stores. It might be too basic for you, but again, it might not.

Investment is a subset of personal finance and by far not the most important one (the most important probably being how to save the most – which means paying you first). Nevertheless, understanding how to invest the money that you do save is important. The following represents a very condensed summary of my views on the topic. There are dozens of others very good books that I could have suggested instead.

The first thing I would do is to get a decent understanding of the relationship between risk and reward that is so pervasive in financial markets. Good books on the topic are The Four Pillars of Investing or A Random Walk Down Wall Street. A new book that I haven’t read, but suspect quite good (and short) is The Little Book of Common Sense Investing. This books try to convince you that it is not possible to ‘beat the market’ and the best you can do is to decide on an asset allocation that represents your risk/reward profile. If you believe that, then it is worth getting to the next step by understanding the concept of mean variance optimization. Good medium level books on that are All About Asset Allocation and the Intelligent Asset Allocator. There are advanced books on the topic as well, but I won’t mention them here. An advanced service in this area is IndexInvestor.

With time, analyzing the amount of evidence, I came to believe that it is indeed possible to beat the market. But it is not easy and 99% of the methods out there are bunk. Moreover it does require a lot of work and a certain passion for the investment field. So, if you don’t have time or passion you are probably better off sticking to the strategies described in the aforementioned books. If you have a small amount of time, but you still want to try and beat the market, an avenue open to you is the selection of mutual funds. Even then, you need to have some strong held view of which kind of investment strategy and character traits are more likely to bring about superior investment results so that you can pick the right manager (or most likely set of managers) for your money. I describe below what are my personal views on the matter. They are all very debatable. A good service that gives sane information about mutual funds, how to select them and assemble them in your portfolio is Morningstar. I certainly suggest you subscribe to it, if you invest in mutual funds.

A time honored and generally successful stock picking method is value investing. The father of value investing is Benjamin Graham and two books written by him are everlasting classics of investment. They are The Intelligent Investor and Security Analysis. Aficionados disagree on which releases are the best, the links provided represent my opinion. Please start with the former, as the latter is long and complex. They are both quite dated, but plenty of folks believe that not much has been discovered after their publication. I disagree. Another, maybe better, introductory book is The Little Book of Value Investing. There are wonderful mutual funds out there that invest in the spirit of value investing. Among others, I like Third Avenue, Pinnacle and Royce.

Warren Buffett expanded on Graham by putting more emphasis on the quality of the companies he invests in instead of mostly the price. A good book describing his approach is The Warren Buffett Way. You can obviously go and read Buffett’s letters directly here, but that is better suited as a second step after reading Graham or the aforementioned book. There are plenty of disciples of this method of investing. If you want to have some of them managing your money, my preferred are Longleaf, Fairholme, Weitz and Clipper among others. A good service that suggest stocks based on this style of investing is the stock section of Morningstar. You also can invest in brk.b directly to buy the services of Buffett himself. I personally believe it is quite undervalued at the current juncture.

In the past 15 years or so, there has been a lot of interesting work done on purely quantitative investing systems. They have several advantages over judgment based ones (essentially the human brain is not wired correctly to fight a complex adaptive system as the stock market). There are two categories of systems I feel I can recommend in this area: fundamental based and momentum based. This is a very complex mathematical topic at the high end, but it is possible to get a profitable understanding of it by reading WWOWS and The Little Book That Beats The Market. These books describe fundamental based systems, for a purely momentum based one the best bet is FundX. The web site is not too informative, but if you go to the library and ask the librarian they have a little book written by the FundX guys that describe their system in a great level of detail. The best quantitative mutual funds I can think of are from Bridgeway (there are also others). Like all the other funds I suggested, they are managed by wonderful people that takes the shareholders interests very seriously.

You can also try to tactically switch between different asset classes (i.e. stocks and cash) based on some indicators. I have mixed feelings about this. It might work, but it is debatable how much do you really gain by doing it, after considering taxes and commissions. It is something I could investigate more. Two good books on the matter are The Only Three Questions That Count and The Research Driven Investor. Good mutual funds that do this (with very different risk profiles) are Hussman and CGM.

A couple of notes to close. Reading these books is an exercise in critical thinking. There are huge amount of contradictory statements on very basic and important concepts. The only way around it is to do your own research and make up your own mind. Even then, you won’t be in a situation of perfect information/clear decision. This cannot be so because the market changes constantly. As soon as something is discovered it gets priced in. This is why it is such an exciting field of study (for some people at least). Also I have my own key rule about investing: I never invest in anything that I don’t understand completely. Every time I’ve done that, I've regretted it. If this is the only thing that you remember after reading this, I’ll be very happy.

A good Emerging Market fund

lucabol — Fri, 02 Feb 2007 21:26:00 GMT

If you are looking for a good fund for this corner of your portfolio, you might want to consider Eaton Vance Structured Emerging Mkt (EAEMX). There is also a much cheaper version (EITEX), but it is open just to folks with a financial advisor.

It is a mechanical fund. Here is how their strategy works (as far as I understand it):

They divide EM market countries in 4 categories, they assign a weight to each category (I.E. 20% to tier1, 30% to tier2 etc...)
For each category they equal weight each country in the category (I.E. in tier1 equal weight Russia, Mexico, etc...)
For each country they equal weight each sector
They actively harvest tax losses

The end result is as follows:

Smaller country have more weight than in a normal Em.Mkt. fund. This is good because:

Smaller markets have had higher returns in the past
Smaller markets are less correlated than large markets both among themselves and against the SP500

Equal weighting the sectors has the following positive effects:

You get more exposure to sectors, like Services, that are going to benefit from economic growth
You avoid the 'commodity effect': the tendency of Em.Markets to fall when commodities prices fall because such a large slice of their economy is based on it

It is extremely tax efficient because:

The mechanical strategy lends itself to low turnover
They actively harvest tax losses

In essence they maximize the positive effects of diversification and tax efficiency, which are very stable effects. They are mathematically provable :-) The real time results confirm the theory as this has been one of the best performing Em. Markets funds with the lowest standard deviation and best tax efficiency. No-one knows what the future holds, but there is a good chance that this will continue to be the case.

I don't own it currently because I don't like the high fees, but the strategy makes so much sense that I might end up buying into it.

A trading/portfolio management Excel Add-in based on the books by Ralph Vince

lucabol — Fri, 05 Jan 2007 01:53:00 GMT

I like to write code and I still manage to do it even now that it is not my primary job. I never post the things that I write because I don't want to maintain them. Lately Charlie convinced me that I don't have to do that. I can just throw the code out, without much preparation or implicit contract of perpetual maintenance.

This one is an Excel add-in that adds functions to Excel to analyze your trading and manage your portfolio. Notice that I'm not a professional trader or statistician, so the whole thing could be wrong, buggy or conceptually absurd. Probably it is all of the above.

I used the extremely good ExcelDna to write the add-in. You need to download it and follow the instructions in HowToInstall.txt on the attached zip file to use it. I based the formulas mainly on the work of Ralph Vince. Please buy and read his books on money management as they are wonderful.

On the statistics side: I made up the "Downside correlation coefficient", I have no idea if it is statistically sound.
On the technical side:

I haven't optimized the algos at all. I.E. I'm sure there is a way to calculate the standard deviation without navigating the array twice, but I didn't bother to look. They end up being fast enough anyhow (apart from the Monte Carlo related functions that I should investigate).
I haven't organized the code correctly. It needs to be rewritten now that I partially know what I'm doing.
I haven't used LINQ. It will be a lot of fun for me to rewrite the code to be more OO and to use LINQ in the process.

I plan to implement OptimalF for a normal distribution and Efficient Frontier calculations in the future. I have no idea when.

I also have some other projects that might be interesting to share (I.E. a blackjack simulator that you can use to try the result of different strategies inspired by Blackjack Attack).

This is the list of Excel functions that you get with this add-in. They ends up in a Trading category in the list of functions on the "Insert Function" Excel dialog:

AnnualizeRet(double, double)
AnnualizeStdevp(double, double)
AnnualStdevpParam(double, double)
ArithMean(object[,], double, double, object, double, double)
AutoCorrel(object[,])
AvgLoss(object[,], double, double, object, double, double)
AvgWin(object[,], double, double, object, double, double)
Chebyshev(double)
Correlat(object[,], object[,])
Describe(object)
DownCorrelat(object[,], object[,], double)
DownStdevp(object[,], double)
EGM(double, double)
EGM2(double, double)
GM(object[,], double, double, object, double, double)
Kelly(double, double) TradingLibrary.Losses(object[,])
MaxDD(object[,])
MaxDDFromEq(object[,])
MaxLoss(object[,])
MaxNegRun(object[,])
MaxWin(object[,])
MCMaxDD(object[,], double)
MCOptimalF(object[,], double, double, double)
OptimalF(object[,], double, double)
OptimalFArray(object[,], double, double, double)
OptimalPos(double, double)
PortAnnualGM(double, double, double, double, double, double, double, double)
PortAnnualTaxGM(double, double, double, double, double, double, double)
PortAnnualTaxGMGivenTrades(object[,], double, double, double, double, double, object, double, double)
PortStdevp(double, double, double)
PRR(object[,], double, double, object, double, double)
PRRParam(double, double, double, double)
RealReturn(double, double, double, double)
RunTest(object[,])
SharpeRatio(double, double, double)
SortinoRatio(double, double, double)
Stdevp2(object[,], double, double, object, double, double)
TWR(object[,], double, double, object, double, double)
Wins(object[,])

To rent or to buy a house, some more evidence of my unpopular thesis ...

lucabol — Wed, 09 Mar 2005 20:01:00 GMT

I discussed this topic in a previous post http://blogs.msdn.com/lucabol/archive/2004/07/30/202394.aspx and I received a number of emails telling me how crazy I was even speculating that, at the current juncture, it may be more convinient to rent than to buy a house.

This week The Economist runs a piece where they make the same exact claim. House prices are so high and rents are so low all over the world that renting might, indeed, be better. Even considering the low interest rate environment, it would seem that the house prices are out of step with reality, very much as equities where in the 90's.

It is all a game of expectations. The harder things to estimate are the future house price gain and the interest rate/gain you can get by investing the difference between rent and mortgage payments. By using difference estimates, you get difference answers. My own opinion is that the probabilities weight against buying an house at this time. Here is a spreadsheet to help you out: http://blogs.msdn.com/lucabol/archive/2004/08/18/216662.aspx .

There are other considerations: I bought a house not to get killed by my wife :) But I've been able to limit the expense and I avoided investing more money in real estate.

"Rent or buy" spreadsheet

lucabol — Wed, 18 Aug 2004 19:13:00 GMT

Someone asked me for the spreadsheet I used for the calculations in my previous posts. Here you have it. The data doesn't reflect my current situation.

Feel free to use it with your own numbers (and tell me of any bug).

I'm buying a house...

lucabol — Mon, 02 Aug 2004 19:57:00 GMT

I think I've been misinterpreted in my previous posts. I'm not suggesting that renting is better than owning. You have to run your numbers and make your own choice. There are many assumptions to consider (financial and not). The not financial ones often are more important. You have to run your numbers, but in general renting looks better if:

You cannot buy ( :) ) and/or
You plan to stay in a place for a short time and/or
You believe houses are vastly overpriced in your area and they'll return will be negative for a while and/or
You don't want the psychological part of having a house (maintenance, tax payments, ...)

In my case the most worrisome scenario is 3.

Back to our situation, we are buying a house for the pleasure of living in a particular location. Financially it is an OK transaction (fair price, I think).

I'll probably post a little rent-buy spreadsheet as soon as I have time to refine it.

Thanks for all the good comments.

To rent or to buy a house??

lucabol — Fri, 30 Jul 2004 18:06:00 GMT

Spurred by some comments to a previous post I quickly run these numbers. They analyze the case of house price not rising in the next five years. I’ll take more time to check them if I really get to a point of deciding, but they should be correct.

This data (fairly typical in my area):

Home price: $200,000

Down payment: $0

Interest rate: 6%

Monthly mortgage: 1210

Rent: 750

Annual gains on investments: 6%

If I run this for 5 years and house appreciation 0% I get:

$33,043 on investing the rent-mortgage difference

$30,609 on paying the mortgage (equity accumulation + tax savings at 28%)

If we assume to put down 20% to buy the house then the numbers are even worse:

$83,542 on investing the down payment and the rent-mortgage difference

$67,548 on paying the mortgage (equity accumulation + tax savings at 28%)

There are other things to consider: house repairs for houses, annual raises in rent, etc… etc…

Are those assumptions reasonable? So it is better to rent than to buy? Well, over the long term houses do tend to appreciate and the more leverage you have, the more you reap the benefits. Over the short term, run your numbers. There are so many assumptions you can make, that it become almost a personal opinion what is best.

Running the numbers help you in making sure than it is at least an informed opinion. It makes your assumptions explicit

My quest for a new home...

lucabol — Thu, 29 Jul 2004 16:10:00 GMT

I’m refining my understanding of how to value residential properties. There is this rule of thumb: buy when you can get a rent for the property which is at least 1.2% of the property price (or in another form, when the home price is 7x the annual rent). This is an approx for a more formal spreadsheet (which I have).

This is fine and dandy, but in the area I’m interested in a typical one bedroom condo sells for around $200,000-300,000 and you can get a monthly rent of $1,000-1,300. This doesn’t work by a very large factor. Hmmm…

Let’s change the target to a cheaper area. Now I can probably get around $800 monthly. To make things work I need to find a condo for $67,200. Not an easy task, almost impossible.

It looks like for me it is hard to find a house in an area I like and at the same time make a good investment (unless the market bails me out by raising home prices at a high rate). There is quite a large disconnect between house prices and rents now. Maybe the residential market is really on the brink of a fall… Time to rent again?