Nathan Brixius : Software Development

Writing high performance code

Nathan Brixius — Fri, 11 Dec 2009 04:29:00 GMT

During Scott Hanselman's podcast we briefly discussed managed code performance. Here are a few of my thoughts about writing high performance code. In the spirit of a cheesy business book (or motivational speaker), I will give away my main points in the form of controversial-sounding statements before proceeding to flog them to death:

Don't trust anyone.
Managed is better.

Some background: before I came to Microsoft I built solvers for both discrete and continuous optimization problems, primarily in C/C++, with a bit of Fortran thrown in. Back in 2002 when I was on the Project Server team we were faced with the decision of whether to rebuild our scheduling engine in C++ or C#. For the past year or so I have worked on the Solver Foundation team, which is pure C#. We build solvers that solve a range of large-scale optimization problems, and they are “industrial grade”: tens or hundreds of thousands of constraints are not uncommon.

Don't trust anyone. Rico Mariani says the first rule of performance is to measure. Listen to Rico. There are many reasons for this. The first is that nobody has ever written code that is doing exactly what you are trying to do. Otherwise (being a smart programmer) you should just call their code or (being a smart business person) you should just buy their code, try to do it better, or do something different entirely (perhaps mixed martial arts). So you should seek to understand how the code behaves, otherwise you will never find out - until it is too late.

The second is that you know your scenarios and priorities better than others do. You should already have an idea of how fast the code is supposed to be. Now sometimes the requirement truly is "make it as fast as possible". For numerical libraries this is often the case because many problems can be made to be arbitrarily complicated. For example, if you are modeling fluid dynamics you could use tighter meshes and get more accurate results. In most other cases (perhaps in a user interface) you have a hard number you need to hit. Of course, there are some simple guidelines that may apply in most situations. If you never go to the database, your server app will be fast. If you never allocate memory, your numeric code will fly. True or false? Well…measure.

The third is that even if someone else has measured code that seems to be similar to yours, their results may be misleading. For example, you depend on different collection classes, or you are running on a different processor. Or your input data is different in some subtle way. To take a very simple example, suppose you want to sort lists of 1 million integers, and somebody says, "hey, I measured different sorting algorithms on arrays of 1 million integers and quicksort is best." And they hand over the code and all of their results. Well, what do those arrays look like? Because if the data is mostly sorted then something like insertion sort may work better.

If you follow this advice you may end up with more questions: doesn't this take a lot of time and effort? How do I know when to stop measuring, or stop optimizing my code? If you start profiling at the very start of development, and build some simple skills, it's not too bad (and you learn a lot). If, for example, you are considering whether to use managed or unmanaged code for a project, prototype a couple of key sections in C++ and C# and measure. I did this on the Project Server team and that was the best way to settle the argument. These things often get way too philosophical – just try it and see. I did standard critical path scheduling in both C# and C++ and found that the C# version was only 10-15% slower. We went with C# and in the course of development we were able to find algorithmic improvements that resulted in a faster implementation that the original C++ codebase. Get good at writing simple programs that answer one specific performance question. For example: should I use a list or an array? How should I read this data set? Be a scientist and seek to answer specific questions by controlling the conditions and varying only one thing at a time. And pick the right thing based on your knowledge of the customer scenario.

Managed or unmanaged? My blunt opinion is: unless your computation is dominated by BLAS-style computations you are better off using C# because of productivity, stability, and maintainability concerns.

Here are a couple of “pros” for C#:

There are clear performance differences, but in many cases when you look at the overall system performance they can be washed away due to other factors: I/O, a webservice call, DB access, etc.
Many programmers (myself included) are much more productive writing C# code. This gives me more time to investigate possible algorithmic improvements, write good unit tests, etc. There are opportunity costs!
I think it is easier to make use of multiple cores. For example, the parallel extensions available for .Net 3.5 (and built into 4.0) are very easy to use.

There are also some clear downsides:

It is easy to get lazy, or to unintentionally use constructs that are not efficient. People start using List<> and Dictionary<> when a good ol’ array will do. It’s also easy to miss unnecessary memory allocations, because everything is garbage collected. So you need to be vigilant about profiling the code.
There is a lot of unnecessary array bounds checking that slows things down. In particular, with LAPACK-style routines with tight, nested loops.
As far as I know, the managed code compilers do not make use of SSE2 instructions.

Others have different opinions and I’m not saying they’re wrong - so long as they have measured. Otherwise they are bullshitting you.

Solver Foundation on Hanselminutes

Nathan Brixius — Wed, 09 Dec 2009 15:57:00 GMT

The latest edition of Scott Hanselman's Hanselminutes podcast features yours truly. We talked about Solver Foundation, numerical optimization, and good old-fashioned code optimization. Scott's a great guy and doing the podcast was a lot of fun.

Those of you new to Solver Foundation (or this blog) may be curious about what you can actually do with Solver Foundation. Here is the download location for the free Express version of Solver Foundation. I hope that by listening to the podcast you already know the basics, so here are a few code-oriented posts:

Here is a short description of “what’s new” in our 2.0 release.
Here is a C# sample that solves traveling salesman problems.
Here is a LINQ to SQL sample using Solver Foundation.
Here is an F# sample. The F# integration features a domain specific language for creating optimization problems. It also supports “units of measure”, meaning you can associate real-life units such as miles, gallons, dollars, or touchdowns to parameters and decisions.
Erwin did a really nice post on doing portfolio management using Solver Foundation and Excel. I like the visuals. In fact Erwin's site has a number of great samples and documents.

And I'd be remiss if I didn't mention that Solver Foundation ships with TONS of samples that span languages, scenarios, and solvers.

The last part of our conversation was about writing high-performance code. I'll share some thoughts about that later this week.

Ctrl+Tab works in Visual Studio, too.

Nathan Brixius — Thu, 02 Apr 2009 20:42:00 GMT

I have been brushing up on keyboard shortcuts ever since I installed the Windows 7 beta on my laptop. Tim Sneath has a great summary of keyboard shortcuts here, in particular I LOVE the window management shortcuts. If you have IE8 you also probably know that Ctrl+Tab switches between browser tabs. What I did not know, and just discovered by accident, is that Ctrl+Tab works in Visual Studio too. You can use it to switch between active files and tool windows (e.g. the Solution Explorer or Class View). The arrow keys work too. I tend to have lots of active files at once, so this is a nice timesaver.

Feel free to share other cool shortcuts in the comments.

A side note: I am a Unix guy by background, and during my first two years at Microsoft I did all my development using the command shell + emacs. But given the improvements that Visual Studio has made over the last 5 years, there is no way I would ever go back now! I am anxiously awaiting their next release.

How to fix bugs (and influence people)

Nathan Brixius — Fri, 04 Jan 2008 22:18:00 GMT

I work on a large team with a lot of dependencies and fairly tight deadlines. I wanted to share some of a few techniques for fixing bugs quickly. Maybe this is all common knowledge but I’ll share it anyway.

Trade bugs. It’s not very efficient if you and another dev to be modifying the same code at the same time. Swap bugs with your teammates to define clear ownership. Sooner is better than later.
Fix areas, not bugs. Don’t just fix one string on a page, fix all the strings on the page. If a validation check was missing in one API, it is probably missing in other APIs too. Proactively look for related bugs assigned to other people and take them on.
Kick off a build every night before you leave. I have known people who scheduled jobs to do this automatically.
Pipeline your work. There is a lot of idle time spent waiting for builds to finish, deploying, etc. Fill that time with something useful by switching over to another bug. My pattern is usually to repro the bug (stepping through the code), code the solution, then do final testing. So I like to work on three bugs (actually three sets of bugs, see above) at once:

            debug   code     test
                        debug   code     test
                                    debug code     test

Sometimes there is a fourth step – “think” – for more difficult bugs.
Write tools. Sometimes it is a big pain in the ass to work through the product to repro and fix bugs. Sometimes building simple tools can really help your productivity, for example a simple winforms app that mimicks a more complicated client application. I have written command line apps that talked to webservices in the past. Once you build the tool, let your teammates know.
Unit test and refactor. Try to think about how you can build a strong foundation of unit tests that are easy to add to. Unit tests prevent regressions. Also, once you have confidence in your unit tests you can fearlessly refactor code if you notice lots of bugs in a certain area.
Schedule uninterrupted time for yourself. It’s hard to work effectively in 15 minute chunks.
Don’t break the f***ing build. That slows down the whole team. If you notice a blocking issue, jump on it immediately even if it is not your area. You will learn something in the process.

What makes a good software developer

Nathan Brixius — Tue, 24 Jul 2007 03:59:00 GMT

A colleague asked me to jot down some thoughts on what makes a good developer at Microsoft. This is by no means complete, but three statements come to mind when I think about strong developers:

We are engineers, not hackers.
A strong dev should be able to fill any engineering role in the company.
Senior devs improve their teams by action and by example.

We are engineers, not hackers. My dev manager in Project (my previous team) said this years ago and it has stuck with me ever since. Engineering is a craft whereas hacking is a hobby, and I expect a Microsoft dev to seek to become a master of the craft. Engineers understand common patterns underlying the problems they solve and seek to make their code predictable and reliable. (This doesn't suck the creativity or joy out of the process, rather it should liberate us.) For an engineer, process is not an end to itself but a tool for arriving at solid designs and trustworthy code. Engineers take great pride in their creations and stand behind them.

Engineering practices apply to all stages of software development. In the design phase I expect a structured approach that leads to a design that clearly meets the needs described in the spec. The deliverable is a design doc that describes how the feature is implemented. A good design doc is an aid to the developer, his/her peers, as well as Test/PM. I don't want to get into details about what makes a good design doc, but it should describe the primary design and implementation challenges and how they are addressed, and why other reasonable approaches were rejected. On a services team it's particularly important to think of the impact the feature has on setup, deployment, and monitoring, and account for these early.

Everyone has things to say about coding guidelines, so I won't dwell on them much here. I assembled a set of team-wide guidelines and I expect those to be followed. I always look for simplicity and consistency. As a services team I particularly value unit tests and tracing.

Finally, I expect a positive handoff to test. By this I mean a detailed TRD (test release document) that describes what the feature does, what works, what doesn't work, and suggestions for testing the feature. I expect code to be structured so that it is easy to test as possible. I don't particularly care for metrics based on the number of bugs a dev fixes. I value developers who prevent bugs from happening in the first place, and it's easiest to do this in the earliest stages of the project before any code is written. This begs the question of how do we measure the quality of a feature relative to the time spent on it. I don't have easy answers except to say that if leads and peers are involved in the entire development process, it is easier to come to a collective understanding of how challenging a feature is and how well the developer has done at implementing it. In other words, it comes down to judgment.

A strong dev should be able to fill any engineering role in the company. The statement applies in two ways. First, a dev should be able to wear their "PM hat" or "test hat" and operate effectively within that role. Perhaps this is statement is too "old school Microsoft", and I certainly don't mean to detract from the roles of Test and PM. After all, Test and PM are also engineering disciplines. I simply mean that devs have a deep understanding of customer requirements and are able to make their own assessments of the relative priority of features and work items, and can make suggestions as to how to improve the experience. Devs should be able to be ruthless testers of their own (and their teammates) features and provide actionable feedback that results in improvements. They write unit tests, which guarantee a certain level of quality. Having the ability to operate in another role also prevents a dev from becoming blocked, and provides broader perspectives which assist devs in interacting with their Test and PM counterparts. Everyone understands where everyone else is coming from, and that makes for a more enjoyable working environment.

Secondly the statement means that a dev is not dependent on domain-specific knowledge to be effective. A deep understanding of specific technologies, code bases, and design patterns is essential for good engineers. But a strong dev is not a "one trick pony" that can only work in one area (though they may prefer to do so, and may be asked to do so from time to time). A dev has technical depth, but also technical breadth, which allows them to see how their code functions to serve a larger purpose. I'd like to think that I could be dropped into the middle of Windows, and though I would probably hate it, I would be effective.

Senior devs improve their teams by action and by example. There are only so many hours in a day, so at a certain point it becomes more difficult to provide more value by simply coding more or working longer. Actions that make others better have a compounding benefit that can have a huge impact over time. Most of us can think of old timers that seem to have an awesome level of productivity, to the extent that it's hard to understand how they can be that effective. I suggest that in most cases, these old timers have mastered the art of making others better. First, they typically do not worry themselves about things like "scope" or "area". Making Microsoft better is their "area", and that extends to fixing bugs in other dev's code, fixing build problems, improving processes, giving feedback to other teams. Master devs share knowledge, through conversations, whiteboard sessions, documents, blogs. They cringe when they hear people talk about "their code"; it's "our code". Their movements are not hurried, but always with purpose. They've got time for other people, and still get their work done. They're able to do this because they identify patterns in their work and automate them, either by defining a design pattern, or by architecting the system appropriately, or by writing tools that do their work for them. Then they share this knowledge with the rest of the team and build a culture where others do the same. Lastly, they continually seek new opportunities to make themselves and their team better. These opportunities come up all the time in a product cycle; a master dev spots them first and steps in to fill a need, for example by forwarding an email to someone who knows the answer; seeing that a new platform component could help implement features in two different projects; adding a new unit test that enforces a coding or stylistic convention.

I love it when devs are passionate about delivering something that works and is useful!

You know you're busy when...

Nathan Brixius — Sat, 21 Jul 2007 00:56:00 GMT

...you start scheduling time on your calendar to 'think'.

Why writing software is like moving

Nathan Brixius — Wed, 20 Jun 2007 03:59:00 GMT

Me and my brother helped my mom move over the weekend. Advice for movers and software developers:

Don't move anything you don't actually want or need. Throw out your SAT study guide, your old copies of Maxim, the Michigan starter jacket, the annotated James Joyce that you've never read but impresses people.
Invite the right number of people. It's a sinking feeling to show up and find out that you and your friend are going to move the whole house. It's fun, but not terribly productive to show up and find 7 other people there - you end up spending a lot of time paging through other people's books, or drinking, or both.
Pack before people show up. Nothing kills me more than showing up to help a friend move to find that they haven't done the prep work. I don't want to show up at 9:00 to find that I'm supposed to help you pack your t-shirts.
Figure out how to load the truck before lifting anything. If it turns out that you don't have enough room then you will waste a lot of time moving stuff around, or worse yet, leaving stuff behind. Taking more than one trip is disastrous. That second trip is never very fun.
Wrap fragile items, or items with sharp edges. It's worth the extra time. Once the truck starts moving objects will tend to bump into each other so it's wise to protect the ones that might break.
Do the heavy stuff first. That way you'll know if you've got enough room in the truck, besides, it's better for morale.
Don't try to do it all by yourself. Yes, you have superhuman strength, and that ottoman isn't all that heavy. But if you've got help then why not take advantage.
Spring for the extra five bucks and unhook the dolly. Then you can stack up all those boxes of books and move them in one trip.
When you get to the new place, just worry about getting the boxes out of the truck and into the right rooms. Worry about arranging the furniture and unpacking boxes later.

And - the appropriate payment is always a six pack.