Random Thoughts and Hints on Software Development

Labor Day weekend in Seattle

2009-09-05T21:37:00Z

"How do you call two rainy days in a row in Seattle?
Weekend!"

For reference, Labor Day is September 7th here, so we have a long weekend: from Saturday September 5th ro Monday September 7th:

As usual from the personal blog...

Legends of Windows Home Server -- Checkin Garfield

2009-06-27T23:47:00Z

It was a logn time ago, so, I guess, it's ok to tell this story at last...

Well, everybody knows what "checkin" is, right? That's when you send your updated, fixed or new code to the depository for safekeeping. There is a lot of systems for that. OpenSource guys usually use CVS for that, there are other commercial ones like PVC, Microsoft has the one of its own called "source depot", as well as SourceSafe.

All they do is just one thing: enable you to restore a previous version if somebody broke the current one. Also, to keep a special version for this very-special-big-corporate-customer, which is not easy, but at least possible with the version control. AFAIK, we don't do that at Microsoft, but one of the companies, I worked before for, had about three major branches for critical customers like Chevron.

Of course, that's not panacea, and when 10+ developers start to checkin their changes left and right,... well... the result is normally not that encouraging. Everybody has his features working standalone, but making it work together becomes a challenge. Actually, we had such a period in our life at the early stage in Windows Home Server. We had a lot to do, and VERY little time to accomplish it. Guess what? Nightly tests become broken and did not want to run no matter what. As a little token of pride, I'd like to mention that tests for my features worked fine -- I was responsible for communications between server and the clients. Which means that when my areas are broken, half of the product tests would fail instantly, so I had to keep my parts in shape. But most of other areas weren't that lucky and overall tests package did not want to run no matter how angry the management was and how actively we as a team were fixing the problems.

Then we found a solution... I had a toy Garfield in my office. After speaking with one of the leads and the manager, we introduced the following rule: to make a checkin, you should have Garfield in your hand. So, you get Garfield, run the tests, if all pass, you do checkin, and give Garfield away to the next one in the line. A sort of a physical mutex preventing concurrent changes.

People hated Garfield. Enough to say that in a couple of weeks my boss put it on the eBay! Don't believe it? See for yourself!

Well, nobody bought it, nevertheless the problem was solved. In three days all the tests were running fine again and never broke again in several weeks. And after the Thansgiving 2006 Garfield was sent to the honorable retirement, while we continued in a more easy way but still without breaking the tests...

By the way, Garfield is still around. It's hanging in my office on the corkboard, reminding the team's veterans of the old times :-)

How to use tripod with the basic Amazon Flip camera

2009-06-15T09:57:00Z

Some time ago I've got an Amazon Flip camera. I loved it, but one thing did not worked well: I've got the basic one and it did not had the jack for a tripod. Not good... The answer happened to be simpler than I expected. You may want to watch this video if you have this camera. You'll enjoy it a lot more afterward. I do now.

Working Paper #666

2008-10-30T05:37:00Z

Recently, in one of the current economic crisis disccusions, I saw a link to a Working Paper #666 from the Federal Reserve Bank of Minneapolis. At first I thought it's a prank. Well, the number alone sounds like some dark humor. Nope. It was not. I went through the link http://www.minneapolisfed.org/publications_papers/pub_display.cfm?id=4062, yeah, really, the site truly belongs to one of the FRS members. For reference, "FRS member" is not any FDIC insured bank, it's one of the very few banks which _ARE_ FRS. And, yes, before this working paper, there are working papers 662, 663, 664, and 665. I guess, just a coincidence. And the actual paper refers to the actual FRS site and their official published data at http://federalreserve.gov/releases/h8/ I checked, yes, it looks like data there really confirm what the paper says...

To be fair, the paper does not argue that US has a financial crisis or likely to get a recession soon. However, it doubts a few statements that were used widely since the crisis was acknowledged. Specifically:

Bank lending to nonfinancial corporations and individuals has declined sharply.
Interbank lending is essentially nonexistent.
Commercial paper issuance by nonfinancial corporations has declined sharply, andrates have risen to unprecedented levels.
Banks play a large role in channeling funds from savers to borrowers.

And once you check the data (again, from the Federal Reserve System site) it really looks like all these statements are false. Credits even increased a bit, interbank credit is about on the same level, commercial papers by nonfinancial corporations are issued as usual at the same rate, in fact, even commercial paper by financial institutions albeit diminished and now at a higher rate, still not reached the rate of 2006. And, most of nonfinancial corporations debt (~80%) are commercial papers bought directly by savers, not banks. Oops...

Wait a minute... Aren't those the same statements that were trumpeting the Doom and forced Senate and Congress to get American taxpayers into $700 bln debt despite tons of calls, letters and faxes of voters demanding not to do so?

What is this? Secretary of Treasure being clueless? Who is lying? Or did we got another PR campaign like in 2001? "Wag the Dog"?

I specially like the conclusion of the report:

Our analysis is based on publicly available data. Policymakers have access to other sources of data as well. Policymakers could well believe that bold action is necessary based on data that are different from that considered here. If so, responsible policymaking requires that they share both the data and the analysis that underlies the need for bold policy with the public.

---
Free-form translated from my Russian blog...

P.S. Added August 6th, 09: Interestingly enough this post was hacked and defaced. I guess, here is one more question "Why?"

Economy crisis 101

2008-10-24T05:25:00Z

Pardon for another non-technical post, but I thought it may be interesting. A friend of mine, a professor at the University of Texas, presented an article of his student, introducing some mathmatical theory explaining the roots of today crisis in economy. And I could not resist to answer. If you know all this, please, understand. Afetr all, if some professors don't get it, it may be interesting to a fair share of people around...

So, here it is:

Do you think that may be this article is a little au contrare to Occam's razor principle?

You see, subprime mortgages per se are not what's the real reason of today's crisis. If it would be so, the cost of handling it would not be $700 bln+, but only around $40-50 bln to help people stay in homes and let the real estate bubble go down slower without big economical or social impact.

The real problems is derivatives based on these subprime mortgages. You don't need a mathematical theory or even Excel to explain it. Although, I admit, there was straightforward and intentional mathematical error in the middle, all right. In a case, somebody missed how it was done, let me explain with an example. Imagine a fleet of 1000 cars going over very long bridge which has a 50% chance to collapse in a next few minutes. Owner of this fleet, scared of the possibility, asks you to buy the whole fleet at a very attractive price. Now, you think, the chances that bridge will collapse is 50%, hence the chance to lose each specific car is 50%, now if I have 1000 cars then the chances that at least 10% of cars survive is much greater than that. You know, say for two cars probability that at least one to survive is p1+p2-p1*p2 = 0.5 + 0.5 - 0.5*0.5 = 0.75, that is 75%. And for a 1000 cars it's much much better. So if the part which will survive will cover the cost, you are good. You see the problem? Dependent events were represented as independent. A mistake unforgivable to a college student, but somehow ok for Wall Street CEOs.

If it's still not clear, let's say the fleet is 10 cars and owner offers you price of 10% of fair value. The chance that at least one car will survive if events are independent is 99.95%, so it looks like a sure shot. So you are entering the game with the expectations of 99.95% chance of not losing money and great expectation of making money. In fact, the chance of losing money is still 50%.

What they did was packaging a lot of subprime mortgages and applying the logic above. Then they issued the bonds based on, say, 10% of those mortgages, _whichever will survive_. And they got AAA rating to these bonds. And then, in expectation of profit, they issued bonds on these bonds with leverage ($1 in original bonds produced $10 in next derivatives) exceeding total annual planetary gross product in times. What was ignored is that the risk of subprime mortgages is not merely financial state of borrowers, but the state of the real estate bubble, which was going to burst with not even 50%, but with 100% probability, everybody knew that ahead. And financial state of the borrowers was not that independent either.

So, you see, no need for extra math or even Excel (although, I used it to calculate probabilities above). What we deal with is cheating and larceny, nothing a good cop could not handle in time without a need for extra math. Unfortunately, neither Greenspan, nor Bernanke proved themselves to be good cops.

A Case Against Code Reviews

2008-08-23T09:17:00Z

I know, it sounds odd in the modern world where code reviews are almost like one of the Sacred Commandments, that is taken religiously and you could be burned at the stake for not respecting it. So, please, hold with me for a little heresy… No, I don’t argue that code reviews are valuable. It’s just everything in this world has cons and pros, and I’d like to discuss one unusual “con” of code reviews that usually does not come to light.

Anyway, imagine a small team very busy doing the work, that is, new features for your product. I stress that: everybody is busy. No kidding, real busy. And, surprise!, just like every other product, yours has bugs. And bugs are to be fixed, right? Right.

Now, how you’d fix a bug? There are generally, two philosophies about fixing the bugs. Well, may be more, and of course, there is that stupid “just fix the bug”, which completely ignores metaphysical side of the things we are discussing right now, but let me ignore this one for a while. So, these two approaches are patching vs. refactoring.

Now, what would you think is the right way to fix bugs? Admittedly, there are cases when patching is acceptable Examples? Quick Fix Engineering patches. Products, which are really done, and the less you touch them the better. Fixing embedded code of a space station passing a Jupiter, where every byte of the fix increases the chance of the whole fix to be corrupted and disable a multi-million irreplaceable spacecraft.

However, in most cases and in what’s considered “good engineering”, I strongly believe that refactoring is far superior to patching. It does not mean that every small bug requires complete rewrite of the product, not at all. But it means that you have to consider a bigger frame and not be afraid to change quite a bit, if necessary.

I know, I know, it sounds theoretical and questionable, so let me give you an example. Considering short size of an article, I’ll have to give a very simplified example, but quite a valid one nevertheless. Suppose, you have a method called DownloadCompletedInstant(), which is a callback at the moment, when some download is completed. In fact, all it does is to allow you to start downloading the next file. You find a bug, when one of our colleagues added to it something really really stupid, because he was confused with “Download completed” part of the name. In fact, it would not be stupid, if it would be in DownloadCompleted() callback instead of DownloadCompletedInstant() one. So, now you have two options:

1. Just move your colleague’s code from DownloadCompletedInstant() to DownloadCompleted()

2. Same as (1) plus renaming DownloadCompletedInstant() into ScheduleNextDownload(), hence avoiding a chance of future confusion.

So, what would you do? (2), clearly, right? However, there is a little problem. The renamed call is used in ten other files. So, although the change is microscopic, it affects much more files than a simple patch.

Now, your team is doing code reviews, so what’s the higher chance of getting it fast – posting a change with just a few lines of changes in a single file, or a changes with several dozen lines changed in a dozen or so files? And don’t forget, you want it fast. In part, because you want to fix the bug and get to the next, in part, because you don’t want the fix sitting on our computer and not being checked in. But! Everybody else is busy too. Everybody else has their own bugs. So, naturally, you would rather get a change with minimum affected lines just to get your colleagues do code review faster.

The result? The product gets a patch, not refactoring, and in a couple of months somebody will again misuse DownloadCompletedInstant() for something else, as bad as this time or even worse.

So, does it look right? Is it just me or is there a real problem somewhere here?

The Long Tail by Chris Anderson

2008-08-07T09:14:00Z

Long Tail, The, Revised and Updated Edition: Why the Future of Business is Selling Less of More by Chris Anderson – Hyperion, 2006/2008, 267 p., ISBN 978-1-4013-0966-4

This book is almost a classic by now, so if you did not heard about it, at least briefly going through pages may be a great idea. Although it’s not the reason I decided to write about it.

The key idea of the book is that for certain markets, where cost of production is low, cost of delivery is low and cost of filters, which let you distinct good things from bad, is low, the majority of revenue may come not from a few hits selling at large numbers, but rather from a huge number of items selling very little each. “Long tail” refers to the long tail of the distribution curve, where these “bottom-sellers” reside. In particular, it’s shown that in markets like digital music, DVD rental, print-on-demand books and so on, the long tail may bring the majority of revenue. Indeed, 100,000 items selling 10 times per quarter is like one bestseller selling one million copies per quarter. The difference is, it’s not easy to find such a bestseller in the modern world, and those 100,000 titles from self-publishing and self-production are easy.

In fact, finding hits become more and more hard, because hits today sell less than they used before. For example, today’s top TV show would not make it even the first 10 in 70s. The author attributes it to rise of Internet and digital technologies, which reduced price of production and distribution and extended the choice. Hence, concludes the author, presented with more choice, customers started to use it, and hence less of the same market went to hits, and more to the long tail, which fits customer demand much better.

Here I’d like if not argue then at least complement that with another reason that the author did not mention or know about. It’s true that presented with more choice people buy more diverse, however there is another very material reason for the rise of long tail. And that reason is the transition from industrial to knowledge society.

Industrial society consists mostly of industrial workers. These are the main category of consumers for industrial society. Let’s consider how industrial workers are raised, in a sense, how industrial society “produces” its main category of consumers.

Industrial worker normally graduates from a high school: a highly uniform institution imprinting millions of children every year with about the same set of basic knowledge, skills and propaganda stereotypes no matter which country it serves. Granted, stereotypes impressed on students in United States were different from the ones in Soviet Union or Western Europe, but within a single economy it was uniform. It was (and still is) essentially highly standardized mass production, like production of bolts and nuts. Yes, in Europe nuts are metric – millimeters, and in US they are in inches, but it’s still the same within a single economy. And so were people produced by the mass school.

And when you have a lot of standardized nuts, you get a huge market for standardized bolts. Because most of your consumers have similar background, prepared by the school, same basic knowledge, same dictionary, same stereotypes, in the end, same memes populating their minds. So the same TV show was good for a lot of them, same music was likable to a lot of them and the same advertising was making a lot of them buy. This was the making of hits: mass markets are created by standardized consumers, which the standardized school system provided.

With the knowledge society, more and more consumers become knowledge workers (the term introduced in 60s by Peter Drucker, sometimes referred as “the father of American corporate management”, see, for example, The Essential Drucker: In One Volume the Best of Sixty Years of Peter Drucker's Essential Writings on Management). And “more” means “majority”. See for example, The Rise of the Creative Class: And How It's Transforming Work, Leisure, Community and Everyday Life by Richard Florida. Knowledge worker is a very different beast than an industrial worker. Knowledge worker normally has at least bachelor degree, and colleges are not uniform, they are very different, and so are professions that they teach. This means that the consumers become segmented, and not just because of different backgrounds like before, but because of the economy, because there is a systematic force in place that fragments them by their background, beliefs and stereotypes.

For example, consider software engineer at software startup and mechanical engineer at Boeing or Ford. First one must innovate; “innovate or die” is pretty much a business model of most software startups. Second ones have to prevent crash, and innovate means for them a risk of crash. Once it comes under the skin, it affect how they react on advertising, politics, everything. It affects what they watch, what they buy, whom do they vote for.

Of course, hits won’t go away, we still have highly uniform basic education system, which still provides quite a bit of common background, but the more segmented will become consumers, the less will go to hits, and more will go to the long tail. And knowledge industry cannot exist without making its workers – and also main consumers – segmented.

Juts thought, it may be worth sharing.

I almost got back to the weight I had when I came to United States....

2008-08-05T10:32:00Z

Just a few days ago I tried, and I fit perfectly into the suite, in which I came into the United States from Russia in now so old 1996 year.

Since that I fattened a lot on Cola, pizzas and other dubious food substitutes from 195 pounds to almost 255 pounds last October (2007). In October vacation at La Push I decided that enough is enough, and began dieting. And now I am back to 200 pounds plus eight new holes in my belt about one inch from one another. And now I fit well into clothes I brought with myself from Russia including the three-piece suite with 19th century style chained watch piece made by St.Petersburg/Lomonosov factory. Yes, I know, it's a small stuff. But it makes me happier.

I am really thinking about reviving this blog...

2008-07-30T08:26:00Z

...but not the way it was before.

My Russian blog on Technet is flooded with readers, and I am wondering if I could bring some of that value to English-speaking audience.

First, while I still will publish an occasional technical piece or two, the topics will be much more diverse and more aligned with my interests. Which includes management (no, I don't want to manage, been there, done that, did not like it, I am just interested in theory), teams dynamics, evolutionary approaches to society and management, corporate parasites (no, that's not just a rude analogy, that's a precise term), and an occasional piece or two on other topics from science to my personal adventures.

And, just in case somebody don't understand it, no Windows Home Server articles. Sorry. When I published my previous article half a year ago, I really meant it. Ciao, Bambino, Sorry. No Windows Home Server for me.

Ciao, bambino, sorry! -- Leaving Windows Home Server

2008-02-23T03:01:00Z

I just wanted to let readers of my blog know that I just left Windows Home Server team and won't be available for questions about this product anymore.

Good resources are:

Windows Home Server team blog: http://blogs.technet.com/homeserver/
Forums: http://forums.microsoft.com/windowshomeserver/default.aspx?siteid=50

Well, everything changes and I am looking forward to challenges in the new group. In fact, once I decided to move and looked around, I felt like a child in a candy store, so many interesting things are happening across Microsoft. When I left my office for the weekend, MP3 player in my car picked the song performed by Mireille Mathieu "Ciao, bambino, sorry!" Well, not really sorry, very happy indeed, but "Ciao!" for sure. :-)