Random Thoughts and Hints on Software Development : Windows Home Server

Legends of Windows Home Server -- Checkin Garfield

EldarM — Sat, 27 Jun 2009 23:47:00 GMT

It was a logn time ago, so, I guess, it's ok to tell this story at last...

Well, everybody knows what "checkin" is, right? That's when you send your updated, fixed or new code to the depository for safekeeping. There is a lot of systems for that. OpenSource guys usually use CVS for that, there are other commercial ones like PVC, Microsoft has the one of its own called "source depot", as well as SourceSafe.

All they do is just one thing: enable you to restore a previous version if somebody broke the current one. Also, to keep a special version for this very-special-big-corporate-customer, which is not easy, but at least possible with the version control. AFAIK, we don't do that at Microsoft, but one of the companies, I worked before for, had about three major branches for critical customers like Chevron.

Of course, that's not panacea, and when 10+ developers start to checkin their changes left and right,... well... the result is normally not that encouraging. Everybody has his features working standalone, but making it work together becomes a challenge. Actually, we had such a period in our life at the early stage in Windows Home Server. We had a lot to do, and VERY little time to accomplish it. Guess what? Nightly tests become broken and did not want to run no matter what. As a little token of pride, I'd like to mention that tests for my features worked fine -- I was responsible for communications between server and the clients. Which means that when my areas are broken, half of the product tests would fail instantly, so I had to keep my parts in shape. But most of other areas weren't that lucky and overall tests package did not want to run no matter how angry the management was and how actively we as a team were fixing the problems.

Then we found a solution... I had a toy Garfield in my office. After speaking with one of the leads and the manager, we introduced the following rule: to make a checkin, you should have Garfield in your hand. So, you get Garfield, run the tests, if all pass, you do checkin, and give Garfield away to the next one in the line. A sort of a physical mutex preventing concurrent changes.

People hated Garfield. Enough to say that in a couple of weeks my boss put it on the eBay! Don't believe it? See for yourself!

Well, nobody bought it, nevertheless the problem was solved. In three days all the tests were running fine again and never broke again in several weeks. And after the Thansgiving 2006 Garfield was sent to the honorable retirement, while we continued in a more easy way but still without breaking the tests...

By the way, Garfield is still around. It's hanging in my office on the corkboard, reminding the team's veterans of the old times :-)

Let me introduce -- Sergey Solyanik, ex-dev manager of Widnows Home Server

EldarM — Sun, 11 Nov 2007 07:12:00 GMT

Sergey was the dev manager of Windows Home Server and one of three people, for whom I decided to leave BizTalk Server and join Windows Home Server team. He is a remarkable person and a friend of mine. Some time ago he left Microsoft for Google, but he is still a remarkable man and a friend of mine! And not far ago he started his own blog!

Here is where he introduces himself: http://softwareandthings.blogspot.com/2007/11/about-myself.html

And here he talks about his experience with Windows Home Server development and his thoughts about comparing Microsoft and Google: http://softwareandthings.blogspot.com/2007/11/intergroup-cooperation.html

I highly recommend his blog and himself. Of course, personally. As Sergey writes in his own blogs:

**This is my personal blog. The views expressed on these pages are mine alone and not those of my employer.**

Home Network Troubleshooter -- Windows Home Server Toolkit V1

EldarM — Mon, 08 Oct 2007 09:03:00 GMT

One of the problematic spots for Windows Home Server are the networks, where it is supposed to work. During Beta-testing we found that home networks are often not up to the task. It's either the name resolution, or UPnP does not go through, or some other problem. Well, usually it works, but problems are a bit too often to be comfortable. A few last months me and a few more members of the team were working on a troubleshooter to help detect and fix those problems.

Here it is: Windows Home Server Toolkit V1: http://www.microsoft.com/downloads/details.aspx?FamilyID=de10c0e9-2d46-4770-91fe-6b84ae06f960&DisplayLang=en

Beside troubleshooting home networks, it also can report the problems to Microsoft and do some fancy things on the server like cleaning up backup database.

Windows Home Server System Builder Edition is now available on NewEgg.com

EldarM — Fri, 05 Oct 2007 00:21:00 GMT

Windows Home Server System Builder Edition is now available on NewEgg.com, and some companies already offering it on Amazon.com site.

As a reminder, System Builder Edition is a standalone software without support. Like OEM edition it is supposed to be used for preinstallation on new computers produced or assembled for sale, but the company does not have to qualify for OEM status to get it. That's the edition used by British Tranquil PC T7-HSA manufacturer, who already ships the systems.

Cool.

[1] http://www.newegg.com/Product/Product.aspx?Item=N82E16832116395

[2] http://www.amazon.com/MICROSOFT-WINDOWS-SERVER-WIN32-ENGLISH/dp/B000VWW8QQ/

More books on Windows Home Server...

EldarM — Wed, 03 Oct 2007 21:15:00 GMT

A new book on WHS was published last month (September 13): Microsoft Windows Home Server Unleashed by Paul McFedries. Very thick, 912 pages.

What's interesting is that the author not merely pointed out which buttons to press -- that's something we tried to make evident without books -- but also described some non-trivial hacks including meddling with registry settings, home network diagnostics and installing Sharepoint Server on WHS. While I cannot officially endorse using undocumented features, it's an impressive investigative work.

And that's not all. The next book Windows Home Server For Dummies by Woody Leonhard comes out in November. This one is more compact, 382 pages. I still did not see it, but according to the Amazon's description, it's purely a user-level book without much complexities or technicalities. Which, considering the target audience for Windows Home Server, is likely to be the right level useful for real users of WHS.

Pardon for using images with ad links -- that was the only way to show the covers legitimately, without violating the copyrigts. Anyway, feels great that they are coming. It's a really pleasant sensation when people start to write books about something that you created :-) Thanks to the authors for their great work!

Terracode Book Server - Server on a bookshelf

EldarM — Wed, 26 Sep 2007 20:50:00 GMT

Site TerraCode.com tells about a project to create a Windows Home Server based system that looks like a book and quitely sits on a bookshelf while serving music and video to PS3 or XBox360.

with permission from (c) Terracode.com

The most fun part of it is that it's "Do-it-yourself" project with details on the parts and steps to perform. Looks real cool to me. I probably would not put clear plastic on top - I am not that concerned that guests will try to stick their fingers into the insides -- but otherwise I'd love to create one for myself, if I would have time. Hats off to Terracode.com for a beautiful idea!

with permission from (c) Terracode.com

5 worst problems of home-grade routers

EldarM — Mon, 30 Jul 2007 22:53:00 GMT

During our crusade for Home Server and Beta programs we faced a multitude of home network configurations. We learned a lot, and some of that we did not like. In fact, that's our beta-participants who did not like that, it's just they did not know what it is until we did investigations on the failing sytems. Now we know.

Let me share a few points on how the choice of your router can impact you, not just with Windows Home Server, but with any computers, Windows or not. Ever had a complain of a child that they need mom's computer to print a school report, because their computer "does not print"?

Routers is the most important piece of home network infrastructure, especially because most people set it right after cable or DSL modem and allow all the home machine to be connected to it. Here are in my opinion the worst things the router may do or have:

Rejection-based firewall
Some routers allow all traffic and only allow block a few specific ports of addresses. To make this worse, they may have a limited space for rules, hence allowing to block only a few ports. Fortunately they are also too dumb to be able to route incoming traffic, which alleviates most of security pains, but still leaves the home network pretty open, compared to permission-based firewalls, where all traffic is porhibited and opened for specific ports, with most popular ports preconfigured.
No name resolution or name resolution that does not include local DHCP-managed hosts
Some routers give out IP addresses over DHCP but don't bother to provide name resolution for them. As a result, home network machines cannot access each other. You can do a few tricks using workgroups with WINS or static IPs, but it's so better when a reasonable local name resolution is available.
Blocking some internal traffic
That's why UPnP may not work. Not just with WHS, but also with your Roku and other media streaming devices. Devices consuming media over network depend on UPnP discovery process. It may also interfere with file and printer sharing - a bad thing on Internet but very important inside the house.
HTTP Proxy and HTTP Proxy configuration
Looks like a cool idea, right? Especially, if you can configure parental control to restrict your child browsing with it. Parental control may be an important thing, but there are other ways to implemnet it on a router. As to the proxy, you need a real good implementation, which should be also bound with local name resolution for everything to work right. It occurs that some proxies in some routers out there are not implemented right.
Limited bandwidth
This one came as a surprise to me. Early in the game we decided that Windows Home Server will not be used as a boundary machine. Really, if you put a lot of sensitive data on it, you don't want it be directly connected to Internet. To my surprise, on a Russian forum on WHS (yes, we have one, WHS Beta was surprisingly popular there), a lot of people were asking if they could do that. When I asked why, the truth revealed itself. It happened that Windows Server 2003, which is the base on which Windows Home Server is built, is sometimes 10 times faster as a router than off-the-shelf gigabit routers. Apparently, gigabit network cards is not the only factor that defines your Internet speed.

I probably missed some problems like UI configuration usability, but it feels to me these are the big five. What would you add t this list?

RTM Sign-off of Windows Home Server is published on Channel 10

EldarM — Tue, 17 Jul 2007 21:49:00 GMT

You can view it here: http://www.on10.net/Blogs/larry/watch-windows-home-server-rtm/

As far as I know, it's the first case at Microsoft when the product sign-off is captured on video and published.

Windows Home Server v.1.0 is Released to Manufacturing!

EldarM — Mon, 16 Jul 2007 23:17:00 GMT

In fact, it was signed off by the end of last week, but the public announcement just went out today. It feels good!

Windows Home Server client join troubleshoooting hints

EldarM — Sun, 29 Apr 2007 09:55:00 GMT

This document is published here because there is no way to publish images on forum. It's written primarily to help our support people, and we are publishing it to help troubleshooting problems with the server join process for Windows Home Server for Beta participants. Please, understand that this document is not official, and provided AS is without any warranties... (see the disclaimer on the sidebar). Also, while I cannot help troubleshoot on this blog (use Connect site for that), I would appreciate comment on how t make this document more useful as well as if you notice any typos or problems in it.

Important notice: the document mentions ports 55000 and 56000, however, if you are using Beta2 build the ports are standard http ports 80 and 443, and for CTP build the ports are 88 and 444.

----------------------------------------------------------------------------

So, server join failed. Now what?

This document is not about troubleshooting client setup or discovery. It’s purely to troubleshoot server join.

Reminder: there are three phases of WHS client software installation:

Install. That’s usual software installation. Files are copied and registered.
Discovery. Client software tried to find WHS over UPNP. That’s the screen with Vista circle cursor going round and round.
Server join. Client software does webservice calls to join WHS. That’s what you see in the very end with either two green marks on the screen (good!) or anything else, usually with some red circles and crosses (bad!)

This document is about handling red circles with crosses.

Steps-by-step

Do trivial stuff: check that your server is up and running, that wires are in place, that your server is connected to network, that your PC is connected to network, and that they are connected to the SAME network. Yes, it’s trivial, but often the case.
Check that your name resolution works. Name resolution is about 9 out of every 10 cases when the problem occurs. An easy way to check name resolution is to go to the command line and type:

nslookup SERVER

where SERVER is the name of your WHS server. It should give you IP address of your server. If it does not work, go to name resolution section.
NB: It's still worth to check steps 3 and 4. Windows uses additional methods (WINS, hosts file) for name resolution which may work even if DNS service failes. If step 3 or 4 work, the name is resolved ok though alternative ways.
If name resolution works, it’s most likely that you have firewall issues. First, try to open your browser and type

http://server

You should get a nice picture of a prehistoric office worker sitting behind his desk in savannah without cubicle walls or sunscreen. If you don’t get it, you will get an IE error message. Read it. If it does not help, go to Accessing public website section.
Now type in browser

http://server:55000/enrollid/id.xml

You should get a nice XML with the number 1 in it. If you don’t, take a note of the error message and go to Accessing internal website section.
Now type in browser

https://server:56000/enroll/id.xml

You should be prompted for the admin user ID and password, and once you give the correct ones, you get a nice XML with the number 2 in it. If you don’t, check the error message and go to Accessing internal site with authentication section.
Now type in browser

http://server:55000/enrollid/id.aspx

You should get a nice XML with a bunch of stuff in it. If you don’t, then it’s either a bug or somebody played with IIS settings on the server and broke it. Anyway, take a note of what error message you will get, it may help. There is no simple troubleshooting from this point. On a positive side, we did not see anybody hitting this in months, so if you have a problem, it’s most likely that you already skipped this text and went to the specific section.

Name resolution

Name resolution is the ability of you machine to figure out IP address by the name. You sit in front of your PC and type in IE http://www.microsoft.com, and the machine converts www.microsoft.com into an IP address like 207.46.19.190. Machines use IP address to reach each other.

Name resolution is critical, because many services that WHS provides are actually standard services of Windows 2003 server, and they all need name resolution to work well to be used over home (or any other) network. In fact, technically we could have make server join work without name resolution, but we decided to break without it, because otherwise many other services of WHS will fail all around.

Unfortunately, most home networks are configured just to share Internet connection, not to use home PCs together. A good example is a home printer. You connect it to one home PC, can you print on it from another home PC? Most people cannot, because their home networks don’t support this. The situation is so bad, that even manufacturers of routers for home network don’t consider it seriously and many even expensive routers don’t do that simple job, stripping their customers from this important and useful functionality.

You see, to use a printer on other machine, your machine needs to be able to get to it. To do so, it need to resolve the name of that PC with the printer, and most home networks are not setup for that. Similarly, WHS disk shares, backup services, health monitoring only work if your PC can get to the server.

Did I get too technical? Well, there is a good news. Vista uses extended name resolution mechanism including UPNP/SSDP protocol. What it means is that Vista client normally can get to WHS even if your router is not up to the task (it still have to support UPNP though). So, upgrading to Vista helps in 95% of cases. But not everybody is ready to upgrade to Vista, and if this is your case, read on.

Why do most home networks fail with internal name resolution? Here is a typical home network:

Name resolution is done by name servers. When your home PC sees a name, it goes to DNS server and asks it, “What’s the IP address for this name?” To share Internet connections you only need to resolve Internet names, like “www.microsoft.com”. In this case all home PCs can be configured to go directly to you ISP provider DNS server on Internet. But once you try to make home PC work with each other, you hit the problem: your ISP DNS server has no clue about the names of you home PCs. And that’s only logical, they are not on Internet!

There is one more complicated version of the same problem. It's when your machine comes to your ISPs DNS and it resolves name. Suppose you name your WHS "SERVER", and for some ridiculous reason your ISP has a machine named "SERVER" on their local (not yours!) network. Then your ISPs DNS server may resolve name to IP address, but it will be a wrong address! So symptoms may be slightly different, but the root cause will be the same -- DNS resolution by DNS server, which does not know about your Home Server.

To resolve home PC names, somebody else should take a job of DNS server. Good routers do that. They present themselves to home PCs as a DNS server, and when in doubt go to your ISP provider’s DNS server for the help. Router is on your home network, so it does know all your home PCs and their IP address. In fact, normally your router is the one who gives IP addresses to your home PCs (DHCP). But it needs to share this knowledge also working as your home DNS server. Many routers out of the box don’t do that. If you can configure your router right, the picture will look very similar but with one critical difference:

Many routers out of box don’t do that, but many modern routers can be configured to do so.

And if instead of a router you have Windows XP Professional, Windows Server, or some Vista SKUs PC with two network cards, it will do that for sure.

There are tricks to make it work without using DNS, mainly around using WINS name resolution.

Trick 1: If all your machines are in the same "workgroup", and router does not block WINS, they will be able to see each other in most cases. If you have OEN headless station, it means that you need to make your client PC to belong to WORKGROUP group. In many cases that should help.

Trick 2: Also, you can potentially assign a static IP to your WHS server and put it hosts table on your PCs. That solution is far from ideal, and you still will have problems with remote access, which will try to go from server to PCs. Also, if you'll ever change the server IP address, you will have to edit hosts tables on all PCs manually again. Essentially, it's a hack, however, it will work. If you are using a headless server, it's even harder and includes a lot of risk of misconfiguring the system into unusable state.

Accessing public website

You came here because you typed in IE

http://server

and got an error message. But you already checked that name resolution works. Now, look at the IE error message.

If it says “cannot find server”, it’s either your server is down, network cable is not inserted somewhere, or firewall. Check cables, see that the server is on (ping server will ensure that), and once it’s ok, go to firewall problems section.

Another popular reason for failing at this step is the setting in Internet Explorer "Internet Options | Connections | LAN Settings | Automatically Detect Settings" or "Use a proxy server" without "Bypass proxy server for local addresses". WHS Connector uses exactly the same Windows code as Internet Explorer and it is affected by these settings. If "Automatically Detect Settings" is checked, your machine may be accidentally (by your router or your Internet sevrice provider) configured to use proxy server for all http calls. If this proxy is on the provider side, or if proxy in the router has problems, you will not be able to access your WHS server over http from local machines. Unchecking this option helps if this was the problem.

Accessing internal website

You’ve come here because name resolution works, public site is accessible fare and square, but when you typed:

http://server:55000/enrollid/id.xml

it failed.

90% chance is that this is a trouble with firewalls, go to firewall problems section.

Accessing internal site with authentication

You came here because name resolution works, public site is accessible, as well as internal site, but when you typed

https://server:56000/enroll/id.xml

it failed.

Read the error message.

If IE says that access is denied, try to recall your password. You typed it incorrectly.

If IE complains about bad certificate, it means that server join was not able to install WHS certificate. It happens in the rare cases of PC misconfiguration, which usually happens on older systems due to malware or user playing with security settings on the system. This is a very rare case.

If IE says it cannot find the site, it’s most likely again firewall issue. Go to firewall problems section.

Firewall problems

Guess, how many firewalls are between your PC and WHS? You have three times to try.

…

Three. At least. In my case four, because I have two firewalls on my home PC. Here is how it looks like in a typical home network setup:

You see, three of them are in the way of your PC communicating to WHS.

WHS firewall

This one is a most harmless of all for WHS communications. After all, it’s WHS firewall and it is configured to be friendly for WHS use. Within a reason, of course. Specifically, most WHS communications are configured to be only allowable on the same subnet. What does it mean?

Suppose you have a typical home network configuration to use private network addresses 192.168.0.*, for example:

Router 192.168.0.1
PC 192.168.0.5
WHS 192.168.0.13

This will work fine. However make it

Router 192.168.0.1
PC 192.168.1.5
WHS 192.168.2.13

and WHS firewall will start blocking attempts by the clients to connect. The rule of thumb is that the first three numbers in IP addresses of WHS, PC and router must be the same. Also, subnets like 192.168.3.* or 192.168.7.* will also work, as long as three first numbers are the same. That’s not technically 100% true, if you are ready to go into technical detail, but if you don’t, don’t even try.

Some people feel uncomfortable with the idea of only 255 computers on their home network (not that they really have 255 devices on it), and use other private address spaces like 10.*.*.* or 172.16.*.*-172.31.*.*, which is potentially ok if you configure subnet mask right, although it is usually configured by default only to let the last number change… Too technical again? Yes, just don’t use these ranges. Stick to ol’good 192.168.0.*.

Actually, that’s the only trouble with WHS firewall that you may encounter (unless you do manual configuration of WHS, in which case you should know all this stuff already anyway).

Router firewall

That’s not necessarily easy, but straightforward. You router should let the intranet traffic through, at least for WHS services. To join you need TCP connections on ports 55000 and 56000. For transport you need the port 1138. There is also a couple of ports that you need for backup and remote access. Also it needs to let UPNP packets through, otherwise server discovery won’t work.

You don’t have to open them to Internet. In fact, you better not open them to Internet. But they should be open for internal home network computers.

PC firewall

Default Windows firewall settings allow WHS client software to go out to the server both from XP and Vista client, no problems. Of course, if you set it manually, see that the same rules as for router firewall apply for outgoing connections and UPNP response. Under no circumstances, except Remote Access, WHS will try to contact your PC. All connections (again, except remote access) goes from the PC to your WHS server.

OneCare firewall is supposed to let signed binaries out, and that’s all WHS client needs, although we’ve seen occasionally OneCare not letting connection out. You may need to open these ports manually.

Most troublesome are third party firewalls. All of them could block some WHS client connections. If you have those, you need to configure them manually to allow the WHS client outgoing connections.

Conclusion

I realize, that more information is needed on the subject, and maybe I’ll be able to come to that and extend this post or write additional posts on the subject. But right now the whole team is very busy, so it was tough to write even this. Still, the plan is to extend this document with more detail and more information/cases as we will find them.

Also, please, understand that I cannot troubleshoot your system through comments on this blog. The right way to submit beta bugs is through the Connect site, where there is a way to get reasonably full troubleshooting information about your problem.

First version: 4/28/07 11:55 pm
Next update: 5/2/07 8:10pm
Next update: 7/9/07
Next update: 7/13/07