I've been asked by a few folks to describe the Power Pack 1 algorithms for "auto-migration", duplication, and balancing. In short the algorithms are the same with a few minor adjustments made based on customer feedback from our first release and the various Power Pack betas.

For reference there is a document located here with more info on the pre-Power Pack 1

http://download.microsoft.com/download/2/F/C/2FC09C20-587F-4F16-AA33-C6C4C75FB3DD/Windows_Home_Server_Drive_Extender.pdf

The decision of which volume a file will be located is based on one of three questions

1. Did the file get created for the first time (auto-migration)
2. Did the user ask for the file to be duplicated (duplication)
3. Does enough free space one volume allow another volume to offload some files (balancing)

Often caused by:

• one of the servers volumes filling up
• bunch of files get deleted
• new hard disk added or removed

Auto-migration:

When a file is first created it's said to be "auto-migrated"  this basically means a volume with sufficient free space was chosen to hold the file. The decision of which volume is designed to keep related files on the same volume in case of disk failure in case the folder was not chosen by the user for duplication. If you imagine a situation where your music isn't duplicated and you have a physical disk failure. You'd probably prefer to have one or two CD's go missing to having one or two songs from several albums go missing.

This is the algorithm Drive Extender uses to decide which volume to place your files

i.Use the volume with the least amount of free space but greater than 10GB

ii.Use the volume with the most amount of free space so long as it has more space than the Primary Volume (D volume)

iii.Use the Primary Volume (D volume)

If you play this out in your head with imaginary file copies I think you'll see how it ends up clustering files together. You'll end up filling up the volume with the least free space first and then moving to the most used filling it up next.

If we would have used a simple "most free" algorithm we'd get into a situation where once all volumes had the same amount of free space we'd interleave every other song/picture across several volumes.

Duplication:

Auto-migration doesn't duplicate your files or balance your files, it just chooses where the file first goes. Duplication occurs (in PP1) every hour with the goal of making sure your chosen files have multiple copies. The algorithm is to inspect every file looking for change since last duplication.  If the file has changed or if the file has not yet been duplicated we create a duplicate copy using the below algorithm.

Duplication preferences for destination volumes

i.Most empty non-primary volume

ii.Primary volume

The algorithm here is a little different from auto-migration because it will end up interleaving between volumes. We did this because the problem of locality isn't as important since we're in the process of duplicating.

Balancing:

Balancing solve the problem of how to handle disk space imbalances. Just like duplication, balancing occurs every hour in PP1. The need to balance occurs typically with addition of a new volume or when the user deletes a bunch of files. The goal of balancing is to move files off any volume that contains less than 10GB of free space. If this condition happens we say the volume has reached a "danger" level because it's possible to have a situation where files cannot be extended (imagine your outlook .pst file getting bigger and bigger over time).

The PP1 balancing algorithm starts only when a volume contains less than 10GB of free space. When this occurs the goal is to start moving files to volumes with more space until 20GB is reached. Think of this as mowing your yard. You wait until your significant other tells you the yard is out of hand and you cut back the yard enough so it looks good and so that you don't have to do it again for a while - it's the same idea, in balancing we start moving files when the volume has less than 10GB free and we stop at 20GB so we don't have to come back for a while.

The algorithm for finding free space is the same as in migration with the only difference being we won't push another volume into an unhappy state in order to achieve self happiness. A quick example would be if during balancing a volume with 5 GB of free space we wont push another volume of its comfort range by making it go under 10GB.

Defining these numbers is tough as there are tradeoffs in all directions. If you've got huge hard disks with lots and lots of free space and gigantic files you may prefer bigger numbers (maybe 30GB and 60GB) but if you've got just two drives you may prefer smaller numbers. We used our extensive beta program to measure 10GB and 20GB as good numbers. However if you understand the description above you can reconfigure the 10GB and 20GB lines with the registry keys

HKLM\Software\Microsoft\DriveExtender (both are DWORDS)

"SecondaryFreeSpaceDangerLevel" (defaults to 10)

"SecondaryFreeSpaceWarningLevel" (defaults to 20)

both values are measured in GB

You’ve built your home server but can’t find identical replacement hardware.

This is a similar problem to what Daniel (my brother in law) suffered.   After purchasing a copy of Home Server Daniel installed the software on to an old desktop computer.  He installed on to a new 500GB disk and later added an older 150GB IDE disk. Long story short the computer was declared unusable when smoke poured out the power supply.

Although much of the hardware was unusable the hard disks seemed fine – he was able to plug both the 500GB and the 150GB disks into a second computer and was able to see his data.

I’ll leave the forensics of how to extract your data to a future post, in this post I’d like to show you how to recover your server if the drives are intact but the motherboard and various components are broken.

Goals:

·         Preserve all ‘data’

o   Shared Folders

o   Backup database

Risks:

·         You will lose all installed software, such as addons

·         You will lose all user settings (remote access on/off), including user accounts

Steps (I’ll have some pictures below):

1.       Remove the disks from your broken server

2.       Open the case to your new server

3.       Install the old disks into your new server – take care to place the disks in the same order as they were on your original system (make master/slave settings and cable order the same for IDE or port numbers for SATA controllers)

4.       Boot and cross your fingers – if this computer reboots and launches Home Server you’re in good shape.  Just go step 9

5.       Should booting fail your new server differs enough from the hardware in your old server that you need to reinstall Home Server.  Do this by placing the Home Server DVD into your DVD player and rebooting.

6.       Follow the normal steps involved with Home Server Install.  *IT’S VERY IMPORTANT* that you choose the “Server Reinstallation” option as seen in the below figure.

7.       Progress along in the Home Server install until it’s complete – then reboot

8.       Complete the Out Of Box Experience (OOBE)

9.       Install needed drivers – follow the guidance that the hardware manufacture gave for your servers hardware.  Typically you’ll just need network and maybe storage if you’ve got a fancy disk controller

The starting place:
I’m first going to create a Home Server install on a server with two SATA disks. One is 300GB, the other 150GB.  On this server I’ve performed one backup of my client computer and have created a Shared Folder named ‘Test Share’.  This share has a few files for demonstration purposes.

Figure 1: Server Storage before the first server broke

Figure 2: Test share (created before the server broke) 

Figure 3: Putting the old disks into the new server - be careful if you're not putting in screws at this point! 

Figure 4: Beginning the Home Server install 

Figure 5: the all important step - "Server Reinstallation" - you must choose this else you will lose your data!

Figure 6: Install has started 

Figure 7: Install Complete

Figure 8: Install Drivers

Figure 9: restored file shares 

Figure 10: permissions on the folder - NOTE that we have no users as they were deleted during the recovery,  your shares will default to no access (so you'll have to reconfigure their settings)

Figure 11: restored backup database

Figure 12: restored server recovery tab

If you’ve purchased and OEM Windows Home Server, you’re in luck there are two likely scenerios and both result in you keeping your data….  I greatly wish Daniel (my brother in law) had an OEM device… so much so that Daniel is getting one for the holidays :)

Should the server not boot there is probably a problem on the primary C: volume.  Examples could include the OEM slightly changing the servers hardware from the time you purchased your old one, the old volume has had a software problem that has prevented booting, or perhaps you’ve installed 3’ed party software that has made the server unhappy.  In the case where simply replacing the hard disks doesn’t work you’re still okay, recovering from a major problem on an OEM device is simple – Home Server has a technology designed to image (refresh to factory conditions) the primary hard disk from the network.  The idea is simple – at time of manufacture the OEM takes a “sysprep”ed image of the C: volume.  This image contains a copy of Windows Home Server that has been put into a special mode,  the mode will “personalize” itself on first boot.  All tech jargon aside the idea is to tweak it’s settings to be customized for you while retaining the fingerprints of your OEM.

The procedure is simple

1.       Press a button on the front of the device

2.       Drop a DVD into your *client* computer

3.       Follow the OEM’s instructions

4.       Wait for the C: volume to be brought to its factory new state

Once reimaged your Home Server will ask for new passwords and will require you to recreate users – but it will preserve your backup database and all your files.

By far – this is the simplest route to recovering from a major failure (you dropped the server, installed some really broken software, or just forgot your administrator password)

One common problems we've encountered while making home gateways (wireless access points) is how do you discover the silly thing?  Most solutions end up shipping a preconfigured IP address or hostname on the device, some people query the DHCP server and I've even heard of people using netmon!  :P

Each of these solutions has its own pro's and con's -- with a gateway its okay to have a user manual where 192.168.1.1 is the preconfigured IP address.  The user just plugs in and uses their browser to navigate to http://192.168.1.1 for configuration.  Other solutions will give a preconfigured hostname so the end user will configure by going to http://hostname.    I think querying the DHCP server is totally acceptable, but my mom didnt send me to college to get that kind of answer ;)

Other solutions are to have a value burnt into the ROM/flash -- this either requires each device to be configured before it leaves the factory or for the device name to be built off something like a MAC address.  Nothing bad here, but it makes for slightly confusing documentation.  “go to http://device_XYZ.. XYZ can be found on a sticker on the back of the device”  its not bad, but its clunky and requires printing a little sticker.

My solution to this problem was to make a little discovery utility that uses UPNP's SSDP (service descovery protocol).  the idea here is when the device boots up it sends out a little UDP packet that announces its presence to the world.  Because both WindowsCE and WindowsXP come with UPNP implementations this wasnt that tricky to setup.  I've included a little sample in WindowsCE 5.0 in the directory %_WINCEROOT%\public\servers\oak\samples\upnp\discoverservice. This utility isnt exactally rocket science, in fact its kind of a joke.  All it does is broadcast a little GUID that corresponds to the service that it supports with a URL for a configration webpage (known as a presentation page).   You can find a little C# util that I wrote to listen for this here http://chrisgray.members.winisp.net/files/discoverapp.zip  -- please note that “My Network Places” will also listen for it if you've got UPNP installed on XP.

The idea of this util is that it would be included in a CD or something that would have an autorun.inf to launch it.  The user would plug in their totally nameless device, it would DHCP, and then it would begin broadcasting UPNP messages announcing its presence.  The XP utility would then pick up the announcment and give the option to configure the device.