The other day my Dad called me on the phone wanting to know if data could be recovered from a freshly formatted disk. It seems that he formatted his primary hard disk and then accidentally plugged in the wrong power connector on his external backup disk. As silly as his actions seem how many of us back up our data (he does. daily). The problem was that he made two mistakes and as a result over 15 years worth of work were gone in an instant.

I'm blogging about this because his desperation got the gears turning and we were able to get all of his important data back - even after a quick format. I thought some of the more tech savvy people may get a kick out of what we did and I thought I'd take the time to plug Windows Home Server as a way to keep yourself out of this situation (hint hint .Dad :))

[NOTE: I'm sure there are better ways to do this task,  I'm also sure what I'm doing isn't completely technically correct.  I'm describing what I did over one weekend because worked for me, if you've lost real data you should consider finding a professional data recovery company]

What exactly did he do?
The problems started when he decided to reformat his computer with the goal of reinstalling from a backup. His backup software requires a fresh install of Windows to be completed before running restore, the idea is your provide a working copy of Windows and then the included restore software gets your computer up and running as it was before the format. His computer had one physical hard disk configured with two volumes C: and D: - C: contained all the operating system, D: contained all his data. His steps were as follows:

Step 1: repartition the primary disk to only have one disk (C:)
Step 2: create a new partition filling the entire disk (note, once completed there is only a C:, the D: had been removed)
Step 3: format the partition NTFS
Step 4: start installing Windows
Step 5: during install call me on the phone and describe what was happening
Step 6: on my instruction he yanked the power cord in order to stop the install

What he wanted:
Because the external hard disk containing all the backup data had been cooked (the electronics were bad) getting data off this disk wasn't possible without paying the manufacture well over $1500 for new electronics. (.we did keep this option in our back pocket in case something went wrong, the data is worth over $1500)

Since the D: volume contained all the important files it's conceivable that the data is still around, even after repartitioning and formatting the C: - here's why

When a disk is partitioned a small piece of data (512 bytes or 1 sector) is placed on sector 0 of your hard disk. This piece of data contains a little code called the Master Boot Record (MBR) as well as the partition table. Inside the partition table are the sector numbers for each of your volumes. In my Dads case there used to be two entries in the partition table, one for the C: volume and one for the D: volume. Think of the partition table as a map where each entry represents a location "to get to your D: go to sector 122945445". Then on sector 122945445 start running the code that you see - in the case of Windows you'll likely find an NTFS boot sector (this is the next step in how Windows boots).

Interestingly, the data he cares about begins at the ending of the old C: where the start of the D: volume began. There is a slight chance that because the disk is large and he yanked the power plug early that the data may still exist - even after a repartition and quick format of the C: volume.

This chance exists because NTFS (or really any filesystem) often won't erase all the data, just its bookkeeping data. Often this is done for performance reasons - erasing all the data on a disk can be really slow (figure 55megabytes a second - for a 500gigabyte disk you're looking at around 2.5 hours to do a full format).

My friend Adam says that I'm miserable at analogies and should never attempt one. but I've rehearsed this one in my head a few times so here goes J To help you understand all this pretend that your partition table is the card catalog at a public library. Contained in each card is enough information to locate the book you want within the library. In the case of your computer, the partition table can only hold four entries. but the idea is the same. Now pretend you lost your card catalog - would you panic? Probably not, you'd sure be annoyed as you'd have to spend hours rebuilding the cards. but the books would still be there and the information in those books would be preserved.

What my Dad effectively did was to destroy the card catalog in his library - each of his data files were right where they should be, he just didn't know where his books were.

The library analogy breaks down a little because the act of formatting and reinstalling Windows did in fact put some of his data in jeopardy. Think of it as destroying the first two floors of the library when the book you really care about are located on the third floor and above. The more he allowed the install to continue, the more floors were destroyed permanently.

What we went hunting for:
In order to rebuild his D: volume we needed to know exactly where his D: volume used to be located so we could modify the partition table. Getting this data isn't easy, the format and creation of the new partition table erased all mappings to the location of his data. We needed a way to sift through hundreds of gigabytes of data automatically looking for something that could identify the beginning of the D: volume. After banging our heads for a while we came up with the following (note there are probably better ways to do this, I'm just describing what worked for us J)

1. Clone the hard disk so if we goof up we'll be okay - this leaves the possibility for a professional to repair the disk and find his data
2. Write a little program to open up the PhysicalDisk and look for an NTFS boot sector if found
    a. Manually open up each sector & see if it points to an MFT
    b. With a sector number from #4 rebuild the partition table

Cloning the hard disk
If anyone cares I can write a blog about this, I wrote a little application that inefficiently copies one disk at the sector level to another disk. I'd be happy to post the source as a rough guide if someone asks.

Program to seek out an NTFS boot sector:
This is where the fun begins, when NTFS creates a volume it always sticks something identifiable at the first sector that describes the size of the volume, how many sectors are in a cluster, etc. This information, along with boot code, lets your computer boot up. We wondered if there could be something that never changed that we could locate programmatically. Turns out that NFTS puts a little signature in the beginning that Windows can later use to identify the disk as NTFS (and not FAT for example). We used a slick tool that ships with WindowsXP named 'dskprobe. Its on the XP install disk in \support\tools.

This simple program lets you read and write sectors on a disk - please do use caution when playing with dskprobe. It's dangerous if you accidentally misuse it! You can easily scramble your disk to a state where it's not usable any longer.

Using DskProbe on a working happy NTFS disk for illustration. Once you get the idea of how everything is put together we'll create a program that looks for our lost NTFS volume

1. Drives->Physical Drive
2. PhysicalDrive0 (double click)
3. "Set Active"
4. Ok
5. Sectors->Read
6. Starting Sector 0, Number of sectors 1
7. Read
8. Notice you've got all kinds of information, this is a working partition table - save it off for later use (File->SaveAs)


9. To figure out where NTFS begins view this data as a partition table (View->Partition Table)

10. Select the partition that you care about, I'm using Partition3 on my computer (so we can see a starting sector that's not for the first volume (not the C: volume), my sample working computer has three partitions

11. Note that the "Relative Sector" is 131057664 (multiply this by 512 bytes per sector to figure out where it begins in bytes). In my case this is about 62.493GB from the beginning of the drive! My first two partitions are approximately 60GB and 2GB - so this adds up to about 62GB. don't worry if the numbers aren't totally what you're expecting, there is a little fudge room in there when aligning to various boundaries). Hit the go button to see the contents of this sector

12. Flip back to the "Bytes" view (View->Bytes) and see what an NTFS Boot Sector looks like

13. See at the beginning you've got NTFS!

14. Hit View->NTFS Boot sector

15. Pay attention here J here in a bit, when you're fixing your disk, you'll use this step to know if you've got the correct boot sector
next to the "Clusters to MFT" button, click "Go"

16. This will jump your to the MFT records, click View->Bytes and look for FILE - if you found it you know that you've found the MFT. This is goodness

Using the below little snippet of code you can scan your disk looking for such signatures.

#include <windows.h>
#include <stdio.h>
#include <conio.h>

int _tmain(int argc, WCHAR* argv[])
{
    HANDLE h = CreateFile(argv[1], GENERIC_READ, FILE_SHARE_READ|FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);

    if(INVALID_HANDLE_VALUE == h)
    {
        wprintf(L"Unable to open %s (GLE:%d)\n", argv[1], GetLastError());
        goto Done;
    }

    // go into a loop reading sectors
    __int64 sectorNumber = 0;
    DWORD dwBytes = 512*2*1024*10;
    BYTE *pTemp = new BYTE[dwBytes];
    DWORD dwRead = 0;

    for(;;)
    {
        LARGE_INTEGER offset;
        offset.QuadPart = (sectorNumber * 512);
        if(INVALID_SET_FILE_POINTER == SetFilePointer(h, offset.LowPart, &offset.HighPart, FILE_BEGIN) && NO_ERROR != GetLastError()) {
            wprintf(L"Error setting file pointer to sector %I64d GLE:%d\n", sectorNumber, GetLastError());
        }

        if(0 == ReadFile(h, pTemp, dwBytes, &dwRead, NULL)) {
            wprintf(L"Error reading sector %I64d GLE:%d\n", sectorNumber, GetLastError());
            goto Done;
        }

        if(dwRead != dwRead) {
            wprintf(L"Short read.\n");   
            goto Done;
        }

        int numSectors = (dwBytes/512);
        BYTE *pTemp2 = pTemp;

        for(int i=0; i<numSectors; ++i)
        {
            if(0 == memcmp(pTemp2+3, "NTFS", 4)) {
                wprintf(L"**HIT on Sector %I64d\n\a\a", sectorNumber+i);
            }

            pTemp2 += 512;
        }
        sectorNumber += numSectors;
    }

Done:
    if(INVALID_HANDLE_VALUE != h)
        CloseHandle(h);
    return 0;
}

I know this code is a little murky to those who aren't programmers out there, since the procedure we're doing is risky I'm only including the code figuring this blog is mostly for fun.

You're almost done!
To wrap this project up you'll need to know where NTFS lays on the formatted disk. Run the program above to locate possibilities - use the sleuthing skills in dskprobe and hope that you can find where on disk your old volume used to be situated. You most likely will get multiple hits, you know when you've got a live one on the hook by browsing around seeking out the FILE bytes in the MFT.

To get your disk back, follow the steps from above where we were reading the contents but this time you're going to overwrite the index on the broken disks partition table based on the sector where NTFS was located by the program above. Write the proper sector value to the partition table, reboot (or unplug the disk), and cross your fingers!

Once you're up an running, should the disk show up, don't throw too big of a party - you may have serious data loss remaining. You should treat this disk as spoiled and should copy anything important to another hard disk.

Good luck - I know disk rebuilding is scary. After this drama with my Dad I bought him a Windows Home Server. He's still breaking computers, but with Home Server he can (.and has :)) recover from almost any problem.