Antimail : Windows XP

The new must-have gadget: Flash laptops

AdiOltean — Fri, 13 Oct 2006 23:31:49 GMT

The entire world is now slowly shifting to Flash-based storage in mobile devices. The latest trend is to get rid of harddisks in your laptop - and you will have longer battery life, faster shutdown/reboot, better shock-resistance. Not to mention that flash memory takes less space than a harddisk.

Fujitsu follows this trend with their NAND-convertible Lifebook laptops:

Fujitsu is estimating that it will receive roughly 20,000 orders for its new NAND-convertible Lifebook laptops through March. If 20 percent of those orders specify a flash drive, as Yamamoto predicts, that figure will amount to 4,000 computers with flash.

Replacing the standard hard disk drive on Fujitsu's new laptops with a flash drive would make the laptops lighter and more shock-resistant, extend battery life by 30 minutes and cut in half the time required to start up Windows. But replacing a 20GB hard drive with a 16GB worth of flash memory would cost an extra $670 (80,000 yen). That figure would be $1,335 to switch to a 32GB flash drive.

"The price is high, but we judged there is adequate demand from companies for NAND PCs, which greatly reduce the risk of losing data," Yamamoto said. "It may be a slow start, but we want to be ready when laptops will be as handy as mobile phones."

[source: CNet]

12-disk SATA storage box for less than a grand

AdiOltean — Fri, 22 Sep 2006 02:34:14 GMT

ExtremeTech has a review of the Norco DS-1220 eSATA storage box, which can accomodate up to 12 SATA disks. The price is pretty low (on Newegg.com it is $840 without disks).

I computed a small table that shows how much it will cost for a simple JBOD configuration. So, for less than $3000 you can get six terabytes in raw storage (therefore, 3 TB with RAID-10). Pretty competitive, and scales down nicely to whatever your storage needs are.

Disk capacity (GB)	Total capacity JBOD (TB)	Disk price	Total cost	$/GB
40	0.47	$32.99	$1,235.88	$2.57
60	0.70	$62.95	$1,595.40	$2.22
80	0.94	$36.95	$1,283.40	$1.34
120	1.41	$50.99	$1,451.88	$1.01
160	1.88	$48.00	$1,416.00	$0.74
200	2.34	$59.99	$1,559.88	$0.65
250	2.93	$63.00	$1,596.00	$0.53
300	3.52	$81.99	$1,823.88	$0.51
320	3.75	$86.00	$1,872.00	$0.49
400	4.69	$123.00	$2,316.00	$0.48
500	5.86	$177.00	$2,964.00	$0.49
750	8.79	$338.50	$4,902.00	$0.54

- Disk prices from http://www.pricewatch.com/hard_drives
- Enclosure price: $839.99 from newegg.com

From this table, it looks like we are approaching nicely (and even passing) the theoretical 50 cents/TB limit, starting with around 160/200 GB disks. So, when fully loaded, this array scales down nicely as expected.

The formulas above should be obvious:

- Total cost = $839.99 + 12 * disk_price

- Total capacity (JBOD) = Disk capacity * 12.

New hard drive price war coming

AdiOltean — Thu, 10 Aug 2006 22:35:00 GMT

Seagate, Hitachi and Samsung will compete for more price cuts for harddisks in the near future.

The toughest competition seems to be on the low-end, 80 GB harddisk models. So, what do you think the new non-rebate price would be for, say, 160 GB drives?

[source: USA Today]

Search Engines and Privacy

AdiOltean — Wed, 09 Aug 2006 21:49:00 GMT

The recent AOL scandal had one and only one learning lesson for any software company. Think 10x, 100x, 1000x about customer privacy. There is a large amount of discussions, opinions, passion on the blogosphere, but I don't think that anyone disagrees with the point above.

However, there is an alternate learning lesson that applies to you, or anyone browsing the internet. It is easy to forget how insecure the internet actually is. There is a simple rule:
1) Any URL that starts with http://xxxx is insecure. Never type private data like credit card numbers, etc in this URL. The data that you type is accessible to the target server but also to any intermediaries on the way (your ISP for example), etc.
2) Any URL that starts with https://xxxx is somewhat secure. (Note the S at the end of the URL moniker I wish we had a different URL moniker for secure HTTP because the two are so similar). By "somewhat secure" I mean that the data that you provide is obviously accessible to the target server, but innaccessible to intermediaries.

You might be worried by the word "somewhat". Well, HTTPS is still not 100% secure as there can be loopholes as well, for example, if the HTTPS link uses a bad/invalid/untrusted/expired certificate. To make sure, just look at the lock icon on the bottom of your browser and make sure that the certificate is valid and trusted by you. IE7 will also display a big, separate warning page if the target certificate is invalid, so you are better protected in this case.

Also, even if you have a valid certificate, the target service itself might do something stupid and release your private data. That's what hapened with AOL. So, to stay on the safe side, don't even assume that the Internet is a safe place to play.

Some might think that Internet anonymizers can be a good workaround. However, not even internet anonymizers can be trusted. Who knows if some internet anonimyzer (used by you) doesn't track all your activity to sell it for profit?

The conclusion? The internet is not a safe place. Nothing new, after all. A public road is not a safe place either. Just make sure you look around when you walk or drive. At least, that's what they are teaching us in the Defense Driving school :-)

Why Flash memory is good for your computer

AdiOltean — Wed, 05 Jul 2006 12:17:00 GMT

Simply put, flash memory will enable a revolution in improving computer performance in daily utilization scenarios. Your computer will boot up faster. It will launch applications significantly faster. (Hey, it will shutdown faster as well.)

The problem

To see why we will have this dramatic performance improvement, let's remember how harddisks work: whenever you have a mixture of random I/O requests, the actuator moves across different tracks to read/write the corresponding data. Switching tracks is a slow operation. For an average SATA drive, this is around 9 milliseconds. This might not seem much, but a few milliseconds per seek means that you can have at most a few hundred random I/Os per second. And this feels like light-years compared with the performance of other components in the system like RAM access speeds or even CPU frequency. So, just to give you an example, a random I/O with 4 KB requests and average of 4 ms seek time per request would mean around 1000/4 * 4 KB = 1 MB per second disk transfer rate. Pretty small, don't you think? Especially when you compare it with sequential I/O, where you can get a much faster transfer rate (say, 60-70 MB/s on a regular harddisk, depending the rotational speed, data density, etc).

One trick to alleviate this performance issue is to minimize seek time by reordering writes and/or serving reads from cached memory. Memory caches can greatly help in this regard, but here is a little problem: applications, the OS, and other components do not expect writes to be reordered. When you a write reordering is detected at the application level, then a data corruption can appear, especially when you reboot the machine in the middle of performingg a set of reordered writes.

For example the applicaiton is performing Write(block1) followed by Write(block2) in one thread, and Read(block1) followed by Read(block2) on a different thread. In the sequence above, the application expects block1 to be written always before writing block2. Having this guarantee simplifies for example applicaiton recovery semantics, assuming that the computer can crash between writing block1 and block2. But if we perform write reordering, and only write to the disk block2, then our application recovery logic cannot be done in any way. And so we get to corruption.

Still, storage controllers perform today all sorts of tricks like maintaining a write-through cache in volatile RAM, coupled with limited reordering. More advanced controllers, or SAN equipment use persistent caches (battery-backed volatile RAM) to perform write reordering, complementing advanced storage features like RAID configurations, etc.

The solution - why flash is good

By now it should be clear how flash can be used in this picture: you can use inexpensive flash as a persistent write-through cache for reads/writes. Also, the fact that this flash is persistent enables reordering I/O requests at an unprecedented level, therefore greatly reducing our nasty seek time bottleneck:

The new 2Gb OneNAND chip doubles the capacity of a OneNAND memory device (from 1Gb) and increases the chip's ‘write' speed from 9.3MByte to 17MByte per second.

”We're seeing a rapidly widening market for our OneNAND memory because of its outstanding performance and capacity that has become even more noteworthy with the application of 60 nm technology,” said Don Barnetson, Director, Flash Marketing, Samsung Semiconductor. [...]

Because of its exceptionally high performance, OneNAND can serve as a catalyst in the development of new product markets. A much-discussed example of this application-creating role is in how OneNAND memory is now being specified as the buffer memory inside a hybrid hard disk.

Samsung successfully demonstrated a commercial Hybrid-HDD prototype for the first time at the MS Developer Conference (WinHEC: Windows Hardware Engineering conference) in Seattle last month.

[source: http://www.physorg.com/news70899414.html]

Flash-based I/O optimizations - already present in Vista

One more thing worth mentioning: Vista already benefits from Flash-based optimization. The feature is called EMD (External Memory Device), and can boost the performance of your computer by simply adding a USB thumbdrive and designate it as an EMD device. Under the cover, it works in a similar way with the technique described above.

Passwords: Bad "best practices"

AdiOltean — Tue, 25 Apr 2006 09:17:00 GMT

So, you think that your password is secure? Let's see: does it contain a mixture of uppercase/lowercase letters, punctuation marks and digits? Yes? Well, even in this case, your password might be still completely insecure. Read ahead why...

To understand the problem, we need first a little rehash of the basic password cracking techniques. The simplest algorithm would be to simply (1) enumerate all English words and names from a given dictionary and (2) check to see if this word matches as your password.

You might say - but in this context we are talking about other characters like punctuation marks and digits that are part of the password. What would an attacker do in this case? Simple - use a little psychology.

The problem is that most people feel that adding digits and other characters is just a burden. When the "password will expire today" dialog comes, they will be in a hurry to get a new password, maybe an easy-to-remember word, and then alter it in a few ways:
1) First, the password needs to have a capital letter. Most people will naturally choose the first letter from our English word to be capitalized. So, a word like "flowers" becomes "Flowers".
2) Second, the password needs to contain some digits. The password would look nicer (and easier to remember) when these digits are appended to the word. Even more, people are usually unimaginative here, and just append one digit, or in more complex cases, digit sequences like "123" or "01" or eventually their birthdate.
3) Third, we need some non-alphanumeric characters. Well, let's see. If we replace an "s" with "$", "a" with "@" or "o" with zero, then we get what we want, right? It is hard to resist the tentation to replace "s" with "$" at least (and not an "a" with "$"), and therefore getting a false sense of security. In some cases also using delimiter characters like "!" or "#" to separate the word from the digit sequence.

So, with the example above, the altered forms of the word "flowers" might be: "Fl0wer$" or "Flower$01" or "Fl0wers#123" and so on and so forth.

The problem with these alteration rules is that they are so predictable. All the attacker has to do is to take the same list of English words, and apply the rules above. He will probably get a longer list by, say a factor of 10-100 which is not that much.

In conclusion, it's not that hard to enter into the minds of regular people, and neither in the minds of attackers. So, if you used any of these rules above, then stop using them. Instead, here are some rules to create strong passwords.

P.S. As for me? I just uuidgen.exe to create a random sequence of digits.

How to run Defrag on a scheduled basis

AdiOltean — Fri, 07 Apr 2006 10:18:00 GMT

On Windows XP, it's simple. Just run this command and you are done:

schtasks /create /TN defrag /RU "NT AUTHORITY\SYSTEM" /SC weekly /TR "defrag %systemdrive%"

Internet Explorer 7 Beta2 available for download

AdiOltean — Tue, 31 Jan 2006 21:45:00 GMT

It's finally here: http://www.microsoft.com/windows/ie/ie7/default.mspx

Using NTBackup to achieve P2V

AdiOltean — Fri, 20 Jan 2006 12:48:00 GMT

I always thought about writing a post about physical-to-virtual migration strategies in combination with VSS, but I never got to it. So it's not surprising that others started to uncover this trail :-)

One particular technique in this area is described in Lindsay Donaghe's blog - she uses NTBackup to do pretty much the following:
1) Backup the physical machine (Windows XP, or something similar). It is important that the backup is done using the ASR mode (Automated System Recovery) because the virtual machine hardware will be different.
2) Create a virtual machine. She uses VMWare but you can use Virtual PC as well.
3) Perform an ASR restore on the newly-created machine. Done!

I would add one more thing. The technique is also useful as a testing procedure to verify that your ASR backup can be actually used to perform a restore. Otherwise, an untested backup is a useless backup, as Murphy would say...

Perpendicular storage arrives on laptops

AdiOltean — Mon, 16 Jan 2006 18:45:00 GMT

http://news.yahoo.com/s/ap/20060116/ap_on_hi_te/seagate_drives

Seagate's new drive, the Momentus 5400.3, was being shipped as of Monday, the Scotts Valley, Calif.-based company said. The shift to perpendicular recording allows it to bump up the maximum capacity of its notebook drive to 160 gigabytes from 120 gigabytes.

The 2.5-inch drive costs $325, compared to about $240 for the 120 gig model. Seagate plans to extend the new recording technology to other notebook drives, as well its 1-inch drives used in handheld gadgets and 3.5-inch drives for desktop PCs.

Windows Backup - highlighted as one of the main Vista features!

AdiOltean — Thu, 05 Jan 2006 23:20:00 GMT

Whoohoo! Windows backup is there. Proeminently:

http://www.microsoft.com/windowsvista/features/default.aspx#wvC2

They also mention the Shadow Copy (aka Previous Versions). This is a standard feature now in Windows Server 2003, which is invaluable for many many customers. Now we are bring it to the masses directly into Vista.

Did you delete a file by mistake? Or did you overwrite it and lost your old work? Just right-click on it (or on the parent directory) and you will see all its previous versions in the last month.

How to stay safe with the new Windows vulnerability

AdiOltean — Tue, 03 Jan 2006 18:35:00 GMT

As previously published here, this is a quick-and-dirty method to get yourself in a relatively safe position:

1) Click Start, click Run, type "regsvr32 -u %windir%\system32\shimgvw.dll" (without the quotation marks), and then click OK.
2) Restart the machine.

[Update] Note that this workaround still does not remove the vulnerability and will not block "modified" attacks.

Impact of Workaround: The Windows Picture and Fax Viewer will no longer be started when users click on a link to an image type that is associated with the Windows Picture and Fax Viewer.

To undo this change, re-register Shimgvw.dll by following the above steps. Replace the text in Step 1 with "regsvr32 %windir%\system32\shimgvw.dll" (without the quotation marks).

MSRC is fully aware about this situation and it is working on a patch to be released in the next days. In the meantime, make sure that your Windows OS is configured for receiving automated updates, and that you have an antivirus program installed.

[Second update - Jan 5] The fix is now available here: http://www.microsoft.com/athome/security/update/bulletins/200601_WMF.mspx

How to do atomic writes in a file

AdiOltean — Thu, 29 Dec 2005 01:51:00 GMT

Let's assume that you want is to write some simple code that writes to a text file. A few assumptions:
1) You need avoid corruptions of any kind.
2) Either all of your writes have to make it to the disk, or none of them.
3) The file is updated serially - no concurrent updates from separate processes are allowed. So only one process writes to the file at a time.
4) No, you cannot use cool new techologies like TxF.

Remember, all you want is just to write to a text file - no fancy code allowed.

What are the possible problems?

Many people mistakenly think that writing to a file is an atomic operation. In other words, this sequence of function calls is not going to cause garbage in your file. Wrong. Can you guess why? (don't peek ahead for the response).

echo This is a string >> TestFile.txt

The problem is that the actual write operation is not atomic. A potential problem is when the machine reboots during actual write. Let's assume that your file write is ultimately causing two disk sectors to be overwritten with data. Let's even assume that each of these sectors is part of a different NTFS clusters, and these two clusters are part of the same TestfFile.txt file. The end of the first sector contains the string "This is" and the beginnning of the second sector "a string". What if one of the corresponding hardware write commands to write these sectors is lost, for example due to a machine reboot? You ended up with only one of these sectors overwritten, but not the other. Corruption!

Now, when the machine reboots, there will be no recovery at the file contents level. This is by design with NTFS, FAT, and in fact with most file systems, irrespective to the operating systems. The vast majority of file systems do not support atomicity in data updates. (That said, note that NTFS does have recovery at the metadata level - in other words, updates concerning file system metadata are always atomic. The NTFS metadata will not become corrupted during a sudden reboot)

The black magic of caching

So in conclusion you might end up with the first sector written, but not with the second sector. Even if you are aware of this problem you might still mistakenly think that the first sector is always written before the second one. In other words, assuming that "this is" is always written before "a string" in the code below:

using System;
using System.IO;

class Test
{
    public static void Main()
    {
        using (StreamWriter sw = new StreamWriter("TestFile.txt"))
        {
            sw.Write("This is");
            sw.Write("a string");
        }
    }
}

This assumption is again wrong. You again can have a rare situation where the machine crashes during your update, and "a string" can end up in the file, but "This is" not saved. Why?

One potential explanation is related with the caching activity. Caching happens at various layers in the storage stack. The .NET Framework performs its own caching in the Write method above. This can interfere with your actual intended order of writes.

So let's ignore .NET and let's present a second example, this time using pure Win32 APIs:

WCHAR wszString1[] = "This is";
WCHAR wszString2[] = "a string";

   fSuccess = WriteFile(hTempFile, wszString1, sizeof(WCHAR) * wcslen(wszString1), &dwBytesWritten, NULL);
   if (!fSuccess)
      ...
   fSuccess = WriteFile(hTempFile, wszString2, sizeof(WCHAR) * wcslen(wszString2), &dwBytesWritten, NULL);

Again, here you can also have caching at the operating system level, in the Cache Manager, where the file contents can be split across several in-memory data blocks. These blocks are not guaranteed to be written in their natural order. For example, the lazy writer thread (a special thread used by Cache Manager that flushes unused pages to disk) can cause an out-of-order flush. There are other considerations that can cause an out-of-order data flush, but in general you need to be aware that any cache layers in your I/O can cause writes to be randomly reordered.

The same reasoning applies to our third example:

echo This is >> TestFile.txt
echo a string >> TestFile.txt

Again, you cannot be sure that the file will not end up corrupted - you can have rare scenarios where the resultant file with contain either the word "This" or the word "string" but not both!

The solution? One idea is to use special write modes like FILE_FLAG_WRITE_THROUGH or FILE_FLAG_NO_BUFFERING, although in these cases you lose the obvious benefit of caching. You have to pass these flags to CreateFile(). Another idea is to manually flush the file contents through the FlushFileBuffers API.

So, how to do atomic writes, then?

From the example above, it looks like it is entirely possible that our writes migth complete partially, even if this case is extremely rare. How we can make sure that these writes are remaining atomic? In other words, my write to this file should either result in the entire write being present in the file, or no write should be present at all. Seems like an impossible problem, but that's not the case.

The solution? Let's remember that metadata changes are atomic. Rename is such a case. So, we can just perform the write to a temporary file, and after we know that the writes are on the disk (completed and flushed) then we can interchange the old file with the new file. Something like the sequence below (I used generic shell commands like copy/ren/del below but in reality you need to call the equivalent Win32 APIs):

Write process (on Foo.txt):
- Step W1: Acquire "write lock" on the existing file. (this is usually part of your app semantics, so you might not need any Win32 APIs here)
- Step W2: Copy the old file in a new temporary file. (copy Foo.txt Foo.Tmp.txt)
- Step W3: Apply the writes to the new file (Foo.Tmp.txt).
- Step W4: Flush all the writes (for example those being remaining in the cache manager).
- Step W5: Rename the old file in an Alternate form (ren Foo.txt Foo.Alt.txt)
- Step W6: Rename the new file into the old file (ren Foo.Tmp.txt Foo.txt)
- Step W7: Delete the old Alternate file (del Foo.Alt.txt)
- Step W8: Release "write lock" on the existing file.

This solution has now another drawback - what if the machine reboots, or your application crashes? You end up either with an additional Tmp or Alt file, or with a missing Foo.txt but with one or two temporary files like Foo.Alt.txt or Foo.Tmp.txt). So you need some sort of recovery process that would transparently "revert" the state of this file to the correct point in time. Here is a potential recovery process:

Recovery from a crash during write (on Foo.txt):
- Step R1: If Foo.txt is missing but we have both Foo.Alt.txt and Foo.Tmp.txt present, then we crashed between Step W5 and Step W6. Retry from Step W6.
- Step R2: If Foo.txt is present but Foo.Tmp.txt is also present, then we crashed before Step W5. Delete the Foo.Tmp.txt file.
- Step R3: If Foo.txt is present but Foo.Alt.txt is also present, then we crashed between Step W6 and Step W7. Delete the Foo.Alt.txt file.

More and more problems...

The sequence of operations above looks good, but we are not done yet. Why? Sometimes shell operations like Delete, Rename can fail for various reasons.

For example, it might just happen that an antivirus or content indexing application randomly scans the whole file system once in a while. So, potentially, the file Foo.Tmp.txt will be opened for a short period which will cause either the step W7 or R1..R3 to fail due to the failed delete. And, not only that, but also Rename can fail if the old file already exists, and someone has an open handle on it. So even the steps W2 or W5 can fail too...

The fix would be to always use unique temporary file names. In addition, during the recovery process, we will want to clean up all the "garbage" from previous temporary file leftovers. So, instead of files like Foo.Tmp.txt or Foo.Alt.txt, we should use Foo.TmpNNNN.txt and Foo.AltNNNN.txt, together with a smart algorithm to clean up the remaining "garbage" during recovery. Here is the overall algorithm:

Write process (on Foo.txt):
- Step W1: Acquire "write lock" on the existing file.
- Step W2: Copy the old file in a new unique temporary file. (copy Foo.txt Foo.TmpNNNN.txt)
- Step W3: Apply the writes to the new file (Foo.TmpNNNN.txt).
- Step W4: Flush all the writes (for example those being remaining in the cache manager).
- Step W5: Rename the old file in a new unique Alternate form (ren Foo.txt Foo.AltNNNN.txt)
- Step W6: Rename the new file into the old file (ren Foo.TmpNNNN.txt Foo.txt)
- Step W7: Delete the old Alternate file (del Foo.AltNNNN.txt). If this fails, simply ignore. The file will be deleted later during the next recovery.
- Step W8: Release "write lock" on the existing file.

Recovery from a crash during write (on Foo.txt):
- Step R1: If Foo.txt is missing but we have both Foo.AltNNNN.txt and Foo.TmpNNNN.txt present, then we crashed between Step W5 and Step W6. Retry from Step W6.
- Step R2: If Foo.txt is present but Foo.TmpNNNN.txt is also present, then we crashed before Step W5. Try to delete all Foo.TmpNNNN.txt files and ignore failures.
- Step R3: If Foo.txt is present but Foo.AltNNNN.txt is also present, then we crashed between Step W6 and Step W7. Try to delete all Foo.AltNNNN.txt files and ignore failures.

That's it!

Windows Local Live is out in the wild!

AdiOltean — Thu, 08 Dec 2005 01:43:00 GMT

Here it is: http://local.live.com

Check out some cool bird's eye images:
- Space Needle
- Microsoft campus.

[Update - Dec 8, 11 PM] It turns out that the service is not exposed to the public yet. Funny enough, it works fine from my dev box, but not from my home computer. Maybe they block all the non-Microsoft IP adresses?

Infoworld says:

Beta 2 of the search service will go live at 9:01 a.m. Pacific Standard Time Thursday at http://local.live.com, according to Microsoft. The beta includes the rebranding of the service and new zoom and "bird's eye" features to the service's satellite imagery, as reported by the IDG News Service Tuesday

[Update - Dec 9, 12 AM] Live.com is now fully launched! (nine hours earlier). Or maybe it was launched all the time and I had a minor glitch on my network and I couldn't see it previously...

Holographic storage for everyone

AdiOltean — Sat, 26 Nov 2005 10:09:00 GMT

Maxell hints that it will ship 300 GB optical media next year, in September 2006 to be more exact. The bandwidth is decent too: 160 Mbps. The technology behind all this? Holographic storage:

Holographic recording technology uses intersecting signal and reference laser beams to store data in a number of 3D holographic images.

According to Maxell, one 13cm optical disc can store up to 150 million pages — more than 63 times the capacity of DVD. [...]

The technology also has an impressive lineage. Hitachi-owned Maxell is working with InPhase Technologies, a subsidiary of Lucent, which has led development of holographic media.