183 : BAS_TODO

Is solid state up to an ESE challenge?

BrettSh — Mon, 05 Mar 2007 19:07:00 GMT

~Eric and I were at Fry's the other day, when we stopped by flash drives ... we were debating how long until we see ESE (well Exchange specifically, though I'm more concered about laptops) on such drives. With the advent of larger and large capacity SSDs, and even commercial level drives like these, becoming ever more imminent, such a possibility looms nearer and nearer. And with a few ESE consumers on the client O.S. I expect we will be seeing our databases on SSDs on laptops within the year (maybe two).

We were discussing the various rumors about how fast they are at read vs. write, how low latency are the reads/writes, +how reliable+ (if you remember there was always this issue with earlier drives, SSD can only be written to so many times), how good at random IO, and other properties that would make them useful as Exchange storage, or at least whether they will be causing us problems on laptops. So curious was I, that of my own accord (and with my own money I'll note) I went ahead and bought about $100 worth of flash drives for running through our little JET engine, see if they survive ...

Here is a list of the drives I acquired (except the last one, being a control of sorts that I already owned):

Manufacturer	Model	Cap (GB)	Cost[1]	Warranty	MBs/$
SanDisk	Cruzer Micro Drive	1.00	$20	2 years	50.0
Kingston	DataTraveler	1.00	$20	5 years	50.0
Sony	MicroVault	0.25[2]	$18	1 year	13.8
PQI	U320 / Traveling Disk	0.50	$15	unknown[3]	33.3
Patriot	XPorter	4.00	$24[4]	5 years	166.7
AComdata	USB Hard Drive	320.00	$300	?3-5 years?	1066.7

[1] For all units, the cost was rounded up by 1 cent to non-retardo-consumer values. Tax was not added, but it applies here in Washington, 8-something% or so.
[2] I really wanted a larger Sony, but the 2 GB was $70 (too much), and I couldn't find a 512 MB or 1 GB unit anywhere on the shelf. Sigh. :P
[3] Documentation claimed it was on web site (www.pqimemory.com), but couldn't find it in a generous 40 seconds of searching.
[4] This one was an incredible value, it was actually $70 in the store, but until 3/5/2007(so only today, yeah sorry for the late notice) you get a $45 rebate if you buy it at Fry's.

All such drives claim USB 2.0 compliance on the packaging, except perhaps the USB Hard Drive, not sure about that one.

I don't want to rate these drives based on arbitrary tests[5], such as whether it comes with a wrist strap, retractable USB plug, etc ... we want our analysis to be on serious benchmarks that tell us if these have the capability to back a real ESE database load / support Exchange, and so we start our tests ...

Test 0; Part A: Getting the packaging off...

I've definately no love of the ridiculously tough plastic modlded packaging that these things come in ... and so I decided as the preliminary test in our evaluation we would test the difficulty of removing the packaging and installing the drive.

(lower is better)
4.0    Drive A (SanDisk Cruzer) had unusually tough to cut plastic, and an asymetrical shape as well (making getting the scissors to purchase hard) as very large borders making the entire thing very tough to cut open.
2.0    Drive B (Kingston DataTraveler) had somewhat better packaging, thin-ish borders, and slightly easier to cut, but 4 sided seals
2.8    Drive C (Sony MicroVault) Pretty easy to cut, reasonable sized shoulder, and only sealed on 3 sides (e.g. where the plastic is like heat fused).
3.4    Drive D (PQI U230) hardish to cut, but only sealed on 3 sides, last side is bent over, so easy to open after 3 sides.
4.4    Drive E (Patriot XPorter) had very tough packaging, the fused seal of plastic was right up against the part where the plastic turns vertical, so there was very little "shoulder" to cut along.
1.5    Drive F (AComdata USB Hard Drive) this was (IIRC) "saran wrapped" cardboard box, which is the ultimate in usable packaging.

Now, you might say this ranks up there with the wrist straps, and claim that my testing is not enough... no, no, if you have got to build an Exchange drive off 10s or 100s of these, you're going to start to care how hard it is to get them out of thier plastic, this would be a serious TCO issue.

Part B: Installing the drive ...
Installing the drive consisted of plugging in the drive, change to the drive ("g:" or "h:" usually),"mkdir \SSDBashData", "cd SSDBashData", and running my test script (I'll publish this soon) to make sure everything is working and it can at least store a results file (no verification that the file written is the file stored). Sort of a pre-race inspection.

3.0    SanDisk gets a base 2.5 for automatically installing (or maybe just running) software. And another .5 doc'd because of how it brought up both a E:\ (CD-ROM RO drive) and an H:\ (USB Mass Storage RW drive), that's just weird.
1.0    Kingston gets as good a score as I will give in this test, it came up promptly, didn't install any software, AND didn't even have any "cool programs" on the root of the drive. Kingston understands its just producing a stick of storage, and I appreciate that.
1.5    Sony gets .5 for having data in the root, but it didn't install / run anything by default that I can tell so that's all they get.
1.0    PQI like Kingston gets the a plain storage device. Good job.
2.0    Patriot gets a .5 for bringing up two drives like the SanDisk. And .5 because one of the drives had software in it. Interestingly both were RW, but the drive with software was only 24 MBs big, the other was 4 GBs.
1.0    AComdata didn't install anything, came right up, removed properly.

I wanted to complete this step also to verify I could use the drives w/o accepting any EULAs and checking the documentation that it didn't have any DeWitt clauses (the term for the clause in EULAs of products like SQL Server, Oracle, etc that say you can't publish benchmark / performance numbers of the product). I do not believe ESE has any such limitation by the way (but one would have to read the Windows OS or Exchange EULAs to be sure).

And as a plus to make sure that terrible spyware they are all bound to install is going to be universally installed when I get to performance tests, surprisingly only one seemed to (not that I looked very hard).

Conclusion:

Total Scores (lower is better):
SanDisk = 7.0
Kingston = 3.0
Sony       = 4.3
PQI   = 4.4
Patriot = 6.4
HD      = 2.5

Since the rest of the tests are likely to be based off metrics, copy times, IOPS, etc, these subjective and arbitrary test results will be used to break any ties, to allow the vendor feels maximally cheated in its ratings. ;) Next test up serial copy performance ...

Cheers,
BrettSh

[5] Yes I do. Seriously, the varying sizes of the drives may throw some of the tests way off, or be difficult to make any claims of scale ... if I were doing real testing, I would be making Microsoft pay for the hardware, getting those larger 32, 64, and 160 GB models mentioned in some of the above articles, and probably wouldn't be blogging about it. ;-) Sooo ... I don't actually intend to answer the question posed in the title, just sort of a check in to see where these cheap flash drives are ... and they all[3] have warranties so I can always recoup my investment if ESE makes them catch on fire.

Show me the backups (win2k3 sp1)...

BrettSh — Thu, 09 Feb 2006 14:50:00 GMT

It recently came to my attention that repadmin + showbackup had no google hits! Well I'd like to fix that.

First a little back story ... around I guess 2003 there had been a growing trend (of PSS whining, er I mean noticing that) customers are not taking any backups, and many customers didn't quite understand how application Naming Contexts (NCs) are not replicated to every Domain Controller (thus the old adage of "backup one DC from every domain" was stale) ... and so people would be missing critical data at restore time ... with restore time being like the 3rd worst time to be missing critical data, but the most likely time to call PSS, PSS asked if we could help, and we / AD dev (were drunk when they asked, had time on our hands, tired of feeling guilty about our in-box monitoring tools story, felt like "putting the feature back in service pack", maybe we were bored... whoa is this my outside voice?) happily said what can we do to help...

So to address this issue, for Win2k3 SP1 we hashed out adding the ability for DCs to log an event (Event ID 2089) if a Naming Context is not being backed up regularly within a certain latency. The default latency is 1/2 the tombstone lifetime (too long IMNHO) ... oh and there is a reason this event won't be logged for an NC, but whatever ... this is not a post about that mechanism / event (more on it someday), besides we're not even sure most admins are capable of reading the event logs (is that too insulting ...where is the line? I can never tell?) ...

OK, event logs are fine, but you want to know now! When I added the 2089 event to AD, I added the /showbackup command to repadmin. This basically can show when backups were taken of various writable NCs the DC hosts. (this block may make the post wide, probably mess stuff up, but anyway here is the output of the command):

C:\bin\rel\win2k3\sp1\x86fre>repadmin.exe /showbackup mycorp-dc-02

Loc.USN                          Originating DC   Org.USN  Org.Time/Date          Ver Attribute
=======                          =============== ========= =============          === =========
DC=DomainDnsZones,DC=MyCorp,DC=com
329205835     084f51ed-d53e-4bad-83db-28694870fdb9 127958011 2006-02-08 02:51:22  197 dSASignature
DC=ForestDnsZones,DC=MyCorp,DC=com
329258203     084f51ed-d53e-4bad-83db-28694870fdb9 127958010 2006-02-08 02:51:22  202 dSASignature
CN=Schema,CN=Configuration,DC=MyCorp,DC=com
330447680     e0cc9580-1546-4da9-af2b-0929c37a378a  68598018 2006-02-09 02:14:56  897 dSASignature
CN=Configuration,DC=MyCorp,DC=com
330447359     e0cc9580-1546-4da9-af2b-0929c37a378a  68598017 2006-02-09 02:14:56  898 dSASignature
DC=MyCorp,DC=com
329205750     084f51ed-d53e-4bad-83db-28694870fdb9 127958006 2006-02-08 02:51:20  205 dSASignature
DC=UnLovedOlderChild,DC=MyCorp,DC=com
DC=UnLovedYoungerChild,DC=MyCorp,DC=com

Obviously the green is showing you can basically see when the DomainDnsZones was last backup. You can probably guess from this output how we are tracking the last backup too. Note: This tracking can only be done if a DC you're taking backups on is upgraded to Win2k3 SP1.

And of course "repadmin /showbackup *" should work if you want to capture the last backup time across all NCs (which means hitting all DCs, thus the *). Don't assume your backup software is smart enough to understand where the NCs are instantiated / replicated to.

It's funny (or embarressing, depending on who you are) 2 years after coding something, you review it, and immediately see everything you screwed up...

The above command should've resolved the Orig DC invocation IDs into DC names, so you could know where that backup was taken. That's just fricken sloppy, sorry about that. I really piss me off sometimes.
In retrospect it would have been better to add another partition test to dcdiag. That would've been way sweeter, fails if over timestamp latency, and /v would print out how old the last backup is, and what DC it was taken on.
Would've been cool to add to ntdsutil the ability to control the backup latency event's sensitity on a per partition basis. The feature has this ability today it is just not exposed, instead you've got to use a reg key (which I don't recommend).
This was actually from a corp DC, but I changed the names ... but it makes me wonder if that's right our child domains aren't being backed up, or there is a bug in the tool / mechanism? Those are partial replicas though, it might not being working on partial replicas ... that's an excercise for the reader ... let me know.

So there you have it repadmin /showbackup, as any self respecting admin, I suggest you move "Try a test restore of our backups" to the bottom of your TODO list, and play with this repadmin command instead. No, no don't worry the restore will just work if you need it, play with this instead.

Well, that is IMNHO only about 1/2 a post ... I didn't even get to the Backup FSMO role (some other time hopefully) ... but tis all I have time for now, sorry.

OK, they CAN NOT be serious, I only have a single sans-serif font to choose from!? Oh, and that font (Arial) even has a sertif on lower case-t. Verdana has serifs on upper case I, but it is mostly serifless. Guh, I'm not sure I can actually live with blogging in this medium, alright Verdana it is, god I miss my Mac ...

Oh and if you're wondering it was Mr "Grillenmeier, Guido" that was the unintentional catalyst to my first post, not the more derisive elements of my life.

Cheers,
BrettSh [msft]
Building #7 Garage Door Operator

P.S. - I still am not quite happy with my categories yet, and I still don't have my Orange theme back, (remorseful voice) it was a really good theme. But at least I can complain now as I'm blogging^H^H^H^Hed.