I decided to check out the size of Eric's DIT ...

BrettSh — Mon, 12 Jun 2006 19:49:00 GMT

... take some time, measuring the exact dimensions of Eric's DIT ... and I must say, I've seen a fair amount of DITs in my time, and I can say with a fair amount of certainty, Eric's DIT is the biggest I've ever seen! I can't believe what a massively huge a DIT Eric has.

Note: while this is about an Active Directory database, Exchange is based on the same database technology, so it would (and does) have similar space hierarchy.

Table of ESE Space usage:

The black in this table is just the output from 3 of the columns of esentutl /ms adamntds.dit (original report), the blue are columns/rows I've added to break out the space usage in a clearer way:

Name	Friendly name	Owned	Available	Owned(GB)	Avail(GB)	b-tree lvls	% of DB	% of Total Idxs
	<calc: DB real>	268179344	6088	0.000	0.046
F:\DNT\adamntds.dit		268179344	218	2046.046	0.002

	<calc: datatable real>	268178751	5689	2046.041	0.043
datatable		268178751	2260	2046.041	0.017
	<calc: Row Data>	186707905	2260	1424.468	0.017	5	69.62%
<Long Values>		42	18	0.000	0.000	2	0.00%
	<sum: Idx Totals>	81470804	3411	621.573	0.026	0	30.38%	100.00%
PDNT_index	Int: PDNT + Name	11482791	7	87.607	0.000	5	4.28%	14.09%
nc_guid_Index	Int: NC + objGuid	10870892	5	82.938	0.000	4	4.05%	13.34%
INDEX_00090002	Att: objectGuid	10791866	3	82.335	0.000	4	4.02%	13.25%
INDEX_00000003	Att: cn	9658638	51	73.690	0.000	4	3.60%	11.86%
INDEX_00090001	Att: name	8999729	83	68.662	0.001	4	3.36%	11.05%
Ancestors_index	Int: Ancestry	7917036	10	60.402	0.000	4	2.95%	9.72%
DRA_USN_index	Int: Repl USN	7083583	251	54.043	0.002	4	2.64%	8.69%
INDEX_0009030E	Att: objectCategory	5274627	21	40.242	0.000	4	1.97%	6.47%
DRA_USN_CREATED_index	Int: Repl Created USN	4479144	34	34.173	0.000	4	1.67%	5.50%
INDEX_00020078	Att: uSNChanged	4279711	0	32.652	0.000	4	1.60%	5.25%
deltime_index	Int: deltime	371267	15	2.833	0.000	4	0.14%	0.46%
INDEX_00020030	Att: isDeleted	261493	2929	1.995	0.022	4	0.10%	0.32%

... deleted about a dozen small indices ...

I'll discuss the permutations I performed on the esentutl /ms output, in the hopes it will be clear ...

First I sum up the owned space for all indices in the datatable, this comes out to 81470804. Note the #'s above may not add up exactly because I deleted a dozen or so super small indices. I summed up all the indices because it makes the next calculation easier, and also so we can get the "% of Total Idxs" column as well.

So first understand that ESE's "owned" space is hierarchical, so the "datatable" owns all the space owned by each of the indices and the LV B-Tree in the datatable. But the primary B-Tree for the datatable also contains (and thus owns) the normal row data. So the real data that is in the regular row data for the datatable is 268178751 (datatable) - 42 (datatable's LV B-Tree owned) - 81470804 (owned by sum of all datatable indices) = 186707905 (i.e. the "<calc: Row Data>" line).

I then created a couple columns to turn this page counts into a usable unit (GBs), i.e. <# of pages> * 8 / 1024 / 1024.

Finally I added a friendly name column, so you'd know roughly what the index was indexing.

Some analysis:

From the above table we can easily see the row data 1,424 GBs and all the indices combined is 621 GBs. This breaks out like so:

Based on the table above this is showing us a full 30% of this database is indices!!! That's a huge amount. This isn't a common space breakdown for most AD objects, as the objects making up Eric's DIT are very very small / light weight. He was just creating containers w/ minimal attributes (see Eric's initial post), and so just the base set of indices on a basic object lead up to a significant portion of the objects overall "footprint" in the DB.

As for the breakup of the individual index usages, it looks something like this:

Of the secondary indices on the datatable, 10 are always updated! And another 2 (the very slender ones) are only updated on delete. Since there are over 2 billion objects in this database, that means we inserted about 22 billion B-Tree entries, kind of neat.

One last, somewhat technical thing that I think a few of you might find interesting, is that even the largest 1,424 GB primary B-Tree is only 5 levels deep. This means that to locate a specific row (by DNT) will only take 5 disk seeks in the worst case (cold cache). B-Trees have this very nice high fan out, that keeps disk seeks minimal.

Interestingly, I dumped the root page, and it only has 3 nodes (TAG 0 doesn't count), what this means is that we could add about 100x more data to this b-tree and there would be no increase in the # of disk seeks to fetch a row from this table.

Anyway that seems like enough for now ...

Show me the backups (win2k3 sp1)...

BrettSh — Thu, 09 Feb 2006 14:50:00 GMT

It recently came to my attention that repadmin + showbackup had no google hits! Well I'd like to fix that.

First a little back story ... around I guess 2003 there had been a growing trend (of PSS whining, er I mean noticing that) customers are not taking any backups, and many customers didn't quite understand how application Naming Contexts (NCs) are not replicated to every Domain Controller (thus the old adage of "backup one DC from every domain" was stale) ... and so people would be missing critical data at restore time ... with restore time being like the 3rd worst time to be missing critical data, but the most likely time to call PSS, PSS asked if we could help, and we / AD dev (were drunk when they asked, had time on our hands, tired of feeling guilty about our in-box monitoring tools story, felt like "putting the feature back in service pack", maybe we were bored... whoa is this my outside voice?) happily said what can we do to help...

So to address this issue, for Win2k3 SP1 we hashed out adding the ability for DCs to log an event (Event ID 2089) if a Naming Context is not being backed up regularly within a certain latency. The default latency is 1/2 the tombstone lifetime (too long IMNHO) ... oh and there is a reason this event won't be logged for an NC, but whatever ... this is not a post about that mechanism / event (more on it someday), besides we're not even sure most admins are capable of reading the event logs (is that too insulting ...where is the line? I can never tell?) ...

OK, event logs are fine, but you want to know now! When I added the 2089 event to AD, I added the /showbackup command to repadmin. This basically can show when backups were taken of various writable NCs the DC hosts. (this block may make the post wide, probably mess stuff up, but anyway here is the output of the command):

C:\bin\rel\win2k3\sp1\x86fre>repadmin.exe /showbackup mycorp-dc-02

Loc.USN                          Originating DC   Org.USN  Org.Time/Date          Ver Attribute
=======                          =============== ========= =============          === =========
DC=DomainDnsZones,DC=MyCorp,DC=com
329205835     084f51ed-d53e-4bad-83db-28694870fdb9 127958011 2006-02-08 02:51:22  197 dSASignature
DC=ForestDnsZones,DC=MyCorp,DC=com
329258203     084f51ed-d53e-4bad-83db-28694870fdb9 127958010 2006-02-08 02:51:22  202 dSASignature
CN=Schema,CN=Configuration,DC=MyCorp,DC=com
330447680     e0cc9580-1546-4da9-af2b-0929c37a378a  68598018 2006-02-09 02:14:56  897 dSASignature
CN=Configuration,DC=MyCorp,DC=com
330447359     e0cc9580-1546-4da9-af2b-0929c37a378a  68598017 2006-02-09 02:14:56  898 dSASignature
DC=MyCorp,DC=com
329205750     084f51ed-d53e-4bad-83db-28694870fdb9 127958006 2006-02-08 02:51:20  205 dSASignature
DC=UnLovedOlderChild,DC=MyCorp,DC=com
DC=UnLovedYoungerChild,DC=MyCorp,DC=com

Obviously the green is showing you can basically see when the DomainDnsZones was last backup. You can probably guess from this output how we are tracking the last backup too. Note: This tracking can only be done if a DC you're taking backups on is upgraded to Win2k3 SP1.

And of course "repadmin /showbackup *" should work if you want to capture the last backup time across all NCs (which means hitting all DCs, thus the *). Don't assume your backup software is smart enough to understand where the NCs are instantiated / replicated to.

It's funny (or embarressing, depending on who you are) 2 years after coding something, you review it, and immediately see everything you screwed up...

The above command should've resolved the Orig DC invocation IDs into DC names, so you could know where that backup was taken. That's just fricken sloppy, sorry about that. I really piss me off sometimes.
In retrospect it would have been better to add another partition test to dcdiag. That would've been way sweeter, fails if over timestamp latency, and /v would print out how old the last backup is, and what DC it was taken on.
Would've been cool to add to ntdsutil the ability to control the backup latency event's sensitity on a per partition basis. The feature has this ability today it is just not exposed, instead you've got to use a reg key (which I don't recommend).
This was actually from a corp DC, but I changed the names ... but it makes me wonder if that's right our child domains aren't being backed up, or there is a bug in the tool / mechanism? Those are partial replicas though, it might not being working on partial replicas ... that's an excercise for the reader ... let me know.

So there you have it repadmin /showbackup, as any self respecting admin, I suggest you move "Try a test restore of our backups" to the bottom of your TODO list, and play with this repadmin command instead. No, no don't worry the restore will just work if you need it, play with this instead.

Well, that is IMNHO only about 1/2 a post ... I didn't even get to the Backup FSMO role (some other time hopefully) ... but tis all I have time for now, sorry.

OK, they CAN NOT be serious, I only have a single sans-serif font to choose from!? Oh, and that font (Arial) even has a sertif on lower case-t. Verdana has serifs on upper case I, but it is mostly serifless. Guh, I'm not sure I can actually live with blogging in this medium, alright Verdana it is, god I miss my Mac ...

Oh and if you're wondering it was Mr "Grillenmeier, Guido" that was the unintentional catalyst to my first post, not the more derisive elements of my life.

Cheers,
BrettSh [msft]
Building #7 Garage Door Operator

P.S. - I still am not quite happy with my categories yet, and I still don't have my Orange theme back, (remorseful voice) it was a really good theme. But at least I can complain now as I'm blogging^H^H^H^Hed.

183 : Comp::Impl::AD

I decided to check out the size of Eric's DIT ...

Show me the backups (win2k3 sp1)...