(....well... in more detail)
This blog posting is for the purposes of explaining the mechanism behind the authoritative restore within Active Directory. While there are many documents out there explaining step-by-step instructions of how to perform authoritative restores and the logic behind it, I feel there’s some information missing from these documents and, hence, this blog posting.
To start off, let’s define some logistics.
This blog does not delve into the mechanics behind Active Directory Replication. It is assumed certain concepts such as USNs, high-watermark, or up-to-dateness vector are already understood. The following article explains these concepts in great detail.
How the Active Directory Replication Model Workshttp://technet.microsoft.com/en-us/library/cc772726(WS.10).aspx
Although I don’t dive into how replication works I’d like to start with some definitions as I feel it is important to this blog.
Update Sequence Number (USN)For every change that occurs on a domain controller (whether replicated inbound or an originating write) the update sequence number is incremented by one.
Highest Committed USNThis is held within RootDSE on every domain controller and is the highest USN committed to the database for that server. This is an attribute of RootDSE and is not replicated to the other domain controllers.
Originating UpdateA domain controller that sets an attribute or builds an object performs an originating update of that attribute or object. The originating update can be seen within the metadata of the object and is used for interpreting which domain controller performed the change to an object.
High-WatermarkEach destination DC contains a high-watermark value of its interpretation of the upstream (or source) DC’s highest committed USN. If DC2’s high-watermark for DC1 is 3000 then DC1 can assume that DC2 already has knowledge of all changes less than or equal to USN 3000. The high-watermark is per replication link and knowledge a DC has of each direct replication partner. Although there may be hundreds of DCs within a domain, it’s possible DC2 only replicates with a handful of them and, therefore, would only have knowledge of the high-watermark of those DCs. The high-watermark prevents irrelevant objects from being considered by the source domain controller with respect to a single destination.
Up-to-Dateness VectorAny DC that has ever made an originating update will have an entry in the up-to-dateness vector. This is per partition. We use the up-to-dateness vector to determine (rather filter out) attributes to objects that a downstream replication partner may have already received from a different source DC. Each DC has its own copy of the up-to-dateness vector for all domain controllers that replicate a specific partition.
VersionFor the purposes of this blog, the term “version” refers to the version number of an attribute. As discussed within this blog, an authoritative restore of an object will increment the version number of every attribute by a specific value (discussed later).
DSA Object GUIDThe Globally Unique Identifier of a domain controller.
DSA Invocation IDThe GUID of the Active Directory database (ntds.dit). The Invocation ID is used to uniquely identify a database and it changes during the restore process of Active Directory.
The Beginning
The environment, discussed in this blog, consists of only two domain controllers (DC1 and DC2) for a domain called contoso.com.
After creating the environment, repadmin /showrepl /v against each DC shows:
DC1
Default-First-Site-Name\DC1DSA Options: IS_GC Site Options: (none)DSA object GUID: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4fDSA invocationID: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4f
(Notice the DC GUID and Invocation ID GUID are the same for DC1. This is because it was the first domain controller promoted into the forest and, therefore, gets the same GUID for both.)
DC2
Default-First-Site-Name\DC2DSA Options: IS_GC Site Options: (none)DSA object GUID: 5ad77be3-1e89-4bba-bd7c-c78a03860204DSA invocationID: e7d0d6bb-056c-45b1-b752-1d6900e3b9e5
Out of the box, the up-to-dateness vector for the DC=contoso,DC=com partition on DC1 showed:
Caching GUIDs.
Default-First-Site-Name\DC1 @ USN 13190 @ Time 2011-10-04 21:11:39Default-First-Site-Name\DC2 @ USN 12755 @ Time 2011-10-04 21:07:39
Out of the box, the up-to-dateness vector for the DC=contoso,DC=com partition on DC2 showed:
Default-First-Site-Name\DC1 @ USN 13185 @ Time 2011-10-04 21:10:14Default-First-Site-Name\DC2 @ USN 12778 @ Time 2011-10-04 21:11:58
One thing to note is that it’s likely that interrogating the up-to-dateness vector on both domain controllers at the same time won’t show the same USN values. This is due to convergence within Active Directory. There’s always something going on. Active Directory replication is loosely consistent and we assume we’ll eventually converge.
First things first, let’s demote DC2. Why? This would be to illustrate a change in Invocation ID. Before the demotion, let’s look at the replication status of DC1:
Repadmin /showrepl DC1 /v shows:
==== INBOUND NEIGHBORS ======================================
DC=contoso,DC=com Default-First-Site-Name\DC2 via RPC DSA object GUID: 5ad77be3-1e89-4bba-bd7c-c78a03860204 Address: 5ad77be3-1e89-4bba-bd7c-c78a03860204._msdcs.contoso.com DSA invocationID: e7d0d6bb-056c-45b1-b752-1d6900e3b9e5
(Notice the DSA invocation ID for the inbound partner DC2.)
After a demotion of DC2 and another DCPromo promotion of DC2, the repadmin /showrepl /v configuration of DC2 shows:
Default-First-Site-Name\DC2DSA Options: IS_GC Site Options: (none)DSA object GUID: 8913255f-89c3-4954-ab3f-3148f7303320DSA invocationID: 43c7f173-db0f-4ddb-aff0-ad50717729c9
Notice the DC GUID and Invocation ID both changed. This comes into play later.
The up-to-dateness vector for the DC=contoso,DC=com partition on DC1 now shows:
Default-First-Site-Name\DC2 @ USN 12399 @ Time 2011-10-04 21:26:04Default-First-Site-Name\DC1 @ USN 13414 @ Time 2011-10-04 21:30:22e7d0d6bb-056c-45b1-b752-1d6900e3b9e5 @ USN 12778 @ Time 2011-10-04 21:12:08
(Recall that the database GUID of DC2 used to be e7d0d6bb-056c-45b1-b752-1d6900e3b9e5. It is still in the up-to-dateness vector.)
The command repadmin /showrepl DC1 /v shows the following information about DC2:
DC=contoso,DC=com Default-First-Site-Name\DC2 via RPC DSA object GUID: 8913255f-89c3-4954-ab3f-3148f7303320 Address: 8913255f-89c3-4954-ab3f-3148f7303320._msdcs.contoso.com DSA invocationID: 43c7f173-db0f-4ddb-aff0-ad50717729c9 SYNC_ON_STARTUP DO_SCHEDULED_SYNCS WRITEABLE USNs: 12329/OU, 12329/PU Last attempt @ 2011-10-04 21:26:07 was successful.
(The above values for USNs represent DC1’s high-watermark knowledge of DC2 for the partition DC=contoso,DC=com.)
(The remainder of the repadmin /showrepl command has been snipped.)
So we just discussed that the up-to-dateness vector keeps records of domain controllers that have been demoted. This is to maintain history of DCs we’ve replicated with in the past. What if I were to restore a backup of DC2 at this point? We’ll cover that in a bit.
First, let’s talk about where this up-to-dateness vector is stored. The up-to-dateness vector is per partition and, therefore, is stored as attributes of a particular partition. In this case, the partition of interest is DC=contoso,DC=com. The attribute we query for is replUpToDateVector. This attribute is not a replicated attribute and looks different depending on which DC is interrogated.
To illustrate this I used LDP and searched for the replUpToDateVector attribute for each DC.
LDP connection to DC1:
***Searching...
ldap_search_s(ld, "DC=contoso,DC=com", 2, "(replUpToDateVector=*)", attrList, 0, &msg)Getting 1 entries:Dn: DC=contoso,DC=comreplUpToDateVector: dwVersion: 2, dwReserved1: 0, V2.cNumCursors: 2, V2.dwReserved2: 0, rgCursors: {uuidDsa: 43c7f173-db0f-4ddb-aff0-ad50717729c9, usnHighPropUpdate: 12459, timeLastSyncSuccess: 10/04/2011 21:31:32}, {uuidDsa: e7d0d6bb-056c-45b1-b752-1d6900e3b9e5, usnHighPropUpdate: 12778, timeLastSyncSuccess: 10/04/2011 21:12:08},;
LDP connection to DC2:
ldap_search_s(ld, "DC=contoso,DC=com", 2, "(replUpToDateVector=*)", attrList, 0, &msg)Getting 1 entries:Dn: DC=contoso,DC=comreplUpToDateVector: dwVersion: 2, dwReserved1: 0, V2.cNumCursors: 2, V2.dwReserved2: 0, rgCursors: {uuidDsa: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4f, usnHighPropUpdate: 13435, timeLastSyncSuccess: 10/04/2011 21:40:17}, {uuidDsa: e7d0d6bb-056c-45b1-b752-1d6900e3b9e5, usnHighPropUpdate: 12778, timeLastSyncSuccess: 10/04/2011 21:12:08},;
Notice both domain controllers have knowledge of the original DC2 (uuidDsa e7d0d6bb….) as well as knowledge of the highest propagated update (12778) and the last time it synchronized (10/04/2011 at 9:12:08 p.m.).
Each DC also has knowledge in the up-to-dateness vector for all other domain controllers that replicate a partition. In the up-to-dateness vector for the partition DC=contoso,DC=com on DC1 you will see it does not hold knowledge of its own uuidDsa. The same is true for DC2.
Domain Controller Restore Process and the Invocation ID
So now that we’ve discussed the mechanics behind it, let’s discuss what happens when a domain controller is restored from backup.
As you may have read, when a restore operation of a domain controller is performed, we keep the DSA Object GUID the same but the DSA Invocation ID is recreated with a random GUID. This process informs downstream replication partners that a DC restore operation has been performed. When the backup was created, the highestCommittedUSN value was at a lower number than what every other domain controller remembered of DC1 before the restore process.
By modifying the Invocation ID it informs the downstream replication partners of this restore operation and we can get around USN rollback scenarios. It also instructs DC1’s replication partners to send all updates from highestCommittedUSN (x) to highestCommittedUSN (y) where X is the value at the time of the backup and Y is the value at the time DC1 went down to perform the restore.
So let’s look at this. I took a system state backup of DC1 and then restored the backup.
Repadmin /showrepl DC2 /v
DC=contoso,DC=com Default-First-Site-Name\DC1 via RPC DSA object GUID: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4f Address: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4f._msdcs.contoso.com DSA invocationID: 1f4b741a-6409-4eb6-996b-0f5f7417646a SYNC_ON_STARTUP DO_SCHEDULED_SYNCS WRITEABLE USNs: 24637/OU, 24637/PU Last attempt @ 2011-10-05 20:11:14 was successful.
(The rest of the repadmin /showrepl output has been snipped.)
Repadmin /showrepl DC1 /v
Default-First-Site-Name\DC1DSA Options: IS_GC Site Options: (none)DSA object GUID: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4fDSA invocationID: 1f4b741a-6409-4eb6-996b-0f5f7417646a
DC=contoso,DC=com Default-First-Site-Name\DC2 via RPC DSA object GUID: 8913255f-89c3-4954-ab3f-3148f7303320 Address: 8913255f-89c3-4954-ab3f-3148f7303320._msdcs.contoso.com DSA invocationID: 43c7f173-db0f-4ddb-aff0-ad50717729c9 SYNC_ON_STARTUP DO_SCHEDULED_SYNCS WRITEABLE USNs: 14407/OU, 14407/PU Last attempt @ 2011-10-05 20:11:20 was successful.
Looking at the /showrepl output from both domain controllers, you can see that the DSA InvocationID for DC1 has changed; however, the DSA Object GUID remains the same.
DSA object GUID: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4fDSA invocationID: 1f4b741a-6409-4eb6-996b-0f5f7417646a
Repadmin /showutdvec DC1 DC=contoso,DC=com /nocache
1f4b741a-6409-4eb6-996b-0f5f7417646a @ USN 24668 @ Time 2011-10-05 20:15:0943c7f173-db0f-4ddb-aff0-ad50717729c9 @ USN 14452 @ Time 2011-10-05 20:11:169628bc8f-fa40-4c72-ac57-99ef34fe3f4f @ USN 13972 @ Time 2011-10-05 18:17:27e7d0d6bb-056c-45b1-b752-1d6900e3b9e5 @ USN 12778 @ Time 2011-10-04 21:12:08
Repadmin /showutdvec DC2 DC=contoso,DC=com /nocache
1f4b741a-6409-4eb6-996b-0f5f7417646a @ USN 24663 @ Time 2011-10-05 20:14:0443c7f173-db0f-4ddb-aff0-ad50717729c9 @ USN 14482 @ Time 2011-10-05 20:14:589628bc8f-fa40-4c72-ac57-99ef34fe3f4f @ USN 13972 @ Time 2011-10-05 18:17:27e7d0d6bb-056c-45b1-b752-1d6900e3b9e5 @ USN 12778 @ Time 2011-10-04 21:12:08
Here’s the recap:
9628bc8f…. was the original DC1 that was just restored. E7d0d6bb… was the original DC2 that was demoted at the beginning.1f4b741a… is the new DC1.43c7f173… is DC2.
Using LDP, I again performed a search on both domain controllers for the replUpToDateVector.
ldap_search_s(ld, "DC=contoso,DC=com", 2, "(replUpToDateVector=*)", attrList, 0, &msg)Getting 1 entries:Dn: DC=contoso,DC=comreplUpToDateVector: dwVersion: 2, dwReserved1: 0, V2.cNumCursors: 3, V2.dwReserved2: 0, rgCursors: {uuidDsa: 43c7f173-db0f-4ddb-aff0-ad50717729c9, usnHighPropUpdate: 14452, timeLastSyncSuccess: 10/05/2011 20:11:16}, {uuidDsa: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4f, usnHighPropUpdate: 13972, timeLastSyncSuccess: 10/05/2011 18:17:27}, {uuidDsa: e7d0d6bb-056c-45b1-b752-1d6900e3b9e5, usnHighPropUpdate: 12778, timeLastSyncSuccess: 10/04/2011 21:12:08},;
ldap_search_s(ld, "DC=contoso,DC=com", 2, "(replUpToDateVector=*)", attrList, 0, &msg)Getting 1 entries:Dn: DC=contoso,DC=comreplUpToDateVector: dwVersion: 2, dwReserved1: 0, V2.cNumCursors: 3, V2.dwReserved2: 0, rgCursors: {uuidDsa: 1f4b741a-6409-4eb6-996b-0f5f7417646a, usnHighPropUpdate: 24679, timeLastSyncSuccess: 10/05/2011 20:17:41}, {uuidDsa: 9628bc8f-fa40-4c72-ac57-99ef34fe3f4f, usnHighPropUpdate: 13972, timeLastSyncSuccess: 10/05/2011 18:17:27}, {uuidDsa: e7d0d6bb-056c-45b1-b752-1d6900e3b9e5, usnHighPropUpdate: 12778, timeLastSyncSuccess: 10/04/2011 21:12:08},;
So, in a nutshell, that’s how the restore process works. There is an additional feature of NTBackup that adds some registry settings during a restore process of the system state. When performing a system state restore using NTBackup, a new key is created called Restore in Progress. This is located under:
HKLM\System\CurrentControlSet\Services\NTDS\Restore In Progress
Information about this key can be found in the following support article:
Description of the “Restore in Progress” Registry Key in Active Directoryhttp://support.microsoft.com/kb/814167
An important registry setting for the restore operation is called New Database GUID located in HKLM\System\CurrentControlSet\Services\NTDS\Parameters. This registry setting supplies the new DSA InvocationID to be used after the restore process is complete.
Authoritative Restore of Joe User
Next let’s create a user (Joe User) and then do some authoritative restores.
Created Joe User (sAMAccountName joeuser) on DC2.
Repadmin /showobjmeta DC2 “CN=joe user,CN=Users,DC=contoso,DC=com”
26 entries.
Loc.USN Originating DSA Org.USN Org.Time/Date Ver Attribute ======= =============== ========= ============= === ========= 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 objectClass 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 cn 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 sn 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 givenName 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 instanceType 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 whenCreated 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 displayName 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 nTSecurityDescriptor
(The rest of the output has been snipped.)
From here I need to explain what happens when an object is authoritatively restored from a system state backup. There are many documents on the Internet stating that the version number of each attribute for an object will increase by 100,000 versions. Some state the version of each attribute will increase 100,000 versions total while others state the version will increase by 100,000 versions per day since the last backup was taken.
Neither of these statements are true.
Without creating a backup at all, using Windows Server 2008, I stopped the Active Directory Domain Services (NOT supported; always reboot into Directory Services Restore Mode) and performed these actions on DC1:
Net stop ntdsChanged the date ahead 2 daysNtdsutilActivate instance NTDSAuthoritative restoreRestore object “CN=Joe User,CN=Users,DC=contoso,DC=com”Restored the objectNet start ntds
Repadmin /showobjmeta DC1 “CN=joe user,CN=users,DC=contoso,DC=com”
27 entries.
Loc.USN Originating DSA Org.USN Org.Time/Date Ver Attribute ======= =============== ========= ============= === ========= 14612 Default-First-Site-Name\DC1 28675 2011-10-07 20:29:35 300001 objectClass 14611 Default-First-Site-Name\DC2 14611 2011-10-07 20:30:23 2 cn 14612 Default-First-Site-Name\DC1 28675 2011-10-07 20:29:35 300001 sn 14612 Default-First-Site-Name\DC1 28675 2011-10-07 20:29:35 300001 givenName 14612 Default-First-Site-Name\DC1 28675 2011-10-07 20:29:35 300001 instanceType 14550 Default-First-Site-Name\DC2 14550 2011-10-05 20:26:16 1 whenCreated 14612 Default-First-Site-Name\DC1 28675 2011-10-07 20:29:35 300001 displayName 14612 Default-First-Site-Name\DC1 28675 2011-10-07 20:29:35 300000 isDeleted
Notice that the version number has increased by 300,000. This is equal to “today + 2 days” or 3 days total.
Deleted Joe UserChanged the date back to the correct dateCreated Joe User on DC2Output of showobjmeta is similar to original Joe User objectNet stop ntdsChanged the date forward 11 daysNtdsutilActivate Instance NTDSAuthoritative restoreRestore object “CN=joe user,CN=Users,DC=contoso,DC=com”Restored the objectNet start ntds
Repadmin /showobjmeta DC1 “CN=joe user,CN=Users,DC=contoso,DC=com”
Loc.USN Originating DSA Org.USN Org.Time/Date Ver Attribute ======= =============== ========= ============= === ========= 14691 Default-First-Site-Name\DC1 32772 2011-10-18 20:33:30 1200001 objectClass 14690 Default-First-Site-Name\DC2 14690 2011-10-05 20:34:37 2 cn 14691 Default-First-Site-Name\DC1 32772 2011-10-18 20:33:30 1200001 sn 14691 Default-First-Site-Name\DC1 32772 2011-10-18 20:33:30 1200001 givenName 14691 Default-First-Site-Name\DC1 32772 2011-10-18 20:33:30 1200001 instanceType 14647 Default-First-Site-Name\DC2 14647 2011-10-07 20:31:33 1 whenCreated 14691 Default-First-Site-Name\DC1 32772 2011-10-18 20:33:30 1200001 displayName 14691 Default-First-Site-Name\DC1 32772 2011-10-18 20:33:30 1200000 isDeleted
(The remainder of the output has been snipped.)
Notice the version number increase is now 1,200,000 (100,000 x 12 days…. “today plus 11 days”).
Here’s where I mention another “feature” of this process. The version number increase is capped at 60 days. I assume this is because the original Tombstone Lifetime was set to 60 days and backups can only be restored < TSL.
Deleted Joe UserChanged the date back to the correct dateCreated Joe User on DC2Output of showobjmeta is similar to original Joe User objectNet stop ntdsChanged the date forward to 2/1/2012NtdsutilActivate Instance NTDSAuthoritative restoreRestore object “CN=joe user,CN=Users,DC=contoso,DC=com”Restored the objectNet start ntds
Loc.USN Originating DSA Org.USN Org.Time/Date Ver Attribute ======= =============== ========= ============= === ========= 14824 Default-First-Site-Name\DC1 36869 2012-02-01 20:42:11 6000001 objectClass 14823 Default-First-Site-Name\DC2 14823 2012-02-01 20:42:37 2 cn 14824 Default-First-Site-Name\DC1 36869 2012-02-01 20:42:11 6000001 sn 14824 Default-First-Site-Name\DC1 36869 2012-02-01 20:42:11 6000001 givenName 14824 Default-First-Site-Name\DC1 36869 2012-02-01 20:42:11 6000001 instanceType 14742 Default-First-Site-Name\DC2 14742 2011-10-05 20:37:28 1 whenCreated 14824 Default-First-Site-Name\DC1 36869 2012-02-01 20:42:11 6000001 displayName 14824 Default-First-Site-Name\DC1 36869 2012-02-01 20:42:11 6000000 isDeleted
(The rest of the output has been snipped.) Notice that the version increase was capped at 60 days (or 6,000,000 version increases). I hope this has helped clear up the understanding of the logic behind the authoritative restore, where we store knowledge of the Invocation ID for the database, and how DC restore process works.
One point to note is that the version increase default is 100,000 versions per day since the last time a change was committed to the database (not since the last backup). This version increase is a default value and can be manipulated using the verinc parameter during the authoritative restore. Restore object “CN=Joe User,CN=Users,DC=contoso,DC=com” verinc 1000
This would increase the version number of every attribute for Joe User by 1,000 each day since the last time a change was committed to the database.