TFS Integration Platform – What is the Lookup Service? Q&A-27

Willy-P. Schaub

10 Apr 2010 2:14 PM

Comments 4

Revised on August 12, 2010.

The Lookup Service has been floating around as part of the TFS Integration Platform tools for some time, but has not received much attention to date. After it has been thoroughly tested and is adding value at one of our proof-of-concepts, it is time to let the service our of the binary bucket.

Quotation from Bob, Visual Studio ALM Ranger, who is running one of our CQ—>TFS proof of concepts: “ClearQuest uses an “open user name format” and so it would be very difficult to develop a standardized algorithm to covert CQ user names to the AD display names used by TFS. However, CQ does maintain the AD alias (account ID) for each user. The LookupService enables us to take the user account for each CQ user, look up the display name in the AD and use that when populating user fields in TFS work items. This means we have out of the box, 100% reliability of mapping user names for our CQ to TFS migration.”

So, what is it and why should we care?

The service is intended to assist us with the translation of the source user information to the target user information, in scenarios where we are dealing with different environments such as ClearQuest on the source and TFS on the target, or cross domain migrations.

The LookupService uses a three phased translation process:

“Lookup”
During the first phase the TFS Integration Platform will ask the source adapter for more information, such as email address, display name or domain name. For example, migrating from CQ, we only have the users alias or account name, from information stored in a record field.
“Transformation”
The second phase is performed by the UserIdentityLookupService in the platform to perform string replacements, string concatenations and formatting. It is basically the user information transformation phase, excluding validation of the user account. The transformation is done during the “Analysis” phase of the migration pipeline and the adapters are not involved.
“Validation and Population”
Phase three performs the target user validation, by talking to Team Foundation Server (TFS), Active Directory (AD) or another custom target lookup provider. It essentially validates the target user information and populates the target system information.

Common Question:

Should all adapters implement support for this service?
- There is out of box support for AD, CQ and TFS 2010 (see UserIdentityLookupService.cs and TFS2010UserIdLookupAdapter.cs for details).
- The lookup service is an add-in, which means the adapters do not have to implement anything. Instead we “add-in” the functionality through configuration and the add-in at run-time.
Why not just use value mapping rules in the configuration?
- The Lookup Service provides additional validation and retrieval of additional run-time information.
- User information stored with an artefacts (Changes, WIT) only contains display name, which would force us to make a lot of assumptions.

Example configurations

Here is an example configuration file (extracts) that configures the service for a migration from ClearQuest to TFS. Let us briefly explore the contents:

Lines 7-9 … we configure the TFS Active Directory User Id Lookup Service addin.
Lines 14-16 … we configure the CQ migration source user identity lookup to use the addin.
Lines 29-31 … we configure the TFS migration source user identity lookup to use the addin as well.
Lines 60 - 67 … we configure stage 1 lookup information. In this case we need to lookup the alias for the owner on CQ side and the DisplayName for the AssignedTo on the TFS side.
Lines 103 – 111 …we configure stage 2, the transformation, using SimpleReplacement in this example.

NOTE 2011-10-26: The UserIdLookupEnabled attribute has been deprecated and replaced with a more readanble EnableValidation attribute with the build dated June 2010.

<?xml version="1.0" encoding="utf-16"?>

<Configuration UniqueId="d853ac81-2582-4c79-9097-4688a9365f18" FriendlyName="BugBash1">

  <Providers>

    <Provider ReferenceName="d9637401-7385-4643-9c64-31585d77ed16" FriendlyName="ClearQuest 7.1.0 Adapter" />

    <Provider ReferenceName="04201d39-6e47-416f-98b2-07f0013f8455" FriendlyName="TFS 2010 Migration WIT Provider" />

  </Providers>

  <Addins>

    <Addin ReferenceName="cdde6b6b-72fc-43b6-bbd1-b8a89a788c6f" FriendlyName="TFS Active Directory User Id Lookup Service Addin" />

  </Addins>

  <SessionGroup CreationTime="2010-03-03T07:08:50.3305827Z" FriendlyName="BugBashDemo" SessionGroupGUID="aa27d9d1-1cbb-47d5-a264-4bb4a2627e0a" Creator="XYZ\testuser" SyncIntervalInSeconds="0" SyncDurationInMinutes="0">

    <MigrationSources>

      <MigrationSource InternalUniqueId="a14dc073-e3da-4cc4-b03e-fdc1a9caa347" FriendlyName="CQ source" ServerIdentifier="Server identifier" ServerUrl="7.0.0" SourceIdentifier="UCM01" ProviderReferenceName="d9637401-7385-4643-9c64-31585d77ed16">

        <Settings>

          <UserIdentityLookup>

          <LookupAddin Precedence="1" ReferenceName="cdde6b6b-72fc-43b6-bbd1-b8a89a788c6f" />

      </UserIdentityLookup>

        </Settings>

        <CustomSettings>

          <CustomSetting SettingKey="LoginCredentialConfigType" SettingValue="UseTextUsernamePasswordPairInConfig" />

          <CustomSetting SettingKey="UserName" SettingValue="testuser" />

          <CustomSetting SettingKey="Password" SettingValue="nicetry" />

          <CustomSetting SettingKey="AdminUserName" SettingValue="" />

          <CustomSetting SettingKey="AdminPassword" SettingValue="" />

        </CustomSettings>

        <StoredCredential />

      </MigrationSource>

      <MigrationSource InternalUniqueId="21761303-70c2-4d6f-84bc-8d58e07581d4" FriendlyName="Test3 (va02-tfs4)" ServerIdentifier="2561585b-d29d-4eec-838a-db773cd9d409" ServerUrl="http://va02-tfs4:8080/collection0" SourceIdentifier="Test3" ProviderReferenceName="04201d39-6e47-416f-98b2-07f0013f8455">

        <Settings>

          <UserIdentityLookup>

         <LookupAddin Precedence="1" ReferenceName="cdde6b6b-72fc-43b6-bbd1-b8a89a788c6f" />

      </UserIdentityLookup>

        </Settings>

        <CustomSettings>

          <CustomSetting SettingKey="EnableBypassRuleDataSubmission" SettingValue="" />

          <CustomSetting SettingKey="DisableAreaPathAutoCreation" SettingValue="" />

          <CustomSetting SettingKey="DisableIterationPathAutoCreation" SettingValue="" />

        </CustomSettings>

        <StoredCredential />

      </MigrationSource>

    </MigrationSources>

    <Sessions>

      <Session CreationTime="2010-03-03T07:08:50.3205827Z" SessionUniqueId="be898261-968c-4078-9ba2-e164e7497d36" FriendlyName="Work Item Tracking session" LeftMigrationSourceUniqueId="a14dc073-e3da-4cc4-b03e-fdc1a9caa347" RightMigrationSourceUniqueId="21761303-70c2-4d6f-84bc-8d58e07581d4" SessionType="WorkItemTracking">

        <EventSinks />

        <CustomSettings>

          <SettingXml>

            <WITSessionCustomSetting>            

    <WorkItemTypes

      &lt;WorkItemType LeftWorkItemTypeName="Defect" RightWorkItemTypeName="Bug" fieldMap="CQType2TFSTypeFieldMap" />        

    </WorkItemTypes>

    <FieldMaps>

      <FieldMap name="CQType2TFSTypeFieldMap">

        <MappedFields>

          <MappedField LeftName="Headline" RightName="System.Title" MapFromSide="Left" valueMap="" />

          <MappedField LeftName="State" RightName="System.State" MapFromSide="Left" valueMap="StateMap" />

          <MappedField LeftName="History" RightName="System.History" MapFromSide="Left" valueMap="" />

          <MappedField LeftName="@@MissingField@@" RightName="System.Reason" MapFromSide="Left" valueMap="ReasonDefault" />

          <MappedField LeftName="Priority" RightName="Microsoft.VSTS.Common.Priority" MapFromSide="Left" valueMap="PriorityMap" />

          <MappedField LeftName="Owner" RightName="System.AssignedTo" MapFromSide="Left" valueMap="" />

        </MappedFields>

        <UserIdentityFields>

          <LeftUserIdentityFields>

            <UserIdField FieldReferenceName="Owner" UserIdPropertyName="Alias" />

          </LeftUserIdentityFields>

          <RightUserIdentityFields>

            <UserIdField FieldReferenceName="System.AssignedTo" UserIdPropertyName="DisplayName" />

          </RightUserIdentityFields>

        </UserIdentityFields>

      </FieldMap>

    </FieldMaps>

    <ValueMaps>

      <ValueMap name="ReasonDefault">

        <Value LeftValue="*" RightValue="New" />

      </ValueMap>

      <ValueMap name="StateMap">

        <Value LeftValue="Submitted" RightValue="Active" />

      </ValueMap>

      <ValueMap name="PriorityMap">

        <Value LeftValue="1-Resolve Immediately" RightValue="1" />

        <Value LeftValue="2-Give High Attention" RightValue="2" />

        <Value LeftValue="3-Normal Queue" RightValue="3" />

        <Value LeftValue="4-Low Priority" RightValue="3" />

      </ValueMap>

    </ValueMaps>

       </WITSessionCustomSetting>

          </SettingXml>

          <SettingXmlSchema />

        </CustomSettings>

        <Filters>

          <FilterPair Neglect="false">

            <FilterItem MigrationSourceUniqueId="a14dc073-e3da-4cc4-b03e-fdc1a9caa347" FilterString="Defect::id='UCM0100002148' or id='UCM0100002149' or id='UCM0100004449' " />

            <FilterItem MigrationSourceUniqueId="21761303-70c2-4d6f-84bc-8d58e07581d4" FilterString="[System.Id] = 0" />

          </FilterPair>

        </Filters>

      </Session>

    </Sessions>

    <Linking>

      <CustomSettings />

      <LinkTypeMappings>

        <LinkTypeMapping LeftMigrationSourceUniqueId="a14dc073-e3da-4cc4-b03e-fdc1a9caa347" LeftLinkType="ClearQuestAdapter.LinkType.Duplicate" RightLinkType="Microsoft.TeamFoundation.Migration.TFS.LinkType.WorkItemToWorkItem" RightMigrationSourceUniqueId="21761303-70c2-4d6f-84bc-8d58e07581d4" />

      </LinkTypeMappings>

    </Linking>

    <WorkFlowType Frequency="ContinuousManual" DirectionOfFlow="Unidirectional" SyncContext="Disabled" />

    <UserIdentityMappings EnableValidation="true">

      <UserIdentityLookupAddins>

        <UserIdentityLookupAddin>cdde6b6b-72fc-43b6-bbd1-b8a89a788c6f</UserIdentityLookupAddin>

      </UserIdentityLookupAddins>

      <AliasMappings DirectionOfMapping="LeftToRight">

        <AliasMapping Left="*" Right="*" MappingRule="SimpleReplacement" />

        <AliasMapping Left="" Right="testuser" MappingRule="SimpleReplacement" />

      </AliasMappings>

    </UserIdentityMappings>

  </SessionGroup>

</Configuration>

Here is a Domain to Domain Alias (TFS to TFS) Mapping Example:

<?xml version="1.0"?>

<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" UniqueId="bb4daf36-a590-473a-8500-1e1bfb3ed532" FriendlyName="DomainAliasMappingTest">

  <Providers>

    <Provider ReferenceName="2F82C6C4-…" FriendlyName="TFS 2008 VC Migration Provider" />

    <Provider ReferenceName="FEBC091F-…" FriendlyName="TFS 2010 VC Migration Provider" />

  </Providers>

  <Addins />

  <SessionGroup FriendlyName="Session Group Friendly Name" SessionGroupGUID="352f930f-…"

                SyncIntervalInSeconds="0" SyncDurationInMinutes="0">

    <MigrationSources>

      <MigrationSource InternalUniqueId="436c872b-…" FriendlyName="TFS 2010 source" 

                       ServerIdentifier="TFS 2010" ServerUrl="serverurl" SourceIdentifier="Test1" 

                        ProviderReferenceName="FEBC091F-…">

        <Settings>

          <DefaultUserIdProperty UserIdPropertyName="DomainAlias" />

        </Settings>

      </MigrationSource>

      <MigrationSource InternalUniqueId="b6639fe5-…" FriendlyName="TFS 2010 target" 

                       ServerIdentifier="TFS 2010" ServerUrl="serverurl" SourceIdentifier="Test2" 

                       ProviderReferenceName="FEBC091F-…">

        <Settings>

          <DefaultUserIdProperty UserIdPropertyName="DomainAlias" />

        </Settings>

      </MigrationSource>

    </MigrationSources>

    <Sessions>

      <Session SessionUniqueId="d7dae825-…" FriendlyName="Version Control session"

               LeftMigrationSourceUniqueId="436c872b-…" RightMigrationSourceUniqueId="b6639fe5-…" 

               SessionType="VersionControl">

        <Filters>

          <FilterPair Neglect="false">

            <FilterItem MigrationSourceUniqueId="436c872b-…" FilterString="$/T1/DomainAliasMappingTest" />

            <FilterItem MigrationSourceUniqueId="b6639fe5-…" FilterString="$/T2/DomainAliasMappingTest" />

          </FilterPair>

        </Filters>

      </Session>

    </Sessions>

    <Linking>

      <CustomSettings />

      <LinkTypeMappings />

    </Linking>

    <WorkFlowType Frequency="ContinuousManual" DirectionOfFlow="Unidirectional" 

                  SyncContext="Unidirectional" />

    <UserIdentityMappings EnableValidation="false">

      <UserIdentityLookupAddins>

        <UserIdentityLookupAddin>CDDE6B6B-…</UserIdentityLookupAddin>

      </UserIdentityLookupAddins>

      <AliasMappings DirectionOfMapping="LeftToRight">

        <AliasMapping Left="*" Right="*" MappingRule="SimpleReplacement" />

      </AliasMappings>

      <DomainMappings DirectionOfMapping="LeftToRight">

        <DomainMapping Left="s-domain" Right="t-domain" MappingRule="SimpleReplacement" />

      </DomainMappings>

    </UserIdentityMappings>

  </SessionGroup>

</Configuration>

Note the DefaultUserIdProperty (line 22) and DomainMappings (lines 51-53) configuration elements, which are different from the previous example.

A bit more information on some of the configuration elements

UserIdPropertyName

MappingRules (enum) Description

DisplayName Users display name, which specific to TFS WIT migration sources

Domain Users domain name.

Alias Users alias name.

DomainAlias Users fully qualified domain alias name, which specific to TFS VC migration sources

EmailAddress Users email address.

UniqueId Users security identifier (SID).

QualifiedName Users distinguished name. (CN=Jeff Smith,OU=Sales,DC=Fabrikam,DC=COM)

MappingRule

MappingRules (enum) Description

SimpleReplacement Simply replace the field value as shown in our example configuration file. Note that Line 109, without a wildcard, takes precedence over line 109, replacing all fields with no value in the source with testuser on the target.

FormatStringComposition See What is the Lookup Service? Q&A-27 Errata 1

FormatStringDecomposition See What is the Lookup Service? Q&A-27 Errata 1

Ignore Ignore the field and perform no mapping.

NOTE: We recommend that you refer to the latest documentation that is included with the TFS Integration Platform for the latest configuration features and rules.

Guidance

Comments

Jaleel

6 Jul 2010 8:52 AM

Hi Willys,

Thanks for the nice post.

I have scenario like where I want to migrate TFS 2010 from one server to another server. Here the domain names are different in both servers. Moreover, we plan to chnage the user names. e.g. the user name , oldusername in Domain1 will be newusername in Domain2. I am primarily concerned about what would happen to the history and Work items assignd to oldusername when migrate to new domain with new user name.Can I use this Lookup Service to tackle this scenario?

Appreciate your help..

Thanks,

Jaleel
Tri Pham

30 Aug 2011 3:51 PM

We're about to go into a migration scenario similar to what Jaleel described above, I'm very interested in hearing any response to the question or any update on Jaleel.

Thanks

-Tri
Willy-P. Schaub

30 Aug 2011 4:45 PM

The Lookup Service can be used to define rules that can easily express a relationship for thousands of users in a cross-domain move, allows you to map the display name instead of domain/user pairs; and also allows you to disable the Active Directory search to enforce a static user transformation. These scenarios are documented in the configuration guide. During the migration the user details are changed as defined by the configuration, both for WIT and VC.

I recommend that you raise your issues and/or questions in the social.msdn.microsoft.com/.../tfsintegration to avoid delays in future. My appologies for mssing this comment for so long :(
Willy-P. Schaub

30 Aug 2011 4:49 PM

By the way, thanks Tri Pham ... your comment alerted me in terms of the missing response.

MappingRules (enum)	Description
DisplayName	Users display name, which specific to TFS WIT migration sources
Domain	Users domain name.
Alias	Users alias name.
DomainAlias	Users fully qualified domain alias name, which specific to TFS VC migration sources
EmailAddress	Users email address.
UniqueId	Users security identifier (SID).
QualifiedName	Users distinguished name. (CN=Jeff Smith,OU=Sales,DC=Fabrikam,DC=COM)

MappingRules (enum)	Description
SimpleReplacement	Simply replace the field value as shown in our example configuration file. Note that Line 109, without a wildcard, takes precedence over line 109, replacing all fields with no value in the source with testuser on the target.
FormatStringComposition	See What is the Lookup Service? Q&A-27 Errata 1
FormatStringDecomposition	See What is the Lookup Service? Q&A-27 Errata 1
Ignore	Ignore the field and perform no mapping.

Willy's Reflections

TFS Integration Platform – What is the Lookup Service? Q&A-27