Jimmie's Sharings : Profile Import

“Hidden” SSP Timer Jobs

jimmiet — Sun, 20 Apr 2008 17:21:00 GMT

Not all timer jobs are visible in the Central Administration timer job definition page. There are MOSS 2007 timer jobs in the SSP application which don’t appear in the Central Administration page. It makes sense that these jobs are not visible, since there is nothing you can modify or disable. All the same, it would be nice to know what these jobs are, and what their schedules are, in case you want to schedule other potentially conflicting activities.

You can see the names of these SSP jobs by using stsadm enumssptimerjobs command, as in the following console sample (view entire article ...)

User Profile Change Logging

jimmiet — Sat, 29 Mar 2008 23:56:00 GMT

Profile Change Logging

There is a table in the SSP database named UserProfileEventLog. The table maintains a history of user profile property changes. By default, it retains 7 days of history. This table can cause problems in a couple of ways.

First issue: size. This table contains one row per change of a property in a user profile. The row contains the old and new property values, along with associated metadata like the datetime the property was changed. The implication is that the table can grow large. Assume you just configured the profile import and are ready to start the first Active Directory import. Further, assume your organization has 100,000 user accounts, and each account be populated with 12 AD attributes. The full import will result in 100,000 X 12 = 1.2 millions rows being inserted into UserProfileEventLog. To extend the example, assume you also have a BDC import connection populating another 20 properties from your company's HR system. That adds another 2 million rows. There are now 3.2 million rows in the event log table. If each row is 100 bytes (old value, new value, plus metadata), the table is now approximately 300 MB in size.

Second issue: deleting old entries. The change history is kept for a configurable number of days. The concern is how many rows will have to be deleted on a given day? The number of days of history is 7 days be default, but can be set using stsadm.exe -o profilechangelog -title <SSP Name> -daysofhistory <number of days> -generateanniversaries (http://technet.microsoft.com/en-us/library/cc263013.aspx). The critical issue is that MOSS has to remove a full day of history every day to honor the daysofhistory setting. Using the numbers in the preceding paragraph, 7 days after the first full import, MOSS is going to delete 3.2 millions rows of data all at once! What makes this an issue is that this is done with a single SQL statement, something like DELETE FROM UserProfileEventLog WHERE EventId < @MinEventTime. Think of the implications. As a single statement, this will hold locks until all 3.2 million rows are deleted. These locks might prevent other database transactions from completing; but it also means the transaction log (even with simple recovery mode) cannot be truncated, and will therefore grow until at least this DELETE statement completes. I have seen this delete statement run for 4 hours, with the transaction log quadrupling in size.

This is little you can do to avoid this. Although you can adjust the number of days of history, MOSS will eventually try to delete an entire day's history at some point. The deletion is done by an internal timer job buried within the SSP. The job is hard coded to run at 10:00 PM daily. You cannot change this scheduled time.

What are the take-aways?

1. Be prepared for an occasional long running DELETE statement. Ensure you have sufficient transaction log space to accommodate the potential transaction log growth.
2. Don't schedule any other timer jobs or database maintenance for 10:00 PM to minimize possible deadlocks and transaction timeouts.
3. Be careful using stsadm -o profilechangelog to reduce the number of days of history, because the next time 10:00 PM comes, MOSS will try to delete multiple days of history all at once.

Problems Importing Profile Data from BDC

jimmiet — Tue, 04 Mar 2008 18:40:00 GMT

By default MOSS imports data from a Master Connection first, which is typically Active Directory but could be another LDAP source. Once a successful master connection import completes, it is possible to import additional data from one or more BDC data sources, also known as supplemental data sources. This is very common, since many organizations keep the bulk of the interesting employee information in HR systems outside of Active Directory. Often these HR systems more accurate and up-to-date than Active Directory, and are therefore considered the official system of record for personnel data. Refer to Importing User Profiles on MSDN for additional information on profile importing.

A relationship between the Master Connection data and the entity methods in the BDC application definition must be established. This is done with an Identifier field. The Identifier is populated by the Master Connection, and then used as the foreign key when executing the BDC methods. In this picture, AccountName is the foreign key; however, most commonly this is a numeric value like PersonnelNumber. (view entire article ...)