Be sure to use paging in membership providers for the MOSS Profile Import Tool...

bgeoffro — Fri, 28 Mar 2008 01:14:00 GMT

Just a quick lesson learned on the MOSS Profile Import Tool to save you import time and resources.

The Profile Import Tool's current release has a hard-coded paging algorithm that attempts to page through any membership provider's GetAllUsers method by retrieving 1000 users at a time. If you DON'T support paging your GetAllUsers method - meaning you try to retrieve all users at once - then the import algorithm will retrieve and import all users N times, where N = (# users / 1000) . This becomes totally unmanageable - but a really good test for memory leaks in your provider - if you have many thousands of users... :)

The best practice is (if able) to add paging to your membership provider, since that will be useful in all calls to the GetAllUsers, FindUsersByName, and FindUsersByEmail methods, all of which will be used by SharePoint.

If you feel the need to alter this aspect of the Profile Import Tool's behavior - i.e. you have 100K users and don't want to take the time to build paging logic into your provider, or don't have access to the membership provider's source code - you'll find the paging logic in the ImportProvider.cs class. All you have to do is change the value of the pageSize constant to a sufficiently large value (i.e. 101000 if you have 100000 users), and you'll bypass the paging:

private const int pageSize = 1000;

If you choose to increase the pageSize value, make sure you have enough RAM to hold all those user records... and don't run the profile import application on a server that's hosting a RAM-intensive role like Query, Web App, or SQL.

MOSS User Profile Delete Tool

bgeoffro — Tue, 18 Mar 2008 17:01:00 GMT

For anyone using a custom authentication provider or non AD/LDAP* directory service, the Profile Import tool at http://www.codeplex.com/sptoolbox is an awesome tool for adding users. Just plug in your provider(s), tweak a few configuration settings, and go. I've used it to add 200,000+ user profiles from my customer's directory service (though I wouldn't recommend attempting to swallow all those records in one gulp - that requires more memory than we had on our import server) to our latest MOSS environment.

Getting the configuration "just right" resulted in my creating lots of extraneous user profiles in the environment. Not wanting to recreate the SSP or build up some serious callouses hitting the "delete" key hundreds of times, I spent quite a bit of time yesterday searching for a tool for doing mass deletes of user profiles, to no avail. I did, however, find a number of postings for other folks out there looking for a similar tool.

So I put together a quick-and-dirty command-line app for nuking your entire user profile DB or a portion thereof based on account name matches. You could modify this pretty easily to include more sophisticated filter criteria, like an existence check in your directory service of choice, but I didn't want to spend more than an hour on the script... :)

* You'd also need to use the Profile Import tool if you need to do an authenticated bind to your LDAP service using something other than Windows credentials.

Configuration File:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
  <add key="test" value="false"/>
  <add key="debug" value="true"/>
  <add key="filterPattern" value="qaldap:"/>
</appSettings>
</configuration>

Configuration Comments:

Here's how to use the config settings:

filterPattern

case-insensitive string used for a "starts with" match on user profile Account Names - i.e. "myprovider:" would be a match for all user profiles that use the provider "myprovider", or "mydomain\" would match all user profiles from the "mydomain" domain

test - used to control whether delete actions occur

true (default) - user profiles matching the filterPattern are reported as matches for deletion, but no delete actions occur
false - all user profiles matching the filterPattern are deleted

debug

true - application info/warning/error events are written to the console
false (default) - application events are written only to the event log

Source Code:

The code is pretty simple. Main is where the action is for retrieving/deleting user profiles and reading the config file; if you need to plug in a different algorithm for identifying profiles to be deleted (like an existence check in your directory service), you'd add it here. GetAnySiteUrl is copied directly from the Profile Import tool to save coding time in establishing a server context; if you have multiple SSPs, you might want to update the way you establish a context.

If you're using only 1 SSP and are OK with the simple "starts with" pattern match algorithm, just put the right references in your project, build it, and you're ready to go. Enjoy!

using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Text;
using System.Diagnostics;
using System.Configuration;

using Microsoft.Office.Server;
using Microsoft.Office.Server.UserProfiles;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;

namespace ProfileDelete
{
    class Program
    {
        static int logLevel = 1;
        static bool debug = false;
        static bool test = true;
        static string logSource = "ProfileDelete";
        static string logDestination = "Application";
        static string filterPattern = "";

        static void Main(string[] args)
        {
            int counter = 0;

            try
            {
                Configuration config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);

                NameValueCollection appSettings = ConfigurationManager.AppSettings;
                logLevel = Convert.ToInt32(appSettings["loglevel"]);
                debug = Convert.ToBoolean(appSettings["debug"]);
                test = Convert.ToBoolean(appSettings["test"]);
                filterPattern = appSettings["filterPattern"];
            }
            catch (Exception e)
            {
                DoLog("Error reading configuration file:" + e.ToString(), EventLogEntryType.Error, 100);
            }

string currentAccountName = String.Empty;

using (SPSite site = new SPSite(GetAnySiteUrl()))
{

ServerContext context = ServerContext.GetContext(site);

UserProfileManager profileManager = new UserProfileManager(context);

                foreach (UserProfile profile in profileManager)
                {
                    currentAccountName = (String) profile[PropertyConstants.AccountName].Value;
                    currentAccountName = currentAccountName.ToLower();

                    if (currentAccountName.StartsWith(filterPattern.ToLower()))
                    {
                        counter++;
                        DoLog(currentAccountName + " matches delete criteria. Count: " + counter, EventLogEntryType.Information, 110);
                        if (!test)
                        {
                            DoLog("Deleting " + currentAccountName, EventLogEntryType.Information, 120);
                            profileManager.RemoveUserProfile(profile.ID);
                        }
                    }
                }
            }

}

        private static string GetAnySiteUrl()
        {
            //the purpose of this function is just to get any site Url so that
            //we can obtain a ServerContext for the UserProfileManager
            string ret = string.Empty;

SPWebServiceCollection wsc = null;

            try
            {
                wsc = new SPWebServiceCollection(SPFarm.Local);
                foreach (SPWebService sw in wsc)
                {
                    foreach (SPWebApplication wa in sw.WebApplications)
                    {
                        if (wa.Sites.Count > 0)
                        {
                            ret = wa.Sites[0].Url;
                            break;
                        }
                        if (!string.IsNullOrEmpty(ret))
                            break;
                    }
                    if (!string.IsNullOrEmpty(ret))
                        break;
                }
            }
            catch (Exception ex)
            {
                throw new Exception("Error getting a site Url: " + ex.Message);
            }
            finally
            {
                wsc = null;
            }

return ret;
}

        private static void DoLog(string msg, EventLogEntryType eventType, int code)
        {
            bool writeEntry = true;
            if (debug) Console.WriteLine("Event Type: {0}; ID: {1}; Message: {2}", eventType.ToString(), code, msg);

            if (logLevel > 0)
            {
                if (!EventLog.SourceExists(logSource))
                    EventLog.CreateEventSource(logSource, logDestination);
                if (eventType.Equals(EventLogEntryType.Warning) && (logLevel < 2))
                    writeEntry = false;
                if (eventType.Equals(EventLogEntryType.Information) && (logLevel < 3))
                    writeEntry = false;
                if (writeEntry) EventLog.WriteEntry(logSource, msg, eventType, code);
            }
        }

}
}

Brett's SharePoint Blog : user profiles

Be sure to use paging in membership providers for the MOSS Profile Import Tool...

MOSS User Profile Delete Tool