Welcome to MSDN Blogs Sign in | Join | Help

Migrating Wiki Pages Remotely – Part 09

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

One of the final touches was to take care of the “but this data here is out of place”, or “this string still has the old server”, etc type complaints.  You may have noticed this, but I wanted to point it out separately (since this works out to be a fall back plan in the code).  When I do the link fix-up button (yet another reason for separating the operations into two different buttons), I looked to see if the Adv Repl (advanced replace) was checked and processed those values—in case you missed it, it was here:

if (chk_AdvRepl.Checked)

{

    modifiedData = modifiedData.Replace(txt_Repl1.Text, txt_Repl2.Text);

}

This was significant because it allows the admin to have a search and replace style tweak.  Ideally this kind of code should never be placed in the hands of an end user—but if you polished it up a bit, it could become an effective tool in allowing folks to make bulk search/replace type changes across wikis.  Dangerous—but hey, that’s up to the implementer to control (that’s you! J).

Another thing that may be of interest was the:

// Kill the linefeeds and \\ because they will be literal since we have to use CDATA.

modifiedData = modifiedData.Replace(@"\r\n", "");

 

I found that when built up the CDATA for use in UpdateListItems from lists.asmx here:

 

string strBatch = "<Method ID='1' Cmd='Update'>" +

    "<Field Name='ID'>" + item.Attributes["ows_ID"].Value + "</Field>" +

    "<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +

    "<Field Name='_CopySource'></Field></Method>";

 

That the call to UpdateListItems would suceed, but I would have massive numbers of \r\n littered throughout the wiki pages.  The replace I was doing there was to counter that and could possibly be destructive, but generally, I did not find it to be.

 

Well that’s all of my thoughts for now.  I hope this wasn’t too excessively long.  It is a new style for me that I wanted to try out—if it is well received I will try it again sometime.  If you’d prefer smaller nuggets—I can try for that to.  I am open to suggestions.

 

Thanks for reading!  You can find the information about the source in Part 10.

Part 10:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-10.aspx

Posted by dwinter | 2 Comments

Migrating Wiki Pages Remotely – Part 08

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

So there are a lot of little things to think about here.  Once you start playing with this—the next one that comes to mind is… what about the images?  It is a good question.  I found in practice that most images would end up in the same Document Library or Images Library for any given segment of Wiki data created by a single person.  However, it can vary… so I worked in something to parse where images are being used and report it back so that the admin knows they need to copy the images as well.  It would be doable to automatically take care of these, but it was out of spec for what I wanted since if told what Document Libraries to grab, I could easily copy everything through explorer view to a list with the same name on the other side.  At the beginning of my copy method I declared:

System.Collections.Hashtable imageLibraries = new System.Collections.Hashtable();


Here’s my code for image detection in the wiki pages:

// Try to find image tags and note their lists

MatchCollection imageMatches = Regex.Matches(sourceWikiField, "<img.*src=\"(.*)\">", RegexOptions.IgnoreCase);

foreach (Match imageMatch in imageMatches)

{

    MatchCollection srcMatches = Regex.Matches(sourceWikiField, "src=\"(.*)\"", RegexOptions.IgnoreCase);

    foreach (Match srcMatch in srcMatches)

    {

        string imageUrl = srcMatch.Value.TrimStart("src=".ToCharArray()).TrimStart("\"".ToCharArray()).TrimEnd("\"".ToCharArray());

        int lastSlash = imageUrl.LastIndexOf("/");

        string imageParentUrl = string.Empty;

        if (lastSlash > 0)

        {

            imageParentUrl = imageUrl.Substring(0, lastSlash - 1);

        }

        Trace.WriteLine("Found image: " + imageUrl);

        if (!string.IsNullOrEmpty(imageParentUrl))

        {

            if (!imageLibraries.Contains(imageParentUrl))

            {

                imageLibraries.Add(imageParentUrl, string.Empty);

                Trace.WriteLine("Added Library: " + imageParentUrl);

            }

        }

    }

}

Then at the end I just loop through the imageLibraries collection and reported it back to the admin.  I thought this was slick because I only would see a distinct list of libraries and not every image in use.  Take a look here:

if (imageLibraries.Count > 0)

{

    Trace.WriteLine("===============================================");

    Trace.WriteLine("The following location(s) contain images used by the coppied wikis.  They should be coppied manually to new locations matching the structure from:");

    Trace.WriteLine("");

    txt_Status.Text += "===============================================\r\nThe following location(s) contain wiki images to be coppied:\r\n";

    txt_Status.Select(txt_Status.Text.Length, 0);

    txt_Status.ScrollToCaret();

    foreach (System.Collections.DictionaryEntry imageLibrary in imageLibraries)

    {

        Trace.WriteLine(imageLibrary.Key);

        txt_Status.Text += imageLibrary.Key + "\r\n";

        txt_Status.Select(txt_Status.Text.Length, 0);

        txt_Status.ScrollToCaret();

    }

    Trace.WriteLine("");

    txt_Status.Text += "\r\n";

    txt_Status.Select(txt_Status.Text.Length, 0);

    txt_Status.ScrollToCaret();

}

 

Part 09:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-09.aspx

Posted by dwinter | 3 Comments

Migrating Wiki Pages Remotely – Part 07

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

Now let us consider the next potential problem in using copy.asmx. There were some instances where I found that the copy.asmx would fail outright. One of these was when I would go between significant schema versions. In my case, I was going between a very early version of o14 and SharePoint 2007. The problem here was that copy.asmx grabs a binary copy of the current object which has all of the information embedded in it. This was a problem because I didn’t have the o14 assemblies (14.x.x.x instead of 12.x.x.x) on my destination server. I will preface the rest of this commentary with the thought that by the time we launch o14, this may not be a problem at all (no promises on what will or won't be in o14 will be comming from this blog--sorry). However, my approach in dealing with the problem is still valuable to share, and that is why I decided to put it here. Here is how I dealt with it:  Going back to the initial code where we successfully got the data from the source server’s list.asmx, I did that again.  However, for this special case scenario, I decided it was appropriate to make the destination a local OM destination.  Then I could run the tool again and go web service to web service form my local server over to my actual destination.  Yes, this is an extra set—but it still allowed me to not have to have access to either of the production servers locally.  This code path is triggered in my code by the DestLocal checkbox.  Here’s what that code looks like:

private bool manualLocalCopy(string sourceWikiField, string itemName, WikiMigrator.Server2CopyWS.FieldInformation[] myFieldInfoArray2, byte[] myByteArray, string[] copyDest, bool copySuccess)

{

    bool manualSuccess = false;

    if (!string.IsNullOrEmpty(sourceWikiField))

    {

        try

        {

            string modifiedData = sourceWikiField.Replace(@"\r\n", "");

            modifiedData = modifiedData.Replace(@"\\", @"\");

                // You need a try-catch block because new SPSite(), OpenWeb(), and GetList() all throw on failure.

 

            try

            {

                // open site, web, list, and file collection

                SPSite site = new SPSite(txt_SiteName2.Text);

                SPWeb web = site.OpenWeb();

                SPList list = web.Lists[txt_SelectedWiki2.Text];

                SPFileCollection files = list.RootFolder.Files;

                SPFile newFile = null;

                try

                {

                    // add new wiki page

                    newFile = files.Add(txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName, SPTemplateFileType.WikiPage);

                }

                catch (Exception exc)

                {

                    newFile = files[itemName];

                }

                // get the list item corresponding to the wiki page and update its content

                SPListItem item = newFile.Item;

                item["WikiField"] = sourceWikiField;

                item.Update();

                manualSuccess = true;

                copySuccess = true;

            }

            catch (Exception exc)

            {

                if (exc.Message.Contains("-2147024816"))

                {

                    Trace.WriteLine("File exists in target");

                }

                else

                {

                    Trace.WriteLine("manualLocalCopy Exception: " + exc.Message);

                }

            }

        }

        catch (Exception exc)

        {

            Trace.WriteLine("***ERROR*** Manual copy failed " + exc.Message);

        }

    }

    if (!manualSuccess)

    {

        Trace.WriteLine("Manual copy of " + itemName + " failed.");

        txt_Status.Text += "Manual copy of " + itemName + " failed.\r\n";

        txt_Status.Select(txt_Status.Text.Length, 0);

        txt_Status.ScrollToCaret();

    }

    else

    {

        Trace.WriteLine("Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName);

        txt_Status.Text += "Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName + "\r\n";

        txt_Status.Select(txt_Status.Text.Length, 0);

        txt_Status.ScrollToCaret();

        copySuccess = true;

    }

    return copySuccess;
} 

Part 08:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-08.aspx

Posted by dwinter | 3 Comments

Migrating Wiki Pages Remotely – Part 06

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

Now that that is done, it is time to consider some potential difficulties of this approach. First, when you use the copy.asmx, the destination file will automatically gain a property called _CopySource that points back to the source item—making a dependency. When you browse the new destination page, it will have a link at the top noting that it is a copy of the other Wiki Page and provide a link to it. In my scenario, I didn’t want this because my source server was going to be going away—and that is not something that would automatically have cleaned up. So, I had to add a step of cleaning _CopySource after the copy operation was complete. I found it was more logical to copy everything first and then fix up the links second with a separate button. The good news is that it works perfectly and will have no side effects. If you clear _CopySource—you are setting it to the same value that a normal Wiki Page would have, so it is as though the copy operation through the copy.asmx never happened and you just manually had created the content. Here is the code to clean up _CopySource:

if (txt_SelectedWiki2.Text.Length > 0)

{

    Server2WS.Lists s2L = new WikiMigrator.Server2WS.Lists();

    s2L.Url = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/lists.asmx";

    s2L.Credentials = System.Net.CredentialCache.DefaultCredentials;

    try

    {

        XmlDocument xmlDocLI = new System.Xml.XmlDocument();

        XmlNode ndQueryLI = xmlDocLI.CreateNode(XmlNodeType.Element,"Query","");

        XmlNode ndViewFieldsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"ViewFields","");

        XmlNode ndQueryOptionsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"QueryOptions","");

 

        ndQueryOptionsLI.InnerXml = "<IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns>" +

            "<DateInUtc>TRUE</DateInUtc>";

        ndViewFieldsLI.InnerXml = "<FieldRef Name='FileRef' />" +

            "<FieldRef Name='WikiField' />" +

            "<FieldRef Name='LinkFilename' />";

        ndQueryLI.InnerXml = "<Where><Eq><FieldRef Name='FileRef' />" +

            "<Value Type='Text'>[server-relative URL of wiki page]</Value></Eq></Where>";

        //

        // Need to provide a large number or we will restrict at the default 100 items if null

        XmlNode ndListItems = s2L.GetListItems(txt_SelectedWiki2.Text, null, null, ndViewFieldsLI, txtNumberRows.Text, null, null);

        XmlNode ndListItemDetail = ndListItems.ChildNodes[1];

        foreach (XmlNode item in ndListItemDetail.ChildNodes)

        {

            try

            {

                if (item.Attributes != null)

                {

                    string itemName = item.Attributes["ows_LinkFilename"].Value;

                    Trace.WriteLine("Fixing: " + itemName);

                    if (!string.IsNullOrEmpty(itemName))

                    {

                        string copySource = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki.Text.Trim().TrimEnd("/".ToCharArray()).TrimStart("/".ToCharArray()) + "/" + itemName;

                        string copyDest = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.Trim().TrimEnd("/".ToCharArray()).TrimStart("/".ToCharArray()) + "/" + itemName;

 

                        string wikiData = item.Attributes["ows_WikiField"].Value;

                        string actualWikiData = string.Empty;

                        if (string.IsNullOrEmpty(wikiData))

                        {

                            Trace.WriteLine("...using ows_MetaInfo instead of ows_WikiField");

                            string itemData = item.Attributes["ows_MetaInfo"].Value;

                            Regex metaProp = new Regex(@"( \w*:\w{2}\|)");

                            string[] regexData = metaProp.Split(itemData);

                            bool prepnextMatch = false;

                            foreach (string data in regexData)

                            {

                                try

                                {

                                    if (data != string.Empty)

                                    {

                                        if (!prepnextMatch)

                                        {

                                            if (data == " WikiField:SW|")

                                            {

                                                prepnextMatch = true;

                                            }

                                        }

                                        else if (prepnextMatch && actualWikiData == string.Empty)

                                        {

                                            actualWikiData = data;

                                            break;

                                        }

                                        else

                                        {

                                            throw new System.Exception("E_FAIL");

                                        }

                                    }

                                }

                                catch {}

                            }

                        }

                        else

                        {

                            actualWikiData = wikiData;

                        }

                        // Locals for replacement operations

                        string sitename = txt_SiteName.Text.TrimEnd("/".ToCharArray());

                        string destsite = txt_SiteName2.Text.TrimEnd("/".ToCharArray());

                        string wiki1 = txt_SelectedWiki.Text.TrimStart("/".ToCharArray());

                        string wiki2 = txt_SelectedWiki2.Text.TrimStart("/".ToCharArray());

                        Uri sourceSiteUri = new Uri(sitename);

                        Uri destSiteUri = new Uri(destsite);

                        string sourceAbsolute = sourceSiteUri.AbsolutePath.TrimEnd("/".ToCharArray());

                        string destAbsolute = destSiteUri.AbsolutePath.TrimEnd("/".ToCharArray());

                        

                        //  http://server1/site/library  to  http://server2/newsite/newlibrary

                        string modifiedData = Regex.Replace(actualWikiData, Regex.Escape(sitename + "/" + wiki1), Regex.Escape(destsite + "/" + wiki2), RegexOptions.IgnoreCase);

                        //  http://server1   to  http://server2

                        modifiedData = Regex.Replace(modifiedData, Regex.Escape(sitename), Regex.Escape(destsite), RegexOptions.IgnoreCase);

                        //   /site/library   to  /newsite/newlibrary   (+ Encoded)

                        modifiedData = Regex.Replace(modifiedData, Regex.Escape(sourceAbsolute + "/" + wiki1), Regex.Escape(destAbsolute + "/" + wiki2), RegexOptions.IgnoreCase);

                        modifiedData = Regex.Replace(modifiedData, Regex.Escape(sourceAbsolute + "/" + Uri.EscapeDataString(wiki1)), Regex.Escape(destAbsolute + "/" + Uri.EscapeDataString(wiki2)), RegexOptions.IgnoreCase);

                        //   /site  to  /newsite

// This is very dangerous and is commented because of it...

// since source could be '/' and if it actually is, we would

// replace every / in the doc

                        //if (sourceSiteUri.AbsolutePath != "/")

                        //{

                        //    modifiedData = modifiedData.Replace(sourceSiteUri.AbsolutePath, destSiteUri.AbsolutePath);

                        //    modifiedData = Regex.Replace(modifiedData, Regex.Escape(), Regex.Escape(), RegexOptions.IgnoreCase);

                        //}

                        // Kill the linefeeds and \\ because they will be literal since we have to use CDATA.  This could flatten \\server\share links to \server\share in the text, but the links should still work.  You could do something more elegant here to protect against that.

                        modifiedData = modifiedData.Replace(@"\r\n", "");

                        modifiedData = modifiedData.Replace(@"\\", @"\");

                        if (chk_AdvRepl.Checked)

                        {

                            modifiedData = modifiedData.Replace(txt_Repl1.Text, txt_Repl2.Text);

                        }

 

                        string strBatch = "<Method ID='1' Cmd='Update'>" +

                            "<Field Name='ID'>" + item.Attributes["ows_ID"].Value + "</Field>" +

                            "<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +

                            "<Field Name='_CopySource'></Field></Method>";

 

                        if (chkDebugFull.Checked)

                        {

                            Trace.WriteLine("***** " + itemName + " *****");

                            Trace.WriteLine(strBatch);