dwinter's [MSFT] WebLog : WikiMigrator

Migrating Wiki Pages Remotely – Part 10

dwinter — Sat, 28 Jun 2008 08:55:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

Here is the full project from this effort:
http://www.codeplex.com/WikiMigrator

Migrating Wiki Pages Remotely – Part 09

dwinter — Sat, 28 Jun 2008 08:53:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

One of the final touches was to take care of the “but this data here is out of place”, or “this string still has the old server”, etc type complaints. You may have noticed this, but I wanted to point it out separately (since this works out to be a fall back plan in the code). When I do the link fix-up button (yet another reason for separating the operations into two different buttons), I looked to see if the Adv Repl (advanced replace) was checked and processed those values—in case you missed it, it was here:

if (chk_AdvRepl.Checked)

{

modifiedData = modifiedData.Replace(txt_Repl1.Text, txt_Repl2.Text);

}

This was significant because it allows the admin to have a search and replace style tweak. Ideally this kind of code should never be placed in the hands of an end user—but if you polished it up a bit, it could become an effective tool in allowing folks to make bulk search/replace type changes across wikis. Dangerous—but hey, that’s up to the implementer to control (that’s you! J).

Another thing that may be of interest was the:

// Kill the linefeeds and \\ because they will be literal since we have to use CDATA.

modifiedData = modifiedData.Replace(@"\r\n", "");

I found that when built up the CDATA for use in UpdateListItems from lists.asmx here:

string strBatch = "<Method ID='1' Cmd='Update'>" +

"<Field Name='ID'>" + item.Attributes["ows_ID"].Value + "</Field>" +

"<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +

"<Field Name='_CopySource'></Field></Method>";

That the call to UpdateListItems would suceed, but I would have massive numbers of \r\n littered throughout the wiki pages. The replace I was doing there was to counter that and could possibly be destructive, but generally, I did not find it to be.

Well that’s all of my thoughts for now. I hope this wasn’t too excessively long. It is a new style for me that I wanted to try out—if it is well received I will try it again sometime. If you’d prefer smaller nuggets—I can try for that to. I am open to suggestions.

Thanks for reading! You can find the information about the source in Part 10.

Part 10:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-10.aspx

Migrating Wiki Pages Remotely – Part 08

dwinter — Sat, 28 Jun 2008 08:52:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

So there are a lot of little things to think about here. Once you start playing with this—the next one that comes to mind is… what about the images? It is a good question. I found in practice that most images would end up in the same Document Library or Images Library for any given segment of Wiki data created by a single person. However, it can vary… so I worked in something to parse where images are being used and report it back so that the admin knows they need to copy the images as well. It would be doable to automatically take care of these, but it was out of spec for what I wanted since if told what Document Libraries to grab, I could easily copy everything through explorer view to a list with the same name on the other side. At the beginning of my copy method I declared:

System.Collections.Hashtable imageLibraries = new System.Collections.Hashtable();

Here’s my code for image detection in the wiki pages:

// Try to find image tags and note their lists

MatchCollection imageMatches = Regex.Matches(sourceWikiField, "<img.*src=\"(.*)\">", RegexOptions.IgnoreCase);

foreach (Match imageMatch in imageMatches)

{

MatchCollection srcMatches = Regex.Matches(sourceWikiField, "src=\"(.*)\"", RegexOptions.IgnoreCase);

foreach (Match srcMatch in srcMatches)

{

string imageUrl = srcMatch.Value.TrimStart("src=".ToCharArray()).TrimStart("\"".ToCharArray()).TrimEnd("\"".ToCharArray());

int lastSlash = imageUrl.LastIndexOf("/");

string imageParentUrl = string.Empty;

if (lastSlash > 0)

{

imageParentUrl = imageUrl.Substring(0, lastSlash - 1);

}

Trace.WriteLine("Found image: " + imageUrl);

if (!string.IsNullOrEmpty(imageParentUrl))

{

if (!imageLibraries.Contains(imageParentUrl))

{

imageLibraries.Add(imageParentUrl, string.Empty);

Trace.WriteLine("Added Library: " + imageParentUrl);

}

Then at the end I just loop through the imageLibraries collection and reported it back to the admin. I thought this was slick because I only would see a distinct list of libraries and not every image in use. Take a look here:

if (imageLibraries.Count > 0)

{

Trace.WriteLine("===============================================");

Trace.WriteLine("The following location(s) contain images used by the coppied wikis. They should be coppied manually to new locations matching the structure from:");

Trace.WriteLine("");

txt_Status.Text += "===============================================\r\nThe following location(s) contain wiki images to be coppied:\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

foreach (System.Collections.DictionaryEntry imageLibrary in imageLibraries)

{

Trace.WriteLine(imageLibrary.Key);

txt_Status.Text += imageLibrary.Key + "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

}

Trace.WriteLine("");

txt_Status.Text += "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

}

Part 09:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-09.aspx

Migrating Wiki Pages Remotely – Part 07

dwinter — Sat, 28 Jun 2008 08:48:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

Now let us consider the next potential problem in using copy.asmx. There were some instances where I found that the copy.asmx would fail outright. One of these was when I would go between significant schema versions. In my case, I was going between a very early version of o14 and SharePoint 2007. The problem here was that copy.asmx grabs a binary copy of the current object which has all of the information embedded in it. This was a problem because I didn’t have the o14 assemblies (14.x.x.x instead of 12.x.x.x) on my destination server. I will preface the rest of this commentary with the thought that by the time we launch o14, this may not be a problem at all (no promises on what will or won't be in o14 will be comming from this blog--sorry). However, my approach in dealing with the problem is still valuable to share, and that is why I decided to put it here. Here is how I dealt with it: Going back to the initial code where we successfully got the data from the source server’s list.asmx, I did that again. However, for this special case scenario, I decided it was appropriate to make the destination a local OM destination. Then I could run the tool again and go web service to web service form my local server over to my actual destination. Yes, this is an extra set—but it still allowed me to not have to have access to either of the production servers locally. This code path is triggered in my code by the DestLocal checkbox. Here’s what that code looks like:

private bool manualLocalCopy(string sourceWikiField, string itemName, WikiMigrator.Server2CopyWS.FieldInformation[] myFieldInfoArray2, byte[] myByteArray, string[] copyDest, bool copySuccess)

{

bool manualSuccess = false;

if (!string.IsNullOrEmpty(sourceWikiField))

{

try

{

string modifiedData = sourceWikiField.Replace(@"\r\n", "");

modifiedData = modifiedData.Replace(@"\\", @"\");

// You need a try-catch block because new SPSite(), OpenWeb(), and GetList() all throw on failure.

try

{

// open site, web, list, and file collection

SPSite site = new SPSite(txt_SiteName2.Text);

SPWeb web = site.OpenWeb();

SPList list = web.Lists[txt_SelectedWiki2.Text];

SPFileCollection files = list.RootFolder.Files;

SPFile newFile = null;

try

{

// add new wiki page

newFile = files.Add(txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName, SPTemplateFileType.WikiPage);

}

catch (Exception exc)

{

newFile = files[itemName];

}

// get the list item corresponding to the wiki page and update its content

SPListItem item = newFile.Item;

item["WikiField"] = sourceWikiField;

item.Update();

manualSuccess = true;

copySuccess = true;

}

catch (Exception exc)

{

if (exc.Message.Contains("-2147024816"))

{

Trace.WriteLine("File exists in target");

}

else

{

Trace.WriteLine("manualLocalCopy Exception: " + exc.Message);

}

catch (Exception exc)

{

Trace.WriteLine("***ERROR*** Manual copy failed " + exc.Message);

}

if (!manualSuccess)

{

Trace.WriteLine("Manual copy of " + itemName + " failed.");

txt_Status.Text += "Manual copy of " + itemName + " failed.\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

}

else

{

Trace.WriteLine("Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName);

txt_Status.Text += "Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName + "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

copySuccess = true;

}

return copySuccess;
}

Part 08:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-08.aspx

Migrating Wiki Pages Remotely – Part 06

dwinter — Sat, 28 Jun 2008 08:47:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

Now that that is done, it is time to consider some potential difficulties of this approach. First, when you use the copy.asmx, the destination file will automatically gain a property called _CopySource that points back to the source item—making a dependency. When you browse the new destination page, it will have a link at the top noting that it is a copy of the other Wiki Page and provide a link to it. In my scenario, I didn’t want this because my source server was going to be going away—and that is not something that would automatically have cleaned up. So, I had to add a step of cleaning _CopySource after the copy operation was complete. I found it was more logical to copy everything first and then fix up the links second with a separate button. The good news is that it works perfectly and will have no side effects. If you clear _CopySource—you are setting it to the same value that a normal Wiki Page would have, so it is as though the copy operation through the copy.asmx never happened and you just manually had created the content. Here is the code to clean up _CopySource:

if (txt_SelectedWiki2.Text.Length > 0)

{

Server2WS.Lists s2L = new WikiMigrator.Server2WS.Lists();

s2L.Url = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/lists.asmx";

s2L.Credentials = System.Net.CredentialCache.DefaultCredentials;

try

{

XmlDocument xmlDocLI = new System.Xml.XmlDocument();

XmlNode ndQueryLI = xmlDocLI.CreateNode(XmlNodeType.Element,"Query","");

XmlNode ndViewFieldsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"ViewFields","");

XmlNode ndQueryOptionsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"QueryOptions","");

ndQueryOptionsLI.InnerXml = "<IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns>" +

"<DateInUtc>TRUE</DateInUtc>";

ndViewFieldsLI.InnerXml = "<FieldRef Name='FileRef' />" +

"<FieldRef Name='WikiField' />" +

"<FieldRef Name='LinkFilename' />";

ndQueryLI.InnerXml = "<Where><Eq><FieldRef Name='FileRef' />" +

"<Value Type='Text'>[server-relative URL of wiki page]</Value></Eq></Where>";

// Need to provide a large number or we will restrict at the default 100 items if null

XmlNode ndListItems = s2L.GetListItems(txt_SelectedWiki2.Text, null, null, ndViewFieldsLI, txtNumberRows.Text, null, null);

XmlNode ndListItemDetail = ndListItems.ChildNodes[1];

foreach (XmlNode item in ndListItemDetail.ChildNodes)

{

try

{

if (item.Attributes != null)

{

string itemName = item.Attributes["ows_LinkFilename"].Value;

Trace.WriteLine("Fixing: " + itemName);

if (!string.IsNullOrEmpty(itemName))

{

string copySource = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki.Text.Trim().TrimEnd("/".ToCharArray()).TrimStart("/".ToCharArray()) + "/" + itemName;

string copyDest = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.Trim().TrimEnd("/".ToCharArray()).TrimStart("/".ToCharArray()) + "/" + itemName;

string wikiData = item.Attributes["ows_WikiField"].Value;

string actualWikiData = string.Empty;

if (string.IsNullOrEmpty(wikiData))

{

Trace.WriteLine("...using ows_MetaInfo instead of ows_WikiField");

string itemData = item.Attributes["ows_MetaInfo"].Value;

Regex metaProp = new Regex(@"( \w*:\w{2}\|)");

string[] regexData = metaProp.Split(itemData);

bool prepnextMatch = false;

foreach (string data in regexData)

{

try

{

if (data != string.Empty)

{

if (!prepnextMatch)

{

if (data == " WikiField:SW|")

{

prepnextMatch = true;

}

else if (prepnextMatch && actualWikiData == string.Empty)

{

actualWikiData = data;

break;

}

else

{

throw new System.Exception("E_FAIL");

}

catch {}

}

else

{

actualWikiData = wikiData;

}

// Locals for replacement operations

string sitename = txt_SiteName.Text.TrimEnd("/".ToCharArray());

string destsite = txt_SiteName2.Text.TrimEnd("/".ToCharArray());

string wiki1 = txt_SelectedWiki.Text.TrimStart("/".ToCharArray());

string wiki2 = txt_SelectedWiki2.Text.TrimStart("/".ToCharArray());

Uri sourceSiteUri = new Uri(sitename);

Uri destSiteUri = new Uri(destsite);

string sourceAbsolute = sourceSiteUri.AbsolutePath.TrimEnd("/".ToCharArray());

string destAbsolute = destSiteUri.AbsolutePath.TrimEnd("/".ToCharArray());

// http://server1/site/library to http://server2/newsite/newlibrary

string modifiedData = Regex.Replace(actualWikiData, Regex.Escape(sitename + "/" + wiki1), Regex.Escape(destsite + "/" + wiki2), RegexOptions.IgnoreCase);

// http://server1 to http://server2

modifiedData = Regex.Replace(modifiedData, Regex.Escape(sitename), Regex.Escape(destsite), RegexOptions.IgnoreCase);

// /site/library to /newsite/newlibrary (+ Encoded)

modifiedData = Regex.Replace(modifiedData, Regex.Escape(sourceAbsolute + "/" + wiki1), Regex.Escape(destAbsolute + "/" + wiki2), RegexOptions.IgnoreCase);

modifiedData = Regex.Replace(modifiedData, Regex.Escape(sourceAbsolute + "/" + Uri.EscapeDataString(wiki1)), Regex.Escape(destAbsolute + "/" + Uri.EscapeDataString(wiki2)), RegexOptions.IgnoreCase);

// /site to /newsite

// This is very dangerous and is commented because of it...

// since source could be '/' and if it actually is, we would

// replace every / in the doc

//if (sourceSiteUri.AbsolutePath != "/")

//{

// modifiedData = modifiedData.Replace(sourceSiteUri.AbsolutePath, destSiteUri.AbsolutePath);

// modifiedData = Regex.Replace(modifiedData, Regex.Escape(), Regex.Escape(), RegexOptions.IgnoreCase);

//}

// Kill the linefeeds and \\ because they will be literal since we have to use CDATA. This could flatten \\server\share links to \server\share in the text, but the links should still work. You could do something more elegant here to protect against that.

modifiedData = modifiedData.Replace(@"\r\n", "");

modifiedData = modifiedData.Replace(@"\\", @"\");

if (chk_AdvRepl.Checked)

{

modifiedData = modifiedData.Replace(txt_Repl1.Text, txt_Repl2.Text);

}

string strBatch = "<Method ID='1' Cmd='Update'>" +

"<Field Name='ID'>" + item.Attributes["ows_ID"].Value + "</Field>" +

"<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +

"<Field Name='_CopySource'></Field></Method>";

if (chkDebugFull.Checked)

{

Trace.WriteLine("***** " + itemName + " *****");

Trace.WriteLine(strBatch);

}

XmlDocument xmlDoc = new System.Xml.XmlDocument();

System.Xml.XmlElement elBatch = xmlDoc.CreateElement("Batch");

elBatch.SetAttribute("OnError", "Continue");

elBatch.SetAttribute("ListVersion", "1");

elBatch.InnerXml = strBatch;

s2L.UpdateListItems(txt_SelectedWiki2.Text, elBatch);

Trace.WriteLine("Updated: " + itemName);

txt_Status.Text += "Updated: " + itemName + "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

}

catch {}

}

catch {}

Part 07:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-07.aspx

Migrating Wiki Pages Remotely – Part 05

dwinter — Sat, 28 Jun 2008 08:44:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

Once you have gotten the source data in memory, the next step is to get it into the destination server. This is where most folks run into problems. In my exploration, I found a few different approaches—which can work better in different scenarios. If you try the obvious method of adding a new item, you may find some difficulty. Let’s walk through what some of the more obvious ways are and show what happens:

string strBatch = "<Method ID='1' Cmd='New'>" +

"<Field Name='ID'>New</Field>" +

"<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +

"</Method>";

if (chkDebugFull.Checked)

{

Trace.WriteLine("***** " + itemName + " ***** MANUAL COPY *****");

Trace.WriteLine(strBatch);

}

XmlDocument xmlDoc = new System.Xml.XmlDocument();

System.Xml.XmlElement elBatch = xmlDoc.CreateElement("Batch");

elBatch.SetAttribute("OnError", "Continue");

elBatch.InnerXml = strBatch;

XmlNode ndReturn = s2L.UpdateListItems(txt_SelectedWiki2.Text, elBatch);

if (ndReturn != null)

{

if (ndReturn.InnerText != "0x00000000")

{

Trace.WriteLine("ErrorCode - " + ndReturn.InnerText);

}

The problem is (and place where most people will find failure until they give up) is that you’ll fall into an error code (the 0x00000000 is a success) the moment you start adding other fields that would be necessary for a wiki. I can successfully create a batch that creates a new Wiki Page using UpdateListItems with the batch method above. However when I start adding other required fields like Title:

string strBatch = "<Method ID='1' Cmd='New'>" +

"<Field Name='ID'>New</Field>" +

"<Field Name='Title'>"+itemName.Split(".".ToCharArray())[0]+"</Field>" +

//"<Field Name='FileLeafRef'>"+itemName+"</Field>" +

"<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +

"</Method>";

I would experience nasty errors back from the web service every time. I did some exploring on existing fields on the source location that had the title/etc in them, and found that:

Content Type = won’t set
Title = won’t set
FileRef = won’t set
FileLeafRef = won’t set
LinkFilename = yes it will set
LinkFilenameNoMenu = yes it will set
WikiField = yes it will set

Again though, I was met with a problem—since I couldn’t get the title or any of the base fields containing the title or derivations of the title to set while creating the item—the new Wiki Pages would be something like <ID#>._ for the title/etc. Since the extension effectively was _, it wouldn’t be linked to the Wiki Pages content type or be recognized as an aspx/etc, so the resulting files are useless. I talked with some of the internal folks who were in charge of the codebase and they noted that if I wanted to create a Wiki Page from scratch like this through a web service, I would likely need to implement my own web service. That was out of spec for me. Needless to say, following a series of failures around that code I re-evaluated my initial approach. The first successful method I found of relocating Wiki Pages was using the copy.asmx Web Service instead of lists.asmx. That was actually the first time I played with the copy.asmx—even though I’ve already shown source above using it. It is quite simple to use, but it does pose a few difficulties which I will discuss later. Here is my code for using the copy.asmx to do the actual copy:

if ((txt_SelectedWiki.Text.Length > 0) && (txt_SelectedWiki2.Text.Length > 0))

{

Server1WS.Lists s1L = new WikiMigrator.Server1WS.Lists();

s1L.Url = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/lists.asmx";

s1L.Credentials = System.Net.CredentialCache.DefaultCredentials;

Server2WS.Lists s2L = new WikiMigrator.Server2WS.Lists();

s2L.Url = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/lists.asmx";

s2L.Credentials = System.Net.CredentialCache.DefaultCredentials;

try

{

// Need to provide a large number or we will restrict at the default 100 items if null

XmlNode ndListItems = s1L.GetListItems(txt_SelectedWiki.Text, null, null, null, txtNumberRows.Text, null, null);

XmlNode ndListItemDetail = ndListItems.ChildNodes[1];

foreach (XmlNode item in ndListItemDetail.ChildNodes)

{

try

{

if (item.Attributes != null)

{

string itemName = item.Attributes["ows_LinkFilename"].Value;

Trace.WriteLine("Copying: " + itemName);

if (!string.IsNullOrEmpty(itemName))

{

Server1CopyWS.Copy myCopyService = new WikiMigrator.Server1CopyWS.Copy();

myCopyService.Url = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/copy.asmx";

myCopyService.Credentials = System.Net.CredentialCache.DefaultCredentials;

Server2CopyWS.Copy myCopyService2 = new WikiMigrator.Server2CopyWS.Copy();

myCopyService2.Url = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/copy.asmx";

myCopyService2.Credentials = System.Net.CredentialCache.DefaultCredentials;

string copySource = txt_SiteName.Text + "/" + txt_SelectedWiki.Text + "/" + itemName;

string[] copyDest = { txt_SiteName2.Text + "/" + txt_SelectedWiki2.Text + "/" + itemName };

WikiMigrator.Server1CopyWS.FieldInformation myFieldInfo = new WikiMigrator.Server1CopyWS.FieldInformation();

WikiMigrator.Server1CopyWS.FieldInformation[] myFieldInfoArray = { myFieldInfo };

byte[] myByteArray;

uint myGetUint = myCopyService.GetItem(copySource, out myFieldInfoArray, out myByteArray);

WikiMigrator.Server2CopyWS.FieldInformation[] myFieldInfoArray2 = new WikiMigrator.Server2CopyWS.FieldInformation[myFieldInfoArray.Length];

string sourceWikiField = string.Empty;

for (int x = 0; x < myFieldInfoArray.Length; x++)

{

if (myFieldInfoArray[x].InternalName == "WikiField")

{

sourceWikiField = myFieldInfoArray[x].Value;

}

if (chkDebugFull.Checked)

{

Trace.WriteLine("SourceProp: " + myFieldInfoArray[x].InternalName + " = " + myFieldInfoArray[x].Value);

}

WikiMigrator.Server2CopyWS.FieldInformation myFieldInfo2 = new WikiMigrator.Server2CopyWS.FieldInformation();

myFieldInfo2.DisplayName = myFieldInfoArray[x].DisplayName;

myFieldInfo2.Id = myFieldInfoArray[x].Id;

myFieldInfo2.InternalName = myFieldInfoArray[x].InternalName;

myFieldInfo2.Type = (Server2CopyWS.FieldType)myFieldInfoArray[x].Type;

myFieldInfo2.Value = myFieldInfoArray[x].Value;

myFieldInfoArray2[x] = myFieldInfo2;

}

// This wont work, and you can't cast the FieldInfo[] object between webservices--hence the above loop

//myFieldInfoArray.CopyTo(myFieldInfoArray2, 0);

// I do some image work here described later in detail

// Are we local OM or remote WS?

if (!chk_DestLocal.Checked)

{

WikiMigrator.Server2CopyWS.CopyResult myCopyResult1 = new WikiMigrator.Server2CopyWS.CopyResult();

WikiMigrator.Server2CopyWS.CopyResult myCopyResult2 = new WikiMigrator.Server2CopyWS.CopyResult();

WikiMigrator.Server2CopyWS.CopyResult[] myCopyResultArray = { myCopyResult1, myCopyResult2 };

try

{

uint myCopyUint = myCopyService2.CopyIntoItems(copySource, copyDest, myFieldInfoArray2, myByteArray, out myCopyResultArray);

if (myCopyUint == 0)

{

int idx = 0;

foreach (WikiMigrator.Server2CopyWS.CopyResult myCopyResult in myCopyResultArray)

{

string opString = (idx + 1).ToString();

if (myCopyResultArray[idx].ErrorMessage == null)

{

Trace.WriteLine("Copied to: " + myCopyResultArray[idx].DestinationUrl);

txt_Status.Text += "Copied to: " + myCopyResultArray[idx].DestinationUrl + "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

copySuccess = true;

}

else

{

Trace.WriteLine("COPY SERVICE FAILURE--TRY LOCAL DEST INSTEAD");

Trace.WriteLine("ERROR: " + myCopyResultArray[idx].ErrorMessage);

txt_Status.Text += "Copy Operation Failure. Try Local Dest instead. Exception: " + myCopyResultArray[idx].ErrorMessage + "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

}

idx++;

}

// In full code, this catch processes CopyResult

catch {}

}

else

{ // Local dest

copySuccess = manualLocalCopy(sourceWikiField, itemName, myFieldInfoArray2, myByteArray, copyDest, copySuccess);

}

catch {}

}

catch {}

}

Part 06:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-06.aspx

Migrating Wiki Pages Remotely – Part 04

dwinter — Sat, 28 Jun 2008 08:42:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

We just looked at using copy.asmx to get WikiField. Now let’s look at getting the field by using just lists.asmx. You may notice that the segment of code I took this from is using the destination web service instead of the source. This one is interesting because if WikiField is empty, you can see how I parsed the ows_MetaInfo propertybag.

if (txt_SelectedWiki2.Text.Length > 0)

{

Server2WS.Lists s2L = new WikiMigrator.Server2WS.Lists();

s2L.Url = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/lists.asmx";

s2L.Credentials = System.Net.CredentialCache.DefaultCredentials;

try

{

// I know a lot of folks want to define their own view

// instead of using the stock ones. I threw this in as

// an example.

XmlDocument xmlDocLI = new System.Xml.XmlDocument();

XmlNode ndQueryLI = xmlDocLI.CreateNode(XmlNodeType.Element,"Query","");

XmlNode ndViewFieldsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"ViewFields","");

XmlNode ndQueryOptionsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"QueryOptions","");

// I’m not actually using ndQueryOptionsLI, it is

// here as an example

ndQueryOptionsLI.InnerXml = "<IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns>" +

"<DateInUtc>TRUE</DateInUtc>";

ndViewFieldsLI.InnerXml = "<FieldRef Name='FileRef' />" +

"<FieldRef Name='WikiField' />" +

"<FieldRef Name='LinkFilename' />";

// I’m not actually using ndQueryLI, it is here as an example

ndQueryLI.InnerXml = "<Where><Eq><FieldRef Name='FileRef' />" +

"<Value Type='Text'>[server-relative URL of wiki page]</Value></Eq></Where>";

// Need to provide a large number or we will restrict at the default 100 items if null

XmlNode ndListItems = s2L.GetListItems(txt_SelectedWiki2.Text, null, null, ndViewFieldsLI, txtNumberRows.Text, null, null);

XmlNode ndListItemDetail = ndListItems.ChildNodes[1];

foreach (XmlNode item in ndListItemDetail.ChildNodes)

{

try

{

if (item.Attributes != null)

{

string itemName = item.Attributes["ows_LinkFilename"].Value;

Trace.WriteLine("Fixing: " + itemName);

if (!string.IsNullOrEmpty(itemName))

{

string wikiData = item.Attributes["ows_WikiField"].Value;

string actualWikiData = string.Empty;

if (string.IsNullOrEmpty(wikiData))

{

Trace.WriteLine("...using ows_MetaInfo instead of ows_WikiField");

string itemData = item.Attributes["ows_MetaInfo"].Value;

Regex metaProp = new Regex(@"( \w*:\w{2}\|)");

string[] regexData = metaProp.Split(itemData);

/* I am going to use prepnextMatch to say that the next iteration in the loop has the WikiField data if it exists. It was a rare condition where WikiField wouldn’t return it and this method would, and I believe it was caused by the Alpha upgrades that nobody else should have gone through on some of our internal servers. None-the-less, it may be interesting to see this alternate method. */

bool prepnextMatch = false;

foreach (string data in regexData)

{

try

{

if (data != string.Empty)

{

if (!prepnextMatch)

{

if (data == " WikiField:SW|")

{

prepnextMatch = true;

}

else if (prepnextMatch && actualWikiData == string.Empty)

{

actualWikiData = data;

break;

}

else

{

throw new System.Exception("E_FAIL");

}

catch {}

}

else

{

actualWikiData = wikiData;

}

Part 05:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-05.aspx

Migrating Wiki Pages Remotely – Part 03

dwinter — Sat, 28 Jun 2008 08:40:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

I wrote before about how you could use lists.asmx to get the data out of a Wiki Library just like any other list type. There is one significant difference though that may not be immediately apparent. If you investigate the internal structure of a Wiki Page, you will find that we do store a file object—but it is an empty stub object. It doesn’t contain the actual data or body of the Wiki. The body or content of the Wiki Page is stored within a metadata property called ows_MetaInfo. The ows_MetaInfo field is a huge propertybag that you ideally would want to use a RegEx to parse and look for data within. It is not something you can simply split and read because each separator is different. While this is something you could absolutely code around, fortunately, there is a shortcut for the Wiki content itself called WikiField. This will make it much easier for you to get the source data from the source server without having to handle the parsing and various different scenarios that could happen within ows_MetaInfo. Here I will show you how I connected up to a source lists.asmx and copy.asmx and got the WikiField value versus getting the ows_MetaInfo.

First I’ll show getting it WikiField with copy.asmx. I’m using lists.asmx to get a list of files in the Library.

if ((txt_SelectedWiki.Text.Length > 0) && (txt_SelectedWiki2.Text.Length > 0))

{

Server1WS.Lists s1L = new WikiMigrator.Server1WS.Lists();

s1L.Url = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/lists.asmx";
// Always set credentials on your webservice

s1L.Credentials = System.Net.CredentialCache.DefaultCredentials;

try

{

// Need to provide a large number or we will restrict at the default 100 items if null

XmlNode ndListItems = s1L.GetListItems(txt_SelectedWiki.Text, null, null, null, txtNumberRows.Text, null, null);

XmlNode ndListItemDetail = ndListItems.ChildNodes[1];

foreach (XmlNode item in ndListItemDetail.ChildNodes)

{

try // This is in a try/catch block so that we silently skip anything that isn’t valid

{

if (item.Attributes != null)

{

// ows_LinkFilename was the most obvious attribute for identifying Wikis page names, you might choose a different field and parse it if you are handling other list types

string itemName = item.Attributes["ows_LinkFilename"].Value;

// I put this trace before the if so I would see it in the log

Trace.WriteLine("Copying: " + itemName);

if (!string.IsNullOrEmpty(itemName))

{

Server1CopyWS.Copy myCopyService = new WikiMigrator.Server1CopyWS.Copy();

myCopyService.Url = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/copy.asmx";

myCopyService.Credentials = System.Net.CredentialCache.DefaultCredentials;

string copySource = txt_SiteName.Text + "/" + txt_SelectedWiki.Text + "/" + itemName;

WikiMigrator.Server1CopyWS.FieldInformation myFieldInfo = new WikiMigrator.Server1CopyWS.FieldInformation();

WikiMigrator.Server1CopyWS.FieldInformation[] myFieldInfoArray = { myFieldInfo };

byte[] myByteArray;

uint myGetUint = myCopyService.GetItem(copySource, out myFieldInfoArray, out myByteArray);

string sourceWikiField = string.Empty;

for (int x = 0; x < myFieldInfoArray.Length; x++)

{

if (myFieldInfoArray[x].InternalName == "WikiField")

{

sourceWikiField = myFieldInfoArray[x].Value;

}

if (chkDebugFull.Checked)

{

Trace.WriteLine("SourceProp: " + myFieldInfoArray[x].InternalName + " = " + myFieldInfoArray[x].Value);

}

Part 04:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-04.aspx

Migrating Wiki Pages Remotely – Part 02

dwinter — Sat, 28 Jun 2008 08:38:00 GMT

Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx

Before we can dig into the data itself, we need to know what is there. I built some simple functionality to enumerate the available Wiki Libraries and allow for selection. I write the name of the Library into a textbox after it is selected so that the user can manually change it if they want, or skip the enumeration step if they know the name of the list and want to save the time from making that trip to the servers for the source or destination values. Here is the enumeration code that you could essentially duplicate for the destination.

private void btn_EnumWikis_Click(object sender, EventArgs e)

{

Trace.AutoFlush = true;

lst_Wikis.Items.Clear();

Trace.WriteLine("Cleared Server 1 WikiList");

txt_Status.Text += "Cleared Server 1 WikiList\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

if (lst_Wikis.Items.Count == 0)

{

Server1WS.Lists s1L = new WikiMigrator.Server1WS.Lists();

s1L.Url = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) +"/_vti_bin/lists.asmx";

s1L.Credentials = System.Net.CredentialCache.DefaultCredentials;

try

{

XmlNode ndLists = s1L.GetListCollection();

foreach (XmlNode list in ndLists.ChildNodes)

{

if (list.Attributes["ServerTemplate"].Value == "119")

{

Trace.WriteLine("Found Source: " + list.Attributes["Title"].Value);

txt_Status.Text += "Found Source: " + list.Attributes["Title"].Value + "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

lst_Wikis.Items.Add(list.Attributes["Title"].Value);

}

catch {}

}

Here is how I was populating the Wiki 1 textbox for manual control—like with the listboxes, you can duplicate this between the source and destination objects:

private void lst_Wikis_SelectedIndexChanged(object sender, EventArgs e)

{

Trace.AutoFlush = true;

if (lst_Wikis.SelectedItem != null)

{

txt_SelectedWiki.Text = lst_Wikis.SelectedItem.ToString().TrimStart("/".ToCharArray());

Trace.WriteLine("Wiki 1 set to: " + txt_SelectedWiki.Text);

txt_Status.Text += "Wiki 1 set to: " + txt_SelectedWiki.Text + "\r\n";

txt_Status.Select(txt_Status.Text.Length, 0);

txt_Status.ScrollToCaret();

}

}

Part 03:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-03.aspx

Migrating Wiki Pages Remotely – Part 01

dwinter — Sat, 28 Jun 2008 08:27:00 GMT

I was doing some work recently where I needed to move a large volume (thousands) of Wiki Pages from one server to another. The problem was that while if I begged and pleaded, I probably could have gotten local admin access to the farms, I didn’t really want to go through that if I didn’t have to. I wanted to be able to manage the relocation of my Wiki Libraries from my client machine and not have to touch the server locally if I could help it. How do you do this kind of thing? Web Services of course… sounds simple right? Well, that’s what I thought until I gave it a try. As it turns out, it is very easy to use lists.asmx to pull all the data you want. You can do it for Wiki Libraries just like you would for any other list type.

In case you didn’t already figure this out, lucky you, I’ve done all the work for you already. If you are an adventurous or savvy developer, you could stop reading right here and go give it a shot. Otherwise keep reading and see how I made this all go. I have split up a much longer document into a series of posts where I will step through the different parts of the code and talk about the logic/etc behind them. I did this so that I was more reasonable on your RSS readers (my docx is 19 pages long). I will finish the final piece with a single post providing all of the code together.

I will say that some of the scenarios I had to deal with are not ones that I would expect many people to encounter. Some of the source servers I was working with had databases that had been around since Beta, even Alpha stages of SharePoint 2007. This situation led to a few things that just wouldn’t work using conventional methods. I don’t expect many readers will have to deal with these same things I ran into that I had to code around. I’ll be the first to say it—my code is a bit bloated as a result. However, I did want to share it in its entirety so you could benefit from seeing how I dealt with various problems. I would encourage you to hunt and peck my code any try the different code paths separately in your environment until you find what works for you instead of throwing everything at it at once.

Before I dig in, here are a few notes you should know and a screenshot of my final UI so that some of the object names will make a little more sense:

Note 1: I only cared about the current version of the Wiki Pages—so none of my code or methodologies will include handling history. This should be accessible though, and I see no reason why wouldn’t be able to migrate if you felt like writing the extra code.

Note 2: ~~I was too lazy~~ It was out of spec :) to make this a multithreaded application, so I put all my logging on the TraceListener. You can use DbgView from sysinternals to watch the Trace.Write statement output fly by.

Note 3: When I use the lists.asmx GetListItems method, I am passing in the value in txtNumberRows because, otherwise we will use the number of rows configured in the view on the list. By default this is 100. If you have less than 100 Wiki Pages, this isn’t a problem. Otherwise it is. A workaround would be to alter the default view to return more rows and continue to pass null instead of txtNumberRows value.

Note 4: In the posts with commentary, I’ve removed all the action in the catch blocks—if you want to see my error tracing/etc, you can look in the final post. You also may notice that most of these code snippets aren’t full methods and don’t close out all of the braces/etc.

Note 5: Server1WS points to lists.asmx. Server1CopyWS points to copy.asmx. I created separate Server2 versions of these because at times I was using multiple at the same time and wanted them to be separate live objects in memory. This could be cleaned up—I thought it made it easier to follow the code if you were seeing it for the first time.

Note 6: Just as a generic disclaimer, as you might imagine this code and resulting tool is not considered production worthy. It was not tested and as a tool would be “unsupported” by Microsoft Product Support. I am sharing it simply to help save you some time in your own development efforts.

We will dive into some of the code with the next post.

Part 02:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-02.aspx