UPDATE: There was an issue with the originally posted download regarding the Batch Site Manager and Permission Reporting tools, this has been fixed. If you experienced a problem, simply uninstall, re-download the package, and reinstall.
I'm not going to cross-post, but check out the announcement over on the team blog:
http://blogs.msdn.com/sharepoint/archive/2009/08/27/announcing-the-fourth-release-of-the-microsoft-sharepoint-administration-toolkit.aspx
I do want to make a special callout that the Permission Reporting Tool (check effective permissions, etc) requires that you be running at least the April 2009 CU for WSSv3. You can be past this, but that is the first release that contained the check effective permissions API that the tool utilizes, so for that tool it is required. If you are just going to use SPDiagv2, etc, you don't need to be on the April 2009 CU for WSSv3 or later. Only the Permissions Reporting Tool carries that requirement.
Enjoy!
If you haven't already seen this yet--please read this announcement from Jeff Teper, Corp VP of SharePoint: http://blogs.msdn.com/sharepoint/archive/2009/05/21/attention-important-information-on-service-pack-2.aspx
On the SharePoint team blog, you can find a more detailed post announcing the release of the December CU and providing detail on a new package that we are shipping from this point forward:
http://blogs.msdn.com/sharepoint/archive/2008/12/17/announcing-december-cumulative-update-for-office-sharepoint-server-2007-and-windows-sharepoint-services-3-0.aspx
While the packages are physically large, they do include everything. It doesn't hurt you if you don't have something installed, it is simply skipped. We believe that this will greatly reduce the complexity of managing the patching of your systems. We are continuing to work to update content in general in this area and will specifically be making an update to the deployment documents in January to reflect this new methodology.
Of course that is referring to
Who moved my cheese?, but the reality behind the analogy is still the same. I no longer am working in Product Support (CSS) as a Senior EE. I am now working as a PM (Program Manager) within the MOSS Product Group in Redmond (I have moved from Texas to Redmond). Specifically I am working with a group of people known as the CAT team (Customer Advisory Team). In this new adventure I am challenged with many things oriented around making the SharePoint offerings better for our customers through various avenues ranging from changes to upcoming products, tools for existing products, or even documentation/whitepapers to fill customer needs. One of my initial challenges is in the areas of Patching and Upgrade, which some of you know I am rather passionate about. As much of my work is internal, I won't be able to share as much here as I have in the past with one exception. When whitepapers or other public documentation are released that I had a hand in, I will be putting notes here with perhaps some additional thoughts around the content. Beyond that, I look forward to seeing you at various conferences/etc and working with you in the Beta programs.
The SharePoint Product Group has provided us with some further guidance on what is currently needed to build a server up so that it is running the most up-to-date code. To get the details, read:
http://blogs.msdn.com/sharepoint/archive/2008/09/29/announcing-august-cumulative-update-for-office-sharepoint-server-2007-and-windows-sharepoint-services-3-0.aspx
Some of you may have noticed that in August my title changed to Senior Escalation Engineer. As I look back over the past four years as an Escalation Engineer for SharePoint, I can't say that I disagree... I certainly am old enough in this product to start getting some discounts at lunch :). More seriously though, it is an honor to get this kind of title, and I wanted to point out a few other folks out in the blogosphere from my team that you may also know who also earned this distinction:
Steve Sheppard (author of the WebDav whitepaper and the fantastic series on overlapped recycling configuration of SharePoint)
Mike McIntyre (creator and maintainer of the SPSReports tool that so many of us use to get information about our environments)
There certainly are other quality engineers who I would love to mention here, but as of yet--do not have a presense in the blog-o-sphere, so I'll keep them anonymous at this point. None-the-less, know that with this new title comes some interesting responsibilities where we are enabled to make changes internally in a number of different areas that not only effect SharePoint, but other products as well. So, exciting times, and congrats to Steve and Mike!
You may have noticed the post on the SharePoint Team blog (http://blogs.msdn.com/sharepoint/archive/2008/08/18/update-on-virtualization-support-for-sharepoint-products-and-technologies.aspx) announcing support for Hyper-V based virtualization. There are however many things to know about the extent of what is supported. Obviously we will be supporting our own Hyper-V solution, but beyond that, we will also be supporting SharePoint running within SVVP certified solutions. As of the time of writing this, we only have one partner that is SVVP certified, Novell, Inc. The list is maintained at http://support.microsoft.com/kb/944987. We do have a number of vendors who are working on becoming certified though, currently:
Cisco Systems, Inc.
Citrix Systems, Inc.
Sun Microsystems
Unisys Corp.
Virtual Iron Software
VMware, Inc.
If things go well, we will see more of these vendors go from one list to the other--which will provide you with more options to suit your needs. None-the-less, this is an exciting time for virtualization and SharePoint, and only paves the way for bigger things.
As I mentioned before, there definitely are some things you should be aware of about running SharePoint in a Hyper-V environment. From a technical perspective there are two large ones. First, you need to be on SP1. There's not much interpretation there--for support and licensing to line up, we are requiring that. The second point is that you cannot take a snapshot of the Hyper-V environment. This largely has to do with how a farm could get out of sync in many areas and not be able to be guaranteed to come back online properly if this was done. In time my hope is that we can change this support stance and allow for more flexibility in this area, but for now, don't snapshot.
We got the video of my patching presentation posted from the SharePoint 2008 conference. There truely is a lot of great information here, and this download will be referenced from many Microsoft locations as a must see when it comes to patching SharePoint.
Here are the links:
Please notice that the Infrastructure Update KB has been updated to include the following:
Known issues discovered after release of this update
Installing the Infrastructure Update in a SharePoint farm that uses Alternate Access Mapping with a
reverse proxy or a network load balancer, such as in an extranet deployment, may cause some
public URLs to become unresponsive.
Microsoft is aware of this issue and is developing a solution. Before installing the Infrastructure Update,
customers who use this configuration should use a test environment to verify that public URLs remain
accessible after the update is installed.
It was just announced over on the SharePoint Team blog:
http://blogs.msdn.com/sharepoint/archive/2008/07/15/announcing-availability-of-infrastructure-updates.aspx
This is an important update and is publically downloadable using the links on the SharePoint team's blog entry. We are recommending that all customers apply these updates. Please schedule a time to test them in your test enviornments, and also time for production upgrades. As with any upgrade, you will need go through an upgrade on all of your content. So the more content you have, the longer it will take. We do recommend that if you have MOSS that you apply both the WSS and MOSS packages and not just WSS or MOSS. The MOSS package (build 6322.5000) contains both the Global and Localized patches for MOSS (you should see 42 MSPs). The WSS package (build 6320.5000) contains only the Global patch (there is 1 MSP). It is recommended that you look to see if you need the latest WSS Localized patches (6309.5000) as well--which at the time of writing this post is: http://support.microsoft.com/kb/953484
Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx
One of the final touches was to take care of the “but this data here is out of place”, or “this string still has the old server”, etc type complaints. You may have noticed this, but I wanted to point it out separately (since this works out to be a fall back plan in the code). When I do the link fix-up button (yet another reason for separating the operations into two different buttons), I looked to see if the Adv Repl (advanced replace) was checked and processed those values—in case you missed it, it was here:
if (chk_AdvRepl.Checked)
{
modifiedData = modifiedData.Replace(txt_Repl1.Text, txt_Repl2.Text);
}
This was significant because it allows the admin to have a search and replace style tweak. Ideally this kind of code should never be placed in the hands of an end user—but if you polished it up a bit, it could become an effective tool in allowing folks to make bulk search/replace type changes across wikis. Dangerous—but hey, that’s up to the implementer to control (that’s you! J).
Another thing that may be of interest was the:
// Kill the linefeeds and \\ because they will be literal since we have to use CDATA.
modifiedData = modifiedData.Replace(@"\r\n", "");
I found that when built up the CDATA for use in UpdateListItems from lists.asmx here:
string strBatch = "<Method ID='1' Cmd='Update'>" +
"<Field Name='ID'>" + item.Attributes["ows_ID"].Value + "</Field>" +
"<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +
"<Field Name='_CopySource'></Field></Method>";
That the call to UpdateListItems would suceed, but I would have massive numbers of \r\n littered throughout the wiki pages. The replace I was doing there was to counter that and could possibly be destructive, but generally, I did not find it to be.
Well that’s all of my thoughts for now. I hope this wasn’t too excessively long. It is a new style for me that I wanted to try out—if it is well received I will try it again sometime. If you’d prefer smaller nuggets—I can try for that to. I am open to suggestions.
Thanks for reading! You can find the information about the source in Part 10.
Part 10:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-10.aspx
Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx
So there are a lot of little things to think about here. Once you start playing with this—the next one that comes to mind is… what about the images? It is a good question. I found in practice that most images would end up in the same Document Library or Images Library for any given segment of Wiki data created by a single person. However, it can vary… so I worked in something to parse where images are being used and report it back so that the admin knows they need to copy the images as well. It would be doable to automatically take care of these, but it was out of spec for what I wanted since if told what Document Libraries to grab, I could easily copy everything through explorer view to a list with the same name on the other side. At the beginning of my copy method I declared:
System.Collections.Hashtable imageLibraries = new System.Collections.Hashtable();
Here’s my code for image detection in the wiki pages:
// Try to find image tags and note their lists
MatchCollection imageMatches = Regex.Matches(sourceWikiField, "<img.*src=\"(.*)\">", RegexOptions.IgnoreCase);
foreach (Match imageMatch in imageMatches)
{
MatchCollection srcMatches = Regex.Matches(sourceWikiField, "src=\"(.*)\"", RegexOptions.IgnoreCase);
foreach (Match srcMatch in srcMatches)
{
string imageUrl = srcMatch.Value.TrimStart("src=".ToCharArray()).TrimStart("\"".ToCharArray()).TrimEnd("\"".ToCharArray());
int lastSlash = imageUrl.LastIndexOf("/");
string imageParentUrl = string.Empty;
if (lastSlash > 0)
{
imageParentUrl = imageUrl.Substring(0, lastSlash - 1);
}
Trace.WriteLine("Found image: " + imageUrl);
if (!string.IsNullOrEmpty(imageParentUrl))
{
if (!imageLibraries.Contains(imageParentUrl))
{
imageLibraries.Add(imageParentUrl, string.Empty);
Trace.WriteLine("Added Library: " + imageParentUrl);
}
}
}
}
Then at the end I just loop through the imageLibraries collection and reported it back to the admin. I thought this was slick because I only would see a distinct list of libraries and not every image in use. Take a look here:
if (imageLibraries.Count > 0)
{
Trace.WriteLine("===============================================");
Trace.WriteLine("The following location(s) contain images used by the coppied wikis. They should be coppied manually to new locations matching the structure from:");
Trace.WriteLine("");
txt_Status.Text += "===============================================\r\nThe following location(s) contain wiki images to be coppied:\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
foreach (System.Collections.DictionaryEntry imageLibrary in imageLibraries)
{
Trace.WriteLine(imageLibrary.Key);
txt_Status.Text += imageLibrary.Key + "\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
}
Trace.WriteLine("");
txt_Status.Text += "\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
}
Part 09:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-09.aspx
Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx
Now let us consider the next potential problem in using copy.asmx. There were some instances where I found that the copy.asmx would fail outright. One of these was when I would go between significant schema versions. In my case, I was going between a very early version of o14 and SharePoint 2007. The problem here was that copy.asmx grabs a binary copy of the current object which has all of the information embedded in it. This was a problem because I didn’t have the o14 assemblies (14.x.x.x instead of 12.x.x.x) on my destination server. I will preface the rest of this commentary with the thought that by the time we launch o14, this may not be a problem at all (no promises on what will or won't be in o14 will be comming from this blog--sorry). However, my approach in dealing with the problem is still valuable to share, and that is why I decided to put it here. Here is how I dealt with it: Going back to the initial code where we successfully got the data from the source server’s list.asmx, I did that again. However, for this special case scenario, I decided it was appropriate to make the destination a local OM destination. Then I could run the tool again and go web service to web service form my local server over to my actual destination. Yes, this is an extra set—but it still allowed me to not have to have access to either of the production servers locally. This code path is triggered in my code by the DestLocal checkbox. Here’s what that code looks like:
private bool manualLocalCopy(string sourceWikiField, string itemName, WikiMigrator.Server2CopyWS.FieldInformation[] myFieldInfoArray2, byte[] myByteArray, string[] copyDest, bool copySuccess)
{
bool manualSuccess = false;
if (!string.IsNullOrEmpty(sourceWikiField))
{
try
{
string modifiedData = sourceWikiField.Replace(@"\r\n", "");
modifiedData = modifiedData.Replace(@"\\", @"\");
// You need a try-catch block because new SPSite(), OpenWeb(), and GetList() all throw on failure.
try
{
// open site, web, list, and file collection
SPSite site = new SPSite(txt_SiteName2.Text);
SPWeb web = site.OpenWeb();
SPList list = web.Lists[txt_SelectedWiki2.Text];
SPFileCollection files = list.RootFolder.Files;
SPFile newFile = null;
try
{
// add new wiki page
newFile = files.Add(txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName, SPTemplateFileType.WikiPage);
}
catch (Exception exc)
{
newFile = files[itemName];
}
// get the list item corresponding to the wiki page and update its content
SPListItem item = newFile.Item;
item["WikiField"] = sourceWikiField;
item.Update();
manualSuccess = true;
copySuccess = true;
}
catch (Exception exc)
{
if (exc.Message.Contains("-2147024816"))
{
Trace.WriteLine("File exists in target");
}
else
{
Trace.WriteLine("manualLocalCopy Exception: " + exc.Message);
}
}
}
catch (Exception exc)
{
Trace.WriteLine("***ERROR*** Manual copy failed " + exc.Message);
}
}
if (!manualSuccess)
{
Trace.WriteLine("Manual copy of " + itemName + " failed.");
txt_Status.Text += "Manual copy of " + itemName + " failed.\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
}
else
{
Trace.WriteLine("Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName);
txt_Status.Text += "Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName + "\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
copySuccess = true;
}
return copySuccess;
}
Part 08:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-08.aspx
Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx
Now that that is done, it is time to consider some potential difficulties of this approach. First, when you use the copy.asmx, the destination file will automatically gain a property called _CopySource that points back to the source item—making a dependency. When you browse the new destination page, it will have a link at the top noting that it is a copy of the other Wiki Page and provide a link to it. In my scenario, I didn’t want this because my source server was going to be going away—and that is not something that would automatically have cleaned up. So, I had to add a step of cleaning _CopySource after the copy operation was complete. I found it was more logical to copy everything first and then fix up the links second with a separate button. The good news is that it works perfectly and will have no side effects. If you clear _CopySource—you are setting it to the same value that a normal Wiki Page would have, so it is as though the copy operation through the copy.asmx never happened and you just manually had created the content. Here is the code to clean up _CopySource:
if (txt_SelectedWiki2.Text.Length > 0)
{
Server2WS.Lists s2L = new WikiMigrator.Server2WS.Lists();
s2L.Url = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/_vti_bin/lists.asmx";
s2L.Credentials = System.Net.CredentialCache.DefaultCredentials;
try
{
XmlDocument xmlDocLI = new System.Xml.XmlDocument();
XmlNode ndQueryLI = xmlDocLI.CreateNode(XmlNodeType.Element,"Query","");
XmlNode ndViewFieldsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"ViewFields","");
XmlNode ndQueryOptionsLI = xmlDocLI.CreateNode(XmlNodeType.Element,"QueryOptions","");
ndQueryOptionsLI.InnerXml = "<IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns>" +
"<DateInUtc>TRUE</DateInUtc>";
ndViewFieldsLI.InnerXml = "<FieldRef Name='FileRef' />" +
"<FieldRef Name='WikiField' />" +
"<FieldRef Name='LinkFilename' />";
ndQueryLI.InnerXml = "<Where><Eq><FieldRef Name='FileRef' />" +
"<Value Type='Text'>[server-relative URL of wiki page]</Value></Eq></Where>";
//
// Need to provide a large number or we will restrict at the default 100 items if null
XmlNode ndListItems = s2L.GetListItems(txt_SelectedWiki2.Text, null, null, ndViewFieldsLI, txtNumberRows.Text, null, null);
XmlNode ndListItemDetail = ndListItems.ChildNodes[1];
foreach (XmlNode item in ndListItemDetail.ChildNodes)
{
try
{
if (item.Attributes != null)
{
string itemName = item.Attributes["ows_LinkFilename"].Value;
Trace.WriteLine("Fixing: " + itemName);
if (!string.IsNullOrEmpty(itemName))
{
string copySource = txt_SiteName.Text.Trim().TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki.Text.Trim().TrimEnd("/".ToCharArray()).TrimStart("/".ToCharArray()) + "/" + itemName;
string copyDest = txt_SiteName2.Text.Trim().TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.Trim().TrimEnd("/".ToCharArray()).TrimStart("/".ToCharArray()) + "/" + itemName;
string wikiData = item.Attributes["ows_WikiField"].Value;
string actualWikiData = string.Empty;
if (string.IsNullOrEmpty(wikiData))
{
Trace.WriteLine("...using ows_MetaInfo instead of ows_WikiField");
string itemData = item.Attributes["ows_MetaInfo"].Value;
Regex metaProp = new Regex(@"( \w*:\w{2}\|)");
string[] regexData = metaProp.Split(itemData);
bool prepnextMatch = false;
foreach (string data in regexData)
{
try
{
if (data != string.Empty)
{
if (!prepnextMatch)
{
if (data == " WikiField:SW|")
{
prepnextMatch = true;
}
}
else if (prepnextMatch && actualWikiData == string.Empty)
{
actualWikiData = data;
break;
}
else
{
throw new System.Exception("E_FAIL");
}
}
}
catch {}
}
}
else
{
actualWikiData = wikiData;
}
// Locals for replacement operations
string sitename = txt_SiteName.Text.TrimEnd("/".ToCharArray());
string destsite = txt_SiteName2.Text.TrimEnd("/".ToCharArray());
string wiki1 = txt_SelectedWiki.Text.TrimStart("/".ToCharArray());
string wiki2 = txt_SelectedWiki2.Text.TrimStart("/".ToCharArray());
Uri sourceSiteUri = new Uri(sitename);
Uri destSiteUri = new Uri(destsite);
string sourceAbsolute = sourceSiteUri.AbsolutePath.TrimEnd("/".ToCharArray());
string destAbsolute = destSiteUri.AbsolutePath.TrimEnd("/".ToCharArray());
// http://server1/site/library to http://server2/newsite/newlibrary
string modifiedData = Regex.Replace(actualWikiData, Regex.Escape(sitename + "/" + wiki1), Regex.Escape(destsite + "/" + wiki2), RegexOptions.IgnoreCase);
// http://server1 to http://server2
modifiedData = Regex.Replace(modifiedData, Regex.Escape(sitename), Regex.Escape(destsite), RegexOptions.IgnoreCase);
// /site/library to /newsite/newlibrary (+ Encoded)
modifiedData = Regex.Replace(modifiedData, Regex.Escape(sourceAbsolute + "/" + wiki1), Regex.Escape(destAbsolute + "/" + wiki2), RegexOptions.IgnoreCase);
modifiedData = Regex.Replace(modifiedData, Regex.Escape(sourceAbsolute + "/" + Uri.EscapeDataString(wiki1)), Regex.Escape(destAbsolute + "/" + Uri.EscapeDataString(wiki2)), RegexOptions.IgnoreCase);
// /site to /newsite
// This is very dangerous and is commented because of it...
// since source could be '/' and if it actually is, we would
// replace every / in the doc
//if (sourceSiteUri.AbsolutePath != "/")
//{
// modifiedData = modifiedData.Replace(sourceSiteUri.AbsolutePath, destSiteUri.AbsolutePath);
// modifiedData = Regex.Replace(modifiedData, Regex.Escape(), Regex.Escape(), RegexOptions.IgnoreCase);
//}
// Kill the linefeeds and \\ because they will be literal since we have to use CDATA. This could flatten \\server\share links to \server\share in the text, but the links should still work. You could do something more elegant here to protect against that.
modifiedData = modifiedData.Replace(@"\r\n", "");
modifiedData = modifiedData.Replace(@"\\", @"\");
if (chk_AdvRepl.Checked)
{
modifiedData = modifiedData.Replace(txt_Repl1.Text, txt_Repl2.Text);
}
string strBatch = "<Method ID='1' Cmd='Update'>" +
"<Field Name='ID'>" + item.Attributes["ows_ID"].Value + "</Field>" +
"<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +
"<Field Name='_CopySource'></Field></Method>";
if (chkDebugFull.Checked)
{
Trace.WriteLine("***** " + itemName + " *****");
Trace.WriteLine(strBatch);
}
XmlDocument xmlDoc = new System.Xml.XmlDocument();
System.Xml.XmlElement elBatch = xmlDoc.CreateElement("Batch");
elBatch.SetAttribute("OnError", "Continue");
elBatch.SetAttribute("ListVersion", "1");
elBatch.InnerXml = strBatch;
s2L.UpdateListItems(txt_SelectedWiki2.Text, elBatch);
Trace.WriteLine("Updated: " + itemName);
txt_Status.Text += "Updated: " + itemName + "\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
}
}
}
catch {}
}
}
catch {}
Part 07:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-07.aspx
