SharePoint Administration Toolkit (version 3.0 and version 4.0):
We released the fully supported toolkit twice this year. The 3.0 release contained the first version of the SPDiag tool.
We published a How-To walkthrough video with the help of Hilton Giesenow on how to use SPDiag 1.0 here: http://technet.microsoft.com/en-us/office/sharepointserver/ee221106.aspx
Then between February and August we worked on the 4.0 release of the toolkit. The 4.0 release contained a number of updates to the already released SPDiag tool as well as a new Permission Reporting solution, updates to other existing tools and a few other odds and ends. You can read more about that release here: http://blogs.msdn.com/sharepoint/archive/2009/08/27/announcing-the-fourth-release-of-the-microsoft-sharepoint-administration-toolkit.aspx
After completing the 4.0 toolkit, Hilton created some additional content for us—first an interview of Luca and I (Permissions Report and SPDiag Program Managers). You can listen to that here: http://www.themossshow.com/2009/10/looking-at-the-sharepoint-administration-toolkit-4-0-with-dan-winter-and-luca-bandinelli/
Hilton also recorded an updated How-To video for us on SPDiag, but with the second version that was in the 4.0 release. This is pending upload and should happen soon (right Hilton? :p).
It is worth noting that the SharePoint Administration Toolkit was not designed (specifically blocked in some instances) from running against SharePoint 2010. This is because of two factors. First, much of the functionality is now a part of the product. Secondly, for the functionality that is not—we are working on a SharePoint 2010 Administration Toolkit as a completely new toolkit.
ULS viewer
I’ve never heard of ULS before, what are ULS trace logs? Why/when would I want to use ULS logs? How can I make ULS logs less complex?
These are all questions I’ve heard before. First, Unified Logging Service (ULS) trace log files are log files stored on every SharePoint server individually that provide a running audit trail showing details (including detailed error information) of events that occur on that server. They are called Unified because all Office Server products log to the same place. You don’t get a separate log file for Excel Service, etc. In most cases, checking the ULS logs can uncover important troubleshooting clues, so it is worth learning a little more about. You could start digging into technet here if you would like (our 2010 documentation on this area is not yet posted): http://msdn.microsoft.com/hi-in/library/aa979590(en-us).aspx
The reason I bring up ULS is because I showed off a new tool at the SharePoint Conference 2009 called ULSViewer. ULSViewer is a tool that was developed internally after the ULS infrastructure was built for the 2007 products. We recognized as we were building 2010 that the majority of the product group was using this tool when doing investigations, so we wanted to release it publically so that our customers could experience the same benefits that we do. The ULSViewer tool works both against SharePoint 2007 and 2010. We will release an update to the tool around the RTM timeframe of 2010 based on feedback we have gotten from you.
There is one important thing to note about ULSViewer. Unlike the SharePoint Administration Toolkit, it is not a supported tool. This means that we will not entertain hotfix requests or cases with product support. You are welcome to submit thoughts on the code gallery that we will think about for future revisions of the tool, but it is an “at your own risk” tool. That said, we feel the risk is very low J
If you haven’t seen it—check it out: http://code.msdn.microsoft.com/ULSViewer
Windows PowerShell
It sounds powerful… therefore I want it. …but how do I use it?
This is not an untypical question for admins who first approach 2010 and get curious about the Management Console. Obviously there are some folks who are Windows PowerShell wizards and have been using it for years to manage Exchange Server, etc… but for the rest of us, we have Zach J. Zach has been posting a number of PowerShell introductory topics to help the rest of us catch up. If you haven’t seen it, it would be worth your time to circle back to his older posts in the PowerShell category and start playing through them. I found this to be a lot of fun and learned a number of things along the way. Don’t fear, Zach isn’t our only PowerShell documentation investment—we have a lot of PowerShell goodness coming for technet that I know you’ll enjoy, but for now—give his content a read: http://sharepoint.microsoft.com/blogs/zach/Lists/Categories/Category.aspx?Name=PowerShell
This leads us to the most exciting part (for me) of this post, the answer to the question of “but I already am comfortable with/know how to use PowerShell… what do you have for me?”. <insert dramatic pause> The answer is SPModule. SPModule is a collection of Windows PowerShell scripts for SharePoint 2010 that cover basic operations such as scripted deployment installation or creating/joining a farm. There are a number of other tasks that also are starting to surface in SPModule such as collecting all of the log files. We have a lot of plans for SPModule, but the thing to take away from it today is that the infrastructure for it is in place and we are sharing SharePoint best practices when using Windows PowerShell through these working examples. If you are playing at all in the SharePoint 2010 PowerShell space, I highly encourage you check this out: http://sharepoint.microsoft.com/blogs/zach/Lists/Posts/Post.aspx?ID=54
Thanks, and happy holidays!
UPDATE: There was an issue with the originally posted download regarding the Batch Site Manager and Permission Reporting tools, this has been fixed. If you experienced a problem, simply uninstall, re-download the package, and reinstall.
I'm not going to cross-post, but check out the announcement over on the team blog:
http://blogs.msdn.com/sharepoint/archive/2009/08/27/announcing-the-fourth-release-of-the-microsoft-sharepoint-administration-toolkit.aspx
I do want to make a special callout that the Permission Reporting Tool (check effective permissions, etc) requires that you be running at least the April 2009 CU for WSSv3. You can be past this, but that is the first release that contained the check effective permissions API that the tool utilizes, so for that tool it is required. If you are just going to use SPDiagv2, etc, you don't need to be on the April 2009 CU for WSSv3 or later. Only the Permissions Reporting Tool carries that requirement.
Enjoy!
If you haven't already seen this yet--please read this announcement from Jeff Teper, Corp VP of SharePoint: http://blogs.msdn.com/sharepoint/archive/2009/05/21/attention-important-information-on-service-pack-2.aspx
On the SharePoint team blog, you can find a more detailed post announcing the release of the December CU and providing detail on a new package that we are shipping from this point forward:
http://blogs.msdn.com/sharepoint/archive/2008/12/17/announcing-december-cumulative-update-for-office-sharepoint-server-2007-and-windows-sharepoint-services-3-0.aspx
While the packages are physically large, they do include everything. It doesn't hurt you if you don't have something installed, it is simply skipped. We believe that this will greatly reduce the complexity of managing the patching of your systems. We are continuing to work to update content in general in this area and will specifically be making an update to the deployment documents in January to reflect this new methodology.
Of course that is referring to
Who moved my cheese?, but the reality behind the analogy is still the same. I no longer am working in Product Support (CSS) as a Senior EE. I am now working as a PM (Program Manager) within the MOSS Product Group in Redmond (I have moved from Texas to Redmond). Specifically I am working with a group of people known as the CAT team (Customer Advisory Team). In this new adventure I am challenged with many things oriented around making the SharePoint offerings better for our customers through various avenues ranging from changes to upcoming products, tools for existing products, or even documentation/whitepapers to fill customer needs. One of my initial challenges is in the areas of Patching and Upgrade, which some of you know I am rather passionate about. As much of my work is internal, I won't be able to share as much here as I have in the past with one exception. When whitepapers or other public documentation are released that I had a hand in, I will be putting notes here with perhaps some additional thoughts around the content. Beyond that, I look forward to seeing you at various conferences/etc and working with you in the Beta programs.
The SharePoint Product Group has provided us with some further guidance on what is currently needed to build a server up so that it is running the most up-to-date code. To get the details, read:
http://blogs.msdn.com/sharepoint/archive/2008/09/29/announcing-august-cumulative-update-for-office-sharepoint-server-2007-and-windows-sharepoint-services-3-0.aspx
Some of you may have noticed that in August my title changed to Senior Escalation Engineer. As I look back over the past four years as an Escalation Engineer for SharePoint, I can't say that I disagree... I certainly am old enough in this product to start getting some discounts at lunch :). More seriously though, it is an honor to get this kind of title, and I wanted to point out a few other folks out in the blogosphere from my team that you may also know who also earned this distinction:
Steve Sheppard (author of the WebDav whitepaper and the fantastic series on overlapped recycling configuration of SharePoint)
Mike McIntyre (creator and maintainer of the SPSReports tool that so many of us use to get information about our environments)
There certainly are other quality engineers who I would love to mention here, but as of yet--do not have a presense in the blog-o-sphere, so I'll keep them anonymous at this point. None-the-less, know that with this new title comes some interesting responsibilities where we are enabled to make changes internally in a number of different areas that not only effect SharePoint, but other products as well. So, exciting times, and congrats to Steve and Mike!
You may have noticed the post on the SharePoint Team blog (http://blogs.msdn.com/sharepoint/archive/2008/08/18/update-on-virtualization-support-for-sharepoint-products-and-technologies.aspx) announcing support for Hyper-V based virtualization. There are however many things to know about the extent of what is supported. Obviously we will be supporting our own Hyper-V solution, but beyond that, we will also be supporting SharePoint running within SVVP certified solutions. As of the time of writing this, we only have one partner that is SVVP certified, Novell, Inc. The list is maintained at http://support.microsoft.com/kb/944987. We do have a number of vendors who are working on becoming certified though, currently:
Cisco Systems, Inc.
Citrix Systems, Inc.
Sun Microsystems
Unisys Corp.
Virtual Iron Software
VMware, Inc.
If things go well, we will see more of these vendors go from one list to the other--which will provide you with more options to suit your needs. None-the-less, this is an exciting time for virtualization and SharePoint, and only paves the way for bigger things.
As I mentioned before, there definitely are some things you should be aware of about running SharePoint in a Hyper-V environment. From a technical perspective there are two large ones. First, you need to be on SP1. There's not much interpretation there--for support and licensing to line up, we are requiring that. The second point is that you cannot take a snapshot of the Hyper-V environment. This largely has to do with how a farm could get out of sync in many areas and not be able to be guaranteed to come back online properly if this was done. In time my hope is that we can change this support stance and allow for more flexibility in this area, but for now, don't snapshot.
We got the video of my patching presentation posted from the SharePoint 2008 conference. There truely is a lot of great information here, and this download will be referenced from many Microsoft locations as a must see when it comes to patching SharePoint.
Here are the links:
Please notice that the Infrastructure Update KB has been updated to include the following:
Known issues discovered after release of this update
Installing the Infrastructure Update in a SharePoint farm that uses Alternate Access Mapping with a
reverse proxy or a network load balancer, such as in an extranet deployment, may cause some
public URLs to become unresponsive.
Microsoft is aware of this issue and is developing a solution. Before installing the Infrastructure Update,
customers who use this configuration should use a test environment to verify that public URLs remain
accessible after the update is installed.
It was just announced over on the SharePoint Team blog:
http://blogs.msdn.com/sharepoint/archive/2008/07/15/announcing-availability-of-infrastructure-updates.aspx
This is an important update and is publically downloadable using the links on the SharePoint team's blog entry. We are recommending that all customers apply these updates. Please schedule a time to test them in your test enviornments, and also time for production upgrades. As with any upgrade, you will need go through an upgrade on all of your content. So the more content you have, the longer it will take. We do recommend that if you have MOSS that you apply both the WSS and MOSS packages and not just WSS or MOSS. The MOSS package (build 6322.5000) contains both the Global and Localized patches for MOSS (you should see 42 MSPs). The WSS package (build 6320.5000) contains only the Global patch (there is 1 MSP). It is recommended that you look to see if you need the latest WSS Localized patches (6309.5000) as well--which at the time of writing this post is: http://support.microsoft.com/kb/953484
Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx
One of the final touches was to take care of the “but this data here is out of place”, or “this string still has the old server”, etc type complaints. You may have noticed this, but I wanted to point it out separately (since this works out to be a fall back plan in the code). When I do the link fix-up button (yet another reason for separating the operations into two different buttons), I looked to see if the Adv Repl (advanced replace) was checked and processed those values—in case you missed it, it was here:
if (chk_AdvRepl.Checked)
{
modifiedData = modifiedData.Replace(txt_Repl1.Text, txt_Repl2.Text);
}
This was significant because it allows the admin to have a search and replace style tweak. Ideally this kind of code should never be placed in the hands of an end user—but if you polished it up a bit, it could become an effective tool in allowing folks to make bulk search/replace type changes across wikis. Dangerous—but hey, that’s up to the implementer to control (that’s you! J).
Another thing that may be of interest was the:
// Kill the linefeeds and \\ because they will be literal since we have to use CDATA.
modifiedData = modifiedData.Replace(@"\r\n", "");
I found that when built up the CDATA for use in UpdateListItems from lists.asmx here:
string strBatch = "<Method ID='1' Cmd='Update'>" +
"<Field Name='ID'>" + item.Attributes["ows_ID"].Value + "</Field>" +
"<Field Name='WikiField'><![CDATA[" + modifiedData + "]]></Field>" +
"<Field Name='_CopySource'></Field></Method>";
That the call to UpdateListItems would suceed, but I would have massive numbers of \r\n littered throughout the wiki pages. The replace I was doing there was to counter that and could possibly be destructive, but generally, I did not find it to be.
Well that’s all of my thoughts for now. I hope this wasn’t too excessively long. It is a new style for me that I wanted to try out—if it is well received I will try it again sometime. If you’d prefer smaller nuggets—I can try for that to. I am open to suggestions.
Thanks for reading! You can find the information about the source in Part 10.
Part 10:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-10.aspx
Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx
So there are a lot of little things to think about here. Once you start playing with this—the next one that comes to mind is… what about the images? It is a good question. I found in practice that most images would end up in the same Document Library or Images Library for any given segment of Wiki data created by a single person. However, it can vary… so I worked in something to parse where images are being used and report it back so that the admin knows they need to copy the images as well. It would be doable to automatically take care of these, but it was out of spec for what I wanted since if told what Document Libraries to grab, I could easily copy everything through explorer view to a list with the same name on the other side. At the beginning of my copy method I declared:
System.Collections.Hashtable imageLibraries = new System.Collections.Hashtable();
Here’s my code for image detection in the wiki pages:
// Try to find image tags and note their lists
MatchCollection imageMatches = Regex.Matches(sourceWikiField, "<img.*src=\"(.*)\">", RegexOptions.IgnoreCase);
foreach (Match imageMatch in imageMatches)
{
MatchCollection srcMatches = Regex.Matches(sourceWikiField, "src=\"(.*)\"", RegexOptions.IgnoreCase);
foreach (Match srcMatch in srcMatches)
{
string imageUrl = srcMatch.Value.TrimStart("src=".ToCharArray()).TrimStart("\"".ToCharArray()).TrimEnd("\"".ToCharArray());
int lastSlash = imageUrl.LastIndexOf("/");
string imageParentUrl = string.Empty;
if (lastSlash > 0)
{
imageParentUrl = imageUrl.Substring(0, lastSlash - 1);
}
Trace.WriteLine("Found image: " + imageUrl);
if (!string.IsNullOrEmpty(imageParentUrl))
{
if (!imageLibraries.Contains(imageParentUrl))
{
imageLibraries.Add(imageParentUrl, string.Empty);
Trace.WriteLine("Added Library: " + imageParentUrl);
}
}
}
}
Then at the end I just loop through the imageLibraries collection and reported it back to the admin. I thought this was slick because I only would see a distinct list of libraries and not every image in use. Take a look here:
if (imageLibraries.Count > 0)
{
Trace.WriteLine("===============================================");
Trace.WriteLine("The following location(s) contain images used by the coppied wikis. They should be coppied manually to new locations matching the structure from:");
Trace.WriteLine("");
txt_Status.Text += "===============================================\r\nThe following location(s) contain wiki images to be coppied:\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
foreach (System.Collections.DictionaryEntry imageLibrary in imageLibraries)
{
Trace.WriteLine(imageLibrary.Key);
txt_Status.Text += imageLibrary.Key + "\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
}
Trace.WriteLine("");
txt_Status.Text += "\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
}
Part 09:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-09.aspx
Note, this series starts at http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-01.aspx
Now let us consider the next potential problem in using copy.asmx. There were some instances where I found that the copy.asmx would fail outright. One of these was when I would go between significant schema versions. In my case, I was going between a very early version of o14 and SharePoint 2007. The problem here was that copy.asmx grabs a binary copy of the current object which has all of the information embedded in it. This was a problem because I didn’t have the o14 assemblies (14.x.x.x instead of 12.x.x.x) on my destination server. I will preface the rest of this commentary with the thought that by the time we launch o14, this may not be a problem at all (no promises on what will or won't be in o14 will be comming from this blog--sorry). However, my approach in dealing with the problem is still valuable to share, and that is why I decided to put it here. Here is how I dealt with it: Going back to the initial code where we successfully got the data from the source server’s list.asmx, I did that again. However, for this special case scenario, I decided it was appropriate to make the destination a local OM destination. Then I could run the tool again and go web service to web service form my local server over to my actual destination. Yes, this is an extra set—but it still allowed me to not have to have access to either of the production servers locally. This code path is triggered in my code by the DestLocal checkbox. Here’s what that code looks like:
private bool manualLocalCopy(string sourceWikiField, string itemName, WikiMigrator.Server2CopyWS.FieldInformation[] myFieldInfoArray2, byte[] myByteArray, string[] copyDest, bool copySuccess)
{
bool manualSuccess = false;
if (!string.IsNullOrEmpty(sourceWikiField))
{
try
{
string modifiedData = sourceWikiField.Replace(@"\r\n", "");
modifiedData = modifiedData.Replace(@"\\", @"\");
// You need a try-catch block because new SPSite(), OpenWeb(), and GetList() all throw on failure.
try
{
// open site, web, list, and file collection
SPSite site = new SPSite(txt_SiteName2.Text);
SPWeb web = site.OpenWeb();
SPList list = web.Lists[txt_SelectedWiki2.Text];
SPFileCollection files = list.RootFolder.Files;
SPFile newFile = null;
try
{
// add new wiki page
newFile = files.Add(txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName, SPTemplateFileType.WikiPage);
}
catch (Exception exc)
{
newFile = files[itemName];
}
// get the list item corresponding to the wiki page and update its content
SPListItem item = newFile.Item;
item["WikiField"] = sourceWikiField;
item.Update();
manualSuccess = true;
copySuccess = true;
}
catch (Exception exc)
{
if (exc.Message.Contains("-2147024816"))
{
Trace.WriteLine("File exists in target");
}
else
{
Trace.WriteLine("manualLocalCopy Exception: " + exc.Message);
}
}
}
catch (Exception exc)
{
Trace.WriteLine("***ERROR*** Manual copy failed " + exc.Message);
}
}
if (!manualSuccess)
{
Trace.WriteLine("Manual copy of " + itemName + " failed.");
txt_Status.Text += "Manual copy of " + itemName + " failed.\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
}
else
{
Trace.WriteLine("Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName);
txt_Status.Text += "Coppied to " + txt_SiteName2.Text.TrimEnd("/".ToCharArray()) + "/" + txt_SelectedWiki2.Text.TrimEnd("/".ToCharArray()) + "/" + itemName + "\r\n";
txt_Status.Select(txt_Status.Text.Length, 0);
txt_Status.ScrollToCaret();
copySuccess = true;
}
return copySuccess;
}
Part 08:
http://blogs.msdn.com/dwinter/archive/2008/06/28/migrating-wiki-pages-remotely-part-08.aspx
