On June 1st, I started the Microsoft Certified Master (MCM) : SharePoint journey with 16 other candidates from across the world. I can proudly say today that I successfully met all requirements to pass the first ‘RTM’ SharePoint Master rotation.
This has been an incredible experience and by far the best professional training I’ve ever attended in nearly 14 years of development experience. It’s challenging, it’s tough, it’s time-consuming – but it’s also extremely rewarding, valuable, and worth the investment (both personal/time and financial). While you will learn a lot on SharePoint (especially how much you didn’t know to begin with), you will also meet several other experts with who you will share a connection – you have been selected through a rigorous review as a candidate for SharePoint Master. This stays. This is true value over time.
I’m also very proud to be amongst the first Canadian to become an MCM SharePoint – I had a fellow Canadian candidate on the RTM rotation – and will be able to share this experience with both French and English audiences.
There was discussions on the value of going for MCM SharePoint 2007 while knowing that SharePoint 2010 is coming out – the first rotation of SharePoint MCM 2010 is scheduled for next year. Personally, I’m very happy to have done this one for several reasons : (1) There are a lot of customers with 2007 today and more will be on SharePoint before 2010 comes out; (2) We will have to migrate to 2010 which means a good knowledge of 2007 to start with; (3) There is a lot to learn on SharePoint 2007 in only 3 weeks, 2010 will also be 3 weeks and there will be even more to learn! (4) it was announced that there will be an upgrade path from MCM SharePoint 2007 to 2010.
If you are wondering if the cost is worth it, especially for non-Microsoft employees – I haven’t heard a single partner mention that they didn’t get their investment’s worth. If you compare this to any other training program – it will still cost several thousands for 3 weeks and it won’t begin to compare. You have access to several worldwide experts on the product. You have access to incredible hardware to test whatever you need on SharePoint (multi-farm / domains included!). You get to not only learn the best from the best, but also share with them. Also, discounts and new dates have been announced on the master’s blog.
As an ending comment : Be prepared – both technically, mentally, and physically. Don’t assume you will breeze through it – you won’t and nobody did. You will get what you will put into it and be very proud of it. Hope to see more of you in the MCM community soon!
There is a lot of discussions and articles on SharePoint Variations which contains good arguments on why the feature isn’t all what it’s prepped up to be. This will not be a post to debate whether they are useful or not but merely how to make them more useful than what I see in the field.
First of all, when discussing variations with a customer, he’s likely to be sold on the ‘multilingual’ bullet from a PowerPoint slide. I sometime hear that the feature should do a complete translation automatically (nothing less :)). The first thing about Variations is to know when to recommend them and when they aren’t useful enough. After that, the next thing is to have a good discussion with the customer about what it means to use variations.
Here’s a few quick and simple facts on Variations:
- Variations are simply a mechanism to provision sites and publishing pages between multiple labels.
- You can have a single set of variations per site collection
- You can only have a one ‘source’ variation
- By default, and in most cases, any published updates in the ‘source’ variation will be replicated on all other labels as a minor version.
- There is also a web part that allows you to select a page in another language.
Problem #1 : Information Architecture usually picks the customer’s default language as the source variation
While this should make sense, the default language is usually also the one with the most content updates. For example, you set ‘English’ as the source label, and you create 2 labels ‘French’ and ‘Spanish’.
For the ‘Go-Live’, you create all your content in English and it will copy it over in the other 2 labels. You will have translators going over and fixing your other labels.
Within a couple days (if not before the Go-Live), you start updating content in your English label and authors start complaining in the other 2 labels as their localized content is ‘reverted back in English’.
What happens is that the French Version 1.0 of a page got bumped by a v.1.1 in English. While the author can edit the page, go back in history and revert the 1.0, he will have to re-approve/publish the page as a 2.0. This is very annoying indeed.
Problem #2 : While you may want to translate everything for a Go-Live, you probably don’t later
Ask your customer if they really want to translate their complete web site all the time? This is usually not the case. They may want their press releases but not all news as they are more targeted. They may want their products or maybe only a segment of products (i.e.: Corp products versus localized offerings).
Problem #3 : Documents aren’t copied over the other labels
Well that' one’s a simple fact, variations are only for publishing pages. In my experience, while some documents are clearly desired in a few languages, all the documents in the sites are unlikely to be required as translated.
Also, I often see more of a need of a ‘document set’ where you want to see all languages available for a document from any site (i.e.: show me the French and English document in both the French and English site).
Problem #4 : Some customers think that it’s all automated
It’s not. Simple. A human is required as the glue in between. The feature does provide a way to export a label (or a section) and there are 3rd-party products that can take that export and provide a system for translators to work more convivially, but there’s a still a human involved.
Problem #5 : Creating labels is buggy
Well that’s true but SP2 fixed it by providing an STSADM to create labels. The problem was that it took too long for the web process to finish the creation and you were left hanging.
Problem #6 (misconception really) : You should not edit content in other labels than the source except to localize content
That’s simply not true. You can author as much as you want in all labels. The worst that can happen is if you create a site or a page in a sub label; and then you try to create a content with the same name in the source variation. What will happen is that content from the source will not be pushed down on that sub label only.
This isn’t necessary bad actually, and the variation log will tell you the issue and you can resolve it if necessary.
Solutions?
Well if you want to think outside the box, you can surely come up with some workflows or custom events to do the work. Is it worth it? Maybe if you really need all the content to be brought on all languages only one time, but hardly.
OOB solution? Create a ‘non-visitor’ source variation!
In my opinion, you usually need roughly 30% of your content in all languages, give or take. But this works even if you have 100%! What this source variation is simply a ‘creator’. If you need content for all labels, that’s the place to go. If you need to update a page’s content with something major for all labels, that’s the place to go and then you’ll have to update all labels.
But when you want to change a typo or add some comments in one language, that’s not the place to go.
So in our previous sample, we had 3 labels required : English, French, and Spanish. Instead of picking up English as the source, create a label called ‘Source’ or ‘GlobalContent’ (or something that makes most sense to your authors). This ‘Source’ will be in English but your visitors will not go there.
In order to do that, you can either pick a label language that isn’t yours (i.e.: if you are in the US only, pick English UK as your source; and then English US for the English sub label). The other way is to customer the Variation Root Landing logic (http://msdn.microsoft.com/en-us/library/ms562040.aspx).
Alright, so now you have 4 variations instead : Source (English UK), English US, French, and Spanish. You will probably have few Contributors in your source variation, maybe only Hierarchy Managers. You would then set appropriate permissions in your labels only.
Remember, only create or update content in the ‘Source’ when it’s global. When it’s not, authors should go in their respective label and simply create it there. This is totally safe and supported.
This way, you will rarely erase ‘French’ or ‘Spanish’ content with ‘English’ at each update. If you define your information architecture and governance right, and you explain the variation process accordingly with your customer and content authors, this will probably be the simplest solution for variations and should cover most of your customer’s needs.
I was helping a customer yesterday that had an unexplained issue in production, a Web Part was behaving as if it had rolled back its code. As it turns out, the following happened:
- WSP containing DLLs to install in the GAC was deployed some time ago
- A bug was discovered in production and there was time constraints in creating the quick fix through a WSP (permission issues)
- The fix was applied in the DLL and they were manually deployed in the GAC
- The WSP was also updated but never redeployed in production
- IT Admins extended the Web Application to debug another non-related situation
- The original WSPs were redeployed in the farm and the 'old' DLL was brought back
Debugging this obviously took some time, until we could have access to the production server to download the DLL in the GAC and disassemble it, but shows how bad a practice it is to have manually updated DLLs in the GAC that were originally deployed from WSP.
Unfortunately, Best Practices goes only so far where we recommend using WSPs and sometimes we understand the benefits but not consequences such as this one. There may be other operations that could redeploy solutions such as adding a farm server or installing a CU or SP but I didn't confirm this.
So all in all, if you have to manually update a deployed-through-WSP-DLL in the GAC, for whatever (political) reasons (which is usually because of permissions or rigid processes), plan to update the WSP as soon as possible.
I had a customer that was having issues setting the anonymous access. While they could access the http://contoso/_layouts/setanon.aspx page, they couldn't actually see the link to that page when navigating to the Site Permissions.
When they were loading up the page, the "Entire Web Site" and "Lists and Libraries" were disabled allowing only the "None" radio button available. As it turns out, they were accessing a correctly configured Alternated Access Mapping as a public URL for their site, however, the 'setanon.aspx' admin page requires the access to be through a zone configured in the "Authentication Providers" for that Web Application.
What this customer had was a Default zone and Internet zone configured correctly (where the Internet zone was a Web Application extension), both of which are under a reverse proxy for external access. The reverse proxy does SSL termination as well so the actual MOSS servers do not have an SSL certificate.
They also had 2 other URLs used internally (Intranet public AAMs) so that it doesn't go through the publishing rule and thus allow administrators to access any server by specifying a host file. While they could have a host file for the published URL, there would be an issue as it expects SSL.
They could have set the anonymous access through the published anonymous URL (while being authenticated). If they would have absolutely liked to have the intranet URL to allow configuring anonymous, they would have had to remove their AAMs and then extend the Web Application for these AAMs. When you extend a Web Application, it creates an Authentication Provider configuration for that zone.
As a site note, if you did configure the anonymous access in the past and then you access through a non-extended URL, the radio buttons will still be disabled but selected (i.e.: Entire Web Site will be disabled but you can only set the value to None and not back to Entire Web Site).
A few months ago, I came across this issue where the number of returned items in a query wasn’t as expected.
Context with normal queries
To contextualize, let’s say you have a home page with several articles section which are in turned served by Content By Queries. Your queries will either target a specific article sub-site or it will use metadata to query something like :
SELECT TOP 5 Article.Title FROM Article-Site-URL [WHERE MetadataA='value'] ORDER BY LastUpdated DESC
This will show the last 5 updated articles from that site, and if necessary filtered by a metadata.
With daily data being added to all article categories, your home page design is designed to display 5 articles in each category. Usually no less, but certainly no more than 5 items.
Context with audiences
Now if you have an authenticated portal and you’d like to have such an article section with targeted content. For example, the articles are regrouped by a logical business meaning but they are targeted to different groups of users. You will use the same Content By Query and check the “Filter results by audiences” to have only the articles for that user’s audiences. You would expect the “Top 5” to work correctly.
Unfortunately, the query resembles more something like :
SELECT Article.Title FROM (SELECT TOP 5 Article.Title FROM Article-Site-URL ORDER BY LastUpdated DESC) WHERE Article.TargetAudiences CONTAINS (Users.AudiencesGUIDs)
If you look closely at the query, it does a standard TOP 5 query in the content without an audience check. This first query may return 5 items but after the audience filtering, 0 to 5 items will be displayed to the user.
The query should rather be something like :
SELECT TOP 5 Article.Title FROM Article-Site-URL WHERE Article.TargetAudiences CONTAINS (Users.AudiencesGUIDs) ORDER BY LastUpdated DESC
**note: the queries aren’t like this in the APIs, I simply simplified them for explanation**
For a customer using audiences, it can be a show-stopper as you cannot effectively plan your home page queries. The only workaround was to “augment the limit of items return” but since you cannot effectively plan how much you will need, it will break the home page’s layout by either showing too many articles or not enough.
Fix!
Originally, it was apparently by-design and wouldn’t be updated until the next release of Office 14. Fortunately, I’m very happy to have received news that it will be released in an upcoming Cumulative Update along with the fix for multiple audiences explained here.
A little while ago, I worked on a performance problem for a WCM portal. We reviewed a lot of parameters, page output caching, object caching, SQL queries, web parts used and their parameters, code review for the custom controls and web parts, etc. I described part of the issue in my previous post at http://blogs.msdn.com/maximeb/archive/2009/02/03/multiple-audiences-on-a-targeted-content-may-lead-to-page-output-cache-flush.aspx.
The context was for a WCM Intranet where 3 WFEs are handling requests for roughly 30,000 users. The WFEs are either sleeping or at 100% CPU, there were no in between. The content is personalized, targeted, and profiled and users are authenticated using NTLM.
We found several little things such as there was a missing SQL Index on the user profiles table after installing SP1, optimized Object Caching for longer periods of time and augmented amount of RAM allowance, optimized Page Output Caching to remove “check for changes” and have a 2 hours caching period.
Unfortunately, it “felt” as if the Page Output Caching wasn’t always working at random intervals. Note that debugging Page Output Caching is tedious, especially if you have multiple WFEs responding. The only built-in tool is to add Debugging Information that will appear if you view the HTML source at the end of the page. This will include which caching profile was used and at what time the cache was made; or it will include why caching wasn’t used.
With multiple WFEs in load balancing (and no affinity), you will have different cache time because you will sometimes hit a different server. What we did was to add the server name in an HTML comment through the master page.
With these 2 information available, I created a (too personalized for the customer to share yet) Windows tool that was creating HttpRequests with multiple users to various page in a site. The tool was reading the server name and caching information. At first, in a testing environment, we could see that the cache was working for 2 hours, as it should. However, when we downloaded content from production to do the same test, we could see a pattern that the cache was sometimes invalidated for all users.
It took us a while but we got a lucky hit: the main difference is that one of the Top Navigation item was targeted to 20+ audiences (while our testing environment only had a single audience for that link). Let’s say you have the following scenario where the cache profile used does have the ‘Vary by user rights’ parameter checked:
A navigation link (NOTE: you can target a navigation link, Web Part, or document/page; the concept applies to all of these) is targeted to Audience1 and Audience2.
- Audience1
- Audience2
- User3 is not in any audience
The expected display results is the following:
- User1 and User2 will see the targeted navigation link and the non-targeted navigation links
- User3 will only see the non-targeted navigation
This means that the page should only have 2 instances of page output caches, one for each 2 display results. (Note: I simplified the scenario but if you have 2 audiences, it’s possible that the same page also contains targeted content for that audience only, which will in turn create a 3rd display set.)
Unfortunately, the following was happening (before the cache expiration of course):
- User1 hits the page, it will create the page in cache (ok)
- User2 hits the page, it will reuse the page in cache (ok)
- User3 hits the page, it will create the page in cache (ok)
- User1 goes back to the page, the cache is recreated again (wrong)
This means that the once a page is accessed and cached, only users in the same “display result set” (or bucket, as coined by the support team) will use the cache. Any other access to the page will flush ALL caches for that page.
If you only have 2 display bucket, and one bucket only contains a few users, it means that the impact may not be seen. Once your buckets contains several hundreds/thousands of users, the problem becomes more apparent. Also, if you have multiple audiences in a single page in a sub-section, the problem may not be seen.
In our case, it was in the Top Navigation bar used by all pages in the site, making the Page Output Cache useless and in fact a burden.
In this scenario, it was a single link that was being made available to groups of users in blocks. An application was being deployed locally to geographical location and each time one of those location was receiving the application, the link was updated to be targeted to Active Directory users of that location. Over time, the display buckets contained several thousand users.
What we noticed was that the issue with “display buckets and audiencing” only occurs when you have multiple audiences set to a single item. If you only have a single audience, it works correctly.
Our workaround was to create a MOSS group that contained the list of Active Directory groups and targeted the content for that MOSS group. Mind you, if you plan to have multiple targeted items with the same list of people, you should do this anyway, but in our case, it was the content team’s choice and it was only used in once place.
Fix coming in!
I’m also very happy to know that a hotfix was approved by the product group and is being implemented for an upcoming Cumulative Update (CU).
We came across this issue lately where performance was degrading gradually and we couldn’t figure out exactly why. What we noticed was that our Page Output Caching that should last 2 hours wasn’t always lasting that long.
The WCM Intranet portal is used by several thousands authenticated (through Active Directory using NTLM) users every day. We have targeted and secured content throughout the site. Page Output Caching is enabled for maximum efficiency (and definitely required for a large audience) along with the other caching mechanisms such as Object Caching (which has been augmented) and application caching.
Unfortunately, in testing environments, we couldn’t originally reproduce the problem as it seemed to be random in production. Also, while we had most of the production content, we hadn’t noticed one critical component : one of the link in the top navigation bar was targeted, as planned, but to multiple audiences instead of a single audience.
Now, this doesn’t seem like a problem, after all, authors are able to do this and we cannot block it. Unfortunately, it seems that as soon as a member of one of those audiences is logging to the site, it will flush all page output caches for that page.
So whenever a page renders a targeted Web Part or a targeted link, if that targeting contains multiple audiences, all caches for that page will be flushed. We are looking at finding a solution with the product and support teams and I will let you know when I have a change of status.
As a workaround, for now, you have to use a single audience to target content. You have 2 choices: compile an audience for that need, or if it’s targeted to multiple active directory groups, create a MOSS group that contains all those AD groups and target the content on the MOSS group.
Hi everyone and sorry for the no-post for several months. My ‘Draft’ list keeps piling but I didn’t take the time to unpile it much.
Anyhow, I came across this problem again last week with the out of the box Content Editor Web Part (CEWP). Basically, if you use the rich html text editing capability, all URLs are absolute.
I came across a similar issue last year while using the HtmlField and Content Deployment where the URLs were still pointing to the authoring web site. The CEWP was also renowned to have the same issue. Back then, we had fixed the general issue with a quick HttpModule installing an HttpFilter which was essentially rewriting URLs. In terms of architecture, I didn’t like the idea but it was a quick fix done in the early hours. It did the job. Since then, we removed Content Deployment so we didn’t have the issue anymore, thus we removed the HttpModule.
While adding new Alternate Access Mappings URLs to the web site, we noticed some link errors with only the CEWP this time. I looked at past support cases to see if the issue had been resolved in any Cumulative Update (up to December 2008) but no, it’s not. Apparently, the fix involves so many changes in the source code that it’s too high a risk. As of right now, it’s not scheduled to be fixed. The official solutions were :
- Update your CEWP for Publishing Html Fields. Anyone doing publishing knows that it’s not possible in all scenarios. If you know all the Html content types, using Html fields is better and recommended, however, in most landing pages, it’s difficult to do.
- After using the Rich Editing capability, go in the source code and update all the links manually. This is beyond practical that I won’t discuss it.
- Look at 3rd party components. The only one I found that was readily SharePoint enabled was the one from Telerik : RadEditor. It’s a great control that I would recommend for whatever else it does or if you start a new web site. Unfortunately, the Lite (and free) edition doesn’t seem to support the properties StripAbsoluteAnchorPaths and StripAbsoluteImagesPaths. The complete version allows you to use these and it’s 1,000$ USD per developer.
While I’d definitely recommend the RadEditor control, it wasn’t fixing my currently existing pages that uses the CEWP. My next thought was back to the HttpModule but then I remembered looking at ASP.NET Control Adapters.
ASP.NET Control Adapters is basically a way to modify the Render of a control. It’s a little bit like inheritance but only for Render AND it allows to inherit Sealed class such as the ContentEditorWebPart class. Since I was a bit rusted with Control Adapters, I typed in a search for “ContentEditorWebPart ASP.NET Control Adapter” and came across this post from Waldek Mastykarz. He lays out most of the essential but he essentially strips all absolute URLs and this was impractical in my scenario.
In most scenario, you’ll want to strip a list of URLs. In the case of Content Deployment, you’ll want to supply the list of AAMs from the authoring environment (if it’s in the same farm, you could do this programmatically much like what I’m doing in this code but you supply a different URL to the SPWebApplication.Lookup method, if it’s a different farm, supply the list in a config file). In the case of a single site with multiple AAMs, this code will work great:
1: using System;
2: using System.Collections.Generic;
3: using System.Text;
4: using System.Web;
5: using System.Web.UI;
6: using System.Web.UI.WebControls;
7: using System.Web.UI.Adapters;
8: using System.IO;
9: using System.Text.RegularExpressions;
10: using Microsoft.SharePoint.Administration;
11:
12:
13: namespace MaximeBBlog
14: {
15: public class ContentEditorWebPartAdapter : ControlAdapter
16: {
17: protected override void Render(System.Web.UI.HtmlTextWriter writer)
18: {
19: StringBuilder sb = new StringBuilder();
20: HtmlTextWriter htw = new HtmlTextWriter(new StringWriter(sb));
21: base.Render(htw);
22: string output = sb.ToString();
23:
24: List<Uri> alternateUrls = GetAlternateUrls();
25: foreach (Uri alternateUrl in alternateUrls)
26: {
27: output = output.Replace(alternateUrl.ToString(), "/");
28: }
29:
30: writer.Write(output);
31: }
32:
33: private List<Uri> GetAlternateUrls()
34: {
35: List<Uri> alternateUrls = (List<Uri>)HttpContext.Current.Cache["AlternateUrls"];
36: if (alternateUrls == null)
37: {
38: alternateUrls = new List<Uri>();
39:
40: SPWebApplication webApp = SPWebApplication.Lookup(System.Web.HttpContext.Current.Request.Url);
41: foreach (SPAlternateUrl alternateUrl in webApp.AlternateUrls)
42: {
43: alternateUrls.Add(alternateUrl.Uri);
44: }
45:
46: HttpContext.Current.Cache.Add("AlternateUrls", alternateUrls, null, DateTime.Now.AddHours(12), System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Normal, null);
47: }
48:
49: return alternateUrls;
50: }
51: }
52: }
In a nutshell, I’m doing the following:
- Create a class inheriting from System.Web.UI.Adapters.ControlAdapter
- Override the Render method and read the stream’s text (Html)
- Get the list of AlternateUrls from the current Web Application
- Keep the list in caching (I did it for 12 absolute hours, you can change that to your liking)
- Do a simple String.Replace() of the AlternateUrls for a relative url
In terms of performance, it’s negligible and it won’t be reprocessed if you have Page Output Caching enabled. The SPWebApplication.Lookup is also very fast and I’m keeping the list in cache. As for the architecture, it ensures you are only stripping the correct URLs for only that control.
You’ll notice that there is no mention of the ContentEditorWebPart so far and that’s because we aren’t inheriting this Web Part. That’s where the Control Adapter magic starts and you could even attach it to multiple controls if there’s a need! To bind it, you need to add (or update if you have one already) a compat.browser file in the App_Browsers folder of your web site. This file requires the following:
1: <browsers>
2: <browser refID="Default">
3: <controlAdapters>
4: <adapter controlType="Microsoft.SharePoint.WebPartPages.ContentEditorWebPart" adapterType="MaximeBBlog.ContentEditorWebPartAdapter, MaximeBBlog, Version=1.0.0.0, Culture=neutral, PublicKeyToken=febc2e2cb2c3d564" />
5: </controlAdapters>
6: </browser>
7: </browsers>
Note: All you need to update is the namespace in both the code and compat.browser file to match your own; as well as signing it with your key.
You’ll notice that the DLL is signed, this allows me to deploy it in the GAC (through a WSP for example) and NOT to require Full trust. I’m still using WSS_Minimal in fact. There is NO requirement for a SafeControl entry in the Web.Config either. Once your DLL is in the GAC and the compat.browser file in the App_Browsers directory of your web site, that web site will render the ContentEditorWebPart through your class.
Note: if you application domain is already loaded, you may need to recycle it for the update to start working, especially if the page was in the page output already.
Thanks again to Waldek for his early start on the issue.
Cheers!
In a customer deployment, we had several custom Site Definitions to create (because they have variations) and we started noticing that the list of sites available in the Create Page is sorted in an odd way. In fact, it's using neither the creation order, the configuration ID, nor the title (which is okay since it's multilingual).
Using Reflector, I noticed that the page was using the GetAvailableWebTemplates() method of the SPWeb class in order to get its list. I tried sorting the list like I wanted and then using the SetAvailableWebTemplate() but the list was still being returned in the original sorted way.
Digging deeper with Reflector, I noticed that it uses the web property "__WebTemplates" and that returns a list of 'All' web templates as well as web templates by LCID.
Here's the method I created to sort the web's site creation list. You need to pass an SPWeb, a flag to determine if you want to be recursive, and the LCID of your web. In my case, I only had All + 1033 in the English, and All + 1036 in French. If you have multiple languages available in the same site, you may need to update the code a little bit.
1: private void SortWebTemplates(SPWeb web, bool recursive, uint lcid)
2: {
3: SPWebTemplateCollection webTemplates = web.GetAvailableWebTemplates(lcid);
4: StringBuilder sb = new StringBuilder();
5:
6: Collection<SPWebTemplate> collection = new Collection<SPWebTemplate>();
7: foreach (SPWebTemplate template in webTemplates)
8: {
9: bool itemAdded = false;
10:
11: for (int i = 0; i < collection.Count; i++)
12: {
13: if (template.Title.CompareTo(collection[i].Title) < 0)
14: {
15: collection.Insert(i, template);
16: itemAdded = true;
17: break;
18: }
19: }
20:
21: if (!itemAdded)
22: collection.Add(template);
23: }
24:
25: sb.Append("<webtemplates><lcid id=\"all\">");
26: foreach (SPWebTemplate webTemplate in collection)
27: {
28: sb.Append("<webtemplate name=\"" + webTemplate.Name + "\" />");
29: }
30: sb.Append("</lcid><lcid id=\"" + lcid.ToString() + "\">");
31: foreach (SPWebTemplate webTemplate in collection)
32: {
33: sb.Append("<webtemplate name=\"" + webTemplate.Name + "\" />");
34: }
35: sb.Append("</lcid></webtemplates>");
36:
37: web.AllProperties["__WebTemplates"] = sb.ToString();
38: web.Update();
39:
40: if (recursive && web.Webs.Count > 0)
41: {
42: foreach (SPWeb subWeb in web.Webs)
43: {
44: SortWebTemplates(subWeb, recursive, lcid);
45: subWeb.Dispose();
46: }
47: }
48: }
Maxime
Whoever worked with the BlobCache has wanted to clear it someday (especially when developing or debugging issues). The BlobCache is the caching mechanism that keeps a local copy of large files. For example, your pictures, styles, and scripts, can be cached on each front-end. This will reduce SQL traffic.
The only visual way to clear the BlobCache is to navigate to http://mywebapp:myport/_layouts/objectcachesettings.aspx and click on "Force this server to reset its disk based cache". Unfortunately, this only works for the server you are on, which is very interesting in a load-balanced scenario (you'll need to use a host file and point to each server, turn by turn, and navigate to the web site).
Then, if you go about reading on Object Caching at http://msdn.microsoft.com/en-us/library/aa622758.aspx, you will notice that there is a note on BlobCache where you can execute the following command to clear the BlobCache "stsadm -o setproperty -propertyname blobcacheflushcount -propertyvalue 11 -url http://mywebapp:port". The note is unclear whether this should work for all servers or just one, but it works for all server ... Unfortunately, it works once! The reason is simple, the BlobCacheFlushCount has to be incremented each time. Now this isn't very practical if you want to script it!
If you look at the Disk Based Caching for Binary Large Objects, http://msdn.microsoft.com/en-us/library/aa604896.aspx, you notice that it mentions you can simply check "Force all servers in the farm to reset their disk based cache". This would be great but where is that CheckBox??? If you look at the ObjectCacheSettings.aspx file (in Layouts), you will indeed find such a checkbox. If you use Reflector to open the class behind the page, you'll notice that, in the OnLoad method, the checkbox's visible property is set to False! Isn't that practical...
On the other hand, by using Reflector, you can see that the available CheckBox calls a FlushCache() method from the BlobCache class, but that one is internal. But you can also notice that the invisible CheckBox's code is available and is actually quite simple. It does what the STSADM command executes, but it increments the value. Here's the code that allows you to clear the BlobCache on demand and for all servers:
1: SPSite site = new SPSite("http://mywebapp:myport");
2: string s = "0";
3:
4: if (site.WebApplication.Properties.ContainsKey("blobcacheflushcount") && site.WebApplication.Properties["blobcacheflushcount"] != null)
5: s = site.WebApplication.Properties["blobcacheflushcount"] as string;
6:
7: site.WebApplication.Properties["blobcacheflushcount"] = (int.Parse(s, CultureInfo.InvariantCulture) + 1).ToString(CultureInfo.InvariantCulture);
8: site.WebApplication.Update();
9: site.Dispose();
I've also created a small Windows application that does just that, or you can create your own custom STSADM extension to do just the same thing. You can download the application here :
Update: I’m also happy to mention that Sean McDonough created a Feature deployed through a WSP that does the same thing with a more convivial web interface. It can be found on Codeplex at this address : http://blobcachefarmflush.codeplex.com/.
Maxime
We have been having this error for quite some time, and very randomly. It's not that the exception happens often, it's that when it does happen, it breaks that Content Deployment Job until you delete it (along with the path).
As it turns out, reading through the May 20th WSS fix, there was the same error but in what seems to be another situation. Basically, the error occurs with Quick Deployments.
So we tried to break the environment and found out how :
- You need to activate Quick Deployments
- While the Quick Deploy job is running (it's fast, good luck!), you need to flag a page to be quick deployed at the same time.
- You will get "Object Reference Not Set to an Instance" for that Quick Deploy job
- The next execution of the job will run fine.
What the article doesn't mention is that if you have the following:
- You have either an Incremental or Full deployment job that is currently running
- You have a Quick Deploy job that is also running at the same time
- You flag a page to be quick deployed during all this
- You will get "Object Reference Not Set to an Instance" for that Quick Deploy job
- You will also get "Object Reference Not Set to an Instance" for the Incremental/Full job
- The Incremental/Full Job will not work until you delete the path/jobs; The subsequent Quick Deploy job executions will run fine.
As for the WSS + MOSS May 20th patches, they are still private and I do not recommend installing a private hotfix in production unless every other option have ran out. When they become public, we'll test them here to see if it also fix the other job types.
Maxime
For the past few weeks, I have been in parental leave (we just had our first baby girl) and unable to write much. However, it seems that there was some issues with Content Deployment at work.
Actually, the installation of SP1 and Blackout had been scheduled for a while but it didn't resolve as many issues as we had hoped. As such, we were still plagued with some "Object Reference Not Set" and "Primary Key constraint violation ..." during the Export phase.
By calling support, my team was directed to another hotfix (950279), http://support.microsoft.com/default.aspx/kb/950279 that had been released just after Blackout and we hadn't noticed it. This is the 3rd hotfix that I remember seeing that fixes "Primary Key constraint violation...".
What you need to know about these errors is that they are mostly generic. Also, it's not by exporting the content and importing it in your dev environment that you will be able to reproduce the issue. The error is "Content Deployment" specific (I didn't check which database hosts the Content Deployment jobs and paths but it's likely to be Configuration, or otherwise Central Administration).
By deleting the Jobs & Paths, you will likely remove the issue that causes the Export to fail. It seems that issues that were still there before the upgrade were still there after. Also, the thing to know is that, after deleting/recreating the jobs & paths, the next incremental deployment will not have any knowledge of what has been deployed previously and will deploy everything.
For some reason, the first incremental gave us some issues where we had several 'file already exists with the same name' issues. I went ahead and deleted the Jobs/Paths again, did a Full Content Deployment once, and all other deployments have been Incremental and running fine.
So unless you have no problem installing hotfixes (privates and/or publics), I would try this simple no-risk step before installing anything in production. While, in some situation, it may only be a temporary solution, it may buy you enough time to test the hotfix deployment in your environments and hopefully wait for a public fix or even a Service Pack.
Maxime
I ran into some more Content Deployment issues lately and I was going through the manifest (see http://blogs.msdn.com/maximeb/archive/2008/02/19/debugging-content-deployment-issues.aspx for more examples) with some difficulties.
First of all, let me give you a quick background. When you use Content Deployment, it will create a series of CAB files that will contain a file called manifest.xml. This is the file that drives the show. Unfortunately, especially with Incremental deployments, the file is deleted at the end of the deployment, even if it failed.
Basically, right after the "transporting" phase, the "importing" phase kicks in. This is the best time to go, with Windows Explorer, on the destination server and make a copy of the Content Deployment's files (that are in the last modified folder).
If you open ExportedFiles.cab (not ExportedFiles1.cab or any other cab files), you will be able to get the Manifest.xml file.
What this tool will require is simply that file as well as the last number of imported objects that you have in the Content Deployment report. Type in the same number and it will show you the next object that was trying to be deployed and failed.
As you can see, I simply get the right object and create an Xml document with only that object. You can then navigate it as if you would in Internet Explorer. The 2nd tab contains the actual text only of the object and you can do a 'select all / copy' if you need:
Note that the manifest location and number of objects imported are saved in the application configuration so that you don't have to enter them all the time.
Last, the 'Solution Helper' tab will be the one that I update over time (or you can send me manifest samples with solutions so that I can test + develop + document the solution in the tab for others.
The first example of this is that I had an issue in an environment where all columns were being deleted and recreated. The Incremental deployment was failing with a message stating that you cannot delete a column that's in use. The Solution Helper, when it sees that the object is of 'DeploymentFieldTemplate' (a field), it will attempt to get all the document libraries that uses the field (according to the manifest) and show them there. (In our case, it turned out that a RetractSolution + AddSolution had occured and was deleting + recreating the same columns. A Full Content Deployment will 'fix' the error.
While it's in no way a complete solution, I hope it may help you see which objects are failing and give you ideas on what to fix. You can download it here :
Maxime
As you probably know if you read my blog, we like to automate as much as we can our WCM portal deployments. One of the thing we do is to update the Publishing Navigation to hide some sites/pages or simply re-order them correctly. Unfortunately, when you have variations and you want to automate this for all languages, you have to specify some kind of "sleep" time so that it waits for the Variation Timer Job to kick in and create the site in all labels.
Obviously, the sleep time period is guess work and is different on all servers (even on the same one!). During the first attempts, our sleep timer was rather low and we noticed that if we tried opening a site in a target variation/label while the variation process was doing its work, the process was simply stopping! I tried restarting the OWSTIMER service, rebooting the server, wait a day ... but no luck, it simply wasn't picking up again. The Timer Job definitions were still 'running' every minutes, they simply didn't check if there was any updates to propagate.
Unfortunately, so far, I haven't found a way to fix it and to be honest, we simply delayed running our updates on all labels to a later stage. While I did take a look at trying to 'kick start' the process, I didn't go very deep and if I have the issue again, I'll try Gary Lapointe's variation relationship fix (which is VERY interesting and practical even if you don't have the issue I outlined here). If that doesn't work, I'll go see if the relationship are created anyway but it's the Timer Job that cannot read them anymore. I'll update the post when I have the time to reproduce the error and debug it.
Maxime
I've been looking a little bit more on finding the easiest way to deploy resource files (.RESX) in the App_GlobalResources directory and I found various solutions that includes Features and Timer Jobs to synchronize these resource files. Mikhail Dikov has a very good article explaining different process and types of resources.
However, I find that having features and timer jobs that copy files aren't very elegant and also impractical when you want to add a farm server, when you extend a web application (new zone), etc. While we could still develop something around it, I couldn't find an elegant solution.
In fact, I think the only elegant solution would be for the WSP to deploy App_GlobalResources correctly and that'd be the end of it. Note that you can use WSP to deploy resources in the 12 hive for all Web Applications and also "feature resources". Since I didn't find a way to have that elegant solution, I reverted to asking myself "How did we do it with out of the box Publishing Sites?". The answer is simple, all resources in the ...12\Resources directory are ALL copied in the App_GlobalResources of each web application during the standard Web Application creation (or extension) process.
This means that simply copying your Resource files in the 12\Resources directory will automatically deploy your resources when you create a web application, extend a web application, and even when you add a server in the farm. Now the next question was "How do I update my resource files?". First of all, you will need to upgrade your WSP with your updated Resource files. This will copy the RESX in the 12\Resources directory again. After this, you will need to run "STSADM -o CopyAppBinContent" on all servers in the farm.
Now, there are 2 things that I find "less elegant" in this solution, one that is fixable, one that depends on your architecture:
- First of all, running that command on all servers isn't fun for your automated deployment scripts since it would require either doing it manually on all servers or use a remote execution (like PSEXEC) process on all servers. You would either use a CMD with a list of servers (not very dynamic), or create a command that lists all farm servers. I'll outline a solution a bit further.
- For the second, it depends how you want your architecture. If you only have one portal, it's fine to deploy it centrally. If you have more than one, it will depend if you "mind" if all resources are available on all web applications. However, in an Intranet scenario, it's probably very acceptable. If you are an ISP, I would stay away from this solution altogether and you will unfortunately have to resort to a less interesting scenario.
What I ended up creating for executing remotely the STSADM command on all servers was of course another STSADM :)! I'll soon create a post with the outline of this STSADM (and add the link here). For now, the extension does the following:
- Accepts a parameter with quotes that includes the command that needs to be executed on all servers. I also noticed that you cannot have a parameters that includes "-o" between quotes so I might modify my command so that it accepts an input file that lists the command to be executed on all servers (just to be safe I can catch all parameters).
- Retrieves the list of servers (of which the server role isn't "Invalid" like the database server)
- For each server, I add an entry in the Central Administration's "Administrator Tasks" list that describes the command (title) with the server name in it, and in the description, I add the command to execute
- For each server, I create a Timer Job with a schedule of SPOneTimeSchedule. That will essentially execute the job now on each servers and they will automatically delete themselves.
- On each server, when the Timer Job kicks in, it goes back in the Central Administration, reads the Administrator Tasks and find the entry with the "local" server name in order to read the Description column and execute the command defined there
The original command also check that there isn't a timer job and task with the same name prior to executing. Last, if the command doesn't execute successfully on a server, the task isn't deleted and stays on the Administrator Tasks list that shows on the first page of the Central Administration. This allows to have a visual knowledge that the task didn't execute correctly.
Cheers!
Maxime
