Recently, I got into issues with using feature upgrade mechanism for adding a new field to a content type. We have been using AddContentTypeField Element inside UpgradeActions Element successfully for some releases and it was adding new content type fields to existing sites properly. However, so far, we did not get into scenario where the same feature was also required to be activated on new sites. When we got into the scenario, we realized this approach was not working at all. The approach we have been using was as per SharePoint Application Lifecycle Management (ALM) guidelines that feature upgrade should take care of both the scenarios:

1. Existing Sites: Whenever the existing features are upgraded, the new functionality should get added to the site
2. New Sites: Whenever the features are activated, the site should get both: the functionality in the previous versions of features as well as that in the current version of the features. 

The approach that we used was as given below:

• Define new field in new elements2.xml
• For New Sites we do this:
• In feature manifest. refer to elements2.xml  (in addition to exiting elements.xml)
• In existing elements.xml (which has content type definition), add additional field to exiting content type definition using new <FieldRef> element for new column
• For existing sites, for Feature Upgrade. We do the following in  <UpgradeActions>,
• add field definition with new elements2.xml using <ApplyElementManifests> and then
• add that new field to existing content type using  <AddContentTypeField> element and with PushDown=”TRUE”

The above approach is in line with the approach suggested in the MSDN article: Application Lifecycle Management in SharePoint 2010. Under the section “Upgrading SharePoint 2010 Features: A High-Level Walkthrough”, the article gives an example of adding a new field to existing content type and mentions the following in step 2:

“Update the existing definition of the content type in the existing feature element file. This update will apply to all sites where the feature is newly deployed and activated.”

However, the approach never worked, and we had various issues with the approach:

• We tried SPFeature.Upgrade(false). We got exception - The object has been updated by another user since it was last fetched
• When we tried SPFeature.Upgrade(true) , the upgrade went through (at least no visible exception). After feature upgrade:  
  • On existing site collection, All new columns for the exiting content types are visible in site collection content type gallery. But not on any list/library either in root or sub sites. New columns are not getting reflected for any inherited content type which is associated with list or library.
  • For New site collections, latest content type definition is available in content type gallery as well as on any list/lib at root or subsite levels.

• We also tried other approach of removing the new <FieldRef> from the existing content type type definition in existing element.xml. In this case, feature upgrade on existing site collection worked fine. But this time, on the new site collections, the new columns are not available. Here feature upgrade can’t be used as the feature version of the new site collection is already the latest version.

After a lot of troubleshooting and trying various options, we finally could find an approach that works.

The working approach is:

• Define new field in new elements2.xml
• For New Sites we do this:
• In feature manifest. refer to elements2.xml  (in addition to exiting elements.xml)
• DO NOT do this: In existing elements.xml (which has content type definition), add additional field to exiting content type definition using new <FieldRef> element for new column
• Add code in feature activation to add columns, which have NOT been added in elements.xml. USE CSV or excel for columns and content type mapping
• For existing sites, for Feature Upgrade. We do the following in  <UpgradeActions>,
• add field definition with new elements2.xml using <ApplyElementManifests> and then
• add code in feature upgrading through custom upgrade action to add new columns. This code would be same as what you used for activation. (You can also add that new field to existing content type using <AddContentTypeField> element and with PushDown=”TRUE”. However, we resorted to code approach as anyway code was being written for feature activation.)

What we figured out for adding new columns to existing content type is this:

Whether we use code (custom upgrade action) OR declarative (using  <AddContentTypeField>) – it’ll never work (sometime throws error, or sometime doesn’t update content types without any error) if we change the existing content type definition xml file.

Later we also realized that Official documentation also states the same thing, however, it was never documented in connection with feature upgrade. Hence, there is confusion across community on this area. Here are the references that state that we should never update content type definition file:

• Updating Content Types http://msdn.microsoft.com/en-us/library/aa543504.aspx

Do not, under any circumstances, update the content type definition file for a content type after you install and activate that content type. SharePoint Foundation does not track all the changes that are made to the content type definition file. Therefore, you have no reliable method for pushing down all the changes made to site content types to the child content types.

• Updating Child Content Types http://msdn.microsoft.com/en-us/library/ms442695.aspx

You cannot add columns to an existing site content type declaratively, in other words, by updating the Feature XML files

The similar issues are also documented by Charles Chen in his blog SharePoint Content Type Lifecycle Management, however, the approach suggested there is to update the original content type definition manifest file, and add a CustomUpgradeAction as well in addition to the AddContentTypeField. I’m not sure what’s official recommendation here, however, personally I am not in favor of this approach for two reasons: first, it updates original content type definition, which is not recommended in official documentation. Second, it is adding two instructions for upgrade – which can cause issue in some later updates. At the same time, the blog has been written well and there’s a lot of sample code in that which applies to the approach suggested here also.  

Update (25-Mar-2012)

Sample Code

As there are plenty other resources available for the samples, I am not providing sample code for this particular approach. Vesa has provided a detailed post on this. Check out section "Task 2 – Updating features and content types". He has also provided zip file containing the complete source code here