It took me a while to come up with everything I needed for this post as I want to make it as accurate as possible with the experience I had on the field. The goal of this post is to provide information on the different ways to work as a team for a WCM web site with MOSS 2007.
I believe that MOSS is large enough that a "one size / fit all" approach is unlikely. There are simply too many variables. Fortunately, I believe that what myself and awesome people I've worked with have overcome most issues and we have automated most deployment procedures. I also believe that most of this knowledge will likely apply to other MOSS scenarios such as Document Management.
This post applies to : Microsoft Office SharePoint 2007 (based on real-life deployments pre-SP1), Microsoft Office SharePoint Designer 2007, Microsoft Visual Studio 2005 Team System, Microsoft Team Foundation Server (not required but nice to have).
Contents
What this post will contain is directions and guidance on how to achieve a working multi-environment team-based development lifecycle for WCM with MOSS 2007. Unfortunately, for legal purposes, I cannot offer all STSADM extensions, Event Handlers, Features, and other code artifacts we developed for specific entities. While I may eventually create a more "blank code version", this isn't available at the moment. However, the how-to knowledge should be relatively easy for developers.
In my experience, there are 2 primary types of development topologies in a MOSS WCM project:
The main differences between the 2 is where your "base environment" is and where you allow "SharePoint Designer users" to act. In an IT oriented lifecycle, every deployment follows a strict cycle of testing, staging, and scheduled deployment to production for all artifacts. In an HTML oriented lifecycle, that may be true for "code artifacts" but you may want to have web integrators use SharePoint Designer directly in the staging environment and/or production.
Obviously, there are advantages and disadvantages for both sides :) I'll try to describe them to the best of my knowledge, but first, I have to explain what Content Deployment is and why it's not meant for development environments:
What I call an IT oriented lifecycle is when you already have a development lifecycle in place. Often case, this is for secure or critical environments where downtime is not acceptable. A secure close-to-production Staging environment is usually available in order to test deployments before doing them in Production. You will want minimal steps to deploy as the deployment may be done by a 3rd party who may not have as much SharePoint experience and who's goal is to maintain the environment, not debug it. This lifecycle is more typical for Intranet web sites.
Developers
When you have this environment in place, are you are likely to have separate developer environments as well as an integrated environment. First of all, let's talk about the separate developer environments. You can take a look at this general Team-Based Development document on MSDN : http://msdn2.microsoft.com/en-us/library/bb428899.aspx for more information. Basically, in order to develop MOSS artifacts such as:
... a developer requires to have MOSS installed on the same machine as Visual Studio. That will allow him to develop and debug his artifacts without impacting anyone else. Note that, often case, in order for a developer to be effective, he will require the same page layouts, master pages, and other "graphical artifacts" to completely test his development. The deployment process has to allow a developer to receive graphical updates without scraping his complete site collection.
Web integrators
For the web integrators doing graphical work and layouts, they do not require a separate development environment. The best solution is for them to have SharePoint Designer (and their graphical design application) on their workstation and all of them should access the same MOSS server. If working on the same Site Collection, SharePoint Designer may be challenging at times and requires some quirks (at least, pre-SP1).
Sometimes, for web integrators who already had their tools prior to developing for SharePoint, I found it easier for them to design their layouts outside first with the tools they know best. Obviously, knowledge of ASP.NET, Master Pages, and SharePoint will help having an easier transition to SharePoint after. Once their layout is as desired in their respective tools, they can then move that content in SharePoint Designer and apply the required updates to work in a SharePoint world.
The MOSS server they work on should also receive development updates when they are at least unit-tested. This update should occurs outside of their working hours if possible and possibly be automated.
Staging
The Staging environment is where you want your customers to first see the end results of their web site. This environment has to be stable and is probably not updated frequently. This is also where you test your complete deployment process. It's not unusual that non-SharePoint resources execute the deployment or it may even be managed by a 3rd party. The process has to be as transparent and automated as possible. This will allow to validate how the deployment to production will occur.
This environment is also often where you have all components similar to the production and hopefully a multi-server MOSS farm. That allows other type of testing (i.e.: deploying WSP across multiple servers, deployment of MOSS hotfixes in a farm, etc.). The network may also be more constrained/secured.
Production
This is the environment where everything has to run smoothly. The deployment process should be the same as the Staging environment. Authoring occurs here, either on the same site collection or a different one (the site collection could even be in a different farm if desired).
Summary
It's clear that, with that many environment, an automated solution to deploy to all of these environments has to be elaborated. Unfortunately, the only one available with the product is not meant for these deployments. SharePoint Features and SharePoint Solution projects are a big key to the solution but may be incomplete depending on your scenarios (as they have been in my experience). I will cover Features, Solutions, and alternatives further.
The main observation on what I call IT focused lifecycle is that the "SharePoint Designer" role is not a heavy role and updates to page layouts are streamlined through tested IT deployments. Designers aren't able to use Designer directly in Staging or Production. Let's see the HTML oriented lifecycle.
What I call an HTML oriented lifecycle is where your web site requires frequent layout updates. Your designers are likely to be close to the authors rather than the developers. This can be typical for highly frequented Internet web sites. Essentially, the same roles seen with the IT lifecycle has to be fulfilled, however, since developers are higher up in the chain, the "bubbling down" of their work may not be as obvious and less controlled.
Developers essentially do the same thing as in the IT lifecycle. Integrated testing is done similarly as before on a centralized server that receives more frequent updates from the designers. The integrated testing environment acts as a staging for the development artifacts.
The main difference of this lifecycle reside here. Web integrators are doing daily updates to layouts on an Authoring Site Collection. They might even be "power authors" that also manage content. They often work directly in the production environment albeit a different Site Collection that may or not be in a different MOSS farm than the Publishing Site Collection.
This allows them to easily deploy layout updates without requiring an IT deployment that is usually more complicated or constrained.
The real staging environment is really the Authoring Site Collection since this is the origin for all layouts.
This environment will be the same as before with the exception that SharePoint Designer is allowed on the Authoring Site Collection.
As you can see, there are similarities in the 2 lifecycles, however, from an architecture perspective, there are differences when you want to extract designer updates. For example, in the IT lifecycle, we can link the integrated environment with a Source Control (Team Foundation Server for example) and generate builds. The same does not easily apply to the HTML lifecycle as your source control environment is unlikely to be in the same network environment and you will rather want to schedule extracts of the graphical artifacts, copy them to the development environment, and then create a build for developers.
Also, while I've seen both topologies in the field, one could debate why would designers need to update layouts so frequently. In my opinion, if you design your layouts correctly and provide the correct level of customization, you should probably not need updates frequently and you might be able to work out with the IT oriented lifecycle. I wanted to discuss about the 2 main scenarios I've experienced and provide solutions for both.
Now that we have seen the 2 lifecycles, I want to discuss Content deployment quickly to assert why it isn't a viable solution for deploying on all environments.
Content Deployment is a SharePoint feature that allows a complete site collection to be deployed from a source to a destination. These 2 environments can be in the same farm or a different farm. This is done through the Operations tab of the Central Administration web site and requires simple configuration. Basically, you create a path between 1 or 2 Central Administration servers (i.e.: 1 or 2 farms), and then you can create jobs to copy content on a scheduled or manual basis.
The primary requirement is that the target or destination has to be a site collection with the Blank Site template (of the same language of the source) and that all activated features to be installed but not activated at the destination. When the first content deployment runs, it will copy all libraries and items to the destination and then activate the same features as the source.
This is very important and to explain it easily, take a look at the Publishing feature. If you activate it, it creates the out of the box master pages, page layouts, and styles for a Publishing Web Site. If you activate it before doing the first content deployment, it will create a "Style Library" with a different GUID than the source. If you were to use content deployment, it would fail as there would be an item with the same name at the source with a different GUID and it won't accept to deploy it. This is the same if you create custom features.
So in essence, since Content Deployment is used to have 2 sites that will be exactly be the same, it's primary use is for having a "staging authoring environment" where you author and test everything. Then Content Deployment is used to update the production environment according to a schedule or a manual request. Note that there are 3 variants of Content Deployment:
A deployment job consists of 3 primary steps :
The Central Administration web site's Content Deployment interface allows you to schedule deployment jobs easily. Unfortunately, you cannot separate the 3 steps so it's good if you only need to deploy to one environment.
Unfortunately, this is usually not an acceptable scenario to update all environments such as Development or even Integrated Testing for the following reasons:
Because of these concerns, other scenarios such as SharePoint Features, SharePoint Solution (WSP) projects, and the SharePoint APIs have to be explored and the following sections will elaborate on my findings for these technologies.
Note, Chris O'brien also have great posts on Content Deployment and Features.
Now that we have a better understanding of what we are up against, let's look at what solutions are available to us. Here are the possible solutions :
There may be other possibilities that I'm unaware of but these are the ones I've tested with. Note that this was done pre-SP1. I'll work by elimination (and reduce the list dramatically :)):
So we are left with 3 plausible solutions:
Let me start by summarizing each of them:
SharePoint Features are basically what SharePoint uses to deploy functionality to a SharePoint site. For example, the Publishing feature is activated on a Blank Site to create a Publishing Web Site. This will add the Style Libraries and the Master Pages gallery along with modifying the look & feel of your web site. For example, the Site Actions will contain different actions than for a Collaboration web site.
A SharePoint Feature consists of a manifest Xml file that is deployed in the 12 hive. That manifest in turn specify what actions need to be done to the web site. These actions can be from simply copying files (i.e.: page layouts, master pages, CSS, images, etc.) for creating the baseline of your web site, it can create site columns and content types, as well as even executing any .NET code you want through Event Handlers. On paper, it's the solution and SharePoint itself uses it.
Unfortunately, creating features can be challenging as it requires knowing CAML. CAML is the language definition for configuring any features. While MSDN provides documentation for lots of the Xml elements available, MSDN is meant more as a reference, not for training. Also, while I still create SharePoint Features today, I do not use is exclusively anymore as I found some bugs & quirks that I couldn't workaround well. Particularly, using it to provision the baseline files. For some other scenarios (Timer Jobs or Site Actions for example), Features are still the best solution.
WSP files are really CAB files that SharePoint knows what to do with it. The goal of SharePoint Solution Packages are to deploy a package once to the MOSS Farm and then it will deploy and execute what's required for each server of the farm. It includes even more features such as an interface in the Central Administration where you can schedule when you want a solution to be deployed or even retracted! And to sugarcoat it, if you add servers to a farm, it will apply the solution packages automatically.
** Update ** : A colleague pointed me to the fact that solution will even automatically deploy after a server crash. His scenario was that a solution was deployed in a farm and one of the servers had crashed. When they fixed the servers, they simply had to bring it back online and the solution was automatically deployed to that server without any manual operation (no wizard execution). ** /end Update**
Common scenarios include copying file to the GAC, modifying the web.config (using SPWebConfigModification class), copying resources files, and copying physical files to the web site. SharePoint Solution Packages are a must to most SharePoint deployments.
SharePoint APIs might seem to be a curious addition to this list as it's not really a deployment mechanism, or at least, not yet a user-friendly one. However, while there may be bugs with SharePoint Features and Content Deployment in some cases, the SharePoint APIs is very robust and complete. More so, strong developers can find more code samples as well as using tools such as Intellisense to guess possible solutions.
In your deployments, you are likely to have to use APIs for Event Handlers or web.config modifications, at the least, but my reason for mentioning it as a deployment alternative is that we ended up replacing some "Feature" scenarios that we had issue with for a complete SharePoint API solution. Here's some operations that we automated, by creating custom STSADM extensions, for deploying complete Publishing web sites:
Since lots of these cannot or should not be done by Features, we had to drill the APIs extensively. When strong (showstopper) issues came up for some Features scenarios, we shifted our focus to the APIs. Let me go into more details on the pros & cons of Features versus APIs.
Features can be a great reusable way of deploying new functionality to a SharePoint site. A SharePoint Feature is a set of functions, settings, and files to be added to a site (there are different levels of features but I'll keep it simple for now) when activated. It's a great reusable mechanism for lots of scenarios such as adding a Custom Site Action, adding a Timer Job, and to some level, adding columns and content types. You can take a look at this MSDN article for starting to work with Features : http://msdn2.microsoft.com/en-us/library/ms460318.aspx.
On one of my projects following the IT Lifecycle, we tried to use Features as much as possible since it was, for us, how reusable deployment artifacts should have been. Unfortunately, we ran into some severe issues when using them for Columns/Content Types and files such as Page Layouts (ASPX), Master Pages, CSS, JavaScripts, and images. While we have successfully used features for other artifacts as mentioned in the previous paragraph, our feature that creates the "web site's baseline" with all the right master pages using custom CSS, images, and scripts; is being phased out. Here are the issues that we ran into:
While number 1 was annoying for the authoring environment, and number 3 wasn't happening often enough; number 2 is the real issue where we hadn't found a workaround yet. Plus we knew that the APIs work great for all of them. What we ended up creating was an STSADM extension that reads the same CAML files that this feature used and:
We still package a WSP that runs the commands, actually, we could still use a Feature that adds and removes the files when activated/deactivated (through code) instead of using the CAML definition of provisioned files. In my experience with Publishing Web Site, it's much safer in the database than in physical files. Also, learning how to create columns and content types in .NET, or to add files in the database, is very easy and you'll have access to Intellisense. Learning CAML and try all the scenarios (updating an environment, updating an environment that uses Content Deployment, versioning of documents deployed through features, etc.) is likely to be more challenging and time consuming.
The next section is for detail's sake and because I have lots of questions on Virtual environments for developing in SharePoint.
While SharePoint can work correctly in a virtual environment, you have to be aware that the solution is not officially supported by Microsoft unless it's on Virtual Server. What Microsoft Support will offer is what they call "Best effort support" where they'll help until they cannot reproduce the error and may ask to reproduce the error on a non-virtual environment.
Also, in my experience, I wouldn't recommend virtual environment for production web sites.
For development and up to staging however, I have seen both successful and very sluggish environments. Personally, if going VM, I recommend the following parameters:
In a coming article, I'll detail how we used Team Foundation Server to synchronize SharePoint Designer users with the main source database which allowed us to easily create WSP packages.
Maxime