Global & Multi Farm Deployments

There have been a bunch of questions lately around Global Deployments. Here are some essential resources and some links.

Multi Lingual Whitepaper for MOSS - Just released! Thanks Luca
Global Deployment Whitepaper for WSS 2.0 & SPS 2003 - Still very relevant, but I'll mention differences below
Import/Export Search Configurations from one farm to another - Angus Logan Blog Post
Network Considerations to Deployment Planning - previous posts (Highly recommended)
Troubleshooting Performance Issues - previous post
What are SSPs? - previous blog post
Global Deployment - short post on how MS IT deals with SSPs
Regional Settings - Site Settings regional options for personal and site

The Keys to the staged deployment and authoring to production environments....

Content Deployment - post by Tyler Butler
Solutions Deployment - post by Chris Johnson

So what's new on this topic in MOSS?

Comparison of WSS 2.0 and SPS 2003 vs WSS 3.0 and MOSS 2007

WSS 2.0 and SPS 2003 were very different products that worked well together to provide a single search and browse experience if you knew what you were doing and used SPS 2003 for Search and setup a directory of your sites. In a dispersed environment it was common and expected to have WSS 2.0 deployments for local team collaboration. What's changed? Let me tell you... The relationship between WSS 3.0 and MOSS is different. MOSS is more than Search and Aggregation or a taxonomy for documents. Now with WSS 3.0 as a platform for collab, the collaboration itself is enhanced with MOSS. For example, auditing, reporting, document library features like major and minor versions, are the beginning. Features themselves within the team sites are enhanced (when activated) when you install MOSS. So what's changed? To have the best collaboration experience, you're going to want MOSS. What about aggregation? It's still there in MOSS, you still want to have an enterprise Intranet portal. Now with the WCM features Internet is a real reality for huge internet sites, but the gist of this paragraph... You're going to want MOSS on all your servers.

SSPs and the WAN

If you haven't seen it elsewhere... You can not consume shared services across the WAN. Not supported. Why? It will cause perf problems, and will cause usability issues for your users. Let's start with audiences, if you have audiences on your site and your SSP is across the country, the audience information is contained in the SSP database and hence the web part can't load until the audience information is reconciled and it can determine if the person loading the page is in the correct audience. Imagine if you have a dozen web parts with different audiences in them. Now search, search requests/results can be easily embedded in web parts. Imagine how poorly those parts will perform. How about profiles on the mysite? The performance on the mysite would likely be worse than any other page and be seriously impacted. You likely haven't heard this description elsewhere, but wanted to say hey... in 2003 we did some tests.... bad results when the Master Portal search servers, were performing slow, some pages were fine, but the mysite and the portal home page would hang until it could get a response. If you understand what goes on between the Query (Search) and the Web Front ends... (I'm suggesting a netmon trace) the DCOM+ connections you'll start to understand why those servers (the WFEs and Query) need to be on the same LAN.

What can you do about it?

There are a few options...

Internet sites - let's say you want to have global availability of your site, but have an authoring environment and the internet site is pretty static to the anonymous users. I'd say you have the most options... What about mutiple production farms, one on the east coast, one on the west coast. Have mapped paths for the east coast with applicable jobs, and a separate job and paths to the other farm. With either DNS round robbin or some cool logic on the WFE's to determine where people should go with then sticky sessions and/or persistent cookies. Caching in this scenario where you have fairly static content and anonymous users you could really, really scale with much less hardware. Going back to SSPs, you'd have SSPs on each farm. Even if some sites weren't static, you could configure caching different on those site collections or if ISA was involved configure different rules for those paths.

Intranet - let's break this down a bit...

Indexing/Search - Index locally. If you want to have a single enterprise search, you determine where that enterprise search comes from and denote it as the enteprise search farm. I recommend making the largest farm the king or master. Then work down the rest of the services, but this is usually the most sensitive. How is this better in 2007? Continuous propegation sends the updates in smaller chunks, those long crawls for a content source with a ton of data can be updating continuously. ACL only crawls are a HUGE improvement. Changing a single ACL on a site collection doesn't mean you need to recrawl everything that inherited from it saving a TON of files that would have had to be recrawled to ensure proper security trimming. Crawl based on change logs - this is HUGE. Imagine only indexing what is needed vs. a look though each site collection to see if the date/time stamp has changed to determine if needs to look for changes. Limit # of Crawl Thread rules - the threads now understand that they are against the same system giving you the ability to throttle your indexing. Farm Configuration - with index/WFE configurations you can now easily target specific systems for indexing against (something that is recognized as a best practice) use host files to do this outside the farm. Crawl Times - start times and stop times can now provide the window that you need to get the crawls going a specific times during the day or night as applicable (use to just be start times)
Profiles - out of the box, you'll configure this to import in each location. MS IT is working on a replication that works off the change logs. They plan to make this available on a codeplex type place where others can take advantage of it
BDC - This makes sense to use locally. Europe may want different sources anyway. Latency/poor bandwidth itself could render a long distance BDC connection inoperable or at least unusable. By the way, Todd Baginski & Nick Swan have made the task of creating these connections so much easier see his BDC Metadata generator post.
Usage - Not a big deal to run usage locally in the farm
My Sites - Here in MS IT they determine who should have my sites based on the domain a user belongs to. Only the users within the relevant countries have rights have the ability to create sites on the applicable farm. Hence users in South America and North America domains can create sites on the farm in the North America farm and users in the Europe in the Europe farm. IT built a clever little app that redirects the user based on a simple lookup to the correct farm, so all they have to do is type http://my to get redirected to their my site. All mysites are indexed by the central farm, so finding data across them is exposed on the central portal and since they are indexed locally within the "region" results come back there as well. People search which is based on profiles works in all farms since all user profiles are imported in each region.
Excel Calculation - I think this makes sense to run locally. Unless you only invested in one compute cluster, this likely will be a non issue.
Audiences - It would be nice to have audiences manged from one location, but with the new rules wizard for creating audiences it isn't too difficult to create rules. You may want to have different audiences in the different deployments around the world anyway. There may be some common ones.

Other Aspects...

Management of Multiple farms & SSPs - use the resources web part in central admin to create links directly to the other central admin interfaces
Monitoring - use MOM to have a centralized view of monitoring of the multiple farm, use Web Sites and Services MP to get a roll up of availability data metrics.

What about a single farm? This holy grail of MOSS deployments is a challenge a curse (all eggs in one basket) and a blessing (single farm/single management). If you do figure out this route, don't put all your eggs in one basket, look at high availability solutions with SQL log shipping with failover to a warm backup or read only copy. If your company is spread out with slow links and poor latency over 200 ms, you'll likely want to consider either a regional/distributed approach (2-4 deployments) or a local deployment/Branch Office approach 5+ deployments. As time goes on you'll see more of your network devices attempt to address transfering deltas, differencing, caching, etc... MS IT has built a tool called copytimer that attempts to capture the upload and download times and can be scheduled. They are planning on sharing this tool. I'm pushing them hard to make this happen in the next month. More on that tool later.

Authoring -> Production

The key to this having your authors understand a schedule and thinking about content deployment jobs, quick deployment options, and the impact of invalidating cache. The links above around content deployment and solutions deployment are what I recommend you understand with this. Building best practices and really control around a locked down production environment that has specific pages that are optimized for performance. Even if you decide that you a collaborative element or collaborative site like a blog, you should look to keep these out of the jobs configured for deployment. It basically takes some planning to determine what you want to be fairly static and highly optimized, vs. what you want to be dynamic and interactive. Don't be afraid as an IT Pro to lay the line on demanding that all code come in as packed solutions in .msi files. It will save you in the end. DR will be a TON easier having these packages. Make sure you keep extra copies of what you've deployed.

Dev -> Test -> Staging -> Production

Dev a development environment where the devs can verify their deployments. I expect it will be more and more common that these environments will be completely virtual environments. From a MOSS perspective if there are multiple Dev groups, they will likely want their own machines to bang on before bringing the code projects together in the common dev environment. From Dev the development projects should be packaged in .msi solutions deployments so they can be easily be deployed to test in packages that can be seen as a complete version and activated in the test environment. The test environment gives the owner of the application or business need a chance to "veryify" the application or fix. The performance testing can happen in either the test or staging, but whatever this environment is it should closely mimic production if perf testing is to be valid. Production, in an intranet can be read/write and be a mimic of staging from a code, feature, and template perspective. If so desired you could use content deployment mapping & jobs to push either from production to staging or visa versa, but it's truly *one way.*

Partners...

Replication:

One partner in this space that I've come across is Syntergy. They have a 2 way replication system. They recently put out an MSD2D article on a similar topic. "Replicator for SharePoint provides the ability to selectively replicate sites and lists between two separate SharePoint instances with full conflict resolution and no special hardware or ports in your firewall."

WAN Acceleration

There's a lot of people getting really active in this space. I've personally looked at a few of them... Certeon, Packeteer (was Tacit Networks), Cisco. There is some compelling technology here, and this landscape will change drastically over the next 3-5 years. I even expect Microsoft to make more moves in this space. Some of these solutions are very SMB / File Server, so make sure it works with not even HTTP/S, but what you want to do with SharePoint Products and Technologies. You should expect/demand these to work with your solution, not the other way around.

Packeteer, Certeon, Silver Peak Systems, Cisco,Juniper Networks, HP, Riverbed, (Future? Microsoft & Citrix)