MOSSy Business

New Medical Condition: SharePointea Sitecollectionfearophobia

You may not have heard of this serious medical condition, but it exists in most people. It is the irrational fear of SharePoint Site Collections (and particularly Self Service Site Creation). The primary driver for this fear is that allowing people to create site collections will “prevent my ability to manage the environment effectively.”  This is pretty ironic, all things considered. Lets compare:

So… from a manageability standpoint, the Site Collection wins. Granted, the Plan for Software Boundaries TechNet article clearly documents the “limit” of 50,000 site collections per content database vs. 250,000 webs per site collection… but if you’re also following the recommendation on content database size, you’re never going to hit either of these “limits” anyway.

In a strange twist of irony, many people seem much more comfortable providing users with the “Full Control” that comes with being either Site Collection Administrator or in the default Owner group. This is ironic because in the process of trying to maintain some sense of control, they’re allowing their users to do things like create unlimited subsites (that they can’t separate or move between databases) on virtually any number of nested paths, allowing permissions to inherit throughout the site collection (occasionally leading to permissions that weren’t planned), and allowing users to activate various features, use SharePoint Designer, and any number of things that are good features implemented badly.

Rather than let everyone create everything everywhere in ways that can’t be reported on, managed, backed up, or migrated, go ahead and turn on Self Service Site Creation and let the Site Directory be your guide!

There are some recommendations that go along with this though:

• Create some managed paths that these sites can be created in. The default “/sites/” path is fine for smaller organizations, but can create some difficulty once a lot of sites have been created with people wanting to use URLs that are already taken. I prefer the following structure: (this is somewhat repeated from this post)
• The “Root” site collection (“/”): This is the intranet homepage, or at least the entry point into SharePoint. It exposes at least search and the site directory, and is the primary means of navigating to all of the site collections in the application. Preferably, the Collaboration Portal template is used for this site.
• a Teams managed path (“/teams/”): In this universe, subsites aren’t created. This also means that the org structure does NOT match the URL path for teams. This is a pretty big change for most environments… but realize that just because I’m on my managers team does not necessarily mean I’m a direct member of my manager’s manager’s team (if that were true, we’d all have access to the CEO’s team site… which we don’t). You’re on your manager’s team, and your manager is on their manager’s team, and all teams are created at the same level… /teams/MyManager and /teams/TheirManager, for example. This also means that if reporting structures change, the sites and URLs stay exactly where they are! You’re insulated from organizational changes!
• a “Projects” managed path (“/projects/”): Only projects… that is, something with a begin date, start date, deliverable, and generally a schedule, will be created in this path. This also means that after a project is completed, it is easy to backup the site collection and either delete it, move it to less expensive storage, or archive it. This also means that if the project owner/sponsor changes, the site and URL stay exactly where it is! You’re again insulated from organizational changes!
• a “Programs” managed path (“/programs/”):
• Use or create a governance tool/feature for cross-site-collection compliance and default configuration. A great example of this is here from Microsoft IT. In particular, pay attention to:
• Master Pages
• Site Collection Content Types/Columns/Values
• Permission Levels (FULL CONTROL IS EVIL! Create a new “Manager” level that matches Contributor, but grants basic items like Override Check-Out instead of Full Control)
• Group membership/creation (a “Site Manager” instead of Owner is a great idea, granted the “Manager” permission level above)
• Version/Audit/Policy settings
• (optional) Revoke “Create Sub-Site” in the permission levels or permissions mask for the web application. This can have the unintentional side effect of preventing meeting workspaces from being created, but a workaround is listed in the first comment.
• Use the Site Directory, enforcing listing, attributes, and activate nightly link validation. It is particularly important to ensure the attributes/columns used in the Site Directory reflect how your users might want to locate sites they’re interested in.
• Use Search web parts, custom scopes, and managed properties to allow content to appear from various site collections. Remember that search results are automatically security trimmed… so if I display a search web part that only shows sites (a bit of extra work) I have access to, and place that on the homepage, I’ve made my own personal site directory!

Yes, this is more initial work… but it is a LOT easier than what I usually hear: “Our site collection is too large, unwieldy, and we need to split it up and we’re not sure how that’s going to work!”.

Go ahead… enable SSSC… done right, it’s much easier to manage than you might think.

It’s 10pm. Do you know what your server is doing?

It’s really surprising how many people really don’t. They might think they do… but they really don’t… or they don’t take a full inventory of the possibilities. Yes, most of us are relatively good at making some attempt to push scheduled activities off to the evening to prevent impacting the user experience… but can you tell me what your server should be doing right NOW? I’d bet no… at least not without a significant amount of research.

Lets go through the possibilities. At any moment in time, your server could be…

…serving user requests during peak usage hours.
…running full crawls daily.
…running incremental crawls every 30 minutes.
…importing user profiles daily.
…doing usage analysis processing daily.
…running your STSADM backup scripts daily.

Or at least that’s what you think it could be doing… but have you tried to make sure that it’s not trying to do all of these things at the same time? Further, have you considered that there are probably other people in your company that are also doing their jobs on your server farm?

…your platforms team is deploying system updates once a week (resulting in unexpected server reboots)
…your DBA is doing SQL backups and running maintenance jobs daily (resulting in significant disk IO)
…your Antivirus team is running full scans of your server once a week (resulting in significant disk and processor IO)
…your Backup team is running server backups daily (resulting in significant disk and network IO)
…your Google appliance is indexing every 6 hours, or 4 times a day (forcing every page, file, and link in your entire farm to be loaded, compiled, and rendered).

So, overall, your system utilization looks something like this:

Notice how many activities are happening at the same time? Backups, profile imports, indexing… all happening at the same time. We (the SharePoint administrators) have done a good job pushing things outside of peak usage (except our incremental crawls)… but we’ve unintentionally set several things to happen at midnight, and we haven’t coordinated our activities with the things that other groups may be doing. Also, because of the order of events we’ve chosen information could either be out of date or could cause significant performance impacts.

A part of this is to understand what we really need to accomplish.

• Do we really need search contents updated every 30 minutes? In most organizations, updating search once daily is perfectly reasonable.
• Do we really need file system backups and SQL backups and SharePoint backups? Frequently, just backing up the content databases once daily is enough. You can certainly back up more, but you should understand what you’re trying to protect against by performing that backup. If your SLA calls for an acceptable loss of less than 1 day, adjust your backup schedule accordingly.
• We probably don’t need incremental crawls to be scheduled at the same time as our full crawls.
• We probably want our crawl to happen after our profile import so that the latest user information is available in search.
• We don’t want any major activity happening during a patching window or when a reboot is likely (this is particularly important with crawling or you can risk corrupting your indexes!)
• It’s possible we don’t need file system backups for SharePoint at all (though backing up select folders can be useful).
• File system antivirus scans in SharePoint are okay, but only protect the server itself from infection. These scans will not scan SharePoint content because this content is not stored on the server, it is stored in the SharePoint databases.
• Do we need both Google and SharePoint indexing content? (I’m not familiar with the Google appliance, but I believe SharePoint is likely more efficient at indexing itself than Google is, particularly when doing incremental updates.)

For demonstration purposes, lets assume we’ve discussed and coordinated our schedules with the various groups (platforms, SQL, AV) and are going to assume the following:

• Patches are deployed on Saturdays at 1:00am.
• Searches only need to be updated once daily.
• Google will NOT index SharePoint, but we will use search federation so that searches in SharePoint will return results from the Google appliance.
• File system backups will happen daily at 10pm, daily full and weekly incremental, and will only include the “12 hive”, InetPub, and IIS/SharePoint log folders.
• SQL backups will include “all user databases” even though several databases are not restorable. This will ensure new content DBs are picked up automatically.
• Our SLA states that search index recovery will be a full recrawl, so search index backup is not necessary.
• SQL and File System backups can happen simultaneously because we’re dealing with separate IO paths and possible network consumption is acceptable.

Taking these things into consideration, we have the following schedule:

1. 8pm-10pm – Antivirus Scanning (Sundays Only)
2. 10pm-12am – File system Backups (Full on Saturday, Incremental Sun-Fri)
3. 10pm-12am – SQL Database Backups (Full on Saturday, Logs only Sun-Fri)
4. 12am-1am – System Patching
5. 2am-3am – Profile Imports
6. 3am-5am – Search Indexing (Full on Saturday, Incremental Sun-Fri)
7. 5am-5:30am – Usage Analysis Processing
8. 5:30-6am – Audit Policy Analysis (optional)

So, now our calendar looks like this:

This is better because we’re not doing any work unnecessarily, and we have some gaps in time that allow some processes to unexpectedly go beyond their planned times. This also ensures that we’re backing up before any significant activities happen that could impact the server or content, such as antivirus scans (which risk corrupting a file if the file must be cleaned).

It’s also better because now I know EXACTLY what my server is doing at 10:00pm. :)

If you’re using virtualization, be sure to also look into what the other virtual machines in your environment are doing. Remember that you are impacted by whatever the other VMs on the same physical host as you are doing. If they’re using all of the disk capacity for example, your processes will take longer and your timelines may need to change. More virtual machines does NOT MEAN more computing resources!!!

SharePoint Code Acceptance Process

While most companies make sincere attempts to stay as “out of the box” as possible, there comes a point at which custom or 3rd party code becomes necessary to meet the needs of the organization. This is NOT a bad thing… SharePoint is designed to be extended. However, there are certain steps that will help ensure deployed code is as well developed as possible.

On Open Source code (Codeplex.com for example): Remember that “Open source” code should be treated essentially the same as “Custom” code. That is, if you want to take advantage of the functionality someone else provided, you should also be willing to take ownership of the enhancements, bug fixes, compilation, and packaging. There is no company or support organization to help you or provide you with fixes if there is a problem, and there is no guarantee that the source code will be available at <insert custom code download site here> forever. (Welcome to the hidden cost of open source… you get what you pay for.)

Generally, the SharePoint development team is separate from the Administration team that supports the system and it’s users, and thus they have different objectives. Developers are trying to make new stuff, while the MOSS administrators are focused on keeping the existing stuff running as well as possible… or to put it another way, developers exist to create change, while administrators are focused on reducing change to increase stability and availability. To be clear, it is a GOOD THING that these two groups have opposing goals… think of it as checks-and-balances for your server environment… but we still need to get something done, so we need a process and clear agreement on what we’re doing and how.

Below are the minimum criteria I would recommend. Note that additional process steps or criteria can be added as appropriate.

1. All code (custom, 3rd party, purchased, whatever) MUST be provided/deployed as a SharePoint Solution Package.
   SharePoint has a software deployment model that puts the majority (if not all) of the responsibility of ensuring SharePoint code (features, templates, etc) is deployed properly into the hands of the SharePoint Developer. This is technically correct… the developer should know the location that files need to be copied, the GUIDs for web parts to be added to the Save Controls list, the naming of specific features, any dependencies in features, any feature activation dependencies, services that should be activated, etc., so SharePoint provides with developer with a way to define this via Solution Packages.
   3rd party vendors may provide code as installation packages (“setup.exe”) instead of pure solution packages. This may be acceptable as long as a solution package is ultimately the deployment methodology being used.
2. All code should return with no issues when analyzed using the SPDisposeCheck tool.
   Because the SharePoint object model relies on several COM components, the rules of “pure” .NET development do not necessarily apply. Specifically, the practice of relying on the .NET CLR to automatically clean up memory does not behave as is typically expected. This means that developers must ensure they specifically dispose of their objects properly in order to prevent significant memory leaks (that will only expose themselves during load testing) from occurring and creating significant problems in your production environment. SPDisposeCheck can detect the most common scenarios that these issues can occur, so provides a simple way to check for the most common errors. However, false-positives are possible, and this tool does not catch every possible code structure, so load testing should still be done… but this is a great way to identify possible issues quickly.
   Tip: You can analyze the code within a solution package by changing the package to have a .CAB extension and extracting the files. You can then run SPDisposeCheck on the extracted files. 3rd party code can usually be checked in this way also.
3. No Microsoft-provided or out-of-the-box files should be modified as part of the deployment (with exception).
   Overwriting or directly manipulating most SharePoint files is generally considered a severely bad practice. Doing so introduces a significant risk that updates to SharePoint will cause such changes to be lost, and/or cause incompatibility as “old” files from custom solution packages overwrite newer files that contain fixes or compatibility elements with updated DLLs or code. There are exceptions (ex., docicon.xml must be updated to include the common PDF icon association), but generally updated Microsoft provided files should not be necessary.
4. All code should be created in a company specific namespace.
   Occasionally, developers believe that in order to properly use the SharePoint object model, their custom code must be in the “Microsoft.SharePoint…” namespace. This is incorrect, and doing so will make troubleshooting significantly more difficult. Instead, the namespace (and related filenames) should reflect the name of the company doing the development. For example, “Contoso Company” might create all of their SharePoint development in the “Contoso.SharePoint…” namespace.
5. All post-deployment configuration must be done through a provide UI interface.
   In many cases there are two steps required: Deployment and Configuration. With deployment having been automated using Solution Packages, it would be ridiculous for us to now have several steps that require going into the configuration files we previously avoided and update numerous settings. Rather, it should be considered a requirement that, if environment-specific configuration is required, an interface for defining that configuration should also be provided. Further, the configuration for that item should be in the appropriate area for where the feature is scoped. For example, if a feature is farm-wide, the configuration should be accessible through Central Administration, whereas if the feature is scoped at the individual web site, access to the configuration pages should be visible in the Site Settings page. Interfaces for programmatically creating these links are available in the SharePoint object model.
6. Code must be reviewed and have sign-off from at least one additional developer prior to test submission.
   Code reviews significantly enhance the development process and team by A) reducing common errors in code that would otherwise be found in testing or development, B) increase knowledge sharing because the reviewing developer may learn new ways to interact with various components or new software development/design patterns, C) decreasing supportability risk if a single developer leaves or is on vacation because multiple developers are familiar and accountable for the code. This will also generally increase the overall quality of the code being generated (because no developer wants to be embarrassed by their code).

While these items may hover back and forth between “policy” and “process”, each item contributes significantly to the ability to catch errors early and/or identify errors easily and quickly.

If you think something is missing, comment below and I may update this post with the suggestions that provide the most value.

Happy coding!

Enable Page Tracing

There are times when you really want to know how your page was processed, what controls were loaded, etc.

Well, there’s an old trick that not every developer knows they can turn on… page tracing output. This is easily done, and I like the idea of doing it on all development environments (but NEVER on a production environment).

In the web.config file for which ever web application you want to enable page tracing output on, add the following line somewhere within the “<system.web>” section:

     <trace enabled="true" pageOutput="true" requestLimit="200" traceMode="SortByTime" />

At the bottom of every page in that web application (at least for SharePoint), you’ll see the complete listing of the page processing, the controls loaded, and every step the system went through to render the pages.

No, this doesn’t show EVERYTHING, but it can be very useful when your page suddenly takes two minutes to load because of your web part… or your deployed web part appears to crash the application pool (especially after numerous loads).

Happy coding!

The “but no changes were made on the server?!” Fallacy

We hear this in the field constantly… a server begins experiencing difficulty or somehow acts “different”… and the first thing we hear is usually something along the lines of “no changes were made!”

Truth be told, there is only one time when no changes are being made on your server… when it’s turned off. And even that is questionable.

…but you haven’t even logged into the server? FYI, just because you aren’t doing something with your server doesn’t mean it isn’t doing anything. Memory is being managed… files are being created, updated, deleted…  indexes are being updated… files are being transferred… files are being scanned for viruses… backups are being run… network connections are being used… policies are being updated… users are being authenticated… logs are growing… all while you were in Tahiti on your vacation. Oh… and don’t forget that other person you gave the Administrator password to the other day? Yeah… they deleted some critical file and didn’t tell you. They also installed utilities, deleted several log files (to make more space of course), made a critical system change, updated a registry key (without checking to find out if the change they made was supported), visited 5 websites (including one that requested permission to run a program… with a virus), and installed a hotfix… which obviously made them too busy to tell you that they made some serious changes to your server.

Oh… by the way… your Active Directory administrators pushed down an updated group policy that added additional security settings on key files… and your AV solution decided to download new signatures (which decided to flag your primary application executable as a virus and block access).

The point is, your servers are constantly changing. Every second of every minute, while you’re sipping your mai tai in Tahiti, your server is working, making people happy, exactly as it has been told to. It has also logged all of that data that you never clean up into the temp folder on the C drive… and it’s using the paging file that you’ve configured on the C drive… and it’s saved that CD ISO to your desktop (on the C drive)… so when it crashes because it can’t make the page file any bigger to accommodate the 10 more users that have made crazy demands of it… don’t blame Microsoft or the server. It was doing exactly what it was supposed to do.

The question is, were you paying attention? Did you have monitoring? When you deployed that new web part and then configured it into your master page so that it appears on every single site in your environment… did you load test it first? Did you ensure that what works fine with 1 user also works fine with 1000 users? At the same time? Over the course of the day? When you’re updating your enterprise search indexes? When you’re unknowingly being scanned by the other enterprise search product you didn’t know was crawling your entire site?

Servers are constantly changing, as is the workload they’re trying to do and the environment they’re operating in. When they can’t keep up with what we’re asking them to do, it’s our jobs to keep an open mind, to understand what might have changed (even if we didn’t change it), and understand the relationship between what failed and what might have changed.

Just because you didn’t change your server doesn’t mean it’s the same today as it was yesterday.

Separation of Roles

We see many instances in which the SQL DBA for SharePoint is the same as the SharePoint administrator, which also happens to be the same person as the security admin, and the support staff, and the etc., etc., etc.

Even though we see these roles blended for various reasons (lack of understanding of the importance of having role separation, not enough money to hire people, not planning the production deployment completely), SharePoint does actually have strong role separation by default. “Role Separation” means that administrators of one piece, tier, or component of the system may not necessarily need administrative access to other pieces, components, or tiers.

This is a valuable consideration for security and for stability. For example, you likely don’t want your enterprise domain administrators to have access to the CEO’s mail. For stability, your SQL DBAs probably don’t want the windows administrators manipulating SQL configuration settings. The same is true for SharePoint… having access to administer the SharePoint farm doesn’t necessarily mean a user should have access to the content within that farm.

So, if you were to design a team with complete role separation (assuming you have enough people to cover each role discretely), what might the team look like and what roles would they have? Here’s my list…

• Platform Administrators – This group has “Local Administrator” access to your Windows machines. They likely own all patching and general system stability. They may potentially own additional elements such as platform security (managing the windows firewall) and antivirus maintenance.
• SQL Database Administrators (DBA) – This group has “SysAdmin” access to your SQL server, but may not necessarily have or require administrator access to the underlying Windows installation. This person is responsible for ensuring SQL is configured optimally, and likely contributes to investigation of any errors or difficulties in the environment.
• SharePoint Farm Administrator – This group is listed in Central Administration in the “Update Farm Administrators Group”, and has complete management rights to manage the logical architecture of the SharePoint environment. This person may not necessarily have administrative rights on the underlying Windows OS, though this would not be unusual. This group is also responsible for managing any SharePoint updates, service packs, or hotfixes, and investigating and resolving SharePoint based errors in the environment.
• SharePoint SSP Administrator – This group can manage settings in an individual SSP, particularly search and business data, that may be confidential or may be specific to web applications that have specialized or independent needs. This person may not necessarily have access to the overall Central Administration website, and will almost certainly not have administrative rights on the underlying Windows OS
• Web Application Policy Members – While this may not have a specific “role”, people with these rights set have the ability to override site collection level settings throughout the application. For example, having “Full read” on the web application means that you have the ability to read content in any of the site collections within that web application, even if you have no specific rights listed in the site collection itself.

All of the roles above are System or Application roles… that is, none of them necessarily have a direct impact on content (with possible exception of web application policy)… but more importantly, none of the roles above are stored with content. If your farm needs to be rebuilt, the above permissions would all need to be rebuilt also.

The roles below are arguably content roles… that is, roles that have a direct impact on content, are visible in the users/groups lists in site collections, can and frequently are managed directly by users, and most importantly, are stored in the content database. Though it may be tempting to attempt to rigorously control these (and there are good reasons to do so), these are more likely best distributed throughout the organization and are frequently managed by people outside of the farm administrators group.

• Site Collection Administrators – These people have overall rights within a given site collection, including certain very specific settings that might otherwise be handled by the Farm Administrators, but make sense to distribute more widely. These people should also be a primary escalation point for SharePoint users (they have certain abilities to override the user activity), and should be included in the governance strategy for SharePoint (since they see user difficulties and needs most).
• SharePoint Group Managers – These people have been listed as owning a SharePoint group, meaning they can add and remove users from those groups. This implies that they also have the ability to grant group members access to things the group itself has been granted permissions to, but do not have the ability to impact permissions directly. This can be very useful in maintaining coherent and manageable permissions while still giving others the ability to provide users with access without needing to call the Site Collection Administrator or create a support request.
• Everyone Else - Below this, we have the permissions that most people are familiar with… groups and users being granted access to SharePoint resources (sites, lists, items), which are again managed directly in the content by various people that have been given the ability to manage those permissions.

The most important points in the above lists are that the roles can be very, very discrete. For example, being the DBA does NOT mean you have access to content in the UI. Yes, it is possible for you to override this… but such an override would generally be visible in the system, either to farm administrators or to users.

As with most security systems, there is (by design) a mechanism for overriding security. For example, it may be reasonable for farm administrators to add themselves to the web application policy for specific investigation purposes… but in a well managed environment, these rights would generally not be required. However, this possibility means that regularly auditing those roles to ensure they haven’t been misused is an excellent activity. Such audits should NOT be scheduled, but should at random intervals, but should still occur with some amount of regularity.

Technically, there are many more distinct roles. For a complete list, look here: Plan for security roles (Office SharePoint Server)

Clarifications: Collaboration vs. Publishing

In SharePoint-Land there are two concepts that people have a hard time separating out: “Collaboration” and “Content Management”. A lot of people like to blend them together, use methods, features, technology, or processes… but the truth is these are separate capabilities, and should be separately managed.

Lets compare…

First, lets look at the communication model that actually happens between the two:

Collaboration	Publishing
Peer based, bi-directional sharing of information	Generally unidirectional, leader-based communication of information.

When we’re collaborating, we’re all sharing information with others in our group, and those people have an equal opportunity to share information as well. In publishing, there is usually a small group of people communicating to a much larger group of people. Think of it as the difference between working with your peers in a conference room, all working together to get the job done, and your CEO doing an announcement to the entire company.

For example, in a publishing site (such as your company’s “www” site), how you say something matters nearly as much as what you say. It’s all part of one larger message. So the layout, the display, colors, pictures, and functionality are all a reflection of the primary message. In collaboration, we’re not worried about how much the site matches our homepage or that the colors fit with our marketing message… these things have no impact on our ability to share information, exchange ideas, update status, or publish documentation.

Microsoft Office SharePoint Server 2007 has numerous site templates to serve varying needs, but the three that tend to matter the most are the Team Site template, the Collaboration Portal template, and the Publishing Site template.

• Team Site template – a simple site template designed to help a group of people (a team?!?!) work together to exchange information and ideas. It’s usually lightly branded (themed), but overall looks and acts like “SharePoint” should.
• Portal Templates -
  • Collaboration Portal – Usually used for an intranet, it supports things like content staging and heavy branding, along with ways to gain access to the additional collaboration (read: Team Sites) site collections below it.
  • Publishing Portal – A publishing template similar to the collaboration portal, but more “blank slate”, designed to be customized to your heart’s (or MarCom director’s) content. This is usually the starting point for building your internet site, which will be published to a read-only SharePoint environment accessible through or on the other side of the firewall, and accessed by your customers.

The big problem I see is people using Team Sites for publishing, and publishing sites badly.

So, how do you know what kind of conversation you’re having, site you’re being asked for, or template you should use?  Here are some sample requests:

• “The CEO needs a site where they can communicate to the rest of the company.” – Collaboration Portal… it’s one person talking to many people. That’s publishing, so a publishing template is appropriate. The CEO will likely want a lot of branding too.
• “Our team needs an area to share documents for our project.” – Team Site… it’s a group of people sharing information together. They don’t care about branding, they care about their documents and delivering the project on time.
• “We want to build a knowledge base site for our customers. We’ll need anyone to be able to create content, have a review and approval process, then have it published as the final step.” – Publishing Site with a staging environment… the internal site will be used for all of the content generation processes and workflows, and the external site would be read-only and available to all users. The heavy branding and content deployment features would be supported by the publishing features.
• “We want our intranet to be collaborative and to provide a seamless experience for our employees.” – Collaboration Portal with collaboration site collections… keep reading…

The last item is one of the most common… and we need to be clear. Yes, we do Web Content Management, including supporting “collaborative” intranets… but we have to reiterate that SharePoint isn’t just a simple website… it’s a complete web application. Would you consider heavily branding Outlook? Does your OS have your branding all over it (except maybe your background image)? Are your ERP, Training, Travel, or HR systems heavily branded? Probably to various degrees… and SharePoint Team Sites can be too… to a certain degree.

Ultimately we need to clearly understand what features we need for publishing vs. collaboration, and what we can and should do to support those needs. Publishing needs heavy branding, high performance read-only access (caching), staging and deployment, while Collaboration needs clear functionality, ease of engagement, consistency between sites and high performance read/write access (no caching, fast servers).

When you talk to your stakeholders, try not to think about how you can bend the product to their whims, but rather how you can effectively translate what they’re trying to accomplish into something SharePoint can support directly. A stakeholder may say “I want the intranet to support collaboration directly.” One approach would be to create one massive site collection with heavy branding throughout… but you don’t really need that. Chances are a main intranet with separate collaboration site collections beneath it (beneath the site directory for example) would be just fine… and if we need to blur the lines a little, search web parts can present content from across the intranet on one page.

…and if you really need to blur the line, you can always custom write something… but that should be your option only after the built in capabilities have been exhausted… and usually they’re not.

So, use team sites for team collaboration… and portal sites for web content management. The less you try to make one act like the other, the happier you and your customers will be in the long term.

Right vs. Wrong (not)

Someone recently posted a comment about one of my previous blog entries, and their comment in their blog reminded me of a point I’ve been meaning to make for some time:

It’s important to understand that when when you make a decision to follow or not follow a suggestion or recommendation, you really are making a business decision that has benefits and trade-offs… and you have to decide if the trade-offs justify the benefit.

For example, it is perfectly reasonable to decide that you don’t need a development environment… but you have to accept that either you won’t be doing any development, or that your developers are going to be developing in a production environment. The benefit of not having a development environment is the cost savings that comes with fewer environments to purchase and manage… but the price of that decision is that your production environment will be constantly changing in unexpected ways as developers deploy unfinished or untested code. Likewise, it is perfectly reasonable to say that you do not need to do load testing… but the price of that decision is to not be able to clearly know how a planned change is going to affect the performance characteristics of your environment. Finally, you may also decide that the additional servers, configuration, and expertise required to create a reasonably fault tolerant environment aren’t important to you… but the price of that decision is the risk of your environment going down when any single link in the chain is broken.

Again… none of these are right or wrong… they’re just decisions that must be made, and benefits and trade-offs that must be acknowledged.

The real problem is in not taking the time to truly understand the benefits and trade-offs of the decision you’re making. If there is anything I would say is truly wrong, it is not taking the time to understand what those benefits and trade-offs are. This blog is not about telling you what right and wrong are… it is simply an attempt to give you the best information available to help you make informed decisions. You may make decisions that are different than those that another might make… but at least you’ll know why you made that decision… and the Why is always more powerful than the What.

Stephen… thanks for the reminder. :)

Portal Site Navigation Decisions

Equal only to branding, your SharePoint portal navigation structure has one of the biggest impacts on the usability of your site. Done correctly, it reflects how people in your organization work (and how you would like them to work), the importance of different aspects of the business/organization, and can very often force you to make decisions that affect the functionality of your environment as well. Unfortunately, because of these things, it is also very, very political. The more powerful an individual is in your organization, the more prominently they feel they should be represented in the portal… and because they are powerful people they frequently get their way. The result is a navigation structure that does more for your organizational chart than it does for the people using the portal.

So you have to get SOMETHING up, and you have basically every CxO calling you wanting their site to be at the top of the portal… so you end up with something similar to the following:

On the surface, this isn’t exactly horrible… and it will likely work well for you initially… but lets consider some perspectives:

• The primary goal of deploying SharePoint (at least the initial deliverable in most cases) is to increase cross-team collaboration.
• Most companies go through at least minor organizational changes at least once every six months… and a major, high-level reorganization every 18 months.
• Most people contributing in non-essential (“Consulted” or “Interested” for those familiar with the common RACI methodology) have absolutely no idea which group actually owns a given project, despite their supposed involvement.
• The vast majority of what any employee does in a company is either project, program, or operationally focused.

So… a year later, you have a problem:

• A major reorganization has happened… the Whatsit division has been merged with the Widget division, and IT (which has been renamed to “Information Services”) will now report to Finance, leaving you with a decision about what to do with those URLs and sites… do you keep them as-is, or do you break people’s bookmarks so that your portal more directly aligns with the organizational structure?
• You’re frequently being told that employees are having difficulty locating what they’re looking for because they don’t know what organization owns a given project, program, or process. For example, is my tax information held by finance or by HR?
• Collaboration has not increased… in fact, people are focusing even more intently on their silos, as identified by the fact that both the Widget and the Thingie divisions have blogs (that no one is reading), and the Whatsit and Thingie groups are finding that a large amount of their Wikis contain duplicate (and frequently conflicting) information.

The problem here is that we are incredibly focused on our organizational structures as our map of how to get things done… but in a world where we’re intentionally trying to work as if those boundaries didn’t exist, building our collaboration space based on them is counterproductive. Instead, we need to focus on how people are working (and how we want them to work), what they’re working on, and what they need to work on and/or eliminating the things that are preventing them from working. Finally, from a SharePoint Farm Administration perspective, we need to find a way to reduce the number of changes to the high-level structure of the environment, and we need to try and maintain the overall performance of the system. Finally, remember that although it is easy and tempting to make URLs match site names, this is not a requirement… and is frequently a less than perfect suggestion.

After working with numerous customers, I’ve come to a general purpose structure that at best can provide a direct model to work off of… and at a worst demonstrates an example of how to create a SharePoint site hierarchy that can flex and bend with the ever changing corporation:

This model presents several significant changes, most intended to refocus the portal and the employees on collaboration:

• The top-most level of the site is now based on how people work and what people work on rather than what group they work with/for. This benefits collaboration because it doesn’t reinforce the idea that a project or program is owned by one particular group… it reinforces the idea that we can all help and work together without regard to our organizational structure. We’re not here to be “Employees of the Widgets Division”. We’re here to get that Widget 2.0 project done!
• The top level of the portal is now fully independent of changes in the organizational structure. Of course, different departments may come and go, and that may have ripple effects as to what projects/programs continue, but for the most part changes in the organization do not have an absolute effect on the projects/programs in place.
• The URLs are intentionally designed to be easier to understand at the portal level, and don’t necessarily match the name or title of the site. “Technology Support” actually links to the IT helpdesk (whether it’s SharePoint or not), and is independent of what IT/IS/I? is formally named.
• The Site Directory is now more heavily used, and every site in the company (Portal or team site, SharePoint or non-SP site) is now listed in the Site Directory. Columns have been added that allow sites to be easily found and organized. For example, every site can be associated with a department/division. This allows ownership to be maintained while reducing cross-departmental boundaries in the sites themselves.
• There is now ONE wiki. This may seem trivial, but it is actually very, very important. The power of wiki is in its ability to serve as a single collectively created and managed, authoritative resource for knowledge. It is tightly interlinked as new terms are found that should be clarified or documented… and generally a wiki is only truly integrated with itself. If you have 100 wikis with 5 pages each, they’re not very useful… but one wiki with 500 pages is truly powerful, useful, and authoritative.
• Each department still has its own site for information that is specific to it… internal processes… meetings that are specific to the organization (department meetings, internal announcements, etc). However, each department site is at the exact same level… flat… organizational structure is irrelevant… meaning that department sites can come, go, and change without dramatic impact to other sites. Further, since a huge amount of the project and program information that was in these departmental sites has been moved, the content in these departmental sites is dramatically reduced.

This structure also provides significant benefits for the system administrator and system maintenance, specifically with regard to content database management.

• Since projects are now each their own site collection, “active” projects can stay in “active” databases. As a project completes/closes, the supporting site collection can be either removed entirely (the contents migrated to a file share), or can be locked, backed up, and moved to a lower availability, lower capacity SQL server. For example, this SQL server might use a simple internal RAID array rather than expensive SAN-based storage. If this server experiences any kind of failure the impact is minimal since these content databases rarely change… restoring from a week-old backup is perfectly acceptable.
• The fact that site collections are used also ensures that as sites grow, they can be shuffled between different content databases with no visible impacts (though this should be done as minimally as possible).

Of course, there’s nothing we can to do stop sites from needing to be created, moved, and deleted… URLs will always change at some level… but the lower you can make those changes, the easier the site is to use for your users… and planning for what might happen, if you take the right steps, is a lot easier than trying to fix what did happen after the fact. Planning is key… including planning for what you don’t know, or what you do know is likely to change.

The Minimal Full-Deployment Infrastructure

Many of my customers ask me questions about what differentiates a successful MOSS deployment from a difficult, troublesome, or problematic environment. My answer is simple to say but difficult to manage well, and can be summarized in two phrases: "Load Testing and Configuration/Patch Management".

These two phrases are fantastic because they describe an end-goal... and in order to do them properly, imply that a lot of prerequisites are achieved. For example, proper load testing requires that you have an appropriate load testing environment, and that you do testing at all (this is common sense that is surprisingly rarely performed). Configuration Management implies that you have a process for moving changes and updates from one environment to another and have clear documentation on those processes (even less rare than testing). Ultimately, having these capabilities also implies having a certain amount of infrastructure to support them.

In a perfect world, here's what your MOSS infrastructure might look like:

Each environment provides support for a specific purpose in the infrastructure. The descriptions are as follows:

As you can see, each environment in this infrastructure provides significant capabilities that are both critical and necessarily isolated. For example, you would never want to perform load testing in your production environment… and yet, should you not have the ability to perform load testing, this is effectively what you’re doing, and making your users suffer through the experience.

Does it seem like a lot of infrastructure? Probably… but the benefits far outweigh the costs. Not following this minimal guideline frequently results in increases in expenses in other areas, such as additional support staff due to the need to support user requests when unforeseen problems arise, culminating in a “reactive” support model. If you find yourself never being able to get ahead, constantly “putting out fires” rather than planning and providing for new or enhanced capabilities, you’re operating in reactive mode.

No… this won’t solve all of your problems, and it doesn’t take into account other needs that could be best served by having yet another completely separate and independent infrastructure (recovery farm?), but following this guideline will move you significantly into the “Proactive” area… and maybe win you a few awards from your customers. :)

The Value of Effeciency

It's sometimes difficult to justify a significant investment in a knowledge management system... it seems that getting a reasonable amount of funds to properly invest in and deploy a tool who's basic promise is to help people work together better (the most basic description of what KM is intended to do) can be a difficult sell... and causes immediate questions. What exactly are you going to do? What is the outcome going to look like? How much is it going to cost? How much time is it going to take? What products do you need? How should it be architected? Ultimately, how much are we going to save by doing what you want? KM is a fuzzy field with frequently ambiguous costs... and the simple act of trying to get answers to these questions... product investigations... Requests for Proposal, knowledge gap analysis, social network and interaction analysis, business process design... all of these things take time for you to do, and time from others that you have to work with to do it. So why would anyone bother?

In my work as a Knowledge Manager in a prior life I figured out a simple calculation to put a number on the potential value of a KM strategy. It's simple, very generic, applies to every company in the world, and uses 2nd grade math at best. No, it's not exact, and there are huge factors that aren't accounted for... but I strongly feel it is the fundamental calculation for understanding what a KM initiative can do at it's most minimal level.... and it's stated as simply as this: How much money can I save the company by saving time... and how much money are you (Mr. CEO) willing to invest to save that money?

Lets pretend for a moment that you're even a relatively small corporation of say, 1,000 people (not just employees... non-employee workers also contribute to company costs). Lets also pretend that the "fully loaded" cost of each employee is $100,000 per year. This isn't just your pay... that's pay, insurance, benefits, taxes, 401K matching, etc... everything that adds up to your total compensation plus the simple overhead to your employer of employing you. Lets also pretend that there's a lot of things that you do every day that aren't ACTUAL work. Things like looking for answers to questions, answering questions of other people, filling out forms, starting and engaging in disconnected or even manual business processes. Now lets do some math...

First, lets break down the actual number of minutes you work in a year (assuming 10 days of paid holiday and 2 weeks paid vacation): 52 weeks - 4 weeks x 5 days a week x 8 hours a day x 60 minutes an hour = 115,200 minutes a year.

Now lets find out how much you cost per minute: $100,000 / 115,200 minutes = $0.868 dollars a minute. (just reading this far has probably cost someone about $3 of time... think about it.)

Okay... now lets pretend we think we can save you maybe 5 minutes an hour... or $4.34 an hour... or $34.72 a day... or $173.60 a week... or $8,332.80 a year.

Already, this is nothing to laugh at... but remember... there are 1,000 of you... so if we saved everyone in the company 5 minutes an hour, we would save the company $8,332,800.00 a year.

8 MILLION DOLLARS.... for 5 minutes an hour.

EVERY YEAR.   (or, lets say $40 Million over 5 years?)

How much are you willing to pay to get $8 Million into your pocket, or $8 Million into your budget?

This is what I was trying to do as a Knowledge Manager... and now, this is what I try to do with Microsoft Office SharePoint Server 2007... a product that is effectively a KM strategy-in-a-box. Getting to spend my time with my customers enabling them to be more effecient, faster, more resilient, more stable, more able to change quickly to evolving business climate. Yeah... maybe I drank the Kool Aid a long time ago... but at that price, it's effectively Cristal. :P

So, other than not doing it, what is the biggest mistake I see my customers make? UNDER INVESTING. The try to do all of this with the lowest possible investment in software, servers, people, training, planning, and strategy, and then find themselves having invested in failure. Please, please, PLEASE let yourself be successful, and your people happy (because they're successful), and your business successful (because it saved $8 million dollars).

We're here if you need any help at all. :)

What vs. Why

Documentation: Saying the word makes most geeks cringe, cry, or run to the comfort of the datacenter for safety. Aside from hating to write it, there's a perception of "What do I need to document for? I know what I did?!".

First answer: No, you probably don't.... but that is really the right answer to the wrong question. Knowing exactly what you did is only about 20% of the value of documentation. Yes, if a clear "Click this, click that" process is provided it's somewhat reasonable to expect that an installation or configuration be done consistently... but much of this can be fixed by using a well managed and stored command-line installation process.

Think about a different question though. Think about how many times you've asked the question "Why do we do it that way?" or "Why is it like that?" or even "Why should I configure it like that?" (despite what is listed in the installation scripts). The truth is that "Why" is much more important than "What". Even as you document the configuration you're implementing, you're doing it based on information that you have at that moment... information that others may not have or understand, and information that not even you are likely to remember a year from now. At that point someone (or again, even you) will take a look at your process and make a change thinking "that's not right" only to have the environment break before your eyes, or worse, create some cascading effect into other systems that cause them to fail as well.

"Why" is what creates the "What". If you correctly document why you're making a decision, you're providing an opportunity for others to understand the configuration well beyond simple button-clicking or script-running. You're helping people understand why the decisions you made are good ones, what options shouldn't be changed, what options can, and how to better interact with and manage the environment. More often than not, if you clearly document why, you can rebuild the "what" later... because all of the reasons that led you to a conclusion in the first place are clear.

So, as you're writing your documentation, consider writing these things down at a minimum:

• The page/screen/command being run
• The options/parameters being selected or provided to that screen
• A capture of the screen with the correct options selected
• The reason this was the correct option or setting in your environment
• The output or outcome (if available) of the command if it was executed properly.

A year from now when you're trying to remember why on earth you made this obviously bad decision, you'll be glad you wrote it down... and glad you didn't break your entire intranet portal in the process. ;)

Closing the Technology vs. Business Gap

I'm Chris Mullendore, a Microsoft Premier Field Engineer focusing on the Microsoft Office SharePoint Server ("MOSS" for those in-the-know) product line. In my prior lives I was Knowledge Manager for one company, and a software developer for another, meaning I've worked on both the technology and the strategy side of things... and there's no question that the technology side is where my true interests lie.

My experience though has led me to the understanding that there is a small gap (Grand Canyon kind of small) between what is technically possible and what is strategically prudent... and I have never seen a product make this as incredibly obvious as MOSS. The reason is that MOSS offers a huge amount of capability and options, but it is in a rare category because almost every action taken has a direct end-user impact. Lets take Active Directory for example... most people don't really even know it's there. Their computer works, they log in, they get access to what they need, they request access to groups, and they use information from it all the time... but they frequently don't know that it's AD they're actually interacting with. Plus, a lot of configuration can happen in AD with almost no visible impact to an end user. OUs can be created, accounts can be managed, permissions can be set, etc., and all the user really knows is that they can get to what they want to get to.

MOSS is different. Though there are certainly exceptions, almost every decision you make has an impact on the user. SSP configuration options, search settings, permissions, BDC configuration, even backups (if you lock the content database) have a direct impact on users. You can do your best to hide these changes as much as possible, but the changes are still there and still have an impact, even if minimized as much as possible.

So that's what this blog is intended to focus on... to call attention to the business impacts of technical decisions in Microsoft Office SharePoint Server 2007. I'll occasionally be more business, occasionally more technical, but my goal will always be to address the relationship between the two.

"With great power comes great responsibility"
As MOSS administrators and technical specialists, we all have huge power in the systems our customers (yours and mine) are using... and it is our responsibility to take action with caution and care, ensuring that the benefits we're trying to achieve come with as few tradeoffs as possible.

primum non nocere