Special Thanks to Tim Walton for Researching And Contributing This.

PROBLEM:

Slow response from custom SharePoint Application components may be happening because we are seeing a timeout going to the Certificate Revocation List.  As I understand the behavior is that we may wait for up to 45 seconds before .NET concludes that the CRL is unavailable.  This conclusion may then be cached for up to a minute before a fresh attempt is made to contact the CRL again.

 THEORY:

To verify that the certificate used to sign the .Net package hasn’t been revoked, .Net will attempt to download the Certificate Revocation Lists from both of these URL’s:
http://crl.microsoft.com/pki/crl/products/CodeSignPCA.crl
http://crl.microsoft.com/pki/crl/products/CodeSignPCA2.crl
If the .Net application cannot reach those end destinations then the .Net package assumes that an attempt to block verification has occurred and that the .Net packages signed by the certificates are not to be trusted.  End result, the .Net packages may run slowly.

PREFERRED RESOLUTION:
Allow the server and the services accounts to access the crl.microsoft.com domain.

RESOLUTION 2:
Turn off CRL checking by creating a turnoffCRL.reg file with this as the content:

Windows Registry Editor Version 5.00
[HKEY_USERS\S-1-5-20\Software\Microsoft\Windows\CurrentVersion\WinTrust\Tru¬st
Providers\Software Publishing
"State"=dword:00023e00

Note that this would be considered to be compromising the integrity of the .Net signed packages stuff as we would have prevented any compromised certificates from being disabled automatically.

RESOLUTION 3:

Manually add the CRLs from the above Urls to the server manually (assumes you’ve downloaded Them locally first):
certutil -addstore CA CodeSignPCA.crl
certutil -addstore CA CodeSignPCA2.crl

I thought I might spend of a few minute with my answer to this question that seems to be coming up with managers more often with each passing day.  My response is complicated to explain to becuase the answer is always you are asking the wrong question and exposing yourself to risk by limiting your attention to the traditional focus.

First, lets start with the my opinion.  SharePoint is a platform similar to ASP.NET or Windows that you build applications and services on.  SharePoint respresents the first true class of products by Microsoft that completely blurs the line of Admin and Developer.  Thinks of it in these terms.  VBScript was the first technology Microsoft developed beyond bacth files that automated the administrators job.  Since then we have made the gigantic leap to .NET, C# and Powershell.  These technologies have changed the game and the sooner Engineers, Developers and Management understand this the better.  This is not to say that the traditional role of application developer or adminstrator is dead.  There will always be the need for specialist that focus at a deep level upon building applications and administratering them.  However, the future Star IT employee is the person that can engineer and write code across all parts of the solution.

So when asked, by my customers how and what should I be looking for in an finding an administrator for SharePoint, my response is always STOP looking for an adminstrator and start searching for the NEXT GENERATION IT STAR that can code and adminster your solution.  The simple fact is that when looking for that star you need someone who understands common adminstration skills (installing windows, patching, monitoring and configuring the server as well as managing the hardware, adminstering SQL, Networking and IIS) and then they need developer skills (.NET Coding, Powershell, WindowsServices Distributed Computing, XML,CAML, SQL Coding, AJAX, JavaScript, HTML, etc...).  This is the perfect solution but it will take time for the world to get to this point and not everybody in IT can accomplish this task.

The second point I make is that the distributed engineering strategy of managing an IT environment by organizing along traditional product responsabilities such as having a Core Windows Team, Client Team, SQL Team and SharePoint Team is not going to result in a cost effective succesful solution.  To be clear I am not saying that having a Core Windows Team doing tasks such as Building Servers and Installing the OS is wrong.  Nor is having a core Client, SQL or SharePoint Team is wrong.  What we see in the real world is the creation of the virtual team made up of members of each team.  In some environment this works fine, but in larger environments this approach will only result in less than agile and fragmented team that creates more problems and costs to the overall operation.  What needs to happen is the opposite approach and adoption of a service based approach.

My suggestion upon this matter is to look at the technologies from a service provided approach.  For example, SharePoint is a collaborative platform and demands in a larger environment that the entire scope of technologies be managed as a distinct unit with a solidified plan under a unified direct management structure.  This trend started with Exchange and has grown into far more by the evolution of the technology.  This is a pretty radical idea for many organizations and push back is always expected.  Regardless, this approach has to happen at some point.  So an appropriate team would include an organization model based upon service rather than product.  Even more important is that finding that IT STAR will be hard at least for a while until the workforce adopts the approach that being an integration expert is where the real money and possibaility of sucess exists.  So until that occurs and even after that occurs organizations need to move from the virtual team based upon products to services where the priorities and goals are unfied.

The answer to these problems is simple and is determined by the politics, size and scope of your organization.  In the large organization there should be a collaboration team which includes Windows, SQL, SharePoint, and Developer Resources under a single manager with a unfied business strategy and goals driven by a strong SLA.  This means the organization needs to start doing the opposite of the product based virtual team but rather reorganize into the service based team where for example the Collaboration SQL expert is a vrtual member of the companies core SQL team.  This doesn't mean that the organization core SQL team drives the collaboration implementation of SQL but rather the Collaborative SQL team member communicates with the core team but the final decision is made by the collaboration SQL expert and project manager.  There might not be even a core product team in the organization, as a true service based model would allow for the SQL team made up of members from all thekey services model based team.

The answer is complicated and the solution is even more complicated requiring organizations to reorganize teams and develop IT STARS.  In short the traditional old school organization is going to flip with the initial idea and work required to implement what I am suggestion, but the end justifies the means and where this approach has been adopted the success is clearly obvious.  The strategy is also well documented and promoted as being the future of IT. 

The proof of this approach is not only well documented by IT Management organizations but also by history in general.  A simple look at hostory shows that the the most agile organization is the always the most sucessful.  For the sake of brevity, I will spare reader the countless examples I could provide from history proving this advice, but if you really want it send me an email and I will provide you the evidence.

What I will provide is a solid proven example of how the United States Marine Corps has taken this strategy to fact by being agile enough to build a services based unit called the Marine Air Ground Task Force (MAGTF).  MAGTFs are a balanced air-ground, combined arms task organization of Marine Corps forces under a single commander that is structured to accomplish a specific mission.which we would define in IT as a Service.  If the government can pull this off then it can certainly be implemented with even greater success by the private sector.

Some good links supporting this advice are:

Workforce Management by Microsoft

Gartner IT Strategy

HP IT Strategy Management

HP Services Mnagement Framework

Microsoft Services Management Framework

ITIL Service Level Management

MOSS SP1 Upgrade Process:

As defined and cleansed by http://technet.microsoft.com/en-us/library/cc263467.aspx#section1

Database Maintenance:

1.      Run MOSS BPA and resolve any problems defined

2.      Stop the www service and disconnect any users

3.      Run DBCC CHECKDB on each database and repair any problems.

4.      Remove Orphaned Items from databases using  stsadm –o databaserepair

5.      Defragment and update statistics of each database

6.      Make sure that there is adequate hard drive space in your database files volumes, tempdb volumes, and Windows temporary folder on the servers running SQL Server, front-end Web servers, and application servers.

7.      After you have backed up all of your databases, use the SQL Server DBCC shrinkfile command to free unused log space, making the logs as empty as possible.  Shrinking the Transaction Log

Upgrade Pre-Steps:

1.      Load Balance Site Collections and ensure your not exceeding site collections limits per content database and make sure content databases do not exceed 100 GB as recommended.

2.      Make sure that you follow the recommendations concerning SQL Server page-fill factor and other storage planning best practices before you begin the upgrade.

3.      Backup           

a.      Use the full backup operation from SharePoint Central Administration or Stsadm to back up search.  The backup should include both the SSP and the index file.

b.      Brick level backup the configuration and central admin content databases

c.       Backup the SSO and every content database

d.      Backup any Front End Customizations such as Templates, features, resources, webparts, etc..

4.      OPTIONAL - In server farms that have a large number of sites, you will find that installing a software update with the content databases attached is not practical in terms of downtime. You may want to pursue this route for any content db > 20 GB as well.  In order to minimize the downtime, we recommend that you perform the additional steps of running the Stsadm preparetomove operation and then detach the content databases. Do not run against the CONFIG database, SSP or ADMINISTRATION CONTENT DATABASE!

a.      Run this command for each content database

stsadm -o preparetomove  -contentdb <db_name>  

If you do not run this operation before you detach the content database, then the membership and profile information in the content database is static and will not be synchronized after upgrade.

b.      If you have multiple Web applications you must run.

stsadm -o deletecontentdb -url http://servername -databasename

c.       In this operation, -url specifies the Web application from which the content databases will be detached and -databasename specifies the name of content database to be detached.

After you upgrade your server farm, you must attach the content databases back to the server farm. You can only attach one content database to the server farm at a time, because when you attach the databases to the upgraded server farm the content database is upgraded automatically.

5.      To deploy the upgrade you must have these memberships

a.      Member of the Administrators group on the Web server computer.

b.      Member of the Administrators group on the server running SQL Server or be granted the fixed database role db_owner to all SharePoint Products and Technologies databases.

Install Sequence

1.      Add the account for the SharePoint Central Administration v3 application pool identity to the Administrators group on each of the local Web servers and application servers and then log on by using that account. These changes are only required for installing the update and then running the SharePoint Products and Technologies Configuration Wizard to complete the upgrade.

2.      The following permissions are required to run psconfig

1.      Member of the Administrators group on the local computer that runs Office SharePoint Server 2007.

2.      In SQL Server, the account must be:

1.      Authorized to access all SharePoint Products and Technologies databases.

2.      Granted the Database Creators (dbcreator) fixed server role.

3.      Granted the Security Administrators (securityadmin) fixed server role.

Install the software update

This section includes all of the procedures required to install a software update successfully in any size server farm. If you are in a large server farm you should read the Large farm optimization section later in this document.

The following procedure provides the steps to:

·         Make all software update files available on all servers in your server farm.

·         Complete the update from one of the servers hosting the Central Administration site.

·         Finish updating the remaining servers in the server farm.

Note

You must perform steps 1 though 8 from the following procedure on every Office SharePoint Server 2007 server in the server farm before you complete the installation on any one Office SharePoint Server 2007 server.

To install a software update

3.      OPTIONAL only If you detached content dbs use the following command line to add them back to the farm

stsadm -o addcontentdb -url <http://backupservername:port> -databasename <ContentDBName>

If you detach and reattach a content database, be aware that the next time the content within that content database is crawled a full crawl will occur even if an incremental crawl is requested. Because a full crawl re-crawls all content that the crawler encounters, regardless of whether that content has been previously crawled, full crawls can take significantly more time to complete than incremental crawls.  CPU will be maxed!

4.      Verify the server install by running the following command against the config database

SELECT * FROM Versions

1.      Release 12.0.4518.1016

2.      October public update 12.0.0.6036

3.      Service Pack 1 12.0.0.6219

4.      Post Service Pack 1 rollup 12.0.6300.5000

·         OPTIONAL Force an upgrade by running one of the following commands if you encounter problems

1.      psconfig –cmd upgrade –inplace b2b –wait –force

2.      psconfig -cmd configdb -connect -server <SQLServerName> -database SharePoint_Config_<dbname> -user <domainusername> -password <password> -cmd helpcollections -installall -cmd secureresources -cmd services -install -cmd installfeatures -cmd applicationcontent –install

·          Next Install Rollups 1 & 2 for both WSS and MOSS and reboot the server

IMPORTANT NOTICE FOR FUTURE INSTALLS:

From this point forward when you add a new server to the farm you should make sure you do the following:

1.      Install MOSS with the SP1 Slipstream Version

2.      Install The Latest Rollup which are presently WSSv3 Blackout rollup & MOSS Blackout rollup

3.      Run PSConfig to attach to the farm.

You have two choices, you can use either of the following stsadm cmds: searchadcustomfilter or setsiteuseraccountdirectorypath.

setsiteuseraccountdirectorypath: 

When people picker resolves the user, it will check whether the user exists in the site collection or not. If it exists, return the user. Otherwise, search the AD or membership provider. For the AD case, if there is “siteuseraccountdirectorypath”, it will only search under that directory path, otherwise, the whole AD is searched.

Suppose the site is empty. After the administrator use

-o setsiteuseraccountdirectorypath

Only users under that path could be added to the site collection and no one else could be added to the site collection. In such case, the user returned will always be under the “siteuseraccountdirectorypath”.

Suppose the site is not empty and there are already some users exists. After the administrator use

-o setsiteuseraccountdiretorypath

To add a new user, the new user must be under the directory path. The people picker will return existing users in the site collection and users under the directory path.

searchadcustomfilter:

This path allows you to control the users shown in the PeoplePicker control by LDAP, BUT a user can still type in a valid alias and click the check name button to be given access.

VMWare could impact Windows services that depend upon the Windows Time Service for maintaining time synchronization across systems.  Apparently the proper way to keep the guest time synced is to use the VMWare TimerSync and not the default Windows Timer Sync.  If the system was to become out of sync by more than 5 minutes let’s say then Kerberos could be a problem.  This could impact user access or Timer jobs performing tasks.  So if a customer has not configured this correctly then there could be problems.  The solution however is an easy workaround described below.

VMware Time Sync and Windows Time Service

http://kb.vmware.com/selfservice/viewContent.do?language=en_US&externalId=1318

“The most accurate way to keep guest operating system time synchronized with real time is to use the VMware Tools time synchronization function. You should not use the Windows Time service or other form of clock synchronization meant for physical machines to set the time in the guest operating system.”

Over the years I have seen a lot of SharePoint deployments from many customers going back to 2003.  This has allowed me to develop a unique perspective and observation on how to deploy this product best.  Here is what I have learned are the top ten things you need to do to make your deployment the success that will provide you with the best ROI and take full advantage of the product.  MOSS and the surrounding products such as Office, Exchange, PerformancePoint, Office Communication Server, Project Server, Excel Server, Forms Server, SQL Analysis and Reporting Server have the potential to provide the same productivity increases seen when email was deployed universally in the 90’s.  Make full use of these technologies AND make Your Company succeed.  That being said, it is important to remember that MOSS is about people and collaboration not products.

1.       “MOSS is not your father’s corporate web!”  When you deploy MOSS you need to think future.  You need to be thinking about not only deploying a portal but also a collaborative integration solution that will dramatically increase your companies productivity potential.  MOSS is not just about document management and lists.  Collaboration/portals has to be about people. If you don’t plan on training your end users about new ways to think and new ways to share information, then the value of the tools will be minimized.

2.       Train your Employees on how they can use MOSS functionality and provide them with the insight they need to see how they can increase their productivity and use all the features of the product.  Don’t allow this opportunity to become a means of using this new technology the same way you would in the past.  Think beyond document libraries and lists.  Think communication, business intelligence such as Dashboards, forms , workflow and collaboration to the fullest potential.  Develop Power Users that allows for your portal to be driven by the users.  They know what they need and if you provide the opportunity and knowledge to allow their ingenuity to drive the process it will allow for greater success and allow your IT staff to focus on IT and not driving the collaboration.  Everyone is equal when collaborating online and senior management of a company needs to buy in to the freedom and flexibility that will come from allowing people to work like this.   Consider spending 25% of your budget on training the power users and 75 percent on training all users.  The Power Users should be those stakeholders in the business and should be instructed upon the possibilities of the technology so they can chose which to implement effectively for their teams.  This is the key to success.

3.       Plan your overall design,  Taxonomy, site structure and navigation, Search, Document Management, Records Management, Security, Server Farms, logical design  with great care, knowing that your deployment will grow far beyond what your initial intentions were.  Think about how you deployed AD and Organizational Units so that they would sustain your deployment lifetime and future upgrades.  Use the Planning Worksheets provided.  Adopt a SharePoint Governance model.  Here is a sample Governance plan.  By implementing a distributed governance model which involves creating power users in the various lines of business for day to day governance tasks like site creation and security you not only mold champions to drive aware