By now most people have discovered that installing Windows 2003 SP2 on your Provisioning Engine servers (MPS01 and MPS02 in the reference architecture) breaks provisioning completely. The problem is that SP2 includes some updates to the MSXML parser that breaks the way MPS transforms XML during its processing of provisioning requests.
Well as it turns out there's a number of security updates that get advertised by Microsoft Update that also include these fixes for folks that don't install SP2. So you can quickly get yourself into the same situation by blindly installing all of the security updates that are offered.
To be clear, Microsoft certainly wants you to install security updates in a timely fashion. However, we strongly recommend that hosters have a lab environment that mirrors production, and that any updates are tested in the lab before being installed in production. We all agree downtime is bad, and it's better to discover these things in a lab instead of production.
That being said, here's a list of updates we've run into problems with in support. The HMC product group is aware of all of these and we are investigating fixing them as we speak. I'll update this post as we uncover more patches or as we fix the ones we already know about.
Patches with Known Problems on MPS
Description of the security update for Microsoft XML Core Services 3.0: August 14, 2007: This one will impact both HMC 3.5 and HMC 4.0 environments. We're currently investigating fixes for this. You will see errors like "Only one top level element is allowed in an XML document." UPDATE: This has been resolved. A hotfix for MPS has been released. You can obtain this fix by contacting Microsoft CSS and requesting the fix for 939216. As of this posting the KB article for the fix has not been published, but it is currently in the process.
Description of the security update for the .NET Framework 2.0 for Windows Server 2003, Windows XP, and Windows 2000: July 10, 2007: This one should only impact HMC 4.0 as it is an update for the 2.0 framework, and 2.0 isn't installed on MPS servers in a 3.5 deployment. This one is also being investigated. You will see an error like "Unable to generate a temporary class (result=1)." UPDATE: This has been resolved. A hotfix for the Exchange 2007 Provider has been released. You can obtain this fix by contacting Microsoft CSS and requesting the fix for 942100. As of this posting the KB article for the fix has not been published, but it is currently in the process.
I strongly urge you to contact support if have any questions about installing a particular patch. If you do install a patch and find it breaks provisioning, please report it through support channels so we can properly investigate it!