Fix: Analysis Services crashes while processing the TFS OLAP cube

Grant Holliday’s blog

Senior Service Engineer, Microsoft Visual Studio Team Foundation Service

Fix: Analysis Services crashes while processing the TFS OLAP cube

  • Comments 0

I was on-site with a customer last week performing a Team Foundation Server Health Check (TFSHC). While I was there, I noticed that their SQL Server Analysis Services 2008 R2 SP1 instance had been crashing. So, I did what any good PFE would do and grabbed a copy of the crash dumps (SQLDmpr0001.mdmp) and analysed them. You can find these in the  \OLAP\Log folder under your instance directory. Normally something like: C:\Program Files\Microsoft SQL Server\MSSQL.2\OLAP\Log.

Here’s the stack trace for the crash:


msmdsrv!PFSetLastError+0x4ea
msmdsrv!PCMinorObjectCollectionBase::GetAt+0x58
msmdsrv!MDInfo::Init+0x1f72
msmdsrv!MDCube::PopulateInfo+0x48
msmdsrv!MDIInfoObject::GetInfo+0x81
msmdsrv!MDCube::PerformRefreshRelatedScopes+0xd8a
msmdsrv!PCDatabase::RefreshDependingSessions+0x35a
msmdsrv!PCDBPerm::OnCommit2+0x59
msmdsrv!PCTransaction::CommitPhase2+0x105f
msmdsrv!PCTransaction::Commit+0x38b

msmdsrv!PXSession::CommitTrans+0x2d0
msmdsrv!PXSession::CommitUserTrans+0x19b
msmdsrv!PCASTDDLTransaction::Dispatch+0x11e
msmdsrv!PCXAExecute::Dispatch+0xe30
msmdsrv!PXSession::InternalExecuteCommand+0x6d8
msmdsrv!PCSession::ExecuteCommand+0x9d

This stack trace was enough to help me find the bug in the internal Microsoft support systems.

The root cause for this bug is a version mismatch – the old cube object is calling GetInfo based on new info. It was at this point, I realised that I’d seen this same issue before. The SSAS bug was raised by a member of the TFS test team after they found the issue on the internal dogfood servers.

Now, before we move on – let’s get one thing clear: SQL, Analysis Services and TFS should never crash – any time they do, that’s a bug that needs to be addressed.

Although this issue can occur with any version of TFS (2005-2012) and SQL2008 or SQL2008R2, this particular bug is the reason why the TFS Installation Guide for TFS2012 has steps to “Configure Analysis Services to Recover on Failure”.

SQL Server Analysis Services - Service Properties for configuring restart on failure

Now that all those versions of SQL Analysis Services have been patched, there’s no reason to change the SSAS service to automatically restart on failure. In fact, if you do restart automatically on failure AND you don’t have adequate monitoring in place, you will mask subsequent failures.

Conclusion

The conclusion here is very simple. Keep your systems patched and up to date, and you will avoid many of the issues which have already been found and fixed.