SQL Server Performance : Analysis Services

SQL Server 2008 is on its way with great performance

Data & SQL Storage Performance Team — Wed, 06 Aug 2008 20:01:00 GMT

Just left the final shiproom meeting for SQL Server 2008, and am happy to say every team has signed off, so the product is now in the hands of manufacturing, and in process toward web and media availability for you. We've shipped! MSDN and TechNet subscriber downloads are now live, with more to come.

It's been less than three years since SQL Server 2005, but we're pleased to give you a great new release of SQL Server which not only adds fantastic new capabilities to your data platform, but also delivers broadly better performance. We used new industry standard benchmark workloads, and customer workloads, to drive us toward delivering better real-world performance...and we didn't take the easy road.

The best example of that is our use of TPC-E, a far more modern, realistic, and challenging benchmark workload than its predecessor. We used TPC-E to improve the scalability of our relational engine, in ways that should be more relevant to your own OLTP database workloads. We're proud of partners, like IBM, NEC, and Unisys, who were able to demonstrate great scalability, up to 64 cores, using SQL Server 2008 running TPC-E, and like Fujitsu-Siemens and Dell, who have shown industry leading price/performance. Results like these should also be more useful for system sizing.

Almost five years of effort by TPC members were invested in the development of TPC-E, and it shows. The workload uses synthetic data which is far more realistic, by modeling real-world data. And compared to its antique predecessor, TPC-E's schema has ~3X more tables and primary keys, 2X as many columns, and 4X more foreign keys. And here's a radical thought for a modern OLTP benchmark workload: include check constraints, referential integrity, and reliable storage. Don't customers actually put DBMS servers into production expecting that?

Because we believe so strongly TPC-E drives us to better meet your needs, I am announcing today this is the first release of SQL Server which will not include published TPC-C benchmark results. Like other great thoroughbreds, TPC-C had a great run, and we were proud to ride it while it was still relevant to customers. But today, we're turning that great old race horse out to pasture for a well-deserved rest.

In SQL Server 2008, we have also invested lots of effort to improve our data warehousing performance, and the performance of our BI services. SSIS, SSRS, and SSAS, each show many double-digit gains in performance, which we hope you will enjoy. The new world record we set with SSIS, loading 1 TB of data in under 30 minutes, gives you a sense of this commitment to BI performance. We are also proud of our first-ever TPC-H 10 TB result.

There are a couple other ways in which this release improves on our past work. First, we've focused more energy on improving 64-bit SQL Server's performance on x64 AMD Opteron and Intel Xeon architectures. Given the price and density of RAM, and great new x64 CPUs available, this is a perfect time to take a closer look at the performance of your database servers, and consider doing fresh deployments, or migrations, on x64 SQL Server, which can use the additional RAM for everything, not just the buffer cache. Second, we invested more heavily in performance regression testing, both in automation and in the breadth of our test coverage. These investments have been reflected in broadly positive feedback from the community as well as internal and external beta sites. While we'll always have more to do, we feel this release marks an important step forward.

Welcome to the beginning of a new era for SQL Server. We hope you enjoy working with SQL Server 2008 and look forward to your feedback.

-David Powell
SQL Performance Engineering

Storage Modes in Analysis Services

Data & SQL Storage Performance Team — Mon, 12 Feb 2007 21:13:00 GMT

Why did we only use the MOLAP storage mode in Project REAL? There is also ROLAP and HOLAP storage, but they weren’t used at all. Also, why doesn’t Project REAL illustrate pro-active caching?

The Analysis Services 2005 Performance Guide contains a more complete discussion of the storage modes in Analysis Services in Appendix B. Because the question comes up from time to time regarding Project REAL, this write-up gives a summary.

Every partition in AS has an associated storage mode. MOLAP partitions store aggregations and a copy of the source data (fact and dimension data) in a multidimensional structure on the Analysis server. This is the optimal storage mode for several reasons:

Compression. When relational data is processed into AS, the storage needed is typically 20% to 40% of the size of the un-indexed relational fact table. Less I/O means that the MOLAP data is faster to access.
Multidimensional structure. The format in which AS data is stored is optimized for the kind of access that OLAP users do – access to the data using arbitrary dimensions and arbitrary slices of the dimension.
Data local to AS. There is no need for AS to call out to another service to obtain data when MOLAP storage is used.
Indexes are not needed on relational fact tables. When AS processes fact data, it generally can do a full table scan, so indexes are not needed. When there are multiple partitions in a measure group, there will be one join to the dimension that is used for partitioning, but that’s all. Then at query time, no requests are sent to the relational database, so no indexes are needed for querying. Given that indexes can multiply the size of data in the relational system, the space savings can be significant.

HOLAP partitions store aggregations in a multidimensional structure on the Analysis server, but do not copy fact data from the original relational database. As a result, whenever Analysis Services needs to resolve a query against fact data stored in a HOLAP partition, Analysis Services must query the relational database rather than using a multidimensional structure stored on the Analysis server. Some administrators choose HOLAP because HOLAP appears to require less total storage space while yielding high performance queries. However, the disadvantages almost always outweigh the advantages:

When a query “misses” the aggregations AS will have to query the relational database for fact-level data. This is necessarily slower than getting data from the compressed and natively multidimensional MOLAP storage.
Because of the above, many administrators will create more aggregations to decrease the number of “misses.” At some point the additional aggregations can become larger than the fact data would have been. But even if it doesn’t, more aggregations take more time to process.
Processing of HOLAP partitions is not significantly faster than processing of MOLAP partitions. In both cases the entire fact table must be read into AS in order to create aggregations. The only difference is whether AS also saves the fact data. However if more aggregations were created because of the previous point, processing will be slower.

ROLAP partitions store aggregations in the same relational database that stores the fact data. ROLAP partitions take longer to process, and because all queries must be redirected to the relational database, ROLAP partitions are slower to query against as well.

Given all this, it sounds like only MOLAP should ever be used! For most implementations that is the case. The exception is when near-real-time access is needed to data that changes throughout the day. Even there, pro-active caching is the first technique of choice. However, if even pro-active caching does not allow frequent enough updates, then the use of one ROLAP partition in the measure group is a way to handle the incoming data.

The recommended way to handle near-real-time scenarios is to take advantage of the fact that the storage mode is set on a per-partition basis. A vast majority of data in an OLAP cube does not change throughout the day. That data should be stored in MOLAP partitions. One partition in the measure group is designated to receive updates as they come in. That way, only that partition needs to be updated. This partition can be updated using proactive caching or it can be defined as a ROLAP partition with zero aggregations. In the latter case, all queries to this partition will be sent to the relational source, and current data will always be as current as that table.

For more details about near-real-time data access, see the section “Near Real-Time Data Refreshes” in the Analysis Services 2005 Performance Guide.

An example of how such a partitioning scheme would be updated might look like this:

Fact tables contain weekly partitions in the RDBMS, and there are corresponding weekly partitions in AS.
Each night, data for the previous day is processed into the correct weekly partition.
As new data comes in during the day, it flows into a relational partition for current updates. AS has a near-real-time partition that maps to that relational partition. Either using pro-active caching or a ROLAP partition with zero aggs, users get access to new data throughout the day.
At night, the collected daily data is moved to the weekly partition and processed (either by full process or using incremental update). The near-real-time partition is now ready to collect the next day’s updates.

Returning to the original question: Why did we only use the MOLAP storage mode in Project REAL? The simple answer is that MOLAP storage performs the best. The slightly extended answer is that Project REAL is based on the scenario in the Barnes & Noble data warehouse. The business model centered on daily updates, not near-real-time access. Therefore, there was no need for proactive caching or ROLAP partitions.

- Len Wyatt

Analysis Services 2005 Performance Guide

Data & SQL Storage Performance Team — Mon, 12 Feb 2007 20:40:00 GMT

Do you want to know how to get the best performance out of Analysis Services? Whether you have an immediate performance issue or just want to know more about what’s going on in there, this guide will be a valuable resource. Many people used the AS 2000 Performance Guide and have been anxiously awaiting the AS 2005 Performance Guide. At last, it’s ready!

This paper (see Table of Contents below) covers a wide variety of topics that are important for AS performance: Writing efficient queries, optimizing the cube design, maximizing processing performance partitioning, aggregation design, and tuning the system for efficient use of resources. It was developed in collaboration with the developers and program managers on the AS team, so it’s the authoritative guide. Take a look! It will be worth your time.

Introduction. 6

Enhancing Query Performance. 8

Understanding the querying architecture. 8

Session management 9

MDX query execution. 10

Data retrieval: dimensions 12

Data retrieval: measure group data. 15

Optimizing the dimension design. 18

Identifying attribute relationships 18

Using hierarchies effectively. 22

Maximizing the value of aggregations 24

How aggregations help. 24