Browse by Tags

The Balanced Data Distributor (BDD) transform has updated and released for SQL Server 2012 . The BDD transform makes it easy to split your data flow up on a buffer-by-buffer basis. The two primary scenarios for it are when you want to do parallel inserts into a destination… … and when you want to parallel...

Three slide decks from the SSIS presentations I did at SQL Bits X in London have been posted online. These include: SSIS Performance Design Patterns – full day training session! What’s New in SSIS for SQL Server 2012 – breakdown of the new features and functionality in SQL Server 2012 SSIS Catalog –...

An interesting case study about how SSIS is used by Canada Post has recently been published . The article mentions how they built an SSIS scale-out solution which processes 10-15 GB of data per day (with four worker nodes). Definitely an interesting read! If you’d like to know more about building your...

Today’s post is from Wee Hyong Tok – a Program Manager on the SSIS team based in Shanghai. It covers the PipelineComponentTime event, which was introduced in SQL Server 2008, and talks about how the event can be used in SQL Server Denali. The PipelineComponentTime event is a very useful custom log entry...

Today’s post is from Wee Hyong Tok, a Program Manager with the SSIS team based in Shanghai. It provides some tips on using T-SQL to analyze performance issues for packages running on the new SSIS Catalog. For more information about the new logging functionality in Denali, see Jamie Thomson’s post about...

Most of this content comes from my Performance Design Patterns series. If you’re interested in learning more about SSIS Design Patterns, I’ll be doing full day pre-conference training sessions at the PASS Summit and SQLBits . ---------- Whenever I’m presenting a session about SSIS performance, I like...

This pattern has been out there for so long I had contemplated taking it out of my Performance Design Patterns talk, but it looks like it still managed to help some people at SQL Saturday #71 last weekend. This pattern is straight forward, and especially important since the data flow threading improvements...

Jamie Thomson recently suggested that you should run your packages using DTEXEC when doing performance testing, as it (typically) performs much better then when the same package is run in BIDS. If you haven’t already read it, go do that now. It has a pretty picture in it and everything. I’ll wait. The...

A new technical note about splitting up transforms to increase performance has been posted on the SQLCAT site . It’s something you want to consider when you’re using synchronous transforms (like Derived Column, or Data Convert) to perform actions on a large number of columns. It seemed like black magic...

As a follow-up to my previous post about SCD processing in SSIS , I thought I’d go deeper into using the built-in Slowly Changing Dimension Wizard . While there are definitely more efficient ways to do SCD processing in SSIS, there are some optimizations you can apply to the components that the wizard...

This post is coming a little late (these have already been announced here and here ), but incase you haven’t seen these before, I wanted to bring attention to four new performance related created by the SQLCAT and SSIS teams. They mostly focus on larger scale/Enterprise level package deployments, but...

A popular SSIS talk at recent conferences has been about the ETL Record that we announced last year. They’ve converted the talk to a white paper, and it is now available on MSDN. It’s a very interesting read. Summary: In February 2008, Microsoft announced a record-breaking data load using Microsoft®...

The SQL CAT team published a Top 10 SQL Server Integration Services Best Practices list back in October, and I thought I’d draw some more attention to it incase anyone missed it the first time around. The list contains performance related tips that were proven during the ETL world record performance...

The Team Development Edition and Team Suite versions of Visual Studio 2008 comes with a performance profiling feature that can used to find bottlenecks in your custom extensions (tasks, data flow components, etc) for SSIS. Although having your extension assemblies in the GAC make profiling a little more...

They mentioned the TPC benchmark results at the SQL Server 2008 launch today. The final paragraph mentions a world record for ETL load performance. The fine folks who maintain the SQLPerf blog have posted details about the result , explaining how they were able to load over 1 TB of TPC-H data in under...