You had me at "Hello World" : Database

How to Find Out Which Column Caused SSIS to Fail?

HelloWorld — Fri, 01 Aug 2008 15:50:00 GMT

In my previous post, I explained how you can find out the error description for the error id that is returned by SSIS.

Of course, that information is only half helpful, debugging the issue also involve knowing which column caused that error. The Error Column returns an integer value which is also a little bit confusing, how can I know which column is represented by this integer value?

The Error Column corresponds to the value of the lineageId attribute inside the SSIS package. As you already know, SSIS package is an xml file. The node that contains lineageId attribute may or may not has a name attribute. The value of the name attribute is the column name.

To query the lineage id and the column id, you can use this code using LINQ.

private Dictionary<long, string> GetLineageIdAndColumnMapping(string SSISFilename)
{
    XDocument xdoc = XDocument.Load(SSISFilename);
    Dictionary<long, string> LineageColumn = new Dictionary<long, string>();

    var LineageNodes = from Nodes in xdoc.Descendants()
                       where Nodes.Attribute("lineageId") != null &&
                             Nodes.Attribute("lineageId").Value != String.Empty &&
                             Nodes.Attribute("name") != null &&
                             Nodes.Attribute("name").Value != String.Empty
                       select new
                       {
                           LineageId = Convert.ToInt64(Nodes.Attribute("lineageId").Value),
                           ColumnName = Nodes.Attribute("name").Value
                       };

    foreach (var Item in LineageNodes)
        LineageColumn.Add(Item.LineageId, Item.ColumnName);

    return LineageColumn;
}

If you are not using .Net framework 3.5, then this XPath code will also do the trick.

private Dictionary<long, string> GetLineageIdAndColumnMapping(string SSISFilename)
{
    XmlDocument doc = new XmlDocument();
    doc.Load(SSISFilename);
    Dictionary<long, string> LineageColumn = new Dictionary<long, string>();

    foreach (XmlNode node in doc.SelectNodes("//*[@lineageId != '' and @name != '']"))
        LineageColumn.Add(Convert.ToInt64(node.Attributes["lineageId"].Value),
                          node.Attributes["name"].Value);
    

    return LineageColumn;
}

That code above, will give a dictionary of lineage id and the actual column name, now using the value from the ErrorColumn field, you can lookup the actual column name.

Someone might say, the XPath is shorter, it is better. Well, there are more than one metrics to evaluate a piece of code. LINQ enables code reviewer who are not familiar with XPath to quickly review the code, understand what is going on, and comment on it.

The beauty of LINQ, your knowledge is almost transferable among different data sources.

How to Decipher/Understand SSIS Error Code?

HelloWorld — Sat, 26 Jul 2008 00:22:46 GMT

If you have worked with SSIS, you may have been doing something like this, in case of errors, you saved the error code and the error column to a text file or some other media.

If you get the error within the IDE, that is cool, as you can get more readable information, but if you run the SSIS package in production environment, you does not get that readable information, only a numeric value.

Now you need to know what caused the error, so you read the error code. Unfortunately, the error code is not that clear.

To understand what is the meaning of the error code, you need these two things, Calculator (calc.exe), and dtsmsg.h file. dtsmsg.h by default is located under “C:\Program Files\Microsoft SQL Server\90\SDK\Include”. If you install SQL Server, you should have this file.

Let says, you get error –1071607689. To see what is the actual error:

First open Calculator and switch to Scientific mode, and make sure the Decimal radio button is selected. Then type that number.
Click the Hex radio button, now you get a hex number, it is prefixed with eight Fs.
Copy the text, excluding the first eight Fs. In that example above, it is C0209077.
Search that code inside dtsmsg.h. Now you get the constant and the description. That example above, the constant is DTS_E_OLEDBDESTINATIONADAPTERSTATIC_CANTCONVERTVALUE, and the description is ‘The data value cannot be converted for reasons other than sign mismatch or data overflow.’

One more error code that I found were –1071607675, using that method above, you can see that that code means there was data truncation.

Hopefully this will help you to figure out what was wrong with the SSIS package.

Boosting Performance with READ_COMMITTED_SNAPSHOT

HelloWorld — Tue, 22 Jul 2008 10:10:00 GMT

I had a problem with one of our database in our testing environment, it performs OK, but we know it can be faster. There are millions of data in that database. Everything are properly indexed and the stored procedure plan is optimum.

I was not aware about read committed snapshot option when the dba suggested this option. I was reading more information and was convinced that this setting would help performance.

We set this configuration, the application is now running really fast, processing hundreds of row per second. Our application has a lot of transactions per second, multiple read and multiple write within the same seconds. With this option, readers won’t block writers, and vice versa.

Take a look at this option, it may give the performance boost that your application needs. Kimberley Tripp has a good article on msdn about this subject. http://msdn.microsoft.com/en-us/library/ms345124.aspx. A long read, but it has good information.

Of course, you should always measure it, and nothing comes for free, it consumes more space in the temp db.

Database Design – Designing Primary Key

HelloWorld — Tue, 24 Jun 2008 15:17:00 GMT

One of the key of designing a database is picking the right primary key. I keep in mind several things when I design a database:

If possible, primary key should not have a business meaning.
Business meaning changes, some attribute that is unique now, may not be unique in the future.
Related to that point above, create a surrogate key.
Most of the time (95%), a surrogate key helps when improving an existing database, there would be less update, since the surrogate key does not have business meaning.
Get a simple data type as primary key
Generally, avoid composite primary key, especially with table that has high volume of transactions.
Having a composite primary key with clustered index may cause pagination, it is also make it more challenging in the future if a child table is added to that table.

There are, of course, exceptions to those thing above, those are just guidelines…