Reed Me : DBPro

Degenerate dimensions in VSTS DB edition GDR R2 (a.k.a. TSD03136)

reedme — Fri, 07 Aug 2009 19:54:42 GMT

This is a common pattern in many BI projects that I’ve been involved with in the past four or five years for deriving degenerate dimensions without expending the effort to build and maintain star schema or snowflake tables in a data mart or data warehouse. (I’m not passing judgment, just making an observation. I’m ambivalent about them.)

CREATE VIEW [dbo].[SomeNewDegenerateDimension]
AS
SELECT DISTINCT([OneColumnOrAnother])
FROM [dbo].[SomeFactTableOrViewOrAnother]

GO

I know that they didn’t work in Data Dude before, but I had hopes that GDR R2 would’ve fixed it. GDR R2 is still fun to say. Yeah, it’s been that kind of week. In the immortal words of the Dread Pirate Wesley, “Get used to disappointment.”

Fortunately, it’s an easy fix. Unless you’ve got ~750 of them to fix in a large project that you’re trying to import into source control for change management and team development purposes... Just copy-n-paste the column name after an AS at the end of the SELECT clause.

CREATE VIEW [dbo].[SomeNewDegenerateDimension]
AS
SELECT DISTINCT([OneColumnOrAnother]) AS [OneColumnOrAnother]
FROM [dbo].[SomeFactTableOrViewOrAnother]

GO

In other news, I forgot today was the department summer party. Everybody else is @ the beach. I’m jousting with Visual Studio. In the immortal words of Dilbert, “Yay, life!”

erR? TSD03006. Doh!

reedme — Fri, 07 Aug 2009 02:17:39 GMT

I’m often skeptical when I read things like this that make absolute statements:

To resolve this problem, upgrade to Microsoft Visual Studio Team System 2008 Database Edition GDR R2.

FIX: When you access a table source in a referenced project by using an alias in VSTS 2008 Database Edition GDR, if you refer to the table source by using a three-part or four-part name, you receive TSD03006 errors or TSD04151 warnings

In my particular case for a new project where I have imported schemas created by Someone Else™ who overzealously fully qualified all the object names in every view definition and stored procedure in eight out of 12 databases in the solution (even when the views reference objects in the same database)... it’s NOT true.

Grrr.

Visual Studio beats me because she loves me. *pout*

Not sure how I missed GDR R2. That’s fun to say. GDR R2. Like magma. Magma is a funny word. GDR R2. Or maybe I left a note2self somewhere that I forgot to read. Huh.

Anyway, I got around to downloading it:

Download Details: Microsoft® Visual Studio Team System 2008 Database Edition GDR R2

But it didn’t magically fix all the errors that GDR reported for tri-part names for me. The following valid object definitions are reported as build errors:

View definitions that contain a three part name that includes the database containing the object. The Add Database Reference interface will not allow me to add a self-reference, so the fix seems to be to remove the database name. Editing ~300 object script files.
View definitions that contain a three part name in which the referenced database (which has a valid database reference in the project) name is “quoted” in square brackets. Removing the brackets allows Data Dude to resolve the reference correctly. Yay! Only ~200 object script files to edit (which incidentally were EXPORTED BY DATA DUDE in the first place).
[Edit] Database reference names are case-sensitive and must match the referenced database project name exactly, even in case-insensitive database projects. Yeah, yeah. Make them match in case and everything’s fine.

It’s apparently too much to expect full parity with the database engine parser or that there’s a way to flag specific errors generated from otherwise valid T-SQL to be ignored without a blanket ignore for an entire error code @ the project level... because some day somebody else might actually produce a bug in that class that I want the build to catch. (Nah, I’d never do that myself. *wink*)

The oversight of quoted identifiers for the database reference name is [censored], but, yes, I agree that self-references are silly in views for the database name (I can imagine several scenarios about how they happened, though), but I’ve 774 more errors in this solution to clean up before I can check in all the schemas that I just imported into Data Dude.

#@%&!!!

Developers MERGE thyselves?

reedme — Sat, 01 Nov 2008 02:05:51 GMT

I was reading this CIO.com article a while back and wondering if “database developers” and “regular developers” agree.

As Microsoft sees it, the roles of the database-centric developer and "regular" developer are less distinct than they once were, so the company is merging its VSTS database and development products. As of October 1, those who belong to the Microsoft Developer Network (MSDN) and currently own Visual Studio Team System 2008 Development Edition or Visual Studio Team System 2008 Database Edition will receive for free the Visual Studio Team System 2008 Development Edition, Visual Studio Team System 2008 Database Edition, Visual Studio 2005 Team System for Software Developers and Visual Studio 2005 Team System for Database Professionals. (CIO.com)

Is it really that simple? Or will it be in the future? Are all (or most) “regular” developers also database developers? And vice versa? Hmmm.

The whole imaginary syntax probably looks something like this:

MERGE Developers.All_Developers AS target
USING
    (
        SELECT
             COALESCE(dbds.DeveloperId, rds.DeveloperId) AS DeveloperId
            ,COALESCE(dbds.FullName, rds.FullName) AS FullName
            ,COALESCE(dbds.IsActive, rds.IsActive) AS IsActive
        FROM Developers.Database_Developers AS dbds
        FULL OUTER JOIN Developers.Regular_Developers AS rds
            ON dbds.DeveloperID = rds.DeveloperID
        GROUP BY COALESCE(dbds.DeveloperId, rds.DeveloperId)
    ) AS source (DeveloperId, FullName, IsActive)
ON (target.DeveloperId = source.DeveloperId)
WHEN MATCHED
    THEN UPDATE SET
         target.IsActive = source.IsActive
        ,target.FullName = target.FullName
WHEN NOT MATCHED
    THEN INSERT (DeveloperId,FullName,IsActive) VALUES
        (source.DeveloperId,source.FullName,source.IsActive)
;

Seems complicated to me… for imaginary syntax of imaginary data constructs. Feel free to nitpick imaginary flaws in it.

Of course, YMMV.

Testing the SQL Database

reedme — Tue, 26 Aug 2008 02:25:30 GMT

Being a big fan of test-driven development of any kind, database or otherwise, I was honored to sit on a panel with some other much smarter people than I at TechEd*2008 entitled "Testing the SQL Database." Not only did I get to opine a bit, but I also got to learn some, too, which happens whenever you get to sit and listen to smart people like Gert, John, Adam and Jamie.

Stream for Low Bandwidth Users | Small WMV | High Quality WMV | High Quality MP4 | MP3 Audio

I do have one observation: I'm not really that fat in person… It's the camera! Heh.

See the TechEd 2008 Video Library for more cool stuff.

DataDude Annoyance #13: Project variables omitted from output script.

reedme — Fri, 07 Dec 2007 05:26:39 GMT

If you're using any software product for a non-trivial purpose, you'll find that it has flaws that will (upon occasion) drive you up a wall. It doesn't make any difference what it is or who produced it. I've even heard Mac users, upon occasion, complain about bugs in their toys.

But sometimes these little annoyances make you ask questions like, "Who the &@#$ tested this thing?!" I've had more than my share of those this week. Don't get me started about WiX and the Windows Installer. Grrr.

Fortunately, I can do something relatively simple about this one with Visual Studio for Database Professionals SR1 (a.k.a. DataDude or DBPro).

"Feature": DataDude projects allow you to specify project-level variables in the project properties on the Variables "tab" (or "sheet" depending on who you ask what it's called) BUT those variables are not written into the project build output (a T-SQL script) as :setvar statements.

In case you wonder why this matters, it really sucks when you're including DDL produced by a DBPro project into an MSI to be deployed as part of a deployed "product" (like an ETL Framework, for example). If those variables are omitted, the DDL script bombs. Not good.

In this case (unlike my WiX blues), I'm not the only one who has complained. It's already logged as feedback @ connect.microsoft.com. If you think that it should be addressed, give it some love. Heh. Yes, people @ Microsoft actually look at stuff like that once in a while.

In the meantime... Enjoy my fix.

"Fix": Use the post-build event to write your own defined variables into the output script. Add a Post-Build.vbs script to your DataDude project that defines the additional variables and writes them into the output script.

Here's some VBScript to do it. (If I were really motivated, I'd read those variables from the project properties through VSTS automation or cheat and read the XML... but my wife is pestering me to go pack up the office to get ready for the move to Redmond.)

Dim fso, ts, line, x, y, script, variables, filepath

Const ForReading = 1, ForWriting = 2, ForAppending = 8

' NOTE: This name must match the DataDude "Build Output File Name"

' in project properties.

filepath = "ETL Repository.sql"

' Append your additional variables here, one per line.

' Include blank lines as desired for spacing.

variables = Array( _

     "" _

    ,"-- Additional variables added by Post-Build.vbs" _

    ,":setvar DatabaseMasterKeyPassword ""Change this password, please.""" _

    ,"" _

    ,"-->> Backup your Service Master Key!!!" _

    ,"-->> Backup your Database Master Key!!!" _

    ,"" _

    ,"-->> Be sure to test carefully!" _

    ,"" _

Set fso = CreateObject("Scripting.FileSystemObject")

Set ts = fso.OpenTextFile(filepath, ForReading)

x = 0

Do Until ts.AtEndOfStream

    line = ts.ReadLine

    x = x + 1

    If x = 7 Then ' Right after the last existing :setvar.

        ' Append variable definitions.

        For Each line In variables

            script = script & line & vbCrLf

        Next

    End If

    script = script & line & vbCrLf

Loop

ts.Close

Set ts = fso.OpenTextFile(filepath, ForWriting)

ts.Write(script)

ts.Close

Set ts = Nothing

Set fso = Nothing

For the record, I *hate* VBScript, but sometimes it's the duct tape that binds the solution together... since my JScript syntax wasn't working (and it wasn't worth 5 minutes to figure it out, so I gave up and rolled back to VBScript)... and I was entirely too lazy to figure out how to make a PowerShell script work as a post-build event. Heh.