Carpe Datum

Quote of the Day – The Definition of an Expert

I’ve been at the PASS conference this week, and I’ve been around a lot of really smart Experts – and this definition definitely fits:

An expert is someone who knows some of the worst mistakes that can be made in their subject, and how to avoid them. - Werner Karl Heisenberg

Learning New Tech on SQL Server Virtualization – From Vendors

I’m at PASS this year, and one of the things I’ve learned this time is that I can learn from – the vendors!  Most of the “big” names are here, like Idera, Redgate, Quest, and more, but I ran into some folks yesterday I hadn’t spoken with before – XKoto.

I’m actually surprised I hadn’t run into them earlier. I talk with folks all the time about virtualizing, which should really be a conversation around consolidation, and the XKoto folks have an incredible product that handles scale-out in a new way. Check them out: http://www.xkoto.com/ 

Looks like even and old dog can learn some new tricks…

Find and learn DMVs

This morning I showed how I find DMVs, and find out how to use them. First, I just run this query:

USE MASTER;

GO

SELECT ‘sys.’ + name

FROM sys.sysobjects

WHERE name LIKE ‘dm%’

Then I copy and paste a result into the “Index” panel of Books Online. I also use a web search to learn more.

I also use the new Activity Monitor, and just hover over a column header in that tool – Microsoft tells you what DMV they are using (if they are using one) and even a little about it. How cool is that!

DBA? No – Data Professionals

I’m here at the Professional Association for SQL Server (PASS) this week, and I’m here in the keynote. A few days ago I ranted about the fact that DBA should be a Data Professional – not an administrator. I’ve heard a lot of response about that blog – and all were in support.

And now here at PASS, what is the theme? Taking a strategic approach, and being professional. w00t! Have we turned the corner? Are other folks realizing that you hold the keys to the kingdom when you’re the DBA? I think so. We’re getting there…

Win7 – Install SQL Server Native or go Virtual PC?

I have Windows 7 on my laptop, and I also teach, demo and use SQL Server 2005 and SQL server 2008. Should I install this “native” on my outside operating system or use the Virtual PC (VPC) software that comes with Windows 7? Well, there are arguments for each.

It’s far simpler to start up SQL Server Management Studio (SSMS) or Business Information Development Studio (BIDS) without having to do anything else on my system. Point: outside operating system. And I can install the developer edition of SQL Server 2005 and 2008 with no trouble on Windows 7. Point: outside operating system.

Using a VPC images lets me install a server-level operating system, SQL Server 2000, 2005 and 2008. That lets me more closely replicate my true production environment. I can also install other things I want on the image and not “pollute” my outside operating system. I can copy the VPC file elsewhere, set recovery points to try something and then “roll it back” to a previous state (very helpful with service packs!) and I can upgrade the VPC to Hyper-V. Point(s): VPC.

So here’s what I do: I have all of the client tools on my PC, along with an Instance of SQL Server Express. But for the most part I use a VPC, for all of the reasons above. It’s the best of both worlds.

Bonus Post: My PASS Schedule

I’ll be speaking again at PASS this year, and I’m heavily involved in the event when I’m not speaking. I would love to meet you – if you’re a reader of this blog, please stop by and say hello! As of right now, here’s where’s I’ll be:

Tuesday: 11:00 AM - 2:00 PM
Ask the Experts Table (Manageability) in the main pavillion

Tuesday: 6:00 10:00 PM
Customer Appreciation at SQL PASS summit

Tuesday: 9:00 11:55 PM
The SQL Sentry MVP / VIP Party at PASS 2009

Wednesday: 7:00 - 8:30 AM (breakfast served until 7:30)
Simplify SQL Server Management with DMVs, Room 611-612 or online Cost: Free

Wednesday: 4:30 - 5:45 PM
DBA-341-M Manageability Series: Understand and Troubleshoot with Policy-Based Management and Data Collector

Thursday: 1:00 - 2:15 PM
DBA-350-M Manageability Series: Microsoft SQL Server Automation on Steroids, Including PowerShell Support

Thursday: 2:30 - 3:45 PM
DBA-342-M Using SQL Server 2008 for Performance Tuning

I’ll probably pick up a few more parties and definitely be working the Microsoft booths and so on, so if you see a guy in a Microsoft shirt with “Buck Woody” on it, that’s probably me.

Quote of the Day: What is Impossible

When I worked at the Space Center in Florida, one of my favorite quotes was posted over a particular rocket engine display near my office. It’s by Werner Von Braun, a German scientist who is largely responsible for the Redstone and Apollo Space Programs. It continues to inspire me today:

“Always use the word ‘Impossible’ with the greatest of prudence.“

Don’t be a DBA – Be a Data Professional

I get asked quite frequently now about the “Cloud” technology and how it will affect the job of the DBA. The answer to that is pretty easy: If you’re “just a DBA”, then it will affect you a great deal. The fact that someone else will add users, take backups and worry about uptime might give you great pause.

Wait a minute – did I just say “just a DBA”?  Isn’t that kind of insulting? Not really. I’ve been in a DBA on one platform or another most of my adult life. It’s been one of the most rewarding, exciting careers I can imagine. But – if you’re only concerned about Feature X, or PL-SQL or Transact-SQL or any SQL, or you only know everything there is to know about bringing a DB/2 system back to life, then you’re very tactical. Those things will eventually be replaced with a fairly sophisticated Perl or PowerShell script. And you have every right to fear disruptive technologies that make major changes to your area, like the cloud.

But – if you realize that your job is not the servers, not the code, but the data – now you’re thinking strategically. You’re no longer a DBA but a Data Professional. Sure, you still have to know how to add a user, restore a database, and configure for performance. But you now realize that those are means to an end – not the destination itself. And when you realize that you hold the company’s second-most important asset – it’s data – you have a different view of your position in your organization. You start thinking about how you use Feature X to help the company, and that’s where your real value is. It’s what sets you apart from being “just a DBA” to someone the business comes to when big decisions need to be made.

So stop being tactical, and start being strategic. It’s no accident I work with SQL Server. I find that the fact that I can back up an Express database and restore it on Enterprise to be very strategic. I find that having every feature in the box without different learning paths to be very strategic. I find that having everything from Visual Studio to web pages and Microsoft Office be in lock-step with my platform to be very strategic. I put those things together for the organization I’m responsible to, and make suggestions on broad strategies. I’m treated like a professional because I act like one. The cloud doesn’t scare me – it’s just another tool in the box. I’m studying it, learning it, and finding out where, when and if it fits.

And you can too. Spend some time finding out what your company actually does, and what their biggest issues are. Then put that knowledge together with what SQL Server does, and how it can help. That’s strategy.

Templates for New Query Window

When I’m in SQL Server Management Studio and I click the “New Query” button, my Query Window comes up like this:

Then I just press CTRL-SHIFT-M and enter the name of the script file, the purpose and so on.

I was recently asked how I have that come up every time.

Just locate the file called Sqlfile.SQL. It’s normally in:

C:\Program Files\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\SqlWorkbenchProjectItems\Sql

And edit it to have whatever you want. If you want the replaceable parameters like I have, just put whatever you want in this format:

<ParameterName, ParameterType, DefaultValue>

Disaster Response and the Physical Plant

Near our home is a Dam about to break. It was built long ago, and even though millions were taxed and spent for it, little to none of that money ever got to the actual shoring up and maintaining the dam  over time (don’t get me started). Anyway, it’s in sad shape, and an entire valley is about to be underwater. It’s very crowded here, and there are hundreds of thousands of homes at risk and many businesses as well. My house isn’t one of those, but I have to drive that way to get to the deathstar.

I was speaking with a gentleman at the gym this morning and he explained some of the expensive, sensitive, one-of-a-kind pieces of equipment in one of those plants, something that just can’t be moved – and they are right in the path of the floodwaters. They aren’t sure what they are going to do.

I’ve seen this in computer data centers before. In one case where I worked as an Oracle DBA on HP-UX (a UNIX variant) the DBA’s and other administrators kept track of the backups, we had off-site tape storage, we practiced our restores, we had a good RPO and RTO defined, everything was ready. Then a pipe leaked in the data center, and we tried to implement the plan. It did not go well.

What we missed was the tape drive. I was young in my career and didn’t know that the tape drive system we were using was unusual. It was also a long time ago and standard tape formatting wasn’t in use – the format was largely a function of the drive. And we couldn’t get another one of those! We finally found one a few days later at a government surplus shop – a very time-consuming, expensive, frustrating experience.

So what am I saying? Put your physical plant into the DR plan. And don’t build something you can’t replicate – if you do, you’ve effectively locked yourself into a location that just might be part of some really poor government maintenance.

How Normal(ized) Should You Be?

Normalization is the process of removing repeating values from your data design, and ensuring that the values depend on the Primary Key in the table. OK, it’s a bit more complicated than that, but this definition will do for this post.

So how far DO you normalize? Consider the following:

Name, Address, City, State, Zip.

You might normalize it to the following:

FirstName, MiddleName, LastName, AddressLine1, AddressLine2, CityOrMunicipality, StateOrProvidence, PostalCode.

That seems pretty well normalized, and ready for a Primary Key. But let’s drill in a bit – the StateOrProvidence column is going to repeat, here in the U.S., quite a bit. Perhaps a Foreign Key is needed there, and another table created with WA, FL, GA, etc?

You could do that. But in most cases you’re too normalized if you take that route. In the case of two-letter state names, you’re not saving much space and there are other ways to enforce the data integrity of that column.

You must consider the reason you’re normalizing the table, and ensure your design fits that requirement. In most cases, it’s enough to tell the developers to code a drop-down list of the acceptable values, and/or put a constraint on the column to enforce what you want.

So apparently you can be too normal(ized). At least in this DBA’s experience.

Women and Cats

I will not explain why I chose this quote for today (it could have to do with the fact that I’m the only male in the house and yes, we do have a cat):

“Women and cats will do as they please, and men and dogs should relax and get used to the idea.” - Robert Heinlein

New Business Metric: DBOM

With all of the travelling, writing, speaking and working I’ve been doing lately, I’ve come across a new trend. My schedule is so fragmented that I communicate primarily through e-mail, and I’ve noticed that I respond to, and hear back from, various people in various tiem increments.

Because of the flights and travel time, I’m almost always 1 day behind on mail – or, a factor of +1 or: Days Behind On Mail (DBOM). But not for everybody…

If my wife or my boss writes me, they get a DBOM multiplier of 0:

0*1 = answer immediately.

If a friend or colleague writes me, they get a DBOM multiplier of 1:

1*1 = 1 Day delay

If a person is asking me to look into an issue, that might get a DBOM multiplier of 2 or 3, and so on.

So what’s your DBOM matrix look like?

The 20 Queries

In the book “The Fourth Paradigm”, Jim Gray describes the modeling process he came up with modeling large databases. He was working with scientists, researchers and others, and the data sets they were generating were huge – terabytes of data at a time. To even come up with an entity model was a daunting task, so he came up with something he called “The 20 Queries”. He asked each researcher the 20 most important queries they would ask of the data, and then he worked from that.

What caught my eye was the quote about the number 20:

“Most selections involving human choices follow a “long tail,” or so-called 1/f distribution, it is clear that the relative information in the queries ranked by importance is logarithmic, so the gain realized by going from approximately 20 (24.5) to 100 (26.5) is quite modest.”

What? They do? So I began my own research – it’s one of the multiple notes I took from the book. The 1/f distribution, or 1/f noise, is all over the place. I won’t go into the formula here since it’s a bit complicated, but it affects everything from signal generation in stereo and other speakers to the probability distributions in the stock market. It’s another one of those “magic formulas” like the golden mean that you see everywhere in nature – and yes, it even works for databases.

Here’s the way I use this:

1. I have the users define their entities – the nouns in their organization or what they want to track.
2. I then have them tell me about the verbs that define what those nouns do, and the relationships between them.
3. Then I ask, “what would be your 20 top queries – in English – for each of those nouns and verbs?”
4. From there, design is a breeze.

Isn’t math cool? 

The Fourth Paradigm

Microsoft Research published a book based on Dr. Jim Gray’s research into Computer Science, called “The Fourth Paradigm”. It’s about 300 pages long, and it fascinated me so much I stayed up reading it last night. I have so many notes to decompress it will take me weeks. If you’re a data professional, I highly advise that you take some time to read it.

The title refers to the Dr. Gray’s theory on scientific paradigms. The first, called the empirical paradigm, deals with what we can see, feel and touch. Greek and Chinese scholars dealt in this realm, all the way up through the times that the Arabic numeral system became prevalent in the West.

The second paradigm was theoretical – meaning that you could take a model or guess about some natural subject and test it.

The third paradigm is computational. Here simulations became paramount, so mathematics becomes the basis for theories and testing.

And so that brings us to the fourth paradigm – a time when we can gather more data than the human mind can effectively deal with, so we use computers to mine and analyze the data. This technology, which Dr. Gray was working on when he died, represents the next scientific frontier. He worked with the SQL Server team and helped us develop and test SQL Server 2008. A version of 2008 is what is used in the Pan-STARRs project at Berkley,  taking in Terabytes of data, well on it’s way to becoming a multi-Petabyte system within a year. (Who says SQL Server can’t scale!)

This book covers natural science, biology and health sciences, and also literary science. I highly recommend it.

There’s a certain section in it that deals with the 1/f distribution, which I’ll talk about tomorrow. It has real-world application in how you develop a database.

You can read the book for free here: http://research.microsoft.com/c/1078/en-us/collaboration/fourthparadigm/default.aspx