Dan's Blog : Policy

PBM – it’s like liquid awesome!

dtjones — Sun, 30 Aug 2009 20:20:10 GMT

Brent Ozar and Thomas LaRock do a 3 part video series on the power of Policy-based Management (PBM). Interestingly SCOM comes up several times in the discussion.

They cover the basics of PBM and Thomas goes into more depth on how he’s using it in his environment. The series is about 35 minutes in total.

Regardless of your PBM experience level I highly recommend watching these videos.

http://thomaslarock.com/2009/08/policy-based-management-podcast-part-1/

http://thomaslarock.com/2009/08/policy-based-management-podcast-part-2/

http://thomaslarock.com/2009/08/policy-based-management-podcast-part-3/

PCI Compliance and SQL Server 2008

dtjones — Mon, 20 Apr 2009 03:22:08 GMT

Today compliance is a huge focus area for IT. Since compliance is written by lawyers for lawyers it takes a fair amount of translation to get it to the point where IT can understand what processes, policies, and configuration to implement to meet the requirements for a given compliance spec.

If you are a SQL Server DBA and face PCI compliance this white paper is a must read and the accompanying TechNet webcast a much watch.

Application & Multi-Server Management

dtjones — Wed, 26 Nov 2008 06:03:45 GMT

@ the SQL PASS Summit on 11/20 we announced our plans to release new capabilities for improving the collaboration between Data-Tier Developers and DBA and to address Multi-Server (or multi-instance) manageability for SQL Server.

Anthony Carrabino has a great posting describing these capabilities. I won’t repeat Anthony here.

In addition, I recorded a session Joey Snow from Technet Edge. This gives you a quick overview along with a demo.

I’ve been working with the team on this for about 4 months. We started work on this early in the Katmai release and had to put it on the shelf for several months (and I mean several months). It’s very exciting to see that early work start to come to life.

There are a couple of points I want to be sure are clearly understood:

1) We are building on our existing technology investments. This includes Policy-based Management, Data Collector, SMO, PowerShell, and others.

2) The first set of functionality – the foundation, if you will – will be available in SQL Server Kilimanjaro. This release is scheduled for the first half (H1) of calendar year 2010.

I’m very excited with what we’re going to deliver in Kilimanjaro and beyond. My early take is this stuff will forever change the database business.

Addressing Compliance with SQL Server 2008

dtjones — Sun, 26 Oct 2008 23:01:17 GMT

JC Cannon, Senior PM on the SQL Server team, presents this webcast where he explores some of the new features in SQL Server 2008 and describes how they can be used to help an organization address their compliance needs.

In the webcast JC covers:

Identity management
Data protection
Separation of duties
Auditing and reporting
Policy-based management

TechEd 2008 is Done!

dtjones — Fri, 13 Jun 2008 23:44:39 GMT

I'm writing this on the plane from Orlando back to Seattle. I have lots of thoughts on TechEd that I'll try to get through before I run out of battery or the guy in front of me snaps my laptop display. I also want to answer a few of the questions that came up that I couldn't immediately answer. I'll post the code from the sessions on another blog post. The PBM code will be posted to the PBM blog.

I am thoroughly impressed by our customers. They have incredible passion for our products and an endless desire to learn how to use our products more effectively, intelligently, efficiently, and completely. I thank each and every one of you for making my job so fun and pleasurable. Thank you for coming up and asking questions (whenever and wherever), thank you for submitting your evals - I take the feedback to heart, and thank you for betting your business on SQL Server. Without you I'd probably be a starving musician.

I did a ton of time at the booth, co-presented one interactive theater on using SQL2K8 for compliance, participated in two Tech-Ed Online Panels (TEOPs): Compliance and Automating DBA Tasks, presented a breakout (BRK) session on PowerShell and co-presented a session on Policy-Based Management.

Interactive Theater: I'm undecided if I like this format or not. Maybe it's just that compliance is a tough topic to discuss since there is so much interpretation that has to happen and there's no one-size-fits-all solution. I co-presented this with Il-Sung Lee a security PM in SQL Server. Il-Sung is so knowledgeable about his space and gave the audience a very good taste of Transparent Data Encryption (don't forget to backup the certificate!) and Auditing. I had directed on person that came to the security kiosk to the session. They came up to me afterward and thanked me.

TEOP: Compliance: This was fun although we didn't have much of an audience. I really like this format and encourage the TechEd planners to extend it. The people on the panel make all the difference.

TEOP: Automating DBA Tasks: Wow, I was up there with Allen White (SQL MVP), Steve Jones (SQL MVP), Kevin Kline (SQL MVP), and Buck Woody (former SQL MVP and now a PM on the SQL Server team). Talk about a humbling experience. It was absolutely fantastic. These guys are terrific and posses such incredible knowledge about SQL Server and the role of the DBA. Great job guys!

BRK: PowerShell: The feedback on this session was that I didn't spend enough time on SQL Server and too much time on vanilla PowerShell. Honestly this was by design and I think after people get back to the office and start using PowerShell they'll understand why I structure the session this way. One question that came up was the use of DAC (Dedicated Admin Connection). This is not support within PowerShell or our PowerShell provider. Another question was the use of SQL Logins. It's possible but not totally clean. It's documented in BOL here. There were more questions which I asked people to email. I'm still waiting for the email! Once I get them I'll post them.

BRK: PBM: I want to thank Peter DeBetta (SQL MVP) for co-presenting with me. I think Peter knows some facets (pun intended) of PBM better than I. BTW: My wife won't let me name the boy Wequel Sequel Jones - probably a good thing. This was, however, a tough session. It'd didn't flow the way I wanted it to. The feedback was that it was a little disjointed. Not to mention I messed up and thought the session started at 4:45 rather than 4:30. Sorry for blowing the first 7 or 8 minutes. But hey who doesn't like a good Bugs Bunny cartoon. The passion for PBM is immense. I remain convinced PBM is going to forever transform the management of the data platform. Y'all will get to say you were there when it all started. There were also some questions during this session that I didn't have the answer to. If you're out there please email your question. If you forgot my email address you can always send it through the blog.

Thank you for making my experience at TechEd valuable and memorable. I don't like taking time away from my family unless it's truly valuable and fruitful. Once again you, the customer, have satisfied both. I hope you got as much out of the experience as I did.

One final note: If you're the guy that came up to me at Universal asking about my round at Pebble Beach I apologize for costing you $10. I pared a few holes (namely 7 and 17!). But I had a few disasters (namely 2 and 8). We don't have much sand on our courses in Seattle so that part of my game is out of practice. Also it was my first full round in 6 months and the first round after breaking my collar bone in January. I won't try snowboarding again!

Announcing the Policy-Based Management Blog

dtjones — Wed, 09 Apr 2008 05:50:03 GMT

Today we launched a new blog dedicated to Policy-Based Management. Take a look and send us your suggestions for topics. My goal is to have people from the PBM development team write and publish postings. Together we can make PBM an amazing feature. I'm really excited by this and I hope you are too.

Policy History Table Exception Columns

dtjones — Sun, 30 Mar 2008 00:39:15 GMT

I was recently asked what the exception_message and exception columns in the msdb.dbo.syspolicy_policy_execution_history_internal table are used for.

There are situations where the policy evaluate can throw an exception. For example, if the policy references a property in a facet which is not available on the given version or edition of the instance. For example the “Server Configuration” facet contains properties that only apply to the Express edition of SQL Server: UserInstancesEnabled and UserInstanceTimeout. If your policy references a condition with these properties the evaluation will throw an exception.

The exception message column contains the friendly exception message. In the example above it’ll contain:

Property value 'UserInstancesEnabled' is not available.

The exception column contains the full stack. In the example above it’ll contain:

Microsoft.SqlServer.Management.Dmf.NonRetrievablePropertyException: Property value 'UserInstancesEnabled' is not available. ---> Microsoft.SqlServer.Management.Smo.UnsupportedFeatureException: UserInstancesEnabled is not supported on this edition of SQL Server. at Microsoft.SqlServer.Management.Smo.Configuration.get_UserInstancesEnabled() at Microsoft.SqlServer.Management.Smo.ServerAdapterBase.get_UserInstancesEnabled() --- End of inner exception stack trace --- at Microsoft.SqlServer.Management.Dmf.Condition.Evaluate(Object target) at Microsoft.SqlServer.Management.Dmf.ObjectSet.CalculateTargets(IEnumerable objectSet, Condition condition, Object[]& conforming, TargetEvaluation[]& violating) at Microsoft.SqlServer.Management.Dmf.ObjectSet.CalculateTargets(SqlStoreConnection targetConnection, Condition condition, String policyCategory, Object[]& conforming, TargetEvaluation[]& violating) at Microsoft.SqlServer.Management.Dmf.Policy.EvaluatePolicyUsingConnections(AdHocPolicyEvaluationMode evaluationMode, SfcQueryExpression targetQueryExpression, ISfcConnection[] targetConnections)

I hope this clears up what these columns are for. Remember, only policies that are enabled on the server have their evaluation logged. If the policy is configured to only be evaluated on demand it's results are not written to the history table.

SQL2K8 Policy-Based Management Interview on TechNet

dtjones — Tue, 25 Mar 2008 22:27:54 GMT

I recently sat down with Joey Snow over at TechNet to discuss Policy-Based Management. You can watch the video here.

Declarative Management is Out!

dtjones — Sat, 22 Dec 2007 23:29:18 GMT

Yes, the title is correct, Declarative Management (DMF) is gone! Did I get your attention? Maybe I should elaborate. About 7 months ago we embarked on a naming project for DMF. We knew that DMF was a little too geeky and didn't necessarily describe the feature in an intuitive and meaningful way. Tehcnically the name is correct as the goal of DMF is to move from a procedural way of managing SQL Server to a declarative way. We kicked around about 10 or so names but couldn't reach any agreement.

That's when we decided to involve an outside agency that does this sort of thing. I knew these companies existed but had never worked with them. Years ago when I was at HP and we spun off the test and measurement business, what later became Agilent, I saw a little bit of the process. Though our budget for this was probably 1% of what it cost to come up with the Agilent name.

But I digress. We gave the agency an overview of DMF along with some key attributes of the feature. They used this information to generate a list of some 50+ names! Each of the names went through an initial legal review - think trademark search - and were divided into three groups: A, B, and C based on the chances of passing a more in-depth legal review. We threw out the B's and C's. We repeated this exercise two more times eventually whittling down the list to our 4 or 5 favorites.

The names were ether descriptive (think HD-DVD) or suggestive (think Blu-Ray). It's interesting in that usually only suggestive names can be trademarked. Our final list of 4 or so names contained a combination of suggestive and descriptive names.

With our final short list in hand the agency conducted a user survey. They surveyed over 350 IT professionals. The structure of the survey was interesting. Some participants were exposed to all of the names while others were only exposed to one of the names.

The agency did a fantastic job collecting and presenting the data. All-in-all it was a very fasinating process. One of our marketing guys, Niraj Nigrani ran the project from our side, he did a splendid job.

Okay, I've kept you in suspense long enough. The official name for DMF is Microsoft ® SQL Server ® 2008 Policy-Based Management.

I know you're probably thinking you could've come up with that name. And you're right. In fact Policy-based Management was on our original list. But just because it's obvious doesn't make it a bad name. IMHO it's a perfect name for the feature and I have the research to prove it. When people hear it they get it; which is exactly what you want in a name.

DMF for your Car

dtjones — Sun, 08 Jul 2007 06:04:01 GMT

@ Tech Ed this year I opened my session talking about how cars have evolved. The analogy is as we don't tinker with our cars anymore the same is going to have to managing data environments - and DMF is part of the initial wave that'll make this happen. In this week's Weekly Roundup of Database News from SQL Server Central there's a link to an article about smarter cars. This makes me think that at some point we're going to have policies governing our cars - when your son or daughter "logs in" it'll apply the policies applicable to them. For example, you could prevent them from going to certain parts of town or driving over a certain speed for more than a specific amount of time. Maybe there's a limit to the number of people they can have in the car at any one time. Lots of good stuff that I'm far more interested as parent than I'd be if I was a teen driver.

Declarative Management Performance

dtjones — Sun, 01 Jul 2007 19:36:27 GMT

There's an interesting thread on the MSDN forums regarding DMF performance or more accurately the overhead of having policies on the system. If you're interesting in using policies on your system and have thoughts on an acceptable overhead go post your thoughts to the forum.

Personally, I'm in the camp that believes DDL is not restricted to the same performance requirements as DML. Granted there are cases where you need to create a number of objects "on-the-fly" and in a certain amount of time. Think of a data loading process where the data is first loaded into "temporary" tables. You want those temporary tables to be created as quickly as possible. But will it really matter if it take 10 or 30 seconds rather than 1 second? I don't think so.

But let us know what you think.

SQL Server 2008 June CTP, Declarative Management LiveMeeting

dtjones — Wed, 27 Jun 2007 02:58:42 GMT

Today I presented a LiveMeeting on the Declarative Management features in the SQL Server 2008 June CTP. It should be posted shortly and is accessible here.

Declarative Management: Behind Enforce

dtjones — Sun, 17 Jun 2007 20:11:18 GMT

The new policy-based admin feature (Declarative Management) in SQL Server 2008 (code-named Katmai) can keep many aspects of the system from falling out of compliance. Some questions that come up are: "how does this work?" and "how come everything can't be enforced?" Let me tackle the first one, well, first. This is sort of a 300-level discussion.

How does this work?

The fundamental magic behind this is the DDL eventing in the engine. In SQL2K8 we've introduced a new type of DDL trigger that listens for all server and database scoped events. This is a public trigger and you'll be able to use it as well. The power in this is we don't have to sprinkle triggers around the system. We create a single trigger and update it with the events we care about. The list of interesting events is driven by the active policies. This means if you don't have any active policies we don't listen to any events. When a policy is activated we grab the events from the facet (which comes from the condition associated with the policy) and update the trigger to listen to those events. Each facet identifies which objects (table, view, database, etc) it works against and which events for those objects it cares about (create_table, alter_table, etc).

Great, now when an action happens on the system we can respond to it. But what do we do with the event once we get it? The main declarative management "engine" runs inside the SQLCLR. We have a stored procedure that executes some logic (I'll define in a minute) and based on the results of that execution it calls into the declarative mgmt engine in the SQLCLR. Before calling the SQLCLR the stored procedure determines if there are any policies that might apply to the object in the event. For example, the policy may filter out certain databases; therefore, we shouldn't run that policy against tables created in a filtered database. This allows us to reduce the chances of unnecessarily executing a policy. Note that even if we pass the pre-check and execute the policy, the policy will only be evaluated against objects that are in its scope. In essence we do the check twice. We do this due to the bit of overhead of the SQLCLR.

If the policy is checked and the object passes the policy we either move on to check the next policy or we return control back to the engine by passing success to the trigger. If the object fails the policy check we pass failure to the trigger (we stop executing any additional policies) and the engine eventing mechanism cancels the transaction and returns the system to its original state.

Declarative management uses the SQLCLR but this doesn't mean you have to enable the SQLCLR. The SQLCLR has two modes "On" and "Off". The "Off" mode still allows assemblies signed by Microsoft and installed in the Resource Database to execute. User assemblies, however, only run when the SQLCLR is "On". If your shop has a "policy" to keep the SQLCLR turned "off" you will still be able to use Declarative Management.

How come everything can't be enforced?

Since Declarative Management is dependent on the DDL eventing in the database engine we can only enforce state for those objects that are true transactions. For example, create table is a true transaction and can be rolled back with the system returned to its original state. However, something like create database is not a transaction. Can you put a trigger on create database and fail it? Yes. But this doesn't cleanly return the system to its original state. When a create database transaction is failed there is "residue" left behind, namely the data files. Because we cannot return the system to its original state we opted to turn off enforce for this class of objects. We're working with the database engine team to support true transactions for all DDL events but we won't get all the way there in SQL2K8.

I hope this brief overview of "Enforce" helps you understand what's going on under the covers.

Steve Jones on TechEd2007 and SQL2K8 Manageability Features

dtjones — Sun, 10 Jun 2007 01:13:13 GMT

@ TechEd 2007 I presented an overview session on new manageability features in SQL Server 2008. SQL Server Central's Steve Jones blogged about the session. Steve provides a pretty accurate account of the session (thanks Steve). This is mostly a straight forward report with a little editorial sprinkled in here and there.

There are a few items I'd like to address:

Backdoor in SQLCLR. The only assemblies that can run when the SQLCLR is off are those signed and shipped by Microsoft. When we build the product we pull in a group of assemblies into the resource database (a system database). This group of assemblies is the only one that is allowed to execute when the SQL CLR is disabled.
Backdoor in SQL Server Agent. Steve hypothesizes that we also have a backdoor in SQL Server Agent. This is not the case. If the service is turned off jobs will not run. This means if the system is configured with policies that are to be checked on schedule, the policies will only execute if SQL Server Agent is up and running. If Agent is down, no scheduled policy checks occur.
Performance Studio. He mentioned a Win32 API. If I said that during the presentation I misspoke. The API will be a managed API (not a native API).

And finally this: Steve writes "The database manages itself. Not totally, but close to it. It was an analogy made by Dan Jones with cars. Auto shop isn't really offered in high school anymore, people don't "tinker" with their cars much anymore.

If this is true, we have more time for data design, data architecture. I'm not sure I agree and as much as I hear, the more I think a DBA needs to be around because when something goes south, it goes way down."

The point isn't that we can eliminate DBAs or push them to the "design room". But rather, DBAs can focus on the harder more interesting problems. Think of it this way. It used to be the case when you put bad gas in your car, or the wrong octane, your car would ping. This is no longer the case. In addition, your car needed a tune-up every 30K miles or so. This is also no longer the case. Does this mean mechanics no longer see cars for pings or tune-ups? Absolutely not, it does mean they see fewer cars and time between visits is greater. In most instances the cases mechanics are handling are more complicated and require more advanced skills. I think this will be true for DBAs as well. The simple stuff will be handled by the system. The more complex stuff, though the system will help, will require the knowledge and skills of a rock star DBA.

It will be fun to see how this all plays out over the next 10 years.

SQL Server 2008 June CTP Bug

dtjones — Thu, 07 Jun 2007 05:29:15 GMT

There is a bug in the SQL Server 2008 June CTP that causes Declarative Management to be inoperable. The fix is very simple. Launch a command line and run the following command:

C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\sn.exe -Vr Microsoft.SqlServer.DmfSqlClrWrapper,89845DCD8080CC91

After running the command you will need to restart the SQL Server service.

The Declarative Management DLL, which runs inside the SQLCLR, did not get signed as part of the build process. Running the above command will disable strong name verification for this assembly. Client-side checking of policies: "Test Policy" and "Run Now" will function as expected without the fix. Policy automation (enforce, check on change, and check on schedule), though, will require applying the above fix.