Mark Brown's Blog : SQL Server

How to access SQL Server on Windows through a secure connection from Perl on Linux

mab — Fri, 05 Jun 2009 03:17:00 GMT

Scenario

Have some perl code that wants to access Microsoft SQL Server 2005. The SQL Server is configured to only allow SSL connections for security reasons.

Solution

Here are the parts you will need:

Perl on Linux (given)
DBI ... http://dbi.perl.org/
Unix ODBC (or equivalent like iODBC) ... http://www.unixodbc.org/
FreeTDS (or equivalent ODBC driver) ... http://www.freetds.org/
OpenSSL ... http://www.openssl.org/
TLS

Resources

Microsoft guidance on accessing SQL Server with Perl
http://technet.microsoft.com/en-us/library/bb497071.aspx

Thanks for Brian on helping figure this out :) ... I am just the messenger.

How to do logic OR AND NOT across rows in SQL

mab — Mon, 15 Dec 2008 20:02:00 GMT

Scenario

You have a normalized table that has values in rows. Let's take an example that has a history table with an approved flag:

Date Approved DollarValue Customer
1/1/2008 Y    13.5    A
2/1/2008 N    12      A
3/1/2008 Y    14      A

1/1/2008 Y    11.5    B
2/1/2008 Y    9        B
3/1/2008 Y    17      B

1/1/2008 Y    13.5    C
2/1/2008 Y    29        C
3/1/2008 Y    18      C

You want to find know if all the customers with all 3 monthly values approved and the values are < 20. Now I know you can get at this result multiple ways but I just want to show how you can apply logic across rows.

First you know that there are 3 monthly records for all the months data and you are also looking for an approved.

SELECT DISTINCT Customer
FROM MyTable
WHERE Approved = 'Y' AND DollarValue < 20
GROUP BY Customer
HAVING COUNT(Customer) = 3

You should see an output with only Customer B.

Similar techniques can be applied for OR and NOT.

Custom Security and SQL Job with SSIS Script Task

mab — Fri, 07 Nov 2008 00:18:00 GMT

Scenario

Recently I ran into a security issue that has a simple fix. Here is the situation:

SSIS Package that contains a Script Task
Run SSIS package as a SQL Job
Use a special credential/proxy for the step of executing the SSIS package

Results

In the SQL Job History Log you may find something like this ...
Code: 0x00000002 Source: XXXXX Script Task Description: The script threw an exception: Unable to generate a temporary class (result=1). error CS2001: Source file 'C:\WINDOWS\TEMP\yyyzzzxxx.N.cs' could not be found error CS2008: No inputs specified

Solution

Simple put,the VBA compiler has permissions to write the file into C:\windows\temp but your credential does not. Now the simple fix is to allow just read/read&execute/list permissions to C:\windows\temp and that will resolve the issue above. However, please not that there is a potential security risk in doing so as that this credential now has access to read all temp files in this directory. So be forewarned that this may not fit your situation.

Conclusion

Although not suggesting the ideal security solution, this may help you troubleshoot and further refine your strategy in running you SSIS package with a Script task with a custom credential/proxy.

Optimizing queries with joins and over statements in SQL 2005

mab — Tue, 29 Jul 2008 09:27:00 GMT

I was working on a common scenario and found an interesting side effect. Applications typically have a listing page that get search criteria from a UI and passes to a stored procedure. In the procedure you often find statements like:

SELECT a,b,c,d
FROM MyTable
WHERE c = @SearchC AND d = @SearchD

Often times you need to join information to the main data that is the most current from a historical table. For example in SQL 2005.

SELECT t.*
FROM
(SELECT
ID
, Amt
, AmtDepositDate
, row_number() over (partition by ID order by ID, AmtDepositDate desc) as rn
FROM MoneyTable
) t
WHERE rn=1

I found that when you join the above query to the main search query above a weird optimization occurs in SQL 2005. I suspect (unconfirmed with product group) that a left join can't optmize the query with the over so it ends up do a sub select on all records every row in the main search.

The solution (in my case where there is small sets of data of < 5000 records), I create a table variable and select into the lasted record from the history table. Then I join the table variable to the main table search.

In my scenario with 2000 main records and 1 million history records the left join with the latest select (no table variable) took 5+ minutes. Using the table variables took 2 seconds for everything.

Here is a snippet of the overall solution:

DECLARE @MyTable table (
      ID int
      ,Amt float
      ,AmtDepositDate datetime
      )

INSERT INTO @MyTable
SELECT
      t.ID
      ,t.Amt
      , t.AmtDepositDate
FROM
      (
      SELECT
      ID
      , Amt
      , AmtDepositDate
      , row_number() over (partition by ID order by ID, AmtDepositDate desc) as rn
      FROM MoneyTable
      ) t
WHERE rn=1

SELECT
      MT2.ID
      , MT2.Name
      , MT2.City
FROM
      MyTable2 MT2
LEFT JOIN @MyTable MT
      ON MT2.ID = MT.ID
WHERE
      c = @SearchC AND d = @SearchD

Anyone experience similar or can offer up an reason for the optimization?

SQL Server 2005 UNPIVOT Command - changings columns to rows (normalizing)

mab — Tue, 17 Jun 2008 00:55:00 GMT

Scenario
A table that contains several columns that you need to take and change into rows in order to normalize the data. With SQL Server 2005, a new T-SQL command UNPIVOT can help. An example table:

**Table called MyTable
FieldID FieldOne FieldTwo FieldThree
1 abc 3.40 2008-03-03 00:00:00.000
2 def 4.00 2008-01-02 00:00:00.000

**Table Design
FieldID is an integer
FieldOne is char(10)
FieldTwo is decimal(14,2)
FieldThree is a datetime

Sample Code
SELECT FieldId, FieldCode, FieldValue
FROM
(
SELECT FieldId,
    CONVERT(varchar(50), RTRIM(FieldOne )) AS FieldOne,
    CONVERT(varchar(50), FieldTwo) AS FieldTwo,
    CONVERT(varchar(50), FieldThree) AS FieldThree
FROM SampleUnpivot
) MyTable
UNPIVOT
(FieldValue FOR FieldCode IN (FieldOne, FieldTwo, FieldThree))AS MyUnPivot

Results

FieldID FieldCode  FieldValue
1       FieldOne   abc
1       FieldTwo   3.40
1       FieldThree Mar 3 2008 12:00AM
2       FieldOne   def
2       FieldTwo   4.00
2       FieldThree Jan 2 2008 12:00AM

Notes

If you have differnt data types make sure they are of the same time in the UNPIVOT IN clause
Make sure to include the "ID" field (FieldID column in example above) to determine original record relationship

How to write SQL to get a record to process in a cluster of processing computers

mab — Sun, 02 Mar 2008 00:59:00 GMT

Working with a friend at work, we were discussing how to create some SQL to go after records in a table and process in a farm of computers (processing cluster). After few minutes of thinking I came up with the following.

BEGIN TRAN MyTran

UPDATE TOP 1 [dbo].[SomeTable] WITH (READPAST,UPDLOCK)
SET [IsCompleted] = 1, @RowId = ID
WHERE [IsCompleted] = 0

-- Do som manual processing on the record @RowId

COMMIT TRAN MyTran

The transaction issues the locks and keeps everything ACID. The update writes or holds a lock on the 1st available record due to the READPAST hint and the UPDLOCK hint holds the lock until the commit tran. If you have 1 machine it works fine and if you add 1000 machines then each machine grabs 1 record and proesses it. It helps with multi-concurrency and blocking that might occur without the hints.

Now this is just a form of load balancing using SQL (smile).

How to backup and truncate all log files in a database

mab — Sat, 26 Jan 2008 20:41:00 GMT

Some things to consider ...

Make sure you can recover your server and database prior to issuing these commands.
Since I am generically calling a rountine you will receive errors on the system databases.
Your database and log files must follow the convention of name=database name_log=log file name.
This will force the reduction of your log files and you may not be able to recover unless you have done the appropriate database backup and log backup operations (I know this is repeating item 1 ... smile )
If you have any better ways or improvements feel free to post a comment.

Enough with the disclaimers, here is the code:

declare @sqlstring nvarchar(1024)
SET @sqlstring='use ?;DBCC SHRINKFILE (?, TRUNCATEONLY);DBCC SHRINKFILE (?_log, TRUNCATEONLY);BACKUP LOG ? WITH TRUNCATE_ONLY;DBCC SHRINKFILE (?_log, TRUNCATEONLY);';
SELECT @sqlstring;
exec master.dbo.sp_MSforeachDB @command1=@sqlstring

Unable to delete or disable SQL Server job due to error regarding MSX server

mab — Sat, 26 Jan 2008 19:54:00 GMT

Have you ever run across this error before when trying to delete or disable a SQL server job (or edit one):
Cannot add, update, or delete a job (or its steps or schedules) that originated from an MSX server.

This sometimes can happen if you rename a server to another name after you have created your jobs and you try to disable a SQL job. This may be fine if you use a management server, however if you are sure you do not have a management server and you want to fix it up continue to read on.

To inspect all your job information use this command.

use msdb
select * from sysjobs

To make all your jobs, owned so to speak, by the server, issue the following command after you make reasonable precautions (backups, backups, etc):

use msdb
DECLARE @srv sysname
SET @srv = CAST(SERVERPROPERTY('ServerName') AS sysname)
select @srv
select * from sysjobs
UPDATE sysjobs SET originating_server = @srv
select * from sysjobs

SQL Server 2005 SP2 - Access Denied on MSSQL\Data directory

mab — Tue, 23 Oct 2007 00:50:00 GMT

This was a strange occurance I found recently. If you encounter the error SQL Server Setup failed to modify security permissions on file C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data\ you might want to look at the permissions on every file (sometimes ownership can be damaged). Don't just set it at the folder level to push the child objects down (this won't always do it if you have ownership issues). Verify that the NT account used to run SP2 has full control priveldges on the data and all the database files (at least the system ones).

Of particulary, make sure there is no distmdl1.* files or mssqlsystemresource1.* files. If they are, back them up somewhere safe, delete them, and try reinstalling SP2.

Hopefuly this will help others as this was particularly fun to figure out.

Issue with SSIS Package calling stored procedure OUTPUT datetime parameter type

mab — Fri, 19 Oct 2007 01:06:00 GMT

Symptom

The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Parameter 3 ("@parameter"): Data type 0xE7 has an invalid data length or metadata length.

Solution

Installing SP2 my help. http://msdn2.microsoft.com/en-us/sql/bb426877.aspx

I was wondering if anyone else has had any issues related to this or found alternatives.

SQL Server 2005 Job, SSIS Step, Data Sources Access Denied

mab — Mon, 15 Oct 2007 04:52:00 GMT

This particular issue was interesting. I was troubleshooting an issue where an individual had rights to run their SSIS Step in a SQL Job. They could also change variables being passed, add/remove steps in job, almost everything except modify the Data Sources. When attempting to change the data source you would get an access denied. If you want to the SQL Server and Remote Desktop in then no access denied. I am going to spoil the ending and tell you it was related to DCOM permissions on the client and server.

By default (unless part of the Admin group) then users on Windows XP (SQL Server installed) do not have access to make DCOM calls. SSIS Step editor in the data sources tab make a call via DCOM to get the data source information. To enable the ability to talk to SQL Server specifically you must enable by following the steps outlined in the following KB article which is related.

Error message when you use SQL Server Management Studio to connect to SQL Server 2005 Integration Services from a client computer: "Connect to SSIS Service on machine '<SSISServer>' failed Access is denied"

http://support.microsoft.com/kb/940232

Selecting the Most Current Record in a table (SQL2000 and 2005 examples)

mab — Thu, 20 Sep 2007 04:45:00 GMT

My friend Alfredo helped with some SQL code and I thought it would be interesting to share. I was trying to select the most current record based on a unique id and a date within a single table. I was familar with the SQL 2000 way of doing things (a sub select) but he helped me with some CTE and WITH functions new with SQL 2005.

The scenario is I have a table with a unique id, interest rate and rate date (effective date). I need to return the latest record for each effective date.

Here is some sample code to evaluate, hope it helps!

Sample Code

if object_id('dbo.t') is not null
drop table dbo.t;
go

create table t(
UniqueID int not null,
InterestRate int not null,
RateDate datetime not null);
go

truncate table dbo.t

insert into dbo.t
select 1,1,'6/1/2007'
union
select 1,2,'6/2/2007'
union
select 1,3,'6/3/2007'
union
select 2,4,'7/1/2007'
union
select 2,5,'8/1/2007'
union
select 2,6,'9/1/2007'

select * from dbo.t
go

--sql 2005 ver 1

select
t.*
from
(
select
UniqueID
, InterestRate
, RateDate
,row_number() over(partition by UniqueID order by UniqueID,RateDate desc) as rn
from dbo.t
) as t
where rn=1
GO

--sql 2005 ver2
with t as
(
select
UniqueID
, InterestRate
, RateDate
,row_number() over(partition by UniqueID order by UniqueID,RateDate desc) as rn
from dbo.t
)

select * from t where t.rn=1
GO

--sql 2000
select
t1.*
from dbo.t t1
inner join
(
select
UniqueID,max(RateDate) as RateDate
from dbo.t
group by UniqueID
) t2
on t1.UniqueID = t2.UniqueID
and t1.RateDate = t2.RateDate

Secure SSIS step execution when running as a SQL Job

mab — Tue, 24 Jul 2007 02:12:00 GMT

Here is the scenario:

SQL Agent account (NETWORK SERVICE or some DOMAIN ACCOUNT) does not have access to database activities in a SSIS package. You want to run a job but need to "proxy" a security account when a SSIS package runs. The following script sets up the information required to run a specific SQL Job step under a certain seucrity account. This is one approach and there may be others. Hope this helps!

/* Wrap everything inside a transaction */
BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
DECLARE @SSISproxy nvarchar(255),@SSIScredential nvarchar(255), @SSIScategory nvarchar(255)
DECLARE @SSIScredentialname nvarchar(255), @SSIScredentialsecret nvarchar(255), @SSISjobname nvarchar(255)
DECLARE @SSISstepname nvarchar(255), @SSISjobdesc nvarchar(255), @SSISstepcmd nvarchar(255)

/* This is the variables used in the script */
SET @SSISproxy = N'My Proxy'
SET @SSIScredential = N'MyCredential'
SET @SSIScategory = N'My SSIS Category'
SET @SSIScredentialname = N'DOMAIN\USER'
SET @SSIScredentialsecret = N'USERPWD'
SET @SSISjobname = N'My SSIS Job'
SET @SSISstepname = N'My SSIS Step'
SET @SSISjobdesc = N'My SSIS Job Description'
SET @SSISstepcmd = N'/DTS "\File System\DTSXNAME" /SERVER SERVERNAME /MAXCONCURRENT " -1 " /CHECKPOINTING OFF'
USE [msdb]

/* Create a credential with the appropriate domain service account */
DECLARE @SQLString nvarchar(500)
SET @SQLString = 'CREATE CREDENTIAL ' + @SSIScredential + ' WITH IDENTITY = ''' + @SSIScredentialname + ''', SECRET = ''' + @SSIScredentialsecret +''';'
EXEC @ReturnCode = msdb.dbo.sp_executesql @SQLString;
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

/* Add a proxy for the SSIS step to use */
EXEC @ReturnCode = msdb.dbo.sp_add_proxy @proxy_name=@SSISproxy, @credential_name=@SSIScredential, @enabled=1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

/* Grant permission to proxy to SSIS subsystem */
EXEC @ReturnCode = msdb.dbo.sp_grant_proxy_to_subsystem @proxy_name = @SSISproxy, @subsystem_name= N'SSIS'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
USE [msdb]

/* Create Job category if does not exist */
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=@SSIScategory AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=@SSIScategory
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
END

/* Create Job */
DECLARE @jobId BINARY(16)
EXEC @ReturnCode = msdb.dbo.sp_add_job @job_name=@SSISjobname,
@enabled=1,
@notify_level_eventlog=0,
@notify_level_email=0,
@notify_level_netsend=0,
@notify_level_page=0,
@delete_level=0,
@description=@SSISjobdesc,
@category_name=@SSIScategory,
@owner_login_name=N'sa', @job_id = @jobId OUTPUT
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

/* Create Job Step */
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=@SSISstepname,
@step_id=1,
@cmdexec_success_code=0,
@on_success_action=1,
@on_success_step_id=0,
@on_fail_action=2,
@on_fail_step_id=0,
@retry_attempts=0,
@retry_interval=0,
@os_run_priority=0, @subsystem=N'SSIS',
@command=@SSISstepcmd,
@database_name=N'master',
@flags=0,
@proxy_name=@ssisproxy
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

COMMIT TRANSACTION
GOTO EndSave

QuitWithRollback:
IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:

Programatic way to determine space used on all database tables

mab — Wed, 18 Jul 2007 03:17:00 GMT

Here is one way I have found in order to get all tables space used. Effectively it queries the sp_tables and sp_spacedused in a loop to return the data.

Declare
@Table_Name varchar(255),
@SQL varchar(max)

---------------------------------------------------------------------------
-- Create a temporary table for storing result of sp_tables
---------------------------------------------------------------------------

Create Table #tServerTables
(
Table_Cat varchar(255),
Table_schem varchar(255),
Table_Name varchar(255),
Table_Type varchar(255),
Remarks varchar(255)
)

CREATE TABLE #tSpaceUsed
(
[name] varchar(255),
rows varchar(255),
reserved varchar(255),
data varchar(255),
index_size varchar(255),
unused varchar(255)
)

---------------------------------------------------------------------------
-- Populate Temporary table with the results of the sp_tables command
-- NOTE: the paramater passed to sp_spaceused MUST be the name of a
-- valid table
---------------------------------------------------------------------------

Insert Into #tServerTables exec sp_tables

---------------------------------------------------------------------------
-- Create cursor for selecting the Table names
---------------------------------------------------------------------------

Declare crsServerTables Cursor For
Select table_name
From #tServerTables
WHERE TABLE_TYPE='TABLE'

-- Open the cursor defined above

Open crsServerTables

Fetch Next From crsServerTables Into @Table_Name

While @@Fetch_Status = 0 Begin -- 0 = more records to process

INSERT INTO #tSpaceUsed exec sp_spaceused @Table_Name
-- Move to the next record
Fetch Next From crsServerTables Into @Table_Name

End

SELECT * FROM #tSpaceUsed

---------------------------------------------------------------------------
-- Clean up (drop temp tables, remove cursors)
---------------------------------------------------------------------------

drop table #tServerTables
drop table #tSpaceUsed
Close crsServerTables
Deallocate crsServerTables

Login Failed for User '' - SQL Server

mab — Thu, 21 Jun 2007 19:49:00 GMT

Periodically, I have seen issues with connecting to a remote SQL Server from SQL Server Management Studio, Asp.Net applications, ODBC connections, etc. Often times this is specific to a machine and not a complete on/off issue with SQL Server security. Often you can connect to other SQL servers but not particular ones. The issue could stem from stored username/passwords for a specific computer. Here is how you may be able to fix it:

Goto User Accounts in Control Panel
Click on the Advanced tab
Click on Manage Passwords
In the list, select the machine name and either edit the properties and correct the stored username and password or remove all together

Your all set. You should now be able to connect to your SQL Server!