Common tips and tricks from a SQL Developer Support perspective

Getting AutoNumber from Access via "SELECT @@IDENTITY" needs to be done in same connection as the INSERT.

2009-12-02T14:39:00Z

There are a few documents out there describing how to get the newly inserted AutoNumber in an Access table.

See for example,

"Walkthrough: Saving Data from Related Data Tables (Hierarchical Update)"

http://msdn.microsoft.com/en-us/library/bb384432.aspx

"Retrieving Identity or Autonumber Values (ADO.NET)"

http://msdn.microsoft.com/en-us/library/ks9f57t0.aspx

"HOW TO: Retrieve the Identity Value While Inserting Records into Access Database By Using Visual Basic .NET"

http://support.microsoft.com/default.aspx?scid=kb;en-us;815629

"Tackle Data Concurrency Exceptions Using the DataSet Object"

http://msdn.microsoft.com/en-us/magazine/cc188748.aspx

However, what some people seems to have missed or do not know is that this must be done on the same connection that executed the insert.

So, to see this, simply follow the steps below:

1. Create a new Access Database (Test.mdb) with a table in it (TestTable) that has two columns "ID" (which is AutoNumber) and "OurTxt" (which is Text).

2. Create a new console application.

DON'T FORGET TO CHANGE "Any CPU" to x86 if running on a x64 machine. Otherwise you will get the:

“System.InvalidOperationException: The 'Microsoft.Jet.OLEDB.4.0' provider is not registered on the local machine.”

See for example:

"System.InvalidOperationException: The 'Microsoft.ACE.OLEDB.12.0' provider is not registered on the local machine."

http://blogs.msdn.com/spike/archive/2009/02/27/system-invalidoperationexception-the-microsoft-ace-oledb-12-0-provider-is-not-registered-on-the-local-machine.aspx

3. Set the code to be:

class Program

{

static string path = @"<your path>";

static string db = @"Test.mdb";

static void Main(string[] args)

{

string cs = String.Format(@"Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0}\{1}", path, db);

// Using the same connection for the insert and the SELECT @@IDENTITY

using (OleDbConnection con = new OleDbConnection(cs))

{

con.Open();

OleDbCommand cmd = con.CreateCommand();

for (int i = 0; i < 3; i++)

{

cmd.CommandText = "INSERT INTO TestTable(OurTxt) VALUES ('" + i.ToString() + "')";

cmd.ExecuteNonQuery();

cmd.CommandText = "SELECT @@IDENTITY";

Console.WriteLine("AutoNumber: {0}", (int)cmd.ExecuteScalar());

}

con.Close();

}

// Using a new connection and then SELECT @@IDENTITY

using (OleDbConnection con = new OleDbConnection(cs))

{

con.Open();

OleDbCommand cmd = con.CreateCommand();

cmd.CommandText = "SELECT @@IDENTITY";

Console.WriteLine("\nNew connection, AutoNumber: {0}", (int)cmd.ExecuteScalar());

con.Close();

}

This should produce the self-explanatory output:

AutoNumber: 1

AutoNumber: 2

AutoNumber: 3

New connection, AutoNumber: 0

How to create a DNS less linked server in Access to SQL Server.

2009-11-24T09:59:00Z

Sometimes you may want to use Access as a front end for SQL Server.

When doing this you typically have linked tables to the SQL Server and you have the .mdb on a shared drive for users to access.

Normally you create a DSN and use that for the linked server. All works well when you are querying the linked tables from the machine that hosts the .mdb.

However, when a user opens the .mdb from the remote share, querying the linked tables fails with:

"ODBC--connection to '<The DSN name>' failed."

Basically this means that the client, i.e. the machine that you open the .mdb file on, not the machine that hosts the .mdb, does not have the ODBC DSN in question.

This is no problem; just create an ODBC DSN with the appropriate name that points to the server on that client machine and you should be good to go.

However, if you have many clients, then maintaining the OBDC DSNs could be a bit tiresome. I.e. if you change the database, then you need to update all clients DSNs.

There is one way to get around this, there may be more ways than this, but I’ve found this one to work.

In short, you create a DSN less ODBC connection via a macro. This way the information will be in the .mdb, and when you change the database, you update

the connection string and rerun the script. Now all clients will ‘automagically’ work.

Example setup:

1. SQL Server Machine

2. Access front end machine (ie. host the .mdb)

3. Access client machine (ie. opens the .mdb from machine 2)

On machine 2, create an ODBC DSN (I've used sqlsrv32.dll) called “ToNorthwind”, pointing to the Northwind database on the SQL Server on Machine 1.

On Machine 2, create two Access databases; WithDSN.mdb and WithoutDSN.mdb

In WithDSN.mdb;

File -> Get External Data -> Link Tables...

Select "ODBC Databases", select "Machine Data Source", select "ToNorthwind" and select (for example) the Employees table.

Double click the Employees table to verify that it works.

- In WithoutDSN.mdb;

Tools -> Macro -> Visual Basic Editor (or simply ALT+F11). This should open the editor.

Tools -> References.

Uncheck any "Microsoft ActiveX Data Objects 2.x Library" unless it is "Microsoft ActiveX Data Objects 2.8 Library"

Select "Microsoft ActiveX Data Objects 2.8 Library" and "Microsoft ADO Ext. 2.8 for DDL and Security"

*** or if on Windows 2008/Vista ***

Select "Microsoft ActiveX Data Objects 6.0 Library" and "Microsoft ADO Ext. 6.0 for DDL and Security"

Tools -> Macros..., enter a name for the macro, for example “DnsLessLinkTable”, and Create.

Enter the following code:

---------------------------------------------------------------------------------------------------------------------------------

Sub DnsLessLinkTable()

Dim con As ADODB.Connection

Dim cat As ADOX.Catalog

Dim tbl As ADOX.Table

Set con = New ADODB.Connection

Set cat = New ADOX.Catalog

Set tbl = New ADOX.Table

cat.ActiveConnection = CurrentProject.Connection

' Create the new (linked) table. Give it an appropriate name,

tbl.Name = "NoDnsEmployees"

Set tbl.ParentCatalog = cat

' Set the properties to create the linked table, Make sure you have a working connection string

tbl.Properties("Jet OLEDB:Link Provider String") = "ODBC;Driver={SQL Server};Server=servername\instancename;Database=Northwind;Uid=user;Pwd=password;"

' Select table in remote database, in this example, Employees table in Northwind Database

tbl.Properties("Jet OLEDB:Remote Table Name") = "Employees"

tbl.Properties("Jet OLEDB:Create Link") = True

' Append the table to the Tables collection.

cat.Tables.Append tbl

Set cat = Nothing

End Sub

---------------------------------------------------------------------------------------------------------------------------------

And Run it (F5)

Now you should have a table in the table list called "NoDnsEmployees". You may have to refresh list to see it.

Double click the “NoDnsEmployees” table to verify that it works.

On Machine 3,

Open the remote database called "WithDSN.mdb". You should see the “Employees” table, double click it. This should give:

"ODBC--connection to 'ToNorthwind' failed."

Open the remote database called "WithoutDSN.mdb". You should see the “NoDnsEmployees” table, double click it.

This should give your employees rows without having a 'ToNorthwind' DSN on Machine 3.

If adding the 'ToNorthwind' DNS to Machine 3, both Access database linked tables work.

Getting metadata about dataset using lambda. Some examples.

2009-11-13T15:30:00Z

This may not be a massive business requirement, but if you have a bunch of DataSets and you want (for some reason) extract information about them

then using Lambda and LINQ queries is one way to do it. I will let the code below speak for itself.

Sample code for meta data collection for DataSet:

class Program

{

static void Main(string[] args)

{

// Create a demo dataset

DataSet ourDataSet = CreateDataSet();

// Display the DataTable info

ShowTableCollectionInfo(ourDataSet.Tables);

}

private static void ShowTableCollectionInfo(DataTableCollection dtc)

{

IEnumerable<DataTable> tbls = dtc.Cast<DataTable>();

int tblCount = dtc.Count;

StringBuilder sb = new StringBuilder();

// Which table(s) has the most rows?

int maxRows = tbls.Max(x => x.Rows.Count);

var maxRowsTableQuery = tbls.Where((x, max) => x.Rows.Count == maxRows);

sb.AppendFormat("Table(s) with highest # of rows ({0}):", maxRows, tblCount);

foreach (DataTable tbl in maxRowsTableQuery)

{

sb.AppendFormat("\n\t{0}", tbl.TableName);

}

// How many rows are there in total, i.e. sum of rows in all datatables?

int totalRows = tbls.Sum(x => x.Rows.Count);

sb.AppendFormat("\n\nTotal # of rows: {0} (in {1} tables)", totalRows, tblCount);

// How many column on average.

double columnAve = tbls.Average(x => x.Columns.Count);

sb.AppendFormat("\n\nAverage # of cols: {0} (in {1} tables)", columnAve.ToString(), tblCount);

//etc.

// Show results

Console.WriteLine(sb.ToString());

}

private static DataSet CreateDataSet()

{

DataTable dt_1 = new DataTable("DataTable_One");

dt_1.Columns.Add(new DataColumn("T1_C1", typeof(int)));

dt_1.Columns.Add(new DataColumn("T1_C2", typeof(int)));

dt_1.Columns.Add(new DataColumn("T1_C3", typeof(int)));

DataTable dt_2 = new DataTable("DataTable_Two");

dt_2.Columns.Add(new DataColumn("T2_C1", typeof(int)));

dt_2.Columns.Add(new DataColumn("T2_C2", typeof(int)));

DataTable dt_3 = new DataTable("DataTable_Three");

dt_3.Columns.Add(new DataColumn("T3_C1", typeof(int)));

DataTable dt_4 = new DataTable("DataTable_Four");

dt_4.Columns.Add(new DataColumn("T4_C1", typeof(int)));

dt_4.Columns.Add(new DataColumn("T4_C2", typeof(int)));

dt_4.Columns.Add(new DataColumn("T4_C3", typeof(int)));

DataSet ds = new DataSet("OurDataSet");

ds.Tables.Add(dt_1);

ds.Tables.Add(dt_2);

ds.Tables.Add(dt_3);

ds.Tables.Add(dt_4);

for (int i = 0; i < 10; i++)

{

dt_1.Rows.Add(new object[] { i, i + 5, i + 10 });

}

for (int i = 0; i < 5; i++)

{

dt_2.Rows.Add(new object[] { i, i * 5 });

}

for (int i = 0; i < 15; i++)

{

dt_3.Rows.Add(new object[] { i * 10 });

}

for (int i = 0; i < 5; i++)

{

dt_4.Rows.Add(new object[] { i, i + 100, i + 50 });

}

return ds;

}

Running this should output the following:

Table(s) with highest # of rows (15):

DataTable_Three

Total # of rows: 35 (in 4 tables)

Average # of cols: 2,25 (in 4 tables)

How a proper physical TCP/IP connection to SQL Server works. Network Monitor

2009-11-06T15:38:00Z

Not about a problem today, today will be about what a successful physical connection from client to SQL Server looks like in Network Monitor.

First download Network Monitor (currently version 3.3):

“Microsoft Network Monitor 3.3”

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=983b941d-06cb-4658-b7f6-3088333d062f

. Install it on the client machine and on the SQL Server machine.

. Start it.

. Select “New Capture”

. Make sure you select the “Capture Filter” tab and not the “Display Filter” tab. Otherwise you will capture all network traffic, not just the TCP we are interested in.

. Set the “Capture Filter” to:

tcp.Port == xxxxx

Where you replace xxxx with the port number your SQL Server is listening on. If you do not know what port it is listening on, the easiest way is to open the SQL Server log, normally found here:

C:\Program Files\Microsoft SQL Server\<your server instance>\MSSQL\Log

At the start of the log, you should see something like this:

2009-11-04 09:06:40.73 Server Server is listening on [ 'any' <ipv6> 60847].

2009-11-04 09:06:40.73 Server Server is listening on [ 'any' <ipv4> 60847].

So here we would replace the xxxx with 60847.

. Then when the filter is set, do not forget to click Apply! This will change <No Active Filter> to tcp.Port == 60847 using the example above.

. Then Start the capture (F5), connect to your SQL Server, execute your query, Stop (F7) the capture on both machines.

. This should produce an output like this (here I have a display filter set to !SSL):

TimeOfDay SourceIP SourcePort DestinationIP DestinationPort Description

3 14:57:51.505 xxx.xx.162.187 51712 (0xCA00) xxx.xx.162.174 60847 (0xEDAF) TCP TCP:Flags=......S., SrcPort=51712, DstPort=60847, PayloadLen=0, Seq=663164647, Ack=0, Win=8192

4 14:57:51.506 xxx.xx.162.174 60847 (0xEDAF) xxx.xx.162.187 51712 (0xCA00) TCP TCP:Flags=...A..S., SrcPort=60847, DstPort=51712, PayloadLen=0, Seq=3468829443, Ack=663164648,

5 14:57:51.509 xxx.xx.162.187 51712 (0xCA00) xxx.xx.162.174 60847 (0xEDAF) TCP TCP:Flags=...A...., SrcPort=51712, DstPort=60847, PayloadLen=0, Seq=663164648, Ack=3468829444,

15 14:57:51.801 xxx.xx.162.174 60847 (0xEDAF) xxx.xx.162.187 51712 (0xCA00) TCP TCP:Flags=...A...., SrcPort=60847, DstPort=51712, PayloadLen=0, Seq=3468830489, Ack=663165781,

19 14:57:52.503 xxx.xx.162.187 51712 (0xCA00) xxx.xx.162.174 60847 (0xEDAF) TCP TCP:Flags=...A...., SrcPort=51712, DstPort=60847, PayloadLen=0, Seq=663165897, Ack=3468830947,

22 14:57:59.513 xxx.xx.162.187 51712 (0xCA00) xxx.xx.162.174 60847 (0xEDAF) TCP TCP:Flags=...A...., SrcPort=51712, DstPort=60847, PayloadLen=0, Seq=663165957, Ack=3468831154,

24 14:58:05.287 xxx.xx.162.174 60847 (0xEDAF) xxx.xx.162.187 51712 (0xCA00) TCP TCP:Flags=...A...., SrcPort=60847, DstPort=51712, PayloadLen=0, Seq=3468831154, Ack=663166035,

26 14:58:05.523 xxx.xx.162.187 51712 (0xCA00) xxx.xx.162.174 60847 (0xEDAF) TCP TCP:Flags=...A...., SrcPort=51712, DstPort=60847, PayloadLen=0, Seq=663166035, Ack=3468831465,

27 14:58:12.845 xxx.xx.162.187 51712 (0xCA00) xxx.xx.162.174 60847 (0xEDAF) TCP TCP:Flags=...A...F, SrcPort=51712, DstPort=60847, PayloadLen=0, Seq=663166035, Ack=3468831465,

28 14:58:12.845 xxx.xx.162.174 60847 (0xEDAF) xxx.xx.162.187 51712 (0xCA00) TCP TCP:Flags=...A...., SrcPort=60847, DstPort=51712, PayloadLen=0, Seq=3468831465, Ack=663166036,

29 14:58:12.846 xxx.xx.162.174 60847 (0xEDAF) xxx.xx.162.187 51712 (0xCA00) TCP TCP:Flags=...A...F, SrcPort=60847, DstPort=51712, PayloadLen=0, Seq=3468831465, Ack=663166036,

30 14:58:12.846 xxx.xx.162.187 51712 (0xCA00) xxx.xx.162.174 60847 (0xEDAF) TCP TCP:Flags=...A...., SrcPort=51712, DstPort=60847, PayloadLen=0, Seq=663166036, Ack=3468831466,

So here we see following chain of events (3, 4 and 5):

.The client (xxx.xx.162.187:51712) sends a SYN (S) to the server (xxx.xx.162.174:60847).

.The server (xxx.xx.162.174:60847) sends an ACK (A) and a SYN (S) to the client (xxx.xx.162.187:51712).

.The client (xxx.xx.162.187:51712) sends an ACK (A) to the server (xxx.xx.162.174:60847).

This is the 3way handshake and once this is complete the physical connection between the client and the server has been setup.

Still, this does not mean that the SQL Server itself has, for example, denied a Login but it proves that the physical connection has been setup.

If there was, for example, a firewall blocking or a router between the client and server that was misrouting, we would not see the S/SA/A sequence.

After this handshake, we will see few packages being sent, this is the data traffic.

Finally we have the following events (27, 28, 29 and 30):

.The client (xxx.xx.162.187:51712) sends an ACK (A) and a FIN (F) to the server (xxx.xx.162.174:60847). FIN tells the server that we will not send more data, ACK to let the server what connection to close.

.The server (xxx.xx.162.174:60847) sends an ACK (A) to the client (xxx.xx.162.187:51712). This acknowledges the FIN received in the previous step. It is not closing yet since the server may have more data to transfer.

.The server (xxx.xx.162.174:60847) sends an ACK (A) and a FIN (F) to the client (xxx.xx.162.187:51712). FIN tells the client that the server will not send more data.

.The client (xxx.xx.162.187:51712) sends an ACK (A) to the server (xxx.xx.162.174:60847). ACK means that the client has acknowledged that the server will send more data.

This is the teardown of the physical connection.

So, if you have a trace then what you should look for is the S/SA/A sequence and the AF/A/AF/A sequence.

If these sequences exist, then you know that the physical network is working. But as mentioned, there could be issues after the physical connection has been setup, for example,

the server is performing slowly but responds, the login fails etc.

As an example, if you run a trace on the client and a trace on the SQL Server and you in the client trace can see the S being in the trace taken on the client, but you do not see

the S in the trace on the server. Then the first step would be to check the firewall so that it is not blocking the port in question. The port in question is clearly seen in the trace.

NOTE; I you are running with connection pooling enabled, which is default from, for example, a .Net client, the S/SA/A will only show on the first connection.

And the AF/A/AF/A sequence will only be shown when the last connection is closed. This is the nature (and purpose) of connection pooling.

Format of the initialization string does not conform to specification starting at index

2009-10-30T15:37:00Z

Short one.

You are letting your user connect to your trusted SQL Server (fairly common scenario J)

They are connecting using SQL Authentication (it is recommended to use Windows Authentication however) an intermittently the application throws an exception:

Unhandled Exception: System.ArgumentException: Format of the initialization string does not conform to specification starting at index 42.

at System.Data.Common.DbConnectionOptions.GetKeyValuePair(String connectionString, Int32 currentPosition, StringBuilder buffer, Boolean useOdbcRules, String& keyname, String& keyvalue)

at System.Data.Common.DbConnectionOptions.ParseInternal(Hashtable parsetable, String connectionString, Boolean buildChain, Hashtable synonyms, Boolean firstKey)

at System.Data.Common.DbConnectionOptions..ctor(String connectionString, Hashtable synonyms, Boolean useOdbcRules)

at System.Data.SqlClient.SqlConnectionString..ctor(String connectionString)

at System.Data.SqlClient.SqlConnectionFactory.CreateConnectionOptions(String connectionString, DbConnectionOptions previous)

at System.Data.ProviderBase.DbConnectionFactory.GetConnectionPoolGroup(String connectionString, DbConnectionPoolGroupOptions poolOptions, DbConnectionOptions& userConnectionOptions)

at System.Data.SqlClient.SqlConnection.ConnectionString_Set(String value)

A common scenario here is that the user has provided a string termination character in the username or password and that character is not correctly escaped.

So, to illustrate:

string cs = @"Data Source=server\instance;User=username;Password='password;Initial Catalog=database";

using (SqlConnection con = new SqlConnection(cs))

{

try

{

con.Open();

Console.WriteLine("We are connected...");

con.Close();

}

catch (Exception ex)

{

Console.WriteLine(ex);

}

Will give the exception above.

So either escape the ‘ correctly (Password=\'password) or do not allow the single quote.

Why is autogenerated UPDATE and DELETE statements complex?

2009-10-22T14:24:00Z

Sometimes people ask why the code generated for INSERT/UPDATE/DELETE is so complex, or verbose if you like, why not just operate on the primary key?

To illustrate what we are talking about here, create a table in SQL Server and insert some rows:

create table Person (Id int primary key, FirstName nvarchar(50), LastName nvarchar(50), City nvarchar(50), Age int)

insert into Person values (1, 'John','Johnson','London', 20)

insert into Person values (2, 'Paul','Paulson','Paris', 30)

insert into Person values (3, 'Sven','Svenson','Stockholm', 40)

Then create a new C# console application and insert the following code:

class Program

{

static void Main(string[] args)

{

string cs = @"Data Source=<your server>;Initial Catalog=<your database>;Integrated Security=SSPI";

using (SqlConnection con = new SqlConnection(cs))

{

try

{

con.Open();

SqlCommand cmd = con.CreateCommand();

cmd.CommandText = "SELECT Id, FirstName, LastName, City, Age FROM Person";

SqlDataAdapter adp = new SqlDataAdapter(cmd);

SqlCommandBuilder scb = new SqlCommandBuilder(adp);

adp.InsertCommand = scb.GetInsertCommand();

adp.UpdateCommand = scb.GetUpdateCommand();

adp.DeleteCommand = scb.GetDeleteCommand();

Console.WriteLine("\n{0}", adp.InsertCommand.CommandText);

Console.WriteLine("\n{0}", adp.UpdateCommand.CommandText);

Console.WriteLine("\n{0}", adp.DeleteCommand.CommandText);

con.Close(); }

catch (Exception ex)

{

Console.WriteLine(ex);

}

And run it, this should give the following output:

INSERT INTO [Person] ([Id], [FirstName], [LastName], [City], [Age]) VALUES (@p1, @p2, @p3, @p4, @p5)

UPDATE [Person] SET [Id] = @p1, [FirstName] = @p2, [LastName] = @p3, [City] = @p4, [Age] = @p5 WHERE (([Id] = @p6) AND ((@p7 = 1 AND [FirstName] IS NULL)

OR ([FirstName] = @p8)) AND ((@p9 = 1 AND [LastName] IS NULL) OR ([LastName] = @p10)) AND ((@p11 = 1 AND [City] IS NULL) OR ([City] = @p12)) AND

((@p13 = 1 AND [Age] IS NULL) OR ([Age] = @p14)))

DELETE FROM [Person] WHERE (([Id] = @p1) AND ((@p2 = 1 AND [FirstName] IS NULL) OR ([FirstName] = @p3)) AND ((@p4 = 1 AND [LastName] IS NULL)

OR ([LastName] = @p5)) AND ((@p6 = 1 AND [City] IS NULL) OR ([City] = @p7)) AND ((@p8 = 1 AND [Age] IS NULL) OR ([Age] = @p9)))

So why this ‘complex’ SQL, why not only operate on the primary key? If we want to delete an entry, then “DELETE FROM Person WHERE id = @id” should do it, right?

Well, yes, that SQL would work, but ADO.Net is designed around a disconnected and optimistic concurrency approach.

For example, UserA retrieves the data from the Person table. This means that the code will open a connection, get the data, close the connection. Then UserA can edit, add, delete as much

as he or she likes. While UserA is doing this, the data remains unchanged on the server. When UserA is done operating on the data the connection opened, the changes are submitted (using the SQL above)

and the connection is closed. So far so good, but what if UserB also operates on the data at the same time. In other words;

UserA retrieves the data for Person with Id 2 to the client.

UserB retrieves the same data (Person with Id 2) to the client and edits it, for example, changes the City and Age.

UserB submits the changes back to the database from UserB client.

UserA deletes the Person with Id 2 and submits the changes back to the database.

If we were only operating on the primary key, it would mean that the edited Person (Id 2) would be deleted. UserB would not like this. Or at least be very surprised when the data is no longer there.

By using UPDATE and DELETE statements that operates on all columns in the table, we make sure that when UserA in the scenario above deletes a row, only the row that exactly matches the row

that UserA originally retrieved is deleted. If the exact matching fails (which it will if UserB has changed City and Age) the row is not deleted.

This is, for example, how LINQ operates in the background.

More info here:

"Optimistic Concurrency (ADO.NET)"

http://msdn.microsoft.com/en-us/library/aa0416cz.aspx

"Generating Commands with CommandBuilders (ADO.NET)"

http://msdn.microsoft.com/en-us/library/tf579hcz.aspx

My take on @@IDENTITY vs SCOPE_IDENTITY()

2009-10-20T13:56:00Z

This may be considered to be somewhat of a ‘classic’, but as mentioned, many of the entries in this blog are reminders and/or point of reference for myself.

Which I am of course happy to share. This one is about a question that comes every now and then;

“There is something wrong with @@IDENTITY, we have been using this for some time and it has always correctly returned the identity value of the last inserted row.

Now it returns what seems to be random numbers. Also, it seems to work sometimes and fails other times. This must be some sort of a bug”.

The short answer here is do not use @@IDENTITY for this purpose, use SCOPE_IDENTITY(). There is a subtle difference between these.

And the behavior described above is usually an effect of using @@IDENTITY instead of SCOPE_IDENTITY() in combination with triggers.

As usual, I’ll show this behavior by example since this makes it easier to understand.

First we need two tables (with some rows) and a stored procedure that takes care of inserting rows to one of the tables.

This procedure will return the new identity of the row being inserted. The emptype here tells us if the employee is a Manager or not.

So from SQL Server Management Studio, run the following:

create table Employee (id int identity, empname nvarchar(100), emptype int)

create table Manager (id int identity, empid int)

insert into Employee values ('John', 0)

insert into Employee values ('Paul', 0)

insert into Employee values ('Jane', 0)

insert into Employee values ('Mike', 1)

insert into Employee values ('Mary', 1)

create procedure InsertPerson(@empname nvarchar(100), @emptype int)

as begin

insert into Employee values (@empname, @emptype)

declare @newId int

set @newId = @@identity

return @newId

end

Check the number of rows in the Employee table (select * from Employee), this should return 5 rows, with 5 being the highest id.

Test the stored procedure:

declare @newId int

exec @newId = InsertPerson 'Neil', 1

select @newId

This should return 6, recheck the number of rows in the Employee table, this should now return 6 rows. So at this stage all is well.

Create a new C# console application that uses the stored procedure in order to insert new Employees. Code as follows:

static void Main(string[] args)

{

string cs = @"Data Source=<your server>;Integrated Security=TRUE;Initial Catalog=<your database>";

using (SqlConnection con = new SqlConnection(cs))

{

try

{

con.Open();

SqlCommand cmd = new SqlCommand("InsertPerson", con);

cmd.CommandType = System.Data.CommandType.StoredProcedure;

cmd.Parameters.Add("@empname", SqlDbType.NVarChar).Value = "Pete";

cmd.Parameters.Add("@emptype", SqlDbType.Int).Value = 1;

cmd.Parameters.Add("@newId", System.Data.SqlDbType.Int).Direction = System.Data.ParameterDirection.ReturnValue;

cmd.ExecuteNonQuery();

int newId = (int)cmd.Parameters["@newId"].Value;

Console.WriteLine("The returned id is {0}", newId);

con.Close();

}

catch (Exception ex)

{

Console.WriteLine(ex);

}

And run it, this should return 7.

Change the name (Pete) to something else and rerun, this should return 8.

So now your application works. You can successfully insert new employees and get back the identity of that new row.

Your application runs fine.

Now, later on someone decides that when a new employee is added, we should also insert a row into the Manager table if the Employee emptype is 1 (which is Manager in this example).

This is solved by using a trigger. So in SSMS add the following trigger:

create trigger MgrTrigger on Employee after insert

if (select emptype from inserted) = 1

begin

declare @id int

set @id = (select id from inserted)

insert into Manager(empid) values(@id)

end

Try this by inserting one row into the Employee table which has emptype = 1 and one row which has emptype = 0

insert into Employee values ('Timm', 0)

insert into Employee values ('Anna', 1)

Now, select * from Employee and select * from Manager.

You should have 10 rows in the Employee table and 1 row in the Manager table, so this seems to work fine.

Go back to the C# application, change the name to something new and rerun it.

What do you expect the return value to be now? There are 10 rows in the Employee table, so it should be 11. Right?

Wrong, the returned value is actually 2.

So you check the number of rows in the Employee table, this returns 11 and the last name is the one you added.

So where does the value 2 come from?

Well, this is where the subtle difference between using @@IDENTITY instead of SCOPE_IDENTITY() comes into play.

If we check the documentation we can see the following:

After an INSERT, SELECT INTO, or bulk copy statement is completed, @@IDENTITY contains the last identity value that is generated by the statement. If the statement did not affect any tables with identity columns, @@IDENTITY returns NULL. If multiple rows are inserted, generating multiple identity values, @@IDENTITY returns the last identity value generated. If the statement fires one or more triggers that perform inserts that generate identity values, calling @@IDENTITY immediately after the statement returns the last identity value generated by the triggers. If a trigger is fired after an insert action on a table that has an identity column, and the trigger inserts into another table that does not have an identity column, @@IDENTITY returns the identity value of the first insert. The @@IDENTITY value does not revert to a previous setting if the INSERT or SELECT INTO statement or bulk copy fails, or if the transaction is rolled back.

Failed statements and transactions can change the current identity for a table and create gaps in the identity column values. The identity value is never rolled back even though the transaction that tried to insert the value into the table is not committed. For example, if an INSERT statement fails because of an IGNORE_DUP_KEY violation, the current identity value for the table is still incremented.

@@IDENTITY, SCOPE_IDENTITY, and IDENT_CURRENT are similar functions because they all return the last value inserted into the IDENTITY column of a table.

@@IDENTITY and SCOPE_IDENTITY return the last identity value generated in any table in the current session. However, SCOPE_IDENTITY returns the value only within the current scope; @@IDENTITY is not limited to a specific scope.

What this then means is that since we have introduced a trigger that does a conditional insert (if emptype is 1) it means that @@IDENTITY in this case returns the last generated identity value

for the Manager table, not the Employee table. This also means that if you change the C# code to insert an Employee that is not a manager (emptype = 0) it will again seem to work as expected.

Whether to use one or the other, I’ll leave that decision to you. This post is only about explaining and exemplifying what is going on.

If you need more guidance, a simple search on SCOPE_IDENTITY and @@IDENTITY will give a lot more information.

DataTable.GetChanges(DataRowState.Modified) returns NULL

2009-10-20T09:06:00Z

The scenario; you have a DataGridView on a Windows Form, you then manually, in code, create a DataTable and set this to be the DataSource of the DataGrid.

You then edit some of the data in the grid and then you need to get hold of a subset of data containing the Modified rows:

DataTable modifiedTable = dt.GetChanges(DataRowState.Modified);

int modifiedRowCount = modifiedTable.Rows.Count;

This however throws an exception

System.NullReferenceException was unhandled

Message="Object reference not set to an instance of an object.";

This is a bit odd; you have clearly edited (Modified) the data, so why are no rows returned from the GetChanges(DataRowState.Modified) method?

In short, what you most likely are experiencing is that you have not called AcceptChanges() on the DataTable before you set it to be the DataSource of the DataGrid.

This will cause all rows (including the one you have edited) to be in the Added state. This will take precedence over the Modified state.

As usual, it is easier to show by example. Create a new Windows Form project; add a Button, a DataGridView and a ListBox to the Form.

Then replace the generated code with the code below.

public partial class Form1 : Form

{

DataTable dt;

public Form1()

{

InitializeComponent();

dt = CreateTable();

dataGridView1.DataSource = dt;

}

private void button1_Click(object sender, EventArgs e)

{

foreach (DataRow dr in dt.Rows)

{

listBox1.Items.Add(String.Format("Id: {0}, RowState: {1}", dr["id"], dr.RowState));

}

DataTable modifiedTable = dt.GetChanges(DataRowState.Modified);

// Row below will fail with the NullReferenceException.

//int modifiedRowCount = modifiedTable.Rows.Count;

}

private DataTable CreateTable()

{

DataTable dt = new DataTable();

dt.Columns.Add(new DataColumn("id", typeof(int)));

dt.Columns.Add(new DataColumn("FirstName", typeof(string)));

dt.Columns.Add(new DataColumn("LastName", typeof(string)));

dt.Rows.Add(new object[] { 1, "Mike", "Mikeson" });

dt.Rows.Add(new object[] { 2, "John", "Johnson" });

dt.Rows.Add(new object[] { 3, "Paul", "Paulson" });

return dt;

}

Run it, edit some values in the columns and hit the button. This should produce the following output in the Listbox:

Id: 1, RowState: Added

Id: 2, RowState: Added

Id: 3, RowState: Added

And this will of course cause the dt.GetChanges(DataRowState.Modified) call to return NULL as specified in the documentation:

“If no rows of the desired DataRowState are found, the method returns null”

So, what you need to do here is simply to call AcceptChanges(), which will set all rows RowState to Unchanged, before setting it to be the DataSource of the grid.

By doing this, the changes you make will be detected and the rows will go into the Modified state. Simply add this before the DataTable is returned.

dt.AcceptChanges();

return dt;

Rerun the application make a change, this should now produce the following, allowing the dt.GetChanges(DataRowState.Modified) to return a datatable with one row.

Id: 1, RowState: Unchanged

Id: 2, RowState: Unchanged

Id: 3, RowState: Modified

XQuery in SQL Server, some examples

2009-09-21T16:35:00Z

I got an XQuery in SQL Server related question from a colleague the other day. I thought I had the answer, but it showed that I forgot most of it.

And of course I hadn’t written down my examples or tests from previous encounters regarding this topic.

So this is not intended to be an explanation of XQuery in SQL Server, just a quick reference that can be used in the future, and it will be expanded when there is a need.

The examples should be pretty self-explanatory.

declare @xml xml

set @xml =

'<RecordStore>

<Artist>Rockers Utd.</Artist>

<Title>Rock Until You Drop</Title>

</Album>

<Artist>Oldies Inc.</Artist>

<Title>Rock Like There Was A Tomorrow</Title>

</Album>

<Artist>ShockRockers</Artist>

<Title>ShockRock</Title>

</Album>

<Artist>Swing Its</Artist>

<Title>Shake and Bake</Title>

</Album>

</RecordStore>'

-- Using @ signifies that we query on attribute, no @ means value of node

-- Get all album titles in Category = "Rock"

select @xml.query('/RecordStore/Album[@category="Rock"]/Title')

-- Get all artists that released albums in 2005

select @xml.query('/RecordStore/Album[ReleaseYear=2005]/Artist')

-- Get the price for the Album called ‘ShockRock’

select @xml.query('/RecordStore/Album[Title="ShockRock"]/Price')

-- Get all albums titles with a price higher than 10

select @xml.query('/RecordStore/Album[Price>10]/Title')

-- Get the titles for the two first albums in list

select @xml.query('/RecordStore/Album[position()<=2]/Title')

-- Note that the above examples returns the nodes as XML, if you wish to get

-- hold of a singular value, then use value() method instead, this returns a scalar so it needs a datatype as the second argument.

-- Get the price for the Album called ShockRock

select @xml.value('(/RecordStore/Album[Title="ShockRock"]/ReleaseYear)[1]', 'int')

-- Get the title for the album with id 4

select @xml.value('(/RecordStore/Album[@id="4"]/Title)[1]', 'varchar(100)')

Running this should produce the following results

RockAlbums

--------------------------------------------------------------------------

<Title>Rock Until You Drop</Title><Title>ShockRock</Title>

(1 row(s) affected)

ReleasesIn2005

--------------------------------------------------------------------------

<Artist>Rockers Utd.</Artist><Artist>Swing Its</Artist>

(1 row(s) affected)

AlbumPrice

-------------------------------------------------------------------------

(1 row(s) affected)

PriceAbove

------------------------------------------------------------------------

<Title>ShockRock</Title><Title>Shake and Bake</Title>

(1 row(s) affected)

FirstTwoAlbums

------------------------------------------------------------------------

<Title>Rock Until You Drop</Title><Title>Rock Like There Was A Tomorrow</Title>

(1 row(s) affected)

AlbumPrice

------------------------------------------------------------------------

2000

(1 row(s) affected)

AlbumByID

------------------------------------------------------------------------

Shake and Bake

(1 row(s) affected)

MemberwiseClone by example

2009-09-17T16:15:00Z

I not ashamed of admitting that I do not know everything about everything J

For example, I didn’t know that well what the method MemberwiseClone() method does.

So, first step was to check the MSDN documentation, found here.

"Object.MemberwiseClone Method"

http://msdn.microsoft.com/en-us/library/system.object.memberwiseclone.aspx

This gives the following explanation;

The MemberwiseClone method creates a shallow copy by creating a new object, and then copying the nonstatic fields of the current object to the new object.

If a field is a value type, a bit-by-bit copy of the field is performed. If a field is a reference type, the reference is copied but the referred object is not; therefore, the original object and its clone refer to the same object.

For example, consider an object called X that references objects A and B. Object B, in turn, references object C. A shallow copy of X creates new object X2 that also references objects A and B.

In contrast, a deep copy of X creates a new object X2 that references the new objects A2 and B2, which are copies of A and B. B2, in turn, references the new object C2, which is a copy C.

Use a class that implements the ICloneable interface to perform a deep or shallow copy of an object.

Which to me is not is that clear and further research didn’t give any good examples, so I created my own in order to see what it does.

And without any attempt to explain what is going on, here it is. Feel free to test as you like.

Hopefully this should be self-explanatory.

class Manager : Employee

{

static void Main(string[] args)

{

Manager manager1 = new Manager();

manager1.Name = "Mary Maryson";

manager1.Age = 45;

manager1.Department.Name = "Accounting";

manager1.Department.City = "Stockholm";

Manager manager2 = (Manager)manager1.MemberwiseClone();

Console.WriteLine("Manager 1 -> {0}", manager1);

Console.WriteLine("Manager 2 -> {0}", manager2);

// Edit the first Manager, notice that this will update the Second managers

// Department data as well since the reference is cloned, not the object.

// For the value types, they are not affected.

manager1.Name = "Mike Mikeson";

manager1.Age = 55;

manager1.Department.City = "London";

Console.WriteLine();

Console.WriteLine("Manager 1 -> {0}", manager1);

Console.WriteLine("Manager 2 -> {0}", manager2);

// And of course, chaning the Department for the manager2 will change the

// Department data for manager1 as well.

manager2.Name = "Paul Paulson";

manager2.Age = 30;

manager2.Department.City = "Paris";

Console.WriteLine();

Console.WriteLine("Manager 1 -> {0}", manager1);

Console.WriteLine("Manager 2 -> {0}", manager2);

}

class Employee

{

public string Name = "<default>";

public int Age = 0;

public Department Department = new Department();

public override string ToString()

{

return String.Format("{0}, age: {1}, is in {2}", Name, Age, Department);

}

class Department

{

public string Name = "<default>";

public string City = "<default>";

public override string ToString()

{

return String.Format("{0} / {1}", Name, City);

}

SGEN gives "Error: The top XML element 'class' from namespace '' references distinct types UserDefinedFunctions.Class and UserDefinedFunctions.Class1."

2009-09-16T17:28:00Z

Another SGEN error that caused me some time to troubleshoot.

In this case customer was again calling a web service from a SQL CLR User Defined Function.

"How to: Create and Run a CLR SQL Server User-Defined Function"

http://msdn.microsoft.com/en-us/library/w2kae45k(VS.80).aspx

If doing this, then you create a reference to the web service and you’ll get a proxy class.

Again, if you are serializing, you need to run SGEN on it. In this case running SGEN caused the following error:

C:\Program Files (x86)\Microsoft Visual Studio 8\VC>sgen /f C:<path>\thedll.dll

Microsoft (R) Xml Serialization support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.42]

Error: The top XML element 'class' from namespace '' references distinct types UserDefinedFunctions.Class and UserDefinedFunctions.Class1.

Use XML attributes to specify another XML name or namespace for the element or types.

The reason for this was that the WSDL used to create the proxy had the not set the XmlTypeAttribute for one class, let’s call it “MyClass”,

but it had set the XmlTypeAttribute to “MyClass” on a class called “MyClass1”.

SGEN gives "Assembly ... does not contain any types that can be serialized using XmlSerializer.

2009-09-15T16:15:00Z

The other day I was working a case where a SQL CLR User Defined Function was used.

A bit more on this here: "How to: Create and Run a CLR SQL Server User-Defined Function"

http://msdn.microsoft.com/en-us/library/w2kae45k(VS.80).aspx

and in this case customer was sending back results from a web service, this was done by serializing the results and sending the data back as XML.

When doing this, you will probably run into a certain type of error (which customer did, I will add an entry on this later) to which the solution is to run SGEN on the built dll.

So, I started the research by creating a new SQL Server Project (called it SgenTest) and I selected the database that I wanted to deploy to.

Then I added a new User Defined Function called MyFunction, the generated code will look like this:

public partial class UserDefinedFunctions

{

[Microsoft.SqlServer.Server.SqlFunction]

public static SqlString MyFunction()

{

// Put your code here

return new SqlString("Hello");

}

};

Then I built the project which generated my .dll (SgenTest.dll) and I was now to run SGEN on it. I did this by using the same syntax as customer.

So, I fired up the “Visual Studio 2008 Command Prompt” and navigated to where the SgenTest.dll was located and ran the following:

C:\Path>sgen /f /p SgenTest.dll

Which gave me:

Microsoft (R) Xml Serialization support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.3038]

Assembly 'C:<path>\sgentest.dll' does not contain any types that can be serialized using XmlSerializer.

Please use /verbose switch for more information.

Ok, adding the verbose output probably would give something, so reran with that switch.

C:\Path>sgen /f /p /v SgenTest.dll

This just gave me the same error. So, did some research and basically found nothing, this is why I’m writing this post.

I then realized that customer was using a web service and therefore was using the /p switch.

This switch means the following: “Generates serialization code only for the XML Web service proxy types.”

"XML Serializer Generator Tool (Sgen.exe)"

http://msdn.microsoft.com/en-us/library/bk3w6240.aspx

So what was happening here was that since I ran with the /p switch it ignored my own code. And because of that it obviously couldn’t find any types that could be serialized.

In short, if you are not using a XML Web service proxy, then do not use /p since this will cause sgen to only try to generate the serialization code for that.

C:\Path>sgen /f SgenTest.dll

Gave me the SgenTest.XmlSerializers.dll that I was looking for.

HTH.

How to save information about DataSet into the XML when serializing it with WriteXML()

2009-09-11T17:36:00Z

I got a question the other day. How do I serialize my dataset with some descriptive information about the meta data included?

Well, ideally we could have a property on the dataset called, for example, Description that would also be serialized. Unfortunately there isn’t.

So I came up with this simple solution.

Basically, since we are serializing to XML, why not use the XmlWriter and add, for example, our information in a XML comment?

Create a new C# console application and add the following code:

class Program

{

static void Main(string[] args)

{

// Create our demodataset.

DataSet ds = CreateDataSet();

// Need to set this if you want the retain then the XML indentation, otherwise it will be a single line.

// So it is not needed if you are not reading it 'manually'. Your call.

XmlWriterSettings settings = new XmlWriterSettings();

settings.Indent = true;

// Do the actual writing using an XmlWriter.

String xmlFile = @"C:\Temp\xmlOut.xml";

using (XmlWriter writer = XmlWriter.Create(xmlFile, settings))

{

writer.WriteComment(String.Format("\n{0}\n", GetDataSetInfo(ds)));

ds.WriteXml(writer);

}

private static string GetDataSetInfo(DataSet ds)

{

// Just get some informational data about the dataset, should be self explanatory.

IEnumerable<DataTable> tbls = ds.Tables.Cast<DataTable>();

int totalRows = tbls.Sum(x => x.Rows.Count);

StringBuilder sb = new StringBuilder();

sb.AppendLine(String.Format("Info for {0}", ds.DataSetName));

sb.AppendLine(String.Format("Dataset contains {0} rows in {1} tables.", totalRows, ds.Tables.Count));

sb.AppendLine("Tables:");

foreach (DataTable dt in ds.Tables)

{

sb.AppendLine(String.Format("\t{0, -10} ({1} row(s))", dt.TableName, dt.Rows.Count));

}

return sb.ToString();

}

private static DataSet CreateDataSet()

{

// Create tables

DataTable dtCustomer = new DataTable("Customer");

dtCustomer.Columns.Add(new DataColumn("CustomerId", typeof(int)));

dtCustomer.Columns.Add(new DataColumn("CustomerName", typeof(string)));

DataTable dtOrder = new DataTable("Order");

dtOrder.Columns.Add(new DataColumn("OrderId", typeof(int)));

dtOrder.Columns.Add(new DataColumn("CustomerId", typeof(int)));

dtOrder.Columns.Add(new DataColumn("CustomerName", typeof(string)));

// Add some data

dtCustomer.Rows.Add(new object[] { 1, "Mikes company" });

dtCustomer.Rows.Add(new object[] { 2, "Petes company" });

dtOrder.Rows.Add(new object[] { 1, 1, "Something of this." });

dtOrder.Rows.Add(new object[] { 2, 2, "Something of that." });

dtOrder.Rows.Add(new object[] { 3, 1, "12 of these." });

dtOrder.Rows.Add(new object[] { 4, 2, "6 of those." });

dtOrder.Rows.Add(new object[] { 5, 1, "More of this." });

DataSet ds = new DataSet("CustomerOrderDataSet");

ds.Tables.Add(dtCustomer);

ds.Tables.Add(dtOrder);

return ds;

}

You could remove the CreateDataSet and the GetDataSetInfo if you wish; they are only there for demonstrational purposes.

You could simply use the XmlWriter like this if all you want is to have the time and date for when the DataSet was serialized.

writer.WriteComment(String.Format("\n{0}\n", DateTime.Now.ToShortDateString()));

Anyways, if running the code above, you will have an XML file with the following content:

<?xml version="1.0" encoding="utf-8"?>

<!--

Info for CustomerOrderDataSet

Dataset contains 7 rows in 2 tables.

Tables:

Customer (2 row(s))

Order (5 row(s))

-->

<CustomerName>Mikes company</CustomerName>

</Customer>

<CustomerName>Petes company</CustomerName>

</Customer>

<Order>

<CustomerName>Something of this.</CustomerName>

</Order>

<Order>

<CustomerName>Something of that.</CustomerName>

</Order>

<Order>

<CustomerName>12 of these.</CustomerName>

</Order>

<Order>

<CustomerName>6 of those.</CustomerName>

</Order>

<Order>

</Order>

</CustomerOrderDataSet>

As simple as that.

There may better ways for your needs, but this is a quick and dirty solution to get meta information into the serialized XML if this is what you wish to do.

"System.Data.Linq.ChangeConflictException: Row not found or changed" Finding the culprit

2009-09-07T17:28:00Z

When updating a data table in SQL Server using LINQ to SQL and the SubmitChanges() method you may encounter the following exception:

System.Data.Linq.ChangeConflictException: Row not found or changed.

at System.Data.Linq.ChangeProcessor.SubmitChanges(ConflictMode failureMode)

at System.Data.Linq.DataContext.SubmitChanges(ConflictMode failureMode)

The error in itself is not too hard to understand, you are simply trying to update by using an UPDATE statement that doesn’t update anything.

The most common reason is that someone (or something) has changed the data in the database between you retrieving the data and you updating the data.

What happens in the background when you call SubmitChanges is that there is an inspection of the rows (objects) in the data context data table.

If it detects any changes, it builds an SQL statement based on the values that it have and then it executes this statement.

This statement is basically taking the values that were retrieved from the database and then uses them in the WHERE clause of the UPDATE, however, this UPDATE will fail

of course if values have changed before you execute the UPDATE, it will be a bit clearer below, this approach is called optimistic concurrency and this is what LINQ to SQL uses.

Now, the problem is that the exception is not very verbose. You may have several tables in your dbml with many columns and rows.

So what column value has changed (and in what table) and is causing us trouble. As usual, let’s do this with a demo.

First create a new table in your SQL Server and insert some rows:

create table Employee (Id int primary key, FirstName nvarchar(20), LastName nvarchar(20), Location nvarchar(20))

insert into Employee values (1, 'Mike', 'Spike', 'Stockholm')

insert into Employee values (2, 'John', 'Johnsson', 'London')

insert into Employee values (3, 'Peter', 'Peterson', 'Paris')

insert into Employee values (4, 'Lars', 'Larsson', 'New York')

Then create a new C# console application. Then Add->New Item->LINQ to SQL classes, call it Employee.dbml

Drag the table you just created from your server onto the designer and Save.

Our objective is to change the location for employees in London to have their location set to Paris.

Our problem is that between we have retrieved the data and made the change and submitted it, a quicker colleague has already done this.

So when we load the data, the London location still exists in the database, when we save it, it does not.

Then the code for the application should be:

static void Main(string[] args)

{

EmployeeDataContext edc = new EmployeeDataContext();

try

{

// Get all Employees that work in London.

var empInLondonQuery = from e in edc.Employees where e.Location.Equals("London") select e;

// Write out employess were are changing location for:

foreach (var Employee in empInLondonQuery)

{

Console.WriteLine("Changing location for {0}, {1}:", Employee.LastName, Employee.FirstName);

// Change location

Employee.Location = "Paris";

}

// Now emulate the other user by changing the Location before we submit our change.

Console.WriteLine("Now update the Employee table by running this is SQL Server Management Studio:");

Console.WriteLine("\t update Employee set Location = 'Paris' where Location = 'London';");

Console.WriteLine("And hit any key...");

Console.ReadKey();

// Finally submit the changes

edc.SubmitChanges();

}

catch (Exception ex)

{

Console.WriteLine(ex);

}

Run it, update the database when prompted, and you should get the “Row not found or changed” exception.

Not very descriptive. So in order to see what is going on, we can first enable the Log for the data context so that we can see what SQL is being used,

secondly we can catch the ChangeConflictException and then query the ObjectChangeConflict and MemberChangeConflict classes.

So, change the code above to look as follows:

static void Main(string[] args)

{

EmployeeDataContext edc = new EmployeeDataContext();

edc.Log = Console.Out;

try

{

// Get all Employees that work in London.

var empInLondonQuery = from e in edc.Employees where e.Location.Equals("London") select e;

// Write out employess were are changing location for:

foreach (var Employee in empInLondonQuery)

{

Console.WriteLine("Changing location for {0}, {1}:", Employee.LastName, Employee.FirstName);

// Change location

Employee.Location = "Paris";

}

// Now emulate the other user by changing the Location before we submit our change.

Console.WriteLine("Now update the Employee table by running this is SQL Server Management Studio:");

Console.WriteLine("\t update Employee set Location = 'Paris' where Location = 'London';");

Console.WriteLine("And hit any key...");

Console.ReadKey();

// Finally submit the changes

edc.SubmitChanges();

}

catch (ChangeConflictException cce)

{

Console.WriteLine(cce);

foreach (ObjectChangeConflict occ in edc.ChangeConflicts)

{

MetaTable metatable = edc.Mapping.GetTable(occ.Object.GetType());

Console.WriteLine("\nTable name: {0}\n", metatable.TableName);

foreach (MemberChangeConflict mcc in occ.MemberConflicts)

{

Console.WriteLine("Member: {0}", mcc.Member);

Console.WriteLine("\tCurrent value: {0}", mcc.CurrentValue);

Console.WriteLine("\tOriginal value: {0}", mcc.OriginalValue);

Console.WriteLine("\tDatabase value: {0}", mcc.DatabaseValue);

Console.ReadLine();

}

catch (Exception ex)

{

Console.WriteLine(ex);

}

and rerun it (you have to change the Location back to London, otherwise there will be no rows selected).

Now the output should be as follows:

SELECT [t0].[Id], [t0].[FirstName], [t0].[LastName], [t0].[Location]

FROM [dbo].[Employee] AS [t0]

WHERE [t0].[Location] = @p0

-- @p0: Input NVarChar (Size = 6; Prec = 0; Scale = 0) [London]

-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.4926

Changing location for Johnsson, John:

Now update the Employee table by running this is SQL Server Management Studio:

update Employee set Location = 'Paris' where Location = 'London';

And hit any key...

UPDATE [dbo].[Employee]

SET [Location] = @p4

WHERE ([Id] = @p0) AND ([FirstName] = @p1) AND ([LastName] = @p2) AND ([Location] = @p3)

-- @p0: Input Int (Size = 0; Prec = 0; Scale = 0) [2]

-- @p1: Input NVarChar (Size = 4; Prec = 0; Scale = 0) [John]

-- @p2: Input NVarChar (Size = 8; Prec = 0; Scale = 0) [Johnsson]

-- @p3: Input NVarChar (Size = 6; Prec = 0; Scale = 0) [London]

-- @p4: Input NVarChar (Size = 5; Prec = 0; Scale = 0) [Paris]

-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.4926

System.Data.Linq.ChangeConflictException: Row not found or changed.

at System.Data.Linq.ChangeProcessor.SubmitChanges(ConflictMode failureMode)

at System.Data.Linq.DataContext.SubmitChanges(ConflictMode failureMode)

at SubmitChangesError.Program.Main(String[] args) in C:\_Blog\Projects\SubmitChangesE

Table name: dbo.Employee

SELECT [t0].[Id], [t0].[FirstName], [t0].[LastName], [t0].[Location]

FROM [dbo].[Employee] AS [t0]

WHERE [t0].[Id] = @p0

-- @p0: Input Int (Size = 0; Prec = 0; Scale = 0) [2]

-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.4926

Member: System.String Location

Current value: Paris

Original value: London

Database value: Paris

This is a bit more verbose, first we can see how the data is selected from the database (all rows where Location is London).

Then we can see the actual update statement, here you will see that the update is built based on all columns using the values that they got when it was retrieved.

Then we get the exception which we then query.

Here we can see that the data table that we are interacting with is Employee.

Then we query the database to get the current value using the primary key.

Finally we display the current value (what we have changed it to), the original value (what it was when we retrieved it) and the database value (what it is in database).

Following the steps above should help you in figuring out what has changed in your database, causing the ChangeConflict exception.

Maybe this is ok for you and you think that whatever is in the database should stay there. You can then call the SubmitChanges method with the ConflictMode set to ContinueOnConflict.

Or you can, after the exception has been caught, call the ObjectChangeConflict.Resolve method. Links related to this below.

"LINQ to SQL - Optimistic Concurrency Overview (LINQ to SQL)"

http://msdn.microsoft.com/en-us/library/bb399373.aspx

"LINQ to SQL - How to: Retrieve Member Conflict Information (LINQ to SQL)"

http://msdn.microsoft.com/en-us/library/bb386985.aspx

"LINQ to SQL - How to: Manage Change Conflicts (LINQ to SQL)"

http://msdn.microsoft.com/en-us/library/bb399389.aspx

"ObjectChangeConflict.Resolve Method"

http://msdn.microsoft.com/en-us/library/system.data.linq.objectchangeconflict.resolve.aspx

"DataContext.SubmitChanges Method (ConflictMode)"

http://msdn.microsoft.com/en-us/library/bb345081.aspx

Why massive inserts using SubmitChanges lack in performance.

2009-09-01T17:45:00Z

Bulk inserting using LINQ to SQL, why is it so slow?

Well, the short answer is that it is not designed to do bulk inserts.

But out of curiosity, why is it slow?

Well, let us do it by example (my preferred way). First create a table in SQL Server like so (this will basically represent a person):

create table OurLinqPerson (cid int primary key, fname nvarchar(10), lname nvarchar(10), age int)

--drop table OurLinqPerson

Then create a new C# console project in Visual Studio.

Right click the project and add a new item, “LINQ to SQL Classes”, call it Person.dbml.

Then use the Server Explorer to find the table (OurLinqPerson) and drag it to the designer surface and “Save”.

Then add the following code:

static void Main(string[] args)

{

int noOfPersonsToInsert = 10000;

long calc = 0;

try

{

PersonDataContext pdc = new PersonDataContext();

Stopwatch sw = new Stopwatch();

sw.Start();

for (int i = 0; i < noOfPersonsToInsert; i++)

{

OurLinqPerson olp = new OurLinqPerson { cid = i, fname = "Peter", lname = "Peterson", age = 33 };

// Add the object to the datacontext.

pdc.OurLinqPersons.InsertOnSubmit(olp);

if (i % 500 == 0)

{

Console.WriteLine("Time (ms): Total / This batch {0, 3} / {1, 3}, Objects using InsertOnSubmit: {2}", sw.ElapsedMilliseconds.ToString(), (sw.ElapsedMilliseconds - calc).ToString(), i);

calc = sw.ElapsedMilliseconds;

}

Console.WriteLine("All InsertOnSubmits done. Total elapsed time (ms): {0}. \n\nNow calling SubmitChanges...", sw.ElapsedMilliseconds.ToString());

calc = sw.ElapsedMilliseconds;

// Now submit changes to the database.

pdc.SubmitChanges();

Console.WriteLine("SubmitChanges done, time for this (ms): {0}", (sw.ElapsedMilliseconds - calc).ToString());

sw.Stop();

Console.WriteLine("\n\nTotal elapsed time (ms): {0}", sw.ElapsedMilliseconds.ToString());

}

catch (Exception ex)

{

Console.WriteLine(ex);

}

The code hopefully is self-explanatory. We want to insert 10000 rows into the OurLinqPerson table, we do this by creating an instance of the OurLinqPerson object.

(Basically, a row in the table in the database is an instance of a class representing a row, the columns are properties on that class).

We then add that instance to the table in the DataContext by using it as an argument to the InsertOnSubmit method.

We write out the time elapsed and the time it took for doing this for every 500 rows.

Finally we submit the changes to the database by calling SubmitChanges.

So, run the application and the output should be something like this:

Time (ms): Total / This batch 23 / 23, Objects using InsertOnSubmit: 0

Time (ms): Total / This batch 27 / 4, Objects using InsertOnSubmit: 500

Time (ms): Total / This batch 30 / 3, Objects using InsertOnSubmit: 1000

Time (ms): Total / This batch 33 / 3, Objects using InsertOnSubmit: 1500

Time (ms): Total / This batch 44 / 11, Objects using InsertOnSubmit: 2000

Time (ms): Total / This batch 47 / 3, Objects using InsertOnSubmit: 2500

Time (ms): Total / This batch 50 / 3, Objects using InsertOnSubmit: 3000

Time (ms): Total / This batch 53 / 3, Objects using InsertOnSubmit: 3500

Time (ms): Total / This batch 58 / 5, Objects using InsertOnSubmit: 4000

Time (ms): Total / This batch 61 / 3, Objects using InsertOnSubmit: 4500

Time (ms): Total / This batch 65 / 3, Objects using InsertOnSubmit: 5000

Time (ms): Total / This batch 73 / 8, Objects using InsertOnSubmit: 5500

Time (ms): Total / This batch 76 / 3, Objects using InsertOnSubmit: 6000

Time (ms): Total / This batch 80 / 4, Objects using InsertOnSubmit: 6500

Time (ms): Total / This batch 83 / 3, Objects using InsertOnSubmit: 7000

Time (ms): Total / This batch 87 / 4, Objects using InsertOnSubmit: 7500

Time (ms): Total / This batch 90 / 3, Objects using InsertOnSubmit: 8000

Time (ms): Total / This batch 94 / 4, Objects using InsertOnSubmit: 8500

Time (ms): Total / This batch 97 / 3, Objects using InsertOnSubmit: 9000

Time (ms): Total / This batch 100 / 3, Objects using InsertOnSubmit: 9500

All InsertOnSubmits done. Total elapsed time (ms): 104.

Now calling SubmitChanges...

SubmitChanges done, time for this (ms): 3396

Total elapsed time (ms): 3500

So it is pretty obvious that adding the rows/objects performs quickly, around 5 ms to insert 500 of them, however, the call to SubmitChanges takes 3396 ms out of a 3500 total.

So, back to the original questions, why is it slow?

Well, when calling SubmitChanges every object/row in the table (OurLinqPersons) has to be inspected in order to decide what action to take on that particular row.

Once the Object State has been determined for the instance it will create the appropriate SQL for that state and then call the database with this SQL.

The rows can be in several states, for example, ToBeInserted, Unchanged, Deleted etc.

In this case the row is in the ToBeInserted state and so the following SQL is created and ran for each row:

INSERT INTO [dbo].[OurLinqPerson]([cid], [fname], [lname], [age]) VALUES (@p0, @p1, @p2, @p3)

So in short, the reason it takes time is because every row/object has to be inspected in order to decide what action to take on it and then that action has to be executed.

It doesn’t matter if the row/object is unchanged; it still has to be inspected.

If you want to insert a large number of row, pure ADO.Net (for example SqlBulkCopy is a better option).

"Table<TEntity>.InsertOnSubmit Method"

http://msdn.microsoft.com/en-us/library/bb763516.aspx

"DataContext.SubmitChanges Method "

http://msdn.microsoft.com/en-us/library/system.data.linq.datacontext.submitchanges.aspx

"Object States and Change-Tracking (LINQ to SQL)"

http://msdn.microsoft.com/en-us/library/bb386982.aspx

"Insert, Update, and Delete Operations (LINQ to SQL)"

http://msdn.microsoft.com/en-us/library/bb386931.aspx