David Szabo's [MSFT] blog : .net

Playing with DLINQ

dszabo — Tue, 19 Jun 2007 11:59:00 GMT

Today, I was playing with DLINQ and would like to share my findings with you. First of all, if you never heard of LINQ or want a basic intro, I suggest starting with the following 3 short videos (in order):

First of all, I generated a C# class file representing my SQL database tables by using a built-in SDK tool. The class is generated from my SQL database, called PfRep: I started an Orcas command prompt and stepped into the C:\Program Files\Microsoft Visual Studio 9.0\SDK\v3.5\Bin folder, then ran the following command:

sqlmetal /server:local host /database:PfRep /code /pluralize C:\blabla\PfRep.cs

This creates a .CS file with the class structure discovered from the database tables.

(by the way, the .CS file can be generated by a Visual Studio designer as well - by adding a "Linq to SQL File" file to the project - sqlmetal is good for re-generating the structure if it has changed.)

The first thing that's a bit weird is that the generated file contains modified table names. For example, I have the following tables in my PfRep database:

Folder
ContentIndex

that (in PfRep.cs) were transformed to the following class names:

Folders
ContentIndexes

I guess, it doesn't want the programmer to mix up the classes (table definitions) with the property methods used to retrieve the records from the tables - or something like that. Let's write some easy code to run a query:

PfRep pfRep = new PfRep("database=PfRep;server=dszabo1");

var bigFolders =

from folder in pfRep.Folders

where folder.ItemCount > 1000

select folder;

foreach (Folder folder in bigFolders)

{

Console.WriteLine(folder.FolderKey + " " + folder.FolderTree.FolderTreeName);

Console.ReadLine();

}

What it does, it queries the Folder table for records with an item count larger than 1,000. Then it iterates through the results, looks up a referenced record (in the FolderTree table, which is referenced by Folder through the FolderTreeID property) and dumps out these data.

I was curious how does it spend the expensive transaction "fees" against SQL Server. :) It turns out that it's pretty clever. Here's what I captured with SQL Profiler. First, it queries for all the Folder records with an ItemCount larger than 1,000:

exec sp_executesql N'SELECT [t0].[FolderID], [t0].[FolderTreeID], [t0].[FolderName], [t0].[FolderKey], [t0].[Indexed], [t0].[ItemCount], [t0].[ParentFolderID], [t0].[ActionPlanID], [t0].[OwnerName], [t0].[OwnerEmail], [t0].[Note]

FROM [Folder] AS [t0]

WHERE [t0].[ItemCount] > @p0',N'@p0 int',@p0=1000

then when in the loop, it queries for the FolderTree record associated with the Folder record:

exec sp_executesql N'SELECT [t0].[FolderTreeID], [t0].[FolderTreeName], [t0].[DateCreated], [t0].[FolderTreeUrl]

FROM [FolderTree] AS [t0]

WHERE [t0].[FolderTreeID] = @p0',N'@p0 int',@p0=1

It checks whether the given FolderTree record was already retrieved and it doesn't run the query twice if already got the record - I'm wondering where does it store the results though ... and how.

sp_executesql is a good choice of running the queries, in my example, it's especially useful when running the FolderTree query (in the loop). Here's how it works:

"sp_executesql can be used instead of stored procedures to execute a Transact-SQL statement a number of times when the change in parameter values to the statement is the only variation. Because the Transact-SQL statement itself remains constant and only the parameter values change, the Microsoft® SQL Server™ query optimizer is likely to reuse the execution plan it generates for the first execution." http://msdn2.microsoft.com/en-us/library/aa933299(SQL.80).aspx

Sounds good, isn't it? I'm still very curious about how is the cached data stored, it could be an important question by Customers on large database projects. After some debugging and reverse-engineering, it turns out that the data is stored by using the type of the underlying entity (FolderTree in my case) and is indexed by a special generic collection.

Have fun with LINQ! :)

How to run a SQL database installer from your .net application

dszabo — Mon, 28 May 2007 13:20:00 GMT

Even if the question itself is very obvious, I got too many problems while implementing it. Here's the situation: I have a WinForms app (C#), I want to offer the user with creating a SQL database and a couple of database objects (SPs, tables) at the first time he/she runs the application.

The CREATE scripts are in a .sql file. It was generated with SQL Management Studio. If I read the file and pass it over to the SqlCommand object (as a string), it fails. If the file were just to contain CREATE TABLE statements, it would work, but ADO.NET doesn't like the CREATE PROC statements. If it's only CREATE PROC statements, it still fails. So, the principle is that every CREATE PROC has to run seperately. CREATE TABLEs can run together. So, what I did was, I was breaking the script to separate statements and ran them one by one by using the SqlCommand object. GO commands can be used to split up the file to multiple parts if there's no "GO" word in your script. Here it is:

// Break the SQL script to statements
// (delimited by the GO command).
// Without doing this, it doesn't work. NOTE: it's important that
// the SQL script shouldn't contain *ANY* GO word except the ones
// used to delimit, otherwise it won't work. We are using GO
// because OSQL also understands GO - so the script remains
// compatible. Another solution would be to use a
// commented delimiter.
string[] delimitedSqlCommand = databaseScript.Split(
    new string[1] { "GO" }, StringSplitOptions.None);

// Creates the db objects
foreach (string sqlCommand in delimitedSqlCommand)
{
    SqlCommand command = new SqlCommand(
        sqlCommand, masterConnection);
    command.CommandType = CommandType.Text;
    command.ExecuteNonQuery();
}

.net Framework 3.0 Installer Crashes

dszabo — Mon, 29 Jan 2007 15:36:12 GMT

Today, I installed the .NET Framework 3.0 Redistributable Package from here at a Customer's site, on a Windows Server 2003 SP1. After the installer started and I accepted the licence agreements, a minute later the installer crashed while tried to download some more data from the internet. It turns out that due to my limited rights on the Customer's firewall, the installer couldn't download the required content thus causing the crash. So, make sure that you have a right to download application/octet-stream content types before running the installer.

How .net stores dates/how to debug them

dszabo — Fri, 17 Nov 2006 01:38:00 GMT

I was debugging an application and had to check a DateTime variable’s actual value. I was expecting something like member variables holding year, month, day, hour, etc values on the object and it will be an easy one to debug, but that was not the case, so I had to find out how to check the actual date value of a DateTime variable and learn how dates are stored in the .net framework 2.0.

First of all, as you might already know, dumping out an object with SOS can be done with the !do (or !DumpObject) command. If you are not familiar with debugging .net, using WinDbg and using the SOS debugger extension, you can find a good starter at http://msdn.microsoft.com/msdnmag/issues/03/06/Bugslayer/default.aspx, you can check the reference of the SOS extension at http://msdn.microsoft.com/msdnmag/issues/03/06/Bugslayer/default.aspx, then if you are still interested, you'll find really good articles at Doug's (http://blogs.msdn.com/dougste) and Tess' (http://blogs.msdn.com/tess) blog. So, returning back to the subject, this is what you get when dumping out a DateTime variable:

0:000> !do 01481d74
Name: System.DateTime
MethodTable: 790ff4c4
EEClass: 790ff424
Size: 16(0x10) bytes
 (E:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields:
Field   Offset                 Type VT     Attr    Value Name
40000f4        4        System.UInt64  0 instance 630992916000000000 dateData
40000f0       30       System.Int32[]  0   shared   static DaysToMonth365
    >> Domain:Value  002401d8:01481cf4 <<
40000f1       34       System.Int32[]  0   shared   static DaysToMonth366
    >> Domain:Value  002401d8:01481d34 <<
40000f2       28      System.DateTime  1   shared   static MinValue
    >> Domain:Value  002401d8:01481cd4 <<
40000f3       2c      System.DateTime  1   shared   static MaxValue
    >> Domain:Value  002401d8:01481ce4 <<

The actual date value is stored in the UInt64 (long) member variable called dateData, which stores the ticks representing the date value. In my example, it is 630992916000000000. That's OK so far, but how can we get the date out from this? To convert it back without using your calculator or writing an expensive algorythm, you have to use the following DateTime constructor:

public DateTime(long ticks)

or the following static method on the DateTime class:

public static DateTime FromBinary(long dateData)

Code that does the job:

DateTime dateValue = new DateTime(630992916000000000);
Console.WriteLine(dateValue);

Exceptions are expensive!

dszabo — Thu, 09 Nov 2006 09:48:00 GMT

Hey! Don't we all use Exceptions in managed code to indicate the caller that something is wrong? That's a really bad habit! - "Hey maaaan, it's the 21st century, passing back HRESULTs is over!" - not yet, you'll see it: did you know that if your caller can systematically force your code to throw an exception (by passing wrong parameters, etc), your application could be DOSed down? It's just a matter of the amount of threads running on the defender's box and the power of the server your code running at. To demonstrate this thingy, I hammered together a small app:

The following is a function that can work in two different ways:

Returning an exception, simulating a function that communicates back to its caller by throwing exceptions on deterministic inputs. MyOwnException is just a class inherited from the ApplicationException class
Returning an integer number, simulating a function that works in an "old-fashioned" HRESULT-way

static int TesterMethod(bool throwException)
{
    if (throwException)
    {
        throw new MyOwnException();
    }
    else
    {
        return 1;
    }
}

(HRESULT is representing an alternative way of passing back the result of a function as a variable or pointer, it's not neccessarily should be the old-fashioned HRESULT, it could be a structure or class containing information about what happened)

... and here's the function that will drive this baby: it's simple: there's a stopwatch we use to measure time, and we call the function in both ways 1,000 times:

static void Main(string[] args)
{
    Stopwatch stopWatch = Stopwatch.StartNew();
    
    // PHASE I: DOING THE EXCEPTION-TEST
    
    // Reset and start the stopwatch
    stopWatch.Reset();
    stopWatch.Start();
    
    // Call the function 1,000 times, an exception will be
    //  thrown at every call
    for (int i = 0; i < 1000; i++)
    {
        try
        {
            TesterMethod(true);
        }
        catch (MyOwnException)
        {
            // Do something to handle ...
        }
    }
    
    // Stop the watch and show the time elapsed
    stopWatch.Stop();
    Console.WriteLine("Exception: " + stopWatch.Elapsed);

    
    // PHASE II: DOING THE "HRESULT" TEST

    // Reset and start the stopwatch again
    stopWatch.Reset();
    stopWatch.Start();
    
    // Calling the function 1,000 times again, but now in
    //  the HRESULT-way
    int retVal = 0;
    for (int i = 0; i < 1000; i++)
    {
        retVal = TesterMethod(false);
    }
    
    // Stop the stopwatch and display the results
    stopWatch.Stop();
    Console.WriteLine("HRESULT " + stopWatch.Elapsed);

}

Surprise!

Exception: 00:00:03.1247094
HRESULT: 00:00:00.0000287

I hope that you noticed the seconds part, not only the milliseconds part! After some time, I was rethinking the whole thing, because it seemed to be too much of a difference, then I found that I made a mistake when measuring the performance: I was running the code from the Visual Studio IDE, meaning that I was debugging the process, which slows down 1st chance exceptions, so a big part of this difference was due to the operating system informing the debugger about the 1st chance exceptions. Release compiled the project, ran it outside the debugger and got a much better performance for exceptions, but still a huge difference:

Exception: 00:00:00.2039267
HRESULT: 00:00:00.0000058

This is all about performance and security. One can say that these are just milliseconds per one thousand cycles. That's true for one thread, but think about an ASP.net application or a service accepting huge amount of TCP connections, with 25 worker threads per processor. If there's a function inside that can be forced from outside to throw an exception, this could be a big performance impact on your service. So, think twice before throwing an exception.