ADO.NET team blog : ADO.NET Orcas

Extending Lightweight Transactions in SqlClient

alazela — Wed, 26 Mar 2008 05:43:00 GMT

We’ve done some work with SqlClient’s System.Transaction support for Sql Server 2008. To explain what we did, and why, I’m going setup some background first, which may also be useful for understanding the existing behavior.

System.Transactions offers a simple, intuitive model for controlling transactions in your .Net code, right? I generally agree with that statement, but there are some devious details, specifically with regard to distributed transactions. They require special setup on the client. They are slow. And probably a few other issues depending on whom you ask.

With version 2.0, SqlClient allowed lightweight transactions when connecting to Sql Server 2005, alleviating the problem for a group of scenarios. But this introduced a new issue – promotion. Promotion is a good thing, as long as you are happy with distributed transactions, but it can be confusing as to when and why it happens.

So why does the transaction promote? A lightweight transaction is an agreement between SqlClient and the transaction, where SqlClient manages transacting the work on the transaction’s behalf. SqlClient starts a local server transaction when enlisting into the transaction, and the transaction sends a request to rollback or commit the work at the end. When a second resource tries to enlist in the connection, the local server transaction cannot be used with it, so the transaction promotes. Ah, you say, that second request may be going to the same server, and with connection pooling I should be able to open another connection and use the same transaction!

To understand why this doesn’t work, let’s take a look inside connection pooling. A connection consists of two parts: the public instance that your code interacts with (the outer connection) and a hidden connection that represents an actual server connection (the inner connection). When you “Open()” the outer connection, it looks for a free inner connection from the pool that is associated with the transaction (or creates a new one if it cannot find one). When you “Close()” the outer connection, it returns the inner connection to the pool. The server doesn’t distinguish between uses of the inner connection so only one outer connection can be used with any one inner connection at a time, and the inner connection must be reset to prevent state from showing up on subsequent uses. Resetting closes cursors, changes set options back to defaults, switches the database context back to the one from the connection string AND… rolls back any outstanding local transactions. Resetting this way provides a known starting state every time you open a connection, which would lead to nasty random behavior in many cases, but it also prevents re-using the original inner connection with the lightweight transaction until said transaction completes. When you close it’s outer connection, this inner connection is set aside pending the rollback or commit request. So the second time you open any outer connection, including the original, the pool is empty, and a new inner connection is created. A quick example:

You open outer connection “A”. The pool has no free appropriate connection, so inner connection “z” is set up and enlisted in the transaction, establishing a lightweight transaction. You now close “A”, which sets aside “z” to wait for the transaction to end. Next you open outer connection “B” (you could also open “A” again and get the same results). “B” looks for a free inner connection in the pool attached to the transaction, doesn’t find one, creates inner connection “y” and tries to enlist it in the transaction. The transaction, now finding two different resources trying to enlist, must promote (resources in general, and sql connections in particular, cannot share local transactions). Finally you end the transaction, which sends the commit or rollback across “z”, disconnects it from the transaction and returns it to the pool.

So this brings us to the extensions we added for Sql Server 2008 support. On the server, we added a new connection reset mode that does not roll back local transactions. This allows SqlClient to return the inner connection to the pool to be reused. In our example, when you open “B”, it will finds “z” waiting in the pool, associated with the transaction where “A” put it when you closed “A”. “B” appropriates and resets “z” (with the transaction-preserving reset) and happily continues working. Neither System.Transaction nor the server are aware that the application sees “z” as two separate connections. As far as they are concerned, there is only one connection, working on a single local transaction and no promotion is necessary.

There are still limitations with this approach. If you try to open a second outer connection BEFORE closing the first one, there won’t be a free connection in the pool, so a second inner connection will have to be enlisted (“A” is using “z” so “B” must again obtain “y”). Ditto if you open a connection with pooling turned off or a slightly different connection string, since neither case will find the original inner connection, even if it is sitting idle in the pool.

The new capability is already in SqlClient v2.0 SP1 (e.g. Visual Studio 2008, Orcas, etc), but requires the changes in Sql Server 2008 to work (CTP5 or later). It is used automatically whenever it can be, so you don’t need to make changes to your code if you are already using pooling and enlisting (both default behaviors). When used against Sql Server 2005, the pooling behavior reverts to that of SqlClient v2.0 RTM.

Alazel Acheson
Developer, ADO.NET

ADO.NET at the Advisor Summit

dpblogs — Wed, 16 May 2007 22:53:00 GMT

Last week at the Advisor Summit conference I gave a talk introducing new LINQ functionality being introduced in the Orcas timeframe (and beyond).

As you probably know if you've been reading this blog regularly, LINQ functionality will begin to ship as part of Visual Studio Codename "Orcas" and the .NET Framework. The Orcas release will include LINQ to SQL, enabling LINQ queries against Microsoft SQL Server databases, LINQ to DataSet and LINQ to XML. The ADO.NET Entity Framework will ship in the first half of 2008 and will include LINQ to Entities, enabling LINQ queries against an Entity Data Model. The Entity Framework will also include other components, such as Object Services and the EntityClient managed provider, and will be supported by new tools for Visual Studio.

If you haven't been reading this blog regularly, or if you're only vaguely aware of the LINQ coming attractions, then all of this is bound to be confusing. Below are some links to help you get started in exploring some of the work being done by the Data Programmability team.

Mary Chipman,

Programmer Writer, ADO.NET

ADO.NET Team Blog

http://blogs.msdn.com/adonet/default.aspx

Data Programmability Team Blog

http://blogs.msdn.com/data/

The LINQ Project

http://msdn2.microsoft.com/en-us/netframework/aa904594.aspx

Download Visual Studio Code Name "Orcas" Downloads

http://msdn2.microsoft.com/en-us/vstudio/aa700831.aspx

http://www.microsoft.com/downloads/details.aspx?FamilyID=5d9c6b2d-439c-4ec2-8e24-b7d9ff6a2ab2&DisplayLang=en

Data Access and Storage Developer Center

http://msdn2.microsoft.com/en-us/data/default.aspx

Data Access Incubation Projects

http://msdn2.microsoft.com/en-us/data/bb419139.aspx

ADO.NET Orcas Forum

http://forums.microsoft.com/msdn/showforum.aspx?forumid=533&siteid=1

.NET Framework Data Access and Storage Forum

http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=45&SiteID=1

LINQ Project General Forum

http://forums.microsoft.com/msdn/showforum.aspx?forumid=123&siteid=1

ADO.NET Entity Framework Update

dpblogs — Sun, 29 Apr 2007 08:09:00 GMT

Microsoft is deepening its investment in the ADO.NET Entity Framework as a critical piece of Microsoft’s Data Platform vision. Based on the need to align with requirements from key internal partners that are building on the Entity Framework, along with the need for a better tool experience, we have decided to ship the ADO.NET Entity Framework and Tools during the first half of 2008 as an update to the Orcas release of the .NET Framework and Visual Studio. We will continue to ship CTPs and Betas of the ADO.NET Entity Framework that align with Orcas throughout the remainder of this year.

Microsoft shipped the first CTP of the ADO.NET Entity Framework in August of 2006. Customers immediately grasped the power of being able to work with their data through a conceptual model, rather than in terms of database schemas optimized around storage concerns like normalization and data partitioning. In February we added the framework into the CTP and Beta 1 builds of the next version of the .NET Framework and Visual Studio codenamed Orcas.

We’ve collected great feedback from you in these early releases, the most significant being that we need better tools to aid in defining a data model and mapping that model to the database. Although we have tools for generating a direct mapping to the storage schema, the real power of the ADO.NET Entity Framework comes in its ability to flexibly map a variety of relational schema representations to a more appropriate conceptual application model. This has reinforced the need to have a graphical designer experience available for the ADO.NET Entity Framework.

At the same time, Microsoft will be leveraging the Entity Data Model in future versions of Microsoft products such as SQL Server. This Data Platform vision enables customers to leverage their investment in a common conceptual entity model across product lines. Stay tuned for announcements starting next week at Mix around new products building on the ADO.NET Entity Framework.

For more information on the ADO.NET Entity Framework and Microsoft’s Data Access Vision, see the Microsoft Data Access Strategy post on the Data blog.

Mike Pizzo
Architect, Data Programmabilty

Entity Framework Orcas Beta 1 - EDM Wizard Fixed

dpblogs — Fri, 27 Apr 2007 02:33:00 GMT

Late last week, after bits had been finalized, we found a bug in the ADO.NET Entity Data Model Wizard that shipped with Visual Studio “Orcas” beta 1. The problem has now been corrected.

Please download and install the patch available at: http://www.microsoft.com/downloads/details.aspx?FamilyID=f69e9eb8-0ebd-4fba-a4cc-2050297ba75b&displaylang=en to fix the problem.

Thank you,

Sanjay Nagamangalam

Program Manager, ADO.NET Tools

ADO.NET Entity Framework - Orcas Beta 1

dpblogs — Sat, 21 Apr 2007 04:02:00 GMT

What else do you need to know about the Entity Framework? We've documented basic syntax and provided a few samples. How can we improve the documentation? Send us scenarios you want to implement.

Orcas Beta 1 contains the ADO.NET Entity Framework. This technology requires new ways of thinking about data applications. The documentation included with Beta 1 introduces design concepts and provides examples to get you up and running as quickly as possible. The ADO.NET team blog will help to answer questions that remain after you've read enough of the documentation to implement an Entity Data Model and write some code using the Entity Client.

We want to answer your questions. As you begin to work with the Entity Framework, add your comments to this post or one of the others on this blog that target specific features of the Entity Framework. This is a direct connection to the people who designed the Entity Framework. You can get answers to your questions and save time getting past the difficulties of working with new technology. The questions you raise will generate a lot of discussion at Microsoft. You'll get fast answers and help us make the next release of the Entity Framework easier to use.

Are there any particular features of the Entity Framework that need more documentation? In order to implement the examples in the documentation, the writers on this project resolved many of the coding issues you will encounter as you experiment with this framework. How can we make it easier for you to understand schema syntax and application code?

Tell us about the data applications you want to implement. Which scenarios for using this framework interest you most? The entity data model exists to increase your productivity. Using this model, you can resolve data access issues and concentrate directly on application scenarios. Tell us how we can help you get started. Your questions will get fast answers, and your feedback will improve the next release of the Entity Framework.

Mike Dodaro

Programmer/ Writer

Orcas Beta 1 - Now Available

dpblogs — Sat, 21 Apr 2007 03:59:00 GMT

Orcas Beta 1 is available for download! Check out the Visual Studio Downloads page for more info.

.NET Rocks with the ADO.NET Team!

dpblogs — Thu, 05 Apr 2007 23:49:00 GMT

Daniel Simmons, ADO.NET Dev Lead, recently sat down with Carl Franklin and Richard Campbell of .NET Rocks to talk in depth about the new ADO.NET Entity Framework. Check out Show #226 and learn more about Microsoft's vision for the Entity Framework long-term and how it is being realized in current development efforts, as well as in releases such as the March CTP.

EDM Wizard and Designer Featured in VSLive! San Francisco Keynote

dpblogs — Wed, 28 Mar 2007 06:42:00 GMT

Britt Johnston, Director, Data Programmability at Microsoft gave a Keynote speech this morning at VSLive in San Francisco. In his presentation, Britt spoke largely about the idea of Conceptual Data Programming and Microsoft's vision for Data Access: raising the level of abstraction around data access to allow developers to be more productive and allow us to write less code.

Britt began by talking about the idea of programming against a Conceptual data model rather than the Logical data model that we currently program to, and ended by delving into the key technoglogies that Microsoft plans to focus on for future innovation including continued investment in the Entity Framework, an Entity-level services ecosystem (Reporting, Synchronization, etc.), and the Enabling of Dynamic Applications (Conceptual Programming, Integrated Development Experience, Device to Cloud Support, etc.).

Britt also featured 5 short videos or screencasts that demonstrated some of the work, specifically around Tools, that the Data Programmability team (including the ADO.NET) has been doing. These videos provide some great information and a preview of the new EDM Wizard coming in Orcas, as well as a sneak preview of a new EDM Designer that we can expect to see released after the upcoming Orcas release.

For links to the other videos featured in Britt's Keynote presentation, see the Data blog.

ADO.NET Orcas: Sample Provider

dpblogs — Sat, 17 Mar 2007 02:19:00 GMT

Last month, the ADO.NET team hosted a number of ADO.NET Data Provider writers on campus to present information on how to enhance an existing provider to support the Entity Framework features in the upcoming Orcas release. To help provider writers who were unable to attend, we’re publishing the material we presented to our team blog.

In this blog post, you’ll find information about how to extend an existing provider. The blog also has a .zip attachment that contains the slide decks presented at the event, a help file that covers the CommandTree class, and the source code for a sample ADO.NET Data Provider.

The sample provider extends the SqlClient provider that’s included with version 2.0 of the .NET Framework. It follows the steps listed in the blog post and includes code to handle SQL generation, commonly referred to as SQLGen. The SQLGen code uses the standard visitor pattern to generate a SqlCommand’s CommandText and Parameters collection based on the CommandTree supplied.

Provider writers can use the code in the provider to enhance and extend their existing providers. The sample provider targets Microsoft SQL Server 2005 and Microsoft SQL Express and contains the SQLGen logic similar to the logic inside of the version of SqlClient that will ship with Orcas. Provider writers can run Entity Framework scenarios with the sample provider, setting breakpoints and stepping into code to better understand the SQLGen logic – including what CommandTree the Entity Framework generated, how SqlClient converts the CommandTree into a SqlCommand.

This version of the sample provider is designed for the March CTP of Orcas. The ADO.NET team plans to update the sample provider for subsequent public releases of Orcas. Please note that the sample provider is not supported.

If you’re a provider writer working on enhancing your provider for Orcas and you’re not already in touch with the ADO.NET team, please let me know.

David Sceppa

ADO.NET Program Manager

David.Sceppa@microsoft.com

_________________________________________________________________________________________

This specification describes how to extend an existing ADO.NET Data Provider to support the Entity Framework in ADO.NET Orcas. Once a provider writer has extended their existing provider to support the Entity Framework, all Entity Framework scenarios will be supported using that provider (assuming the SQL Gen logic in the provider supports generating the appropriate store-specific queries) including generation of mapping files and classes via EdmGen.Exe.

1 Supporting instantiation via DbProviderFactories.GetFactory

Note: This section refers ADO.NET 2.0 features that the Entity Framework relies upon.

The primary Entity Framework scenarios involve supplying the provider's name as part of the connection string.

string providerConnectionString = @"Data Source=.\SQLExpress;Initial Catalog=Northwind;" +

"Integrated Security=True";

string entityConnectionString =

string.Format(@"Metadata=.\NorthwindModel.csdl|.\NorthwindModel.ssdl|.\NorthwindModel.msl;" +

"Provider=System.Data.SqlClient;" +

"Provider Connection String=\"{0}\"", providerConnectionString);

using (EntityConnection connection = new EntityConnection(entityConnectionString))

{

}

The Entity Framework parses the connection string and extracts the provider's name and the connection string for the underlying provider, then instantiates the database connection using code like:

DbProviderFactory factory = DbProviderFactories.GetFactory(providerName);

DbConnection connection = factory.CreateConnection();

connection.ConnectionString = providerConnectionString;

This means that for a provider to plug into the Entity Framework, its factory class must be accessible via DbProviderFactories.GetFactory. Doing so has three main requirements:

1.1.1 The provider is listed in the combined configuration file

The provider may be listed in the machine configuration file, the application configuration file, etc. The provider's configuration entry should be under configuration / system.data / DbProviderFactories and typically looks like this.

<add name="SqlClient Data Provider"

invariant="System.Data.SqlClient"

description=".Net Framework Data Provider for SqlServer"

type="System.Data.SqlClient.SqlClientFactory, System.Data, Version=2.0.0.0,

Culture=neutral, PublicKeyToken=b77a5c561934e089" />

Note: The simple way to generate the value for the type attribute is to store typeof(ProviderFactory).AssemblyQualifiedName.

1.1.2 The provider's factory class is accessible via Type.GetType()

Calling DbProviderFactories.GetFactory also requires that provider factory's assembly qualified name can be used to access the provider factory class type via Type.GetType(). The two main ways to ensure that your provider can be instantiated in this fashion are to either register the assembly in the GAC (using regasm.exe) or to include the assembly in the application's directory.

1.1.3 The provider's ProviderFactory class has a public static Instance field

Each provider included with the .NET Framework exposes a public static Instance field on the provider factory class. ADO.NET relies on this pattern in the call to DbProviderFactories.GetFactory. If the desired provider does not follow this pattern, the call to GetFactory will throw an InvalidOperationException with the following message:

The requested .Net Framework Data Provider's implementation does not have an Instance field of a System.Data.Common.DbProviderFactory derived type.

2 Implementing ICloneable on the Command class

As part of the SQL Gen layer, described later, the Entity Framework requires that the underlying provider is able to generate a DbCommand given a CommandTree. The Entity Framework may execute the query specified in the DbCommand multiple times. To ensure that executing the query does not side effect the Command, the Entity Framework requires that the Command objects that a provider generates implements ICloneable and clones the Command and only uses the Execute methods of the cloned Commands.

Here is some sample code for cloning a Command object. The code assumes the SampleCommand already implements the ADO.NET 2.0 features and that the SampleParameter class follows a similar pattern for cloning.

public partial class SampleCommand : ICloneable

{

object ICloneable.Clone()

{

SampleCommand clone = new SampleCommand();

clone.Connection = this.Connection;

clone.CommandText = this.CommandText;

clone.CommandType = this.CommandType;

clone.CommandTimeout = this.CommandTimeout;

clone.DesignTimeVisible = this.DesignTimeVisible;

clone.Transaction = this.Transaction;

clone.UpdatedRowSource = this.UpdatedRowSource;

foreach (SampleParameter p in this.Parameters)

clone.Parameters.Add(((ICloneable) p).Clone());

return clone;

}

3 Overriding DbConnection.DbProviderFactory

There are many areas where the Entity Framework's API requires only a Connection object. However, the Entity Framework needs to access the ProviderFactory class for the provider given only an instance of its Connection class.

As of ADO.NET Orcas, there is no public way to access a ProviderFactory given a Connection. In ADO.NET Orcas, the DbConnection class has a new private ProviderFactory property, which calls a protected DbProviderFactory property. This is the same pattern established in ADO.NET 2.0 for properties like DbCommand.Connection. Note: The ProviderFactory property will likely be made public in a subsequent version of ADO.NET.

Once you've overridden the DbProviderFactory property, the Entity Framework will be able to access your provider's ProviderFactory class given an instance of the Connection class.

public partial class SampleConnection

{

protected override DbProviderFactory DbProviderFactory

{

get { return SampleFactory.Instance; }

}

4 Implementing IServiceProvider on the ProviderFactory class

The Entity Framework relies on a new class called DbProviderServices, which will be discussed later, as a starting point for retrieving database metadata, type information, SQL Generation, etc. The Entity Framework relies on the IServiceProvider pattern for accessing the DbProviderServices class given ProviderFactory, using code like:

DbProviderServices providerServices = null;

if (providerFactory as IServiceProvider != null)

{

IServiceProvider iServiceProvider = (IServiceProvider) providerFactory;

providerServices = (DbProviderServices) iServiceProvider.GetService(typeof(DbProviderServices);

}

In order for this code to succeed, you must implement IServiceProvider for your ProviderFactory, and return an instance of your provider's DbProviderServices class on calls to GetService that request DbProviderServices using code like:

public partial class SampleFactory : IServiceProvider

{

object IServiceProvider.GetService(Type serviceType)

{

if (serviceType == typeof(DbProviderServices))

return new SampleProviderServices();

else

return null;

}

For more information on IServiceProvider, see the MSDN documentation at:

http://msdn2.microsoft.com/en-us/library/system.iserviceprovider.getservice.aspx

5 Implementing DbProviderServices

The DbProviderServices class is the starting point for surfacing provider-specific types, metadata and functions (also known as the provider manifest), a DbCommand given a CommandTree (also known as SQL Generation), and mapping information required by the Entity Framework tools to generate classes and mapping files based on the tables and columns available.

The DbProviderServices class is used within the Entity Framework and is not expected to be accessed directly by users. There is no requirement to make the class public in ADO.NET Orcas.

internal class SampleProviderServices : DbProviderServices

{

}

5.1 Implementing GetDbInformation

The Entity Framework relies on the DbProviderServices class to access metadata about the underlying provider in order to understand the various data types and functions available through the provider. This information is commonly referred to as the provider manifest.

The Entity Framework tools also rely on the DbProviderServices class to access a store schema definition and a store schema mapping that describes how to access the schema information for the data store. The tools then use this mapping information to retrieve the store schema information to generate mapping files and class files for applications.

You can surface this information via the protected GetDbInformation method using code like the following. In this example, we expect that the XML information has been compiled into the provider as resources.

protected override XmlReader GetDbInformation(string informationType, DbConnection connection)

{

if (informationType == DbProviderServices.ProviderManifest)

{

if (connection == null)

throw new ArgumentNullException("Expected non-null connection on call to GetDbInformation");

if (connection.GetType() != typeof(SampleConnection))

throw new ArgumentException(string.Format("Wrong connection type. Expecting SampleConnection, received {0}",

connection));

return this.GetXmlResource("OrcasSampleProvider.Resources.SampleProviderServices.ProviderManifest.xml");

}

else if (informationType == DbProviderServices.StoreSchemaDefinition)

{

return this.GetXmlResource("OrcasSampleProvider.Resources.SampleProviderServices.StoreSchemaDefinition.ssdl");

}

else if (informationType == DbProviderServices.StoreSchemaMapping)

{

return this.GetXmlResource("OrcasSampleProvider.Resources.SampleProviderServices.StoreSchemaMapping.msl");

}

throw new NotSupportedException(string.Format("SampleProviderServices does not support informationType of {0}",

informationType));

}

private XmlReader GetXmlResource(string resourceName)

{

Assembly executingAssembly = Assembly.GetExecutingAssembly();

Stream stream = executingAssembly.GetManifestResourceStream(resourceName);

return XmlReader.Create(stream);

}

Note: This portion of the ADO.NET API is still under development and does not yet appear in publicly available builds. Earlier builds exposed the provider manifest using the following code.

[Obsolete("Please use GetInformation(DbProviderServices.ProviderManifest, connection) instead")]

public override XmlReader GetProviderManifest(DbConnection connection)

{

if (connection == null)

throw new ArgumentNullException("connection");

else

return this.GetXmlResource("OrcasSampleProvider.Resources.SampleProviderServices.ProviderManifest.xml");

}

5.2 Surfacing SQL Generation

As noted earlier, the DbProviderServices class is also the starting point for transforming a CommandTree into a store-specific query (DbCommand). The provider is responsible for generating both the CommandText and the Parameters for the Command.

Once the provider has implemented and surfaced this transformation logic, often called SQL Generation, the Entity Framework is able to execute queries generated via the EntityCommand, ObjectQuery<T> etc.

You can use code like the following to surface your SQL Generation layer via DbProviderServices.

public override DbCommandDefinition CreateCommandDefinition(DbCommand prototype)

{

return base.CreateCommandDefinition(prototype);

}

public override DbCommandDefinition CreateCommandDefinition(DbConnection connection, CommandTree commandTree)

{

DbCommand prototype = CreateCommand(connection, commandTree);

DbCommandDefinition result = this.CreateCommandDefinition(prototype);

return result;

}

internal DbCommand CreateCommand(DbConnection connection, CommandTree commandTree)

{

//SQL Generation logic goes here!

throw new NotImpelementedException("SQL Generation logic not yet supplied!");

}

Note: Generating a DbCommand based on a CommandTree is non-trivial. Use the SQL Generation logic that's part of the sample provider in the attachment as an example.

6 Transactions

The Entity Framework supports both local (DbTransaction) and distributed (System.Transaction) transactions. For example, the work done within a call to ObjectContext.SaveChanges is implicitly wrapped in a DbTransaction unless a transaction (DbTransaction or System.Transaction) is already active. In each case, the Entity Framework uses the transactional functionality of the underlying provider.

6.1 DbTransaction

As noted above, the Entity Framework implicitly wraps submitting changes in transactions. Therefore, providers must support DbTransactions.

6.2 System.Transactions

Using the Entity Framework with System.Transactions requires that the underlying provider also supports System.Transactions by supporting the EnlistTransaction method on the Connection class.

Inheritance in the Entity Framework

dpblogs — Thu, 15 Mar 2007 19:00:00 GMT

One of the coolest features of the ADO.NET Entity Framework is the ability to use inheritance in your database! Very cool, but what does it mean, and why is it important?

War of the worlds

Relational

The world of relational data is based around tuples, which have an inherent two dimensional structure. Think of a database table with rows and columns; it is a very two dimensional idea. Relationships, UDTs, views, etc are all extensions of this two dimensional view. This structure allows for the power we know and love in products like SQL Server, with features like query optimization, indexes, views, etc.

Objects

Object oriented programming (OOP) drives higher developer productivity and allows for applications to better represent the world in which they operate. One of the ways that they do this is by utilizing inheritance. Inheritance allows for an application to better model the data, and also provides the ability to extend an existing model and application in interesting ways.

Union of worlds

While the relational model has a lot of power, its way of looking at the world is somewhat at odds with the gestalt of object oriented programming.

The Entity Framework helps bridge these worlds by bringing the power of inheritance to the world of data access in ways that make sense and allow for the best features of both to shine through. To illustrate this, we will look at some of the benefits of OOP, and see how the Entity Framework helps bring these benefits to the store.

Why OOP?

Problem Modeling

In OOP, inheritance is often used to represent objects that have a IS-A relationship. This simply means that one object is a specialization of the other. For example, think of an online store with Products. A Product has certain attributes and data associated with it. Now, there is also a Discontinued Product which has all the same data as a regular Product, but also something more, namely a discontinued date. You could create a new object to represent the Discontinued Product, but it makes a lot more sense to create a new object that inherits from Product, and adds the additional field. After all, programmers are lazy, and why would I want to write more code then I need to!

Application Extension

You’ve done such a great job, and this online store has such a great database and application driving it, it’s ready to expand into new markets. The store is ready to branch out into seasonal products!

This scenario really shows off the extension power of OOP. You simply need to write a new object that inherits from Product and applies some different logic in its methods. For example, you might only show summer products in the summer or automatically provide a 10% discount for summer products in the fall. And all of the existing systems to manage inventory, shopping carts, etc just continue to work, as everything is still made up of Products!

How the Entity Framework bridges the gap

The ideas presented above are fairly basic; generally introduced in OOP 101. So what is the big deal? Let’s look at how we apply these scenarios in the Entity Framework, so we can see how powerful it is. This will also give us an opportunity to dive into the specifics of how the translation of OOP ideas to the relational world works.

To make the scenario specific, let’s use the example of an online hardware store, with both Discontinued Products, and Seasonal Products. The Entity Data Model (CSDL) for the Products might look something like the following.

</EntityType>

</EntityType>

</EntityType>

The CLR code for this situation is not shown, but we would have the same inheritance hierarchy in the generated CLR classes, with the full capabilities that this implies. The interesting thing with respect to this post is the OO to relational mapping.

Table per Hierarchy

Let’s start with the Discontinued Product, and see how you might model this in a database. You know that you will need to maintain a list of discontinued products (to maintain referential integrity), so you might create a table that contains all products with data like the following.

ProductID	Name	Price	Discontinued	Discontinued Date
1	Nail	$1.40	No	Null
2	Bubble level	$5.50	Yes	2/2/07

How do you use inheritance with data like this? Looking at the data, you know that when the Discontinued field is true, we’re dealing with a discontinued product. This is exactly the way that Table per Hierarchy (TPH) works! The idea is that the entire OO hierarchy is contained in one table, with some column(s) acting as discriminator. The value of the discriminator is what informs the Entity Framework what OO type each row contains.

For this setup, the mapping file might look something like the following. The Condition tag identifies the discriminator for the type.

</TableMappingFragment>

</EntityTypeMapping>

</TableMappingFragment>

</EntityTypeMapping>

Table per Type

Perhaps we did not initially intend to sell Seasonal Products, so they were not introduced until after the application was deployed. Modifying a core table of a live application is a good way to get the adrenaline flowing, and risk that new boat you have been saving for. We do not want to modify the Products table to add any new information!

We can create a new table with the Seasonal Product information, and create a one-to-one relationship between the two. Then, we simply need to modify the mapping file to let the Entity Framework know that the additional information associated with Seasonal Products can be found in another table. This is the core of Table per Type (TPT), where each table represents a type, but with only those fields associated with that type in the table. The mapping for this situation might look like the following.

</TableMappingFragment>

</TableMappingFragment>

</EntityTypeMapping>

Table per Concrete Type

We are going to have a long and successful run; our store is going to grow and prosper. The housing boom continues and we have millions of users. Competitors are entering the market, and so we want to make sure we do not have issues with scaling. One issue is that the number of products will increase and decrease over time, but it won’t change by a huge amount. However, the number of discontinued products is only going to increase with time.

To deal with this, we might want to move all the discontinued products to their own Discontinued Product table, perhaps on a linked server or on another filegroup. However, we don’t want to change the application to make this change. With the Entity Framework we don’t need to!

The Discontinued Product table contains all fields that exist on Discontinued Products, not just those that do not apply to other products (as was the case with TPT). This is the essence of Table per Concrete Type (TPC). Each table represents the entire Entity, and so when they are loaded, you only need to load from that one table. The mapping for TPC might look something like the following.

</TableMappingFragment>

</EntityTypeMapping>

Wrapping up…

As you can see from all this, the Entity Framework’s separation of the model from the database allows for some very cool scenarios to work. By using the different inheritance models supported, you can have the best of both the relational and the OO worlds. Viva la difference!

Please leave a comment with any questions that you have, or if you’d like to see more detail on any of the technologies mentioned.

Thanks,

Erick Thompson

ADO.NET Program Manager

101 LINQ Samples Updated

dpblogs — Wed, 14 Mar 2007 00:15:00 GMT

The 101 LINQ Samples that were included in the March CTP have now been updated to include the 101 LINQ to Entities Samples.

To check out the newest samples (updated April 17th) download from here.

Cheers,

Elisa Johnson
Data Programmability Community Program Manager

Using Stored Procedures for Change Processing in the ADO.NET Entity Framework

dpblogs — Fri, 09 Mar 2007 04:31:00 GMT

Most people who’ve played with the ADO.NET Entity Framework eventually ask whether they can replace the SQL statements it generates with stored procedures of their own. This is important in many applications because direct table access is not allowed. Current builds of the Entity Framework support using stored procedures for inserting, updating, and deleting entities. We are actively working on supporting stored procedures for reading entities – this should be available in an upcoming build.

In this post, I cover the use of stored procedures for change processing: taking changes (inserts, updates, and deletes) made to entities at the object layer and propagating them to the database. The examples used in this post are based on the Orcas March CTP. Readers unfamiliar with the ADO.NET Entity Framework should first read the ADO.NET Entity Framework Overview before proceeding – in this post, I assume you already have a working knowledge of Entity Framework basics, including conceptual models, mapping, and programming with Object Services. In addition to the overview, readers new to the Entity Framework may benefit from looking at the ADO.NET Samples for the Orcas October CTP.

Context: The Sample Database, Conceptual Model, Mapping, and Client Code

I’ll illustrate the use of stored procedures in the context of a sample application. For clarity, the database and conceptual model I use are exceedingly simple, but they are sufficient to illustrate the key concepts involved in working with stored procedures in the Entity Framework.

This section describes the sample database, the conceptual model and its mapping to the database, and the client code used to manipulate entities with the ADO.NET Entity Framework. At first, the operations on the entities will translate to direct insert, update, and delete statements against the tables. In the next section, we’ll look at how to change the mapping so that the Entity Framework calls a set of stored procedures to perform these operations, rather than manipulating the tables directly.

Logical Database Schema

Our database is called ProductInventory and has tables that encapsulate product and category information. The logical schema of this database is shown in Figure 1.

Figure 1: Logical schema of the ProductInventory database

The Categories table stores the information about known categories. The table has columns that store an auto-generated CategoryID (the primary key of this table) and a string CategoryName. The Products table stores product information: an auto-generated ProductID (the primary key of this table), a string ProductName, and a CategoryID (a foreign key into the Categories table). The CategoryID value for a given Product row specifies the category in which that product belongs.

My SSDL file reflects this logical schema, as shown in Listing 1 (attributes on the root <Schema> element have been omitted from this and subsequent listings for brevity; an ellipsis appears in place of the missing attributes):

<AssociationSet Name="FK__Products__Catego__7F60ED59"

Association="dbo.FK__Products__Catego__7F60ED59">

</AssociationSet>

</EntityContainer>

<Property Name="CategoryID" Type="int" Nullable="false"

StoreGeneratedPattern="identity" />

</EntityType>

<Property Name="ProductID" Type="int" Nullable="false"

StoreGeneratedPattern="identity" />

</EntityType>

<ReferentialConstraint FromRole="Categories" ToRole="Products"

FromProperty="CategoryID" ToProperty="CategoryID" />

</Association>

</Schema>

Listing 1: SSDL declaration of the ProductInventory logical schema

Conceptual Model

Because the purpose of this post is to illustrate stored procedures and not the richness of the Entity Framework’s mapping capabilities, I’ll use a conceptual model that is a simple 1:1 mapping to the logical database schema. That model is shown in Figure 2.

Figure 2: Conceptual model for products and categories

This model has two entity types: Category and Product. The properties of these entity types map to the columns in the Categories and Products tables respectively. The model also has an association type, Category_Product that describes a 1:* association relationship between the two entity types.

This conceptual model can be declared in CSDL as shown in Listing 2.

<AssociationSet Name="Category_Products"

Association="ProductInventory.Category_Product">

</AssociationSet>

</EntityContainer>

<Property Name="CategoryName" Type="String" Nullable="false"

MaxLength="4000" />

<NavigationProperty Name="Products"

Relationship="ProductInventory.Category_Product"

FromRole="Category" ToRole="Product" />

</EntityType>

<Property Name="ProductName" Type="String" Nullable="false"

MaxLength="4000" />

<NavigationProperty Name="Category"

Relationship="ProductInventory.Category_Product"

FromRole="Product" ToRole="Category" />

</EntityType>

</Association>

</Schema>

Listing 2: CSDL declaration of the conceptual model

Mapping

The conceptual model can be mapped 1:1 to the logical database schema via the MSL declaration shown in Listing 3. Here, each entity set and association set is mapped to the corresponding tables in the underlying database.

<cs:EntityContainerMapping cs:StorageEntityContainer="dbo"

cs:CdmEntityContainer="ProductInventoryContainer">

<cs:EntitySetMapping cs:Name="Categories">

<cs:EntityTypeMapping cs:TypeName="ProductInventory.Category">

<cs:TableMappingFragment cs:TableName="Categories">

<cs:ScalarProperty cs:Name="CategoryID" cs:ColumnName="CategoryID" />

<cs:ScalarProperty cs:Name="CategoryName"

cs:ColumnName="CategoryName" />

</cs:TableMappingFragment>

</cs:EntityTypeMapping>

</cs:EntitySetMapping>

<cs:EntitySetMapping cs:Name="Products">

<cs:EntityTypeMapping cs:TypeName="ProductInventory.Product">

<cs:TableMappingFragment cs:TableName="Products">

<cs:ScalarProperty cs:Name="ProductID" cs:ColumnName="ProductID" />

<cs:ScalarProperty cs:Name="ProductName"

cs:ColumnName="ProductName" />

</cs:TableMappingFragment>

</cs:EntityTypeMapping>

</cs:EntitySetMapping>

<cs:AssociationSetMapping cs:Name="Category_Products"

cs:TypeName="ProductInventory.Category_Product"

cs:TableName="Products">

<cs:EndProperty cs:Name="Category">

<cs:ScalarProperty cs:Name="CategoryID" cs:ColumnName="CategoryID" />

</cs:EndProperty>

<cs:EndProperty cs:Name="Product">

<cs:ScalarProperty cs:Name="ProductID" cs:ColumnName="ProductID" />

</cs:EndProperty>

<cs:Condition cs:ColumnName="CategoryID" cs:IsNull="false" />

</cs:AssociationSetMapping>

</cs:EntityContainerMapping>

</Mapping>

Listing 3: MSL declaration of the mapping from the conceptual model to the logical database schema

Client Code

With the database, model, and mapping in place, it’s time to look at some client code that reads and manipulates entities using the Entity Framework’s Object Services. Listing 4 shows an example of code that obtains an entity container, adds a new category and two new products, then changes one of the products, and finally deletes the category and both products.

// Obtain an context object for the ProductInventory entity container.

using (ProductInventoryContainer container = new ProductInventoryContainer())

{

// Create a new category.

Category hardware = new Category();

hardware.CategoryName = "Hardware";

container.AddObject(hardware);

// Create some products in the new category.

Product hammer = new Product();

hammer.ProductName = "Hammer";

hammer.Category = hardware;

container.AddObject(hammer);

Product screwDriver = new Product();

screwDriver.ProductName = "Screwdriver";

screwDriver.Category = hardware;

container.AddObject(screwDriver);

// Persist changes.

container.SaveChanges();

// Change one of the products.

screwDriver.ProductName = "Electric Screwdriver";

container.SaveChanges();

// Remove the hardware category and all its products.

container.DeleteObject(screwDriver);

container.DeleteObject(hammer);

container.DeleteObject(hardware);

container.SaveChanges();

}

Listing 4: Client code that uses ADO.NET Object Services to read and manipulate entities

If we examine the SQL that the Entity Framework generates and sends to the server when the SaveChanges() method is called, we see that inserts, updates, and deletes are performed directly against the database tables. Listing 5 shows the SQL produced for the operations performed in this code sample. Don’t worry too much about the details of this SQL – just notice that the operations are performed directly on the tables.

exec sp_executesql N'insert [dbo].[Categories]([CategoryName])

values (@0)

select [CategoryID]

from [dbo].[Categories]

where @@ROWCOUNT > 0 and [CategoryID] = scope_identity()',N'@0 nvarchar(8)',@0=N'Hardware'

exec sp_executesql N'insert [dbo].[Products]([ProductName], [CategoryID])

values (@0, @1)

select [ProductID]

from [dbo].[Products]

where @@ROWCOUNT > 0 and [ProductID] = scope_identity()',N'@0 nvarchar(6),@1 int',@0=N'Hammer',@1=7

exec sp_executesql N'insert [dbo].[Products]([ProductName], [CategoryID])

values (@0, @1)

select [ProductID]

from [dbo].[Products]

where @@ROWCOUNT > 0 and [ProductID] = scope_identity()',N'@0 nvarchar(11),@1 int',@0=N'Screwdriver',@1=7

exec sp_executesql N'update [dbo].[Products]

set [ProductName] = @0

where ([ProductID] = @1)

',N'@0 nvarchar(20),@1 int',@0=N'Electric Screwdriver',@1=18

exec sp_executesql N'delete [dbo].[Products]

where (([ProductID] = @0) and ([CategoryID] = @1))',N'@0 int,@1 int',@0=18,@1=7

exec sp_executesql N'delete [dbo].[Products]

where (([ProductID] = @0) and ([CategoryID] = @1))',N'@0 int,@1 int',@0=17,@1=7

exec sp_executesql N'delete [dbo].[Categories]

where ([CategoryID] = @0)',N'@0 int',@0=7

Listing 5: SQL generated by the Entity Framework to perform inserts, updates, and deletes on the database tables

In the following section, we’ll define stored procedures to perform the insert, update, and delete operations. We’ll then change the mapping to instruct the Entity Framework to use these stored procedures rather than manipulating the tables directly.

Using Stored Procedures to Manipulate Entities

In this section, we’ll change our application to use stored procedures to manipulate Category entities. We start with Category because it the simpler of the two entity types and will not require us to think about relationships yet. In the next section, we’ll look at using stored procedures when relationships are involved.

Stored Procedure Definitions

Let’s create stored procedures in the database for creating a new category, updating a category, and deleting a category. Listing 6 shows the definition of these procedures.

CREATE PROC [CreateCategory](

@CategoryName NVARCHAR(255)

)

BEGIN

-- Insert the category data.

INSERT INTO [Categories](CategoryName) VALUES (@CategoryName)

-- Get back the id of the newly created category.

SELECT CategoryID

FROM [Categories]

WHERE @@ROWCOUNT > 0 and CategoryID = scope_identity()

END

CREATE PROC [EditCategory](

@CategoryID INT,

@CategoryName NVARCHAR(255))

UPDATE [Categories]

SET CategoryName=@CategoryName

WHERE CategoryID=@CategoryID

CREATE PROC [RemoveCategory](

@CategoryID INT)

DELETE [Categories]

WHERE CategoryID=@CategoryID

Listing 6: Definition of change processing stored procedures for the Categories table

The CreateCategory stored procedure inserts a new row into the Categories table and retrieves the auto-generated CategoryID. The EditCategory procedure enables changing a category name by updating the Categories table for a given CategoryID. The RemoveCategory procedure deletes a category by removing the row with a given CategoryID from the Categories table.

Stored Procedure Declaration in SSDL

In order to use these stored procedures within the Entity Framework, they must be declared in the SSDL file that describes the database’s logical schema. Listing 7 shows the SSDL snippet that declares the three Categories-related stored procedures.

...

</Function>

</Function>

</Function>

</Schema>

Listing 7: SSDL declaration of stored procedures related to modifying the Categories table

These SSDL declarations simply describe the procedure names and parameters. Notice that the same <Function> SSDL element is used to declared stored procedures and user-defined functions. Stored procedures are distinguished from functions by the IsComposable attribute being set to false.

Mapping the Stored Procedures

Declaring the stored procedures in the SSDL file simply lets the Entity Framework know they exist in the database. But the SSDL declaration does not tell Entity Framework how and when to use these stored procedures. For that, we need to go back to the MSL file and change the declaration of the entity set mapping for categories. Listing 8 shows the MSL changes required (the boldface elements are new):

<cs:EntityContainerMapping cs:StorageEntityContainer="dbo"

cs:CdmEntityContainer="ProductInventoryContainer">

<cs:EntitySetMapping cs:Name="Categories">

<cs:EntityTypeMapping cs:TypeName="ProductInventory.Category">

<cs:TableMappingFragment cs:TableName="Categories">

<cs:ScalarProperty cs:Name="CategoryID" cs:ColumnName="CategoryID" />

<cs:ScalarProperty cs:Name="CategoryName"

cs:ColumnName="CategoryName" />

</cs:TableMappingFragment>

<cs:ModificationFunctionMapping>

<cs:InsertFunction cs:FunctionName="dbo.CreateCategory">

<cs:ScalarProperty cs:Name="CategoryName"

cs:ParameterName="CategoryName" />

<cs:ResultBinding cs:ColumnName="CategoryID" cs:Name="CategoryID"/>

</cs:InsertFunction>

<cs:UpdateFunction cs:FunctionName="dbo.EditCategory">

<cs:ScalarProperty cs:Name="CategoryID"

cs:ParameterName="CategoryID"

cs:Version="current"/>

<cs:ScalarProperty cs:Name="CategoryName"

cs:ParameterName="CategoryName"

cs:Version="current"/>

</cs:UpdateFunction>

<cs:DeleteFunction cs:FunctionName="dbo.RemoveCategory">

<cs:ScalarProperty cs:Name="CategoryID"

cs:ParameterName="CategoryID"/>

</cs:DeleteFunction>

</cs:ModificationFunctionMapping>

</cs:EntityTypeMapping>

</cs:EntitySetMapping>

...

</cs:EntityContainerMapping>

</Mapping>

Listing 8: Changes to the Categories entity set mapping to enable stored procedures

Within the <EntityTypeMapping> element for Category, a new <ModificationFunctionMapping> element is added. In this element we specify the functions to use for inserts, updates, and deletes (note that we say “function” instead of “stored procedure” following the convention, established in the SSDL, of treating stored procedures as non-composable functions).

For each function, we specify the function name and then mappings from the entity type’s properties to the function parameters. In the case of the insert function, we also specify a result binding: a mapping from columns in the stored procedure’s result set to the entity’s properties. This allows the Entity Framework to obtain the values of server-generated values, like the CategoryID in our example.

The property-parameter mappings for the update and delete functions can specify a Version attribute with a value of either “original” or “current”. In the update function mappings, Version is required; in the delete function mappings, it is optional. The Version attribute specifies whether the parameter is mapped to the original value of the entity’s property (as originally read from the database and maintained by the Object Services change tracking facilities) or the current value (potentially changed by the client code in-memory). This mechanism allows stored procedures to take both old and new values as parameters; this can be useful when performing optimistic concurrency control.

A NOTE ABOUT OPTIMISTIC CONCURRENCY CONTROL: The examples in this document do not illustrate performing optimistic concurrency control within stored procedures. The mechanism for passing original and changed values of parameters (described in this section) allows for optimistic concurrency control checks to be performed within a stored procedure’s implementation. The Entity Framework determines whether a concurrency violation occurred by examining the number of rows affected by the stored procedure’s execution: a non-zero number of rows affected means the operation succeeded; zero indicates a concurrency violation.

With the mappings shown in Listing 8 in place, the Entity Framework will translate insert, update, and delete operations performed on Category entities through Object Services into calls to the declared stored procedures. At the end of the next section (in which we’ll look at stored procedure mappings for Product entities), we’ll examine the SQL trace to see these procedures being invoked.

Using Stored Procedures to Manipulate Entities and Relationships

In mapping the Product entities to stored procedures for change processing, we need to think about relationships. In our conceptual model, Product entities are related to Category entities via the Product_Category relationship. In the database, this relationship is implemented with a foreign key from the product table to a categories table.

In terms of the logical database schema, we can think of the CategoryID foreign key like any other property of the product row. But in the EDM, relationships are first class constructs: a row in the Products table represents not just a Product entity but also an instance of the Product_Category relationship. When I delete a row from the Products table, I am removing two things that Object Services tracks: an entity and a relationship. If I update a Products table row and change only the value of the CategoryID column then I am not really updating a Product entity: I am deleting one Product_Category relationship and replacing it with a new one.

Relationships in the EDM can take a little time to get one’s head around. While this may seem abstract, it’s import to understand how the EDM treats relationships in order to make sense of what’s required to map stored procedures that manipulate them. As we look at the stored procedures that do change processing for Product entities and the corresponding MSL declarations, this will probably become clearer.

Stored Procedure Definitions

Listing 9 shows the stored procedures we’ll create in the database to support change processing for Product entities:

CREATE PROC [CreateProduct](

@ProductName NVARCHAR(255),

@CategoryID INT)

BEGIN

-- Insert the product data

INSERT INTO [Products](ProductName, CategoryID)

VALUES (@ProductName, @CategoryID)

-- Get back the id of the newly created product.

SELECT ProductID

FROM [Products]

WHERE @@ROWCOUNT > 0 and ProductID = scope_identity()

END

CREATE PROC [EditProduct](

@ProductID INT,

@ProductName NVARCHAR(255),

@CategoryID INT)

UPDATE [Products]

SET ProductName=@ProductName, CategoryID=@CategoryID

WHERE ProductID=@ProductID

CREATE PROC [RemoveProduct](

@ProductID INT,

@CategoryID INT)

DELETE [Products]

WHERE ProductID=@ProductID

Listing 9: Stored procedures for creating, editing, and removing rows in the Products table

These follow much the same patterns as the stored procedures we defined for the Categories table in Listing 6. The one noteworthy point here is that the RemoveProduct stored procedure takes both ProductID and CategoryID parameters but only uses the ProductID parameter. The CategoryID parameter could certainly be used to implement optimistic concurrency control within the procedure, but for this example, it is present in order to support the mapping required to handle the Category_Product relationship (shown in the next section).

Like before, we need to declare these stored procedures in the SSDL file in order to expose them to the Entity Framework. Listing 10 shows the required declarations.

...

</Function>

</Function>

</Function>

</Schema>

Listing 10: SSDL declaration of stored procedures related to modifying the Categories table

Mapping the Stored Procedures

After declaring the stored procedures in the SSDL file, we need to map them in the MSL file. Listing 11 shows the required changes to the mapping of the Products entity set:

<cs:EntityContainerMapping cs:StorageEntityContainer="dbo"

cs:CdmEntityContainer="ProductInventoryContainer">

...

<cs:EntitySetMapping cs:Name="Products">

<cs:EntityTypeMapping cs:TypeName="ProductInventory.Product">

<cs:TableMappingFragment cs:TableName="Products">

<cs:ScalarProperty cs:Name="ProductID" cs:ColumnName="ProductID" />

<cs:ScalarProperty cs:Name="ProductName"

cs:ColumnName="ProductName" />

</cs:TableMappingFragment>

<cs:ModificationFunctionMapping>

<cs:InsertFunction cs:FunctionName="dbo.CreateProduct">

<cs:ScalarProperty cs:Name="ProductName"

cs:ParameterName="ProductName" />

<cs:AssociationEnd cs:AssociationSet="Category_Products"

cs:From="Product" cs:To="Category">

<cs:ScalarProperty cs:Name="CategoryID"

cs:ParameterName="CategoryID"/>

</cs:AssociationEnd>

<cs:ResultBinding cs:ColumnName="ProductID" cs:Name="ProductID"/>

</cs:InsertFunction>

<cs:UpdateFunction cs:FunctionName="dbo.EditProduct">

<cs:ScalarProperty cs:Name="ProductID" cs:ParameterName="ProductID"

cs:Version="current"/>

<cs:ScalarProperty cs:Name="ProductName"

cs:ParameterName="ProductName"

cs:Version="current"/>

<cs:AssociationEnd cs:AssociationSet="Category_Products"

cs:From="Product" cs:To="Category">

<cs:ScalarProperty cs:Name="CategoryID"

cs:ParameterName="CategoryID"

cs:Version="current" />

</cs:AssociationEnd>

</cs:UpdateFunction>

<cs:DeleteFunction cs:FunctionName="dbo.RemoveProduct">

<cs:ScalarProperty cs:Name="ProductID"

cs:ParameterName="ProductID"/>

<cs:AssociationEnd cs:AssociationSet="Category_Products"

cs:From="Product" cs:To="Category">

<cs:ScalarProperty cs:Name="CategoryID"

cs:ParameterName="CategoryID"

cs:Version="original" />

</cs:AssociationEnd>

</cs:DeleteFunction>

</cs:ModificationFunctionMapping>

</cs:EntityTypeMapping>

</cs:EntitySetMapping>

</cs:EntityContainerMapping>

</Mapping>

Listing 11: Changes to the Products entity set mapping to enable stored procedures

In many ways, the modification function mappings look similar to those in the previous example (Listing 8). The only substantial difference is in the way the stored procedures’ CategoryID parameters are mapped. Rather than mapping directly to a property of the Product entity type, these parameters are mapped to an association end in the Category_Products association set. Specifically, they are mapped to the CategoryID property of the Category end of the association.

This declaration informs the Entity Framework that the CategoryID parameters refer to an end of the relationship. The Entity Framework uses this information to obtain the parameter values from the relationship instances that it tracks.

Because the CategoryID parameter is not used within the body of the RemoveProduct stored procedure, it’s natural to wonder why it needs to be declared and mapped. The reason is that the Entity Framework needs to know that the stored procedure deletes both a Product entity and a Category_Product relationship. The mapping of the CategoryID parameter to the relationship end lets the Entity Framework know it must call the stored procedures in the correct order to maintain the foreign key constraints in the database. Without this information, when several products and categories have been deleted via Object Services, the Entity Framework might generate the stored procedure calls to RemoveProduct and RemoveCategory in the wrong order, resulting in an attempt to remove a category before related products are deleted.

Stored Procedures in Action

With all the changes to the database, SSDL files, and MSL files described in the previous sections, we can now run the original client code (Listing 4) again to see the result. Note that we need not make any changes to the code – from the Object Services perspective, the operations don’t change. It is simply the modified mapping that results in the stored procedures being invoked, instead of the direct table manipulations. Listing 12 shows the results:

exec [dbo].[CreateCategory] @CategoryName=N'Hardware'

exec [dbo].[CreateProduct] @ProductName=N'Screwdriver',@CategoryID=8

exec [dbo].[CreateProduct] @ProductName=N'Hammer',@CategoryID=8

exec [dbo].[EditProduct] @ProductID=19,

@ProductName=N'Electric Screwdriver',@CategoryID=8

exec [dbo].[RemoveProduct] @ProductID=20,@CategoryID=8

exec [dbo].[RemoveProduct] @ProductID=19,@CategoryID=8

exec [dbo].[RemoveCategory] @CategoryID=8

Listing 12: Stored procedure calls generated by the Entity Framework to perform inserts, updates, and deletes

As this trace shows, the operations performed in Object Services translate to the stored procedures we declared.

I hope this post has been helpful and look forward to your feedback through comments.

Thanks

Shyam Pather, Development Lead – ADO.NET Entity Framework

LINQ - XElement and Entities

dpblogs — Tue, 06 Mar 2007 20:58:00 GMT

With LINQ we get a common data programming experience across multiple data sources. We can query in-memory collections, relational tables and XML. The other day I was looking at the various LINQ-XML examples and thought to myself that I would like to play around with loading my database from a remote XML source. Using XElement this becomes quite easy… I can load XML from a remote source, filter out the data that I care about and use this data to instantiate and persist data in my local database. Being a basketball fan I figured I would try this out with sports information, following is a discussion of my little experiment in this regard:

Loading Data From A Remote Data Source:

The first thing I needed was some sports information from a remote source, I went on live.com and did a search for sports information. I came across an XML format for sports information called SportsML and an URL with sample basketball data. For this exercise I figured that this would be a reasonable source for my endeavor.

Retrieving the information in a form that I can use LINQ over it is trivial, given a URL:

private String _sportsML = @"http://www.sportsml.org/IPTC/SportsML/1.7/examples/sportsml-basketball_standings.xml";

I can use the Load method on the XElement class as my starting point:

XElement sports = XElement.Load(_sportsML);

I can now perform LINQ queries over this to retrieve the data I want. The sportsML schema provides results, schedules and more, I just wanted to pull out schedules which I did as follows:

protected IEnumerable<XElement> GetSchedule()

{

XElement sports = XElement.Load(_sportsML);

var schedule = from s in sports.Elements()

where s.Name.LocalName == "schedule"

select s;

return schedule;

}

Given a schedule I can further use LINQ to pull out the pieces of the schedule that I want. For this exercise I will use LINQ to pull the date & time plus the participating teams:

Retrieve the Date:

protected String GetEventDateTime(XElement sportsEvent)

{

var dateTimeResult = from eventMetadata in sportsEvent.Elements()

where eventMetadata.Name.LocalName == "event-metadata"

from evDateTime in eventMetadata.Attributes() where evDateTime.Name.LocalName == "start-date-time"

select evDateTime;

return (dateTimeResult.Count() > 0) ? dateTimeResult.First().Value : null;

}

Retrieve the Teams:

protected String[] GetTeams(XElement sportsEvent)

{

var teamNames = from teams in sportsEvent.Elements()

where teams.Name.LocalName == "team"

from teamMetadata in teams.Elements()

where teamMetadata.Name.LocalName == "team-metadata"

from nameElement in teamMetadata.Elements()

select nameElement.Attribute("first").Value +" "+ nameElement.Attribute("last").Value;

return teamNames.ToArray<String>();

}

With this information at hand I can now create persistent instances that I will save using the ADO.NET Entity Framework:

protected SportsEvent CreateSportsEvent(XElement sportsEvent)

{

SportsEvent entitySportsEvent = null;

String dateTimeAsString = GetEventDateTime(sportsEvent);

if (null != dateTimeAsString)

{

String[] teamNames = GetTeams(sportsEvent);

entitySportsEvent = new SportsEvent();

entitySportsEvent.EventDateTime = DateTime.Parse(dateTimeAsString);

entitySportsEvent.AwayTeam = teamNames[0];

entitySportsEvent.HomeTeam = teamNames[1];

}

return entitySportsEvent;

}

Saving this is merely a matter of adding the instance to the ObjectContext and saving:

public void ScheduleReader()

{

BlogRssContext context = new BlogRssContext();

IEnumerable<XElement> schedule = GetSchedule();

foreach (XElement sportsEvent in schedule.Elements())

{

SportsEvent entitySportsEvent = CreateSportsEvent(sportsEvent);

if (null != entitySportsEvent)

{

context.AddObject(entitySportsEvent);

}

context.SaveChanges();

}

With that we have pulled data from a remote service and persisted it in a local database.

Tim Mallalieu

Program Manager, ADO.NET

Object Services – Write less code

elisaj — Fri, 02 Mar 2007 05:35:00 GMT

One of the main features of the Entity Framework revolves around Object Services. The main points of Object services are to write less code and allow programmers to program against objects to access the data that’s stored in a database or possibly anywhere. We can consider this feature as a road to CRUD; where CRUD represents Create, Read, Update, and Delete operations against data, except this feature provides more than just a simple road.

How this works?

Object Services is the top layer in the Entity Framework. The framework consists of three district layers: EDM, mapping, and the source layer. The important layer for Object Services is the EDM layer (also label CSDL). The Entity Data Model (EDM) layer brings the concepts of entities to stored data. Currently in version 1.0 of the ADO.NET Entity Framework there is a 1:1 relationship between the EDM and Objects Services. Object Services is the realization of Objects from those described Entities. The table below shows the different layers.

Another important feature for Object Services and the Entity Framework in general comes with the knowledge that the Objects Services gives access to the entity type information in the lower layers of the Framework. By providing access to the lower layers of the Entity Framework, customers gain access to specific type information, relationship information, and basic data readers and writers to the data. One particular scenario may be translating between data sources, and one way to do this would be to gain access to the type information.

What can you do with Object Services?

Here’s a bullet list of some of the normal operations which object services can provide:

· Query using LINQ or Entity SQL

· CRUD

· State Management – change tracking with events

· Lazy loading

· Inheritance

· Navigating relationships

More features of Object Services allow for data conflicts resolution. For example, Object Services supports:

· Optimistic concurrency

· Identity resolution - keeping on copy of the object even from several different queries

· Merging

· Refreshing data with overwrite options (ie: server wins or client wins)

These are important concepts to understand because the Entity Framework isn’t just designed to do CRUD operations; the Entity Framework is designed to scale to large or even mission critical applications. Given a large or small program, programming against a service model with consistent error handling, data handling, and easy access to data layers allow programs to have a robust experience. These features also allow programmers to work on features instead of core application plumbing.

Let’s dive into some examples:

Here’s a simple sample using a query with Entity SQL that does a foreach over objects materialized from the EDM.

using (NorthwindEntities northWind = new NorthwindEntities())

{

// Calling CreateQuery materializes objects

ObjectQuery<Products> products =

northWind.CreateQuery<Products>(

"SELECT VALUE p FROM Products AS p");

foreach (Products p in products)

{

Console.WriteLine(p.ProductName);

} // foreach

} // using Northwind Entities

Same sample using LINQ.

using (NorthwindEntities northWind = new NorthwindEntities())

{

var products = from p in northWind.Products select p;

foreach (Products product in products)

{

Console.WriteLine(product.ProductName);

} // for each

} // using

CRUD Examples:

The next example does a LINQ query retrieving all the products that belong to a category called beverages and then just for kicks, changes the discontinued products that have unit prices to $1.25. When looking at the samples, notice that there’s a call to save changes which only pushes the modified Objects back to the the data source.

using (NorthwindEntities northWind = new NorthwindEntities())

{

// do a LINQ query which finds all the products that fall

// under the category beverages

var products = from p in northWind.Products

where p.Categories.CategoryName == "beverages"

select p;

foreach (Products product in products)

{

// with lazy loading, object references must

// be loaded first, otherwise an exception is thrown.

// this allows for a reduced number of trips to the source.

Console.WriteLine("{0}\tDiscontinued: {1}",

product.ProductName,

product.Discontinued);

// just for fun, if the product is discontinued,

// then let's set the unit price to $1.25

if (product.Discontinued)

{

product.UnitPrice = (decimal)1.25;

} // if

} // for each

// notice, that until we call save changes

// the data in the database or source is not updated.

northWind.SaveChanges();

} // using

One more example with conflict:

The final example is a little longer, but shows optimistic concurrency and one way to deal with the conflict. conflicts. Also note that to make this work, which also shows the flexibility in the EDM, the entity set Categories must have an attribute flagged with ConcurrencyMode set to “fixed”.

// Property set in the EDM or CSDL file.

Code example:

// save the category id out for retrieval later in this example.

int cateID = 0;

// create an instances of a new category and add to the list

// of categories.

using (NorthwindEntities northWind = new NorthwindEntities())

{

// create a new category

Categories catNew = new Categories();

catNew.CategoryName = "Blog Example";

catNew.Description = "Blog description";

// add to object context, and save changes

northWind.AddObject(catNew);

northWind.SaveChanges();

// cache away the category ID.

cateID = catNew.CategoryID;

} // using

// create different object context variables

using (NorthwindEntities northWind1 = new NorthwindEntities())

using (NorthwindEntities northWind2 = new NorthwindEntities())

{

// using a direct query from the categories item from

// northwind 2, get the id out.

Categories cat1 = northWind1.CreateQuery<Categories>(

"SELECT VALUE c FROM Categories AS c WHERE c.CategoryID=@id",

new ObjectParameter("id", cateID)).First();

cat1.CategoryName = "category 1";

// using a direct query from the categories item from

// northwind 2, get the id out.

Categories cat2 = northWind2.CreateQuery<Categories>(

"SELECT VALUE c FROM Categories AS c WHERE c.CategoryID=@id",

new ObjectParameter("id", cateID)).First();

cat2.CategoryName = "category 2";

// save northwind 2 so we can create a conflict.

northWind2.SaveChanges();

try

{

// with the second save changes, the services notices the data

// that’s out of sync and throws an exception.

northWind1.SaveChanges();

} // try

catch(OptimisticConcurrencyException)

{

Console.WriteLine("Caught optimistic concurrency exception");

} // catch

// One way of fixing the violation is to

// refresh the first object context, then

// change the category name again

// and save changes.

northWind1.Refresh(RefreshMode.StoreWins, cat1);

cat1.CategoryName = "category 1";

northWind1.SaveChanges();

// for the sake of our database, let's delete the

// object, and not to forget to refresh

// and save changes

northWind2.Refresh(RefreshMode.StoreWins, cat2);

northWind2.DeleteObject(cat2);

northWind2.SaveChanges();

} // using

To Sum it up:

This briefly describes the Object Services layer of the Entity Framework. There is a lot more, but the basic concepts are: writing less code and get more powerful services.

Watch for more blogs to detail the subject of Object Services in the near future.

Brian Dawson

ADO.NET Program Manager

RSS Feeds with ADO.NET Entity Framework and the ASP.NET RSS Toolkit

elisaj — Thu, 01 Mar 2007 00:17:00 GMT

The ASP.NET team has released the RSS toolkit beta on sandbox.asp.net. This led me to want to tinker with exposing some RSS feeds using ADO.NET Entity Framework to retrieve the data. I pulled the toolkit and within 10 minutes had my first couple of feeds going.

Pretty simple, yet cool stuff. The RSS toolkit essentially provides a specialized HttpHandler that one then derives from to expose the RSS functionality.

I created a new WAP project and added an EDM model on top of the Northwind Database.

From their I added a new HttpHandler class and changed the base class to be the GenericRssHttpHandlerBase class from the RSS Toolkit.

At this point it was merely a matter of retrieving the data that I wanted to return and setting it accordingly.

The only thing one need do to use the RSS Toolkit is to provide an override of the PopulateChannel method.

In this case my override uses the EntityFramework to retrieve all products from the database.

/// <summary>

/// Summary description for $codebehindclassname$

/// </summary>

[WebService(Namespace = "http://tempuri.org/")]

[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]

public class NorthwindProducts : GenericRssHttpHandlerBase

{

protected override void PopulateChannel(string channelName, string userName)

{

Channel["title"] = "Products";

using (EntityNorthwindContext context = new EntityNorthwindContext())

{

foreach(Product product in context.Products.Execute(MergeOption.NoTracking))

{

GenericRssElement item;

item = new GenericRssElement();

item["title"] = product.ProductName;

item["description"] = BuildProductDescription(product);

Channel.Items.Add(item);

}

protected String BuildProductDescription(Product product)

{

StringBuilder bldr = new StringBuilder();

bldr.AppendLine(String.Format("ProductID={0};", product.ProductID));

bldr.AppendLine(String.Format("ProductName={0};", product.ProductName));

bldr.AppendLine(String.Format("QuantityPerUnit={0};", product.QuantityPerUnit));

bldr.AppendLine(String.Format("ReorderLevel={0};", product.ReorderLevel));

bldr.AppendLine(String.Format("UnitPrice={0};", product.UnitPrice));

bldr.AppendLine(String.Format("UnitsInStock={0};", product.UnitsInStock));

bldr.AppendLine(String.Format("UnitsOnOrder={0};", product.UnitsOnOrder));

return bldr.ToString();

}

From there one can just hit the page to get back an RSS representation of the products in the Northwind database.

Tim Mallalieu

Program Manager, ADO.NET