meek : stored procedures

Stored Procedures and First-Class Relationships in the Entity Framework

cmeek — Sun, 07 Dec 2008 16:49:00 GMT

A user of the EFExtensions library asked me how to retrieve a set of ‘categories’ and their related ‘products’ through a single stored procedure call. I made some changes to the sample application to illustrate this pattern, but I thought I should also spend some time explaining how it works. The full explanation takes a little bit of patience, because it gets into some of the unique features of the Entity Data Model (EDM).

The EDM has “first-class relationships”. What does this mean? Take a look at products and categories in the EFExtensions sample application. The dbo.Products table has a cid column with a foreign key constraint. The corresponding Product entity type does not have a property corresponding to that column. Where did the foreign key go? Typically, foreign keys are mapped to a separate "association set" in your model, in this case ProductCategoryAssociations, so they would be redundant if included in the entity type as well. Consider the following table contents:

dbo.Categories
cid	name
1	Beverage
2	Food

dbo.Products
pid	name	cid
1	Beer	1
3	Water	1

In our entity model, there are three independent sets:

Categories
CategoryID	Name
1	Beverage
2	Food

Products
ProductID	Name
1	Beer
3	Water

ProductCategoryAssociations
Products<ProductID>	Categories<CategoryID>
<1>	<1>
<3>	<1>

The products and the relationships between products and categories are actually separate ‘things’ in the conceptual model, whereas in the database they’re represented by a single table. The nice thing about this independence in the EDM is that you can define a relationship between two entity types without modifying the entities. There are also some potentially unintuitive side effects to consider. I’ll talk about a few of them here.

Let’s say you decide that ‘Beer’ is a meal in and of itself and should be classified as a ‘Food’ rather than a ‘Beverage’. You start off by loading the relevant category and product:

Category food = context.Categories.Where(c => c.Name == "Food").First();

Product beer = context.Products.Where(p => p.Name == "Beer").First();

So where is our "first-class" relationship hiding? Let’s look at the contents of the state manager (context.ObjectStateManager.GetStateEntries):

ProductCategoryAssociations(Unchanged)

<CategoryID=1, ProductID=1>

Categories(Unchanged)

C2: Food

Products(Unchanged)

P1: Beer

Categories(Unchanged)

Stub entity CategoryID=1

Although we only queried two entities, there are four state entries! As expected, we have the ‘Beer’ product and the ‘Food’ category. We also have a ProductCategoryAssocations entry, representing the relationship between ‘Beer’ and ‘Beverage’, and a so-called stub entry representing the ‘Beverage’ category (at this point we only know that the key value is ‘1’). This is a little bit strange. After all, I didn’t query the association set. This is because the Entity Framework (EF) implicitly “spans” in reference relationships when you issue certain queries. In this case, Product has a reference (as opposed to collection) relationship with Category, so the query issued automatically includes the relationship information as well.

Warning: implicit relationship span does not happen for EntityCommand, ObjectQuery (LINQ or Entity-SQL) with MergeOption.NoTracking, or FunctionImport stored procedures (though I demonstrate how to work around this last limitation using EFExtensions in this post).

What happens when I change the category?

beer.Category = food;

Although it may look like I’m modifying the product, I’m really deleting one relationship and adding another. If you examine the state manager again, you’ll notice there are now two relationship state entries:

ProductCategoryAssociations(Added)

<CategoryID=2, ProductID=1>

ProductCategoryAssociations(Deleted)

<CategoryID=1, ProductID=1>

When I call context.SaveChanges, the EF is smart enough to issue an UPDATE rather than attempting to INSERT a new row and DELETE the existing row. The moral of the story: things are not necessarily what they seem. The ObjectContext does a pretty good job of sheltering us from the complexities of first-class relationships, but it’s good to understand them in case something goes wrong. Which brings me back to the original point of this post… What if I want to call a stored procedure that returns multiple categories and multiple products? Most likely, I’d like to know which categories and products are related. My original samples showed how you could return a single category and its related products. If there are multiple categories, we no longer know how the products line up with categories, and we end up with a disconnected graph. After attaching products and categories to my context, I still don’t know about the relationships, obvious when you examine the state manager:

Categories(Unchanged)

C1: Beverage

Categories(Unchanged)

C2: Food

Products(Unchanged)

P1: Beer

Products(Unchanged)

P3: Water

If we change the definition of our product ‘materializer’ (I describe these techniques in detail here), we can include the relationship data with the product entity:

new Product {

ProductID = r.Field<int>("pid"),

Name = r.Field<string>("name"),

CategoryReference = {

EntityKey = new EntityKey("MiniNWEntities.Categories", new[] { new EntityKeyMember("CategoryID", r.Field<int>("cid")) }),

}

Now we’re mimicking the “relationship span” behavior of the EF by including the EntityKey for our category reference along with the product, and the state manager reflects the relationships as well as the entities:

ProductCategoryAssociations(Unchanged)

<CategoryID=1, ProductID=1>

ProductCategoryAssociations(Unchanged)

<CategoryID=1, ProductID=3>

In general, I think it’s useful to adopt a “first-class relationships” view of the world when you work with the EF because it is central to the design of the product.

EFExtensions update

cmeek — Sun, 07 Dec 2008 16:46:00 GMT

The EFExtensions helper library has a few updates:

- CreateCommand now respects the CommandTimeout property on ObjectContext. A user of the library reported this bug to me a while ago, and I’m finally getting around to fixing it.

- One tricky scenario involves multiple stored procedure result sets that need to be combined to produce a single object graph. No new code in the library is required, but I have added a new sample illustrating how you can return a set of categories and their related products through a single stored procedure call. See the Materializer_MultipleResultSetsMerge sample for details. See this post for background information.

- We get frequent requests for LINQ patterns involving InvocationExpressions. I’ve added a couple of helper ExpandInvocations overload to simplify use of these patterns in the Entity Framework (EF). See the Misc_ExpandInvocationExpressions sample for details. See this post for details of the implementation soon.

ADO.Entity Framework: Stored Procedure Customization

cmeek — Wed, 26 Mar 2008 18:52:00 GMT

The ADO.NET Entity Framework (EF) allows you to map stored procedures to functions that return typed results in the conceptual model. However, when stored procedures results don't match the patterns supported by the EF, reading and tracking results is quite difficult. Fortunately, it is possible to run arbitrary stored procedures and Transact-SQL commands through the ObjectContext. This post describes these facilities and leverages a new utility library recently posted on Code Gallery (EFExtensions) to make the job much easier. In most cases, I will show the coding patterns directly against the EF and then illustrate the same behavior using EFExtensions, e.g.:

// EF example

// EFExtensions example

The following patterns are discussed:

· Creating and executing store commands.

· Materializing typed results.

· Tracking results.

· Multiple result sets.

There’s lots of code between here and the end, so as a motivation I’ll start by showing you what all of these extensions methods working in concert can do. Without the EFExtensions library the code to call a stored procedure in the database (including properly handling connection lifetime), create a set of objects and identity resolve them against the context would be surprisingly large. With EFExtensions, that code is reduced to:

var results = context

.CreateStoreCommand("GetCategories", CommandType.StoredProcedure)

.Materialize<Category>()

.Bind(categorySet);

Store commands

You can reuse the connection on an ObjectContext to create a store command as follows:

using (MyContext context = new MyContext())

{

// In V1 of the EF, the context connection is always an EntityConnection

EntityConnection entityConnection = (EntityConnection)context.Connection;

// The EntityConnection exposes the underlying store connection

DbConnection storeConnection = entityConnection.StoreConnection;

DbCommand command = storeConnection.CreateCommand();

command.CommandText = "GetCategory";

command.CommandType = CommandType.StoredProcedure;

command.Parameters.Add(new SqlParameter("cid", 1));

}

First you’ll need to add the library namespace to your sources files in order to use the extension methods provided by this library:

using Microsoft.Data.Extensions;

Now you can use the CreateStoreCommand method which extends ObjectContext and packages the code shown above:

using (MyContext context = new MyContext())

{

DbCommand command = context.CreateStoreCommand("GetCategory",

CommandType.StoredProcedure, new SqlParameter("cid", 1));

}

The ObjectContext does a nice job of managing the store connection for you when you run a query. If the connection is not already open, it will be opened for the duration of the query. I’ve included a general purpose extension method called CreateConnectionScope on DbConnection to support the same behavior, so instead of writing:

bool openingConnection = command.Connection.State == ConnectionState.Closed;

if (openingConnection) { command.Connection.Open(); }

int result;

try

{

result = command.ExecuteNonQuery();

}

finally

{

if (openingConnection && command.Connection.State == ConnectionState.Open) { command.Connection.Close(); }

}

I can write

int result;

using (context.Connection.CreateConnectionScope())

{

result = command.ExecuteNonQuery();

}

Alazel Acheson, a developer on the ADO.NET team, wrote a more extensive connection scope utility that you can find here.

Materializing typed results

Given a store command, you can manually produce typed results, which EF team calls “result materialization”: turning relational records we get from the store into “real” objects!

Here’s some code that takes results from the command we created in the last section and uses them to populate Category instances:

using (DbDataReader reader = command.ExecuteReader())

{

while (reader.Read())

{

int categoryIDOrdinal = reader.GetOrdinal("cid");

int nameOrdinal = reader.GetOrdinal("name");

yield return new Category

{

CategoryID = reader.GetInt32(categoryIDOrdinal),

Name = reader.IsDBNull(nameOrdinal) ? (string)null : reader.GetString(nameOrdinal),

};

}

EFExtensions includes a Materializer class and several Materialize extension method overloads that make this a little bit easier.

command.Materialize(r =>

new Category {

CategoryID = r.Field<int>("cid"),

Name = r.Field<string>("name"),

});

Internally, all Materialize overloads taking commands leverage CreateConnectionScope and handle the lifetime of the reader. Materialize overloads optionally take a “shaper” delegate (as shown in the above example) which maps from IDataRecord to the result type. Note the Field method overload as well which handles the magic around type conversion and DBNull handling from data records (similar to the LINQ to DataSet method).

When no materialization delegate is specified, we generate a default shaper which assigns column values to public writable properties of the result type of the same name. In the following example, the store command explicitly projects two columns with the names of CLR properties on the Category type:

context.CreateStoreCommand("select cid as CategoryID, [name] as [Name] from dbo.Categories")

.Materialize<Category>();

Why are shapers provided as Expression<Func<IDataRecord, T>> rather than just Func<IDataRecord, T>? We plan on exploiting the expression representation of the shaper in a future release of EFExtensions (details to follow).

While the Materialize extension methods on DbCommand and DbDataReader are convenient, it is more efficient to create a single Materializer instance and reuse it. Here are a couple of materializers that I will reuse in the rest of this post:

// Materializer with column renames.

private static readonly Materializer<Category> s_categoryMaterializer = new Materializer<Category>(r =>

new Category

{

CategoryID = r.Field<int>("cid"),

Name = r.Field<string>("name"),

});

// Materializer returning different types based on a condition.

private static readonly Materializer<Product> s_productMaterializer = new Materializer<Product>(r =>

r.Field<DateTime?>("discontinued_date").HasValue ?

(Product)new DiscontinuedProduct

{

ProductID = r.Field<int>("pid"),

Name = r.Field<string>("name"),

DiscontinuedDate = r.Field<DateTime>("discontinued_date")

} :

new Product

{

ProductID = r.Field<int>("pid"),

Name = r.Field<string>("name")

});

Tracking results

Using the techniques I’ve described so far, you can retrieve arbitrary CLR objects from store commands, including entities. Entities aren’t very useful if they aren’t tracked however – you cannot modify them or delete them using the context’s state manager. To facilitate tracking, EFExtensions includes an EntitySet class which manages activities related to Entity Data Model (EDM) entity sets. The class includes a FindOrAttach method which either attaches the given entity to the state manager for tracking purposes or returns an existing entity with the same key. This simulates the identity resolution behavior that the EF provides when running queries.

In the following example, we run a command returning a single category and track it

Category category = s_categoryMaterializer.Materialize(command).SingleOrDefault();

// Create a category set

EntitySet<Category> categorySet = new EntitySet<Category>(context, "categories");

// Track command result

category = categorySet.FindOrAttach(category);

An EntitySet<T> similar to an ObjectQuery<T> in the EF but it is bound to a specific EDM EntitySet instance (which you can examine by looking at the EntitySet<T>.Metadata property), and encapsulates various services related to the set. I will cover this class in greater detail in a future post.

For convenience, a Bind extension method that attaches a sequence of entities is also included. This makes it easy to stream results, as in the following example:

IEnumerable<Product> products = s_productMaterializer.Materialize(command).Bind(productSet);

I recommend creating EntitySet properties on your ObjectContext partial class definition. These properties can also be used as roots for LINQ and Entity-SQL queries.

public partial class MyObjectContext : ObjectContext

{

private EntitySet<Product> productSet;

/// <summary>

/// Gets alternative entry point for the Products entity set.

/// </summary>

public EntitySet<Product> ProductSet

{

get

{

if (null == this.productSet)

{

this.productSet = new EntitySet<Product>(this, "Products");

}

return this.productSet;

}

…

}

Multiple result sets

I’ll give one final example that ties together many of the concepts described in this post. Assume we have a stored procedure returning two result sets. The first result set includes a Category instance and the second result set includes all related Products. For convenience, I’ll define a method on my object context that takes a category ID, and returns that category with all related products attached. We’ve already defined the necessary materializers (s_categoryMaterializer and s_productMaterializer), as well as our entity set properties (ProductSet and CategorySet). Note that we leverage an existing EF method, EntityCollection.Attach, to associate the returned category with the related products.

public Category GetCategoryAndRelatedProducts(int categoryID)

{

DbCommand command = this.CreateStoreCommand("GetCategoryAndProducts", CommandType.StoredProcedure, new SqlParameter("cid", categoryID));

Category category;

using (command.Connection.CreateConnectionScope())

using (DbDataReader reader = command.ExecuteReader())

{

// first result set includes the category

category = s_categoryMaterializer

.Materialize(reader)

.Bind(this.CategorySet)

.SingleOrDefault();

// second result set includes the related products

if (null != category && reader.NextResult())

{

category.Products.Attach(s_productMaterializer

.Materialize(reader)

.Bind(this.ProductSet));

}

return category;

}

Expect more posts on the EFExtensions library over the months leading up to the release of the ADO.NET Entity Framework V1!