Diego Vega

Rowan on Entity Framework Events and Alex’s Tips

I don’t post much on my blog lately (too busy working on Entity Framework for .NET 4!), but this post from my teammate Rowan struck me as something that would help lots of customers, so I wanted to link to it. It explains basically everything you need to know about events available in the Object Services API of Entity Framework.

While I am here, the tips series in my other teammate Alex James’ blog probably don’t need much publicity from me, but they are an awesome resource for customers.

Third post about POCO, first post about Code Only

It is always busy here with all the improvements we are doing in Entity Framework to make your code work better with it. That is why I haven’t been posting to my blog much in the last months. Today however, there are two important posts from people that sit very close to me, so I am going to link to them.

Faisal posted the third part in a series on the POCO experience with EF4. His post delves into the details of how snapshot change tracking compares with notification based change tracking and on some of the API considerations for it.

Alex, who sits in my office (although he likes to think I sit in his :)) made the first post about the Code Only experience we are working on. I like to think of Code Only as “POCO on steroids”, because it not only gives you the right level of decoupling between your domain classes and the persistence framework, but it also puts mapping artifacts out of the way. I am especially fond of the way you can customize mapping using LINQ queries, although that feature is not going to be included in the first preview.

Please go read the posts, play with the bits (you will need to wait a few weeks to play with code-only) and tell us what you think!

Server queries and identity resolution

I answered a Connect issue today that deals with a very common expectation for users of systems like Entity Framework and LINQ to SQL. The issue was something like this:

When I run a query, I expect entities that I have added to the context and that are still not saved but match the predicate of the query to show up in the results.

Reality is that Entity Framework queries are always server queries: all queries, LINQ or Entity SQL based, are translated to the database server’s native query language and then evaluated exclusively on the server.

In sum, Entity Framework does not include a client-side or hybrid query processor.

MergeOption and Identity resolution

There are chances that you have seen unsaved modifications in entities included in the results of queries. This is due to the fact that for tracked queries (i.e. if the query’s MergeOption is set to a value different from NoTracking) Entity Framework performs “identity resolution”.

The process can be simply explained like this:

1. The identity of each incoming entity is determined by building the corresponding EntityKey.
2. The ObjectStateManager is looked up for an entity already present that has a matching EntityKey.
3. If an entity with the same identity is already being tracked, the data coming from the server and the data already in the state manager are merged according to the MergeOption of the query.
4. In the default case, MergeOption is AppendOnly, which means that the data of the entity in the state manager is left intact and is returned as part of the query results.

However, membership of an entity in the results of a given query is decided exclusively based on the state existing on the server. In this example, for instance, what will the query get?:

The answer is:

1. The modified entity customer1 won’t show up in the query because its LastName is still Tiger on the database.
2. The deleted entity customer2 will be returned by the query, although it is a deleted entity already, because it still exists in the database.
3. The new entity customer3 won’t make it, because it only exists in the local ObjectStateManager and not in the database.

This behavior is by design and you need to be aware of it when writing your application.

Put in some other way, if the units of work in your application follow a pattern in which they query first, then make modifications to entities and finally save them, discrepancies between query results and the contents of the ObjectSateManager cannot be observed.

But as soon as queries are interleaved with modifications there is a chance that the server won’t contain an entity that exist in the state manager only and that that would match the predicate of the query. Those entities won’t be returned as part of the query.

Notice that the chances that this happens has to do with how long lived is the Unit of Work in your application (i.e. how much does it take from the initial query to the call to SaveChanges).

Hope this helps,
Diego

EntityDataSource and Bind: What are those square brackets?

This post is about a small issue that I have seen in the forums and that arises often in cases in which EntityDataSource is used in combination with bound controls that use templates, like FormsView or a GridView with template based columns.

If you can create your page correctly, simply using the different drag & drop features and context menus provided by the design surface. However, in the end, some Bind expressions will get enclosed in square brackets that produce compile or run-time errors, then your code may look like this:

This annoyance may keep you blocked until you realize how simple the workaround is: just remove the square brackets!

People that watched my video here, may have noticed that I had to remove the square brackets manually around minute 8:10. I wish I had blogged about it then, but it wasn’t clear at the time I recorded that video that the issue would still be there in the RTM version.

Here is a short explanation:

In order to adapt entity objects that do not have foreign key properties to work better with ASP.NET databinding, we decided to wrap each entity returned with a databinding wrapper object that, among other things, flattens the members of EntityKeys in related entities (for more details on how and why the EntityDataSource returns wrapped entities you can go here).

The name of flattened reference keys are usually composed by the corresponding navigation property, plus the name of the property in the related end. For instance, in a Product, the key of the associated Category, will be exposed as “Category.CategoryID”. We decided to use dots to delimit the parts of the name because that plays well and is consistent with the use of dots in most programming languages, and also in Eval expressions, like the one used in the following code:

The problem is that for the particular case of Bind (often used in template based databound controls to perform two-way databinding) the design time component of ASP.NET will catch property names that contain dots, and it will try to escape them by surrounding them with square brackets (leading to ta Bind expression of the form “[Categories.CategoryID]”).

Unfortunately, the rest of the ASP.NET components which need to evaluate the binding expression are actually not capable of parsing the square brackets escaping notation (i.e. for DataBinder.Eval() the brackets actually indicate that a lookup in an indexed property is necessary, and there is no indexed property in this case).

Given that the EntityDataSource in fact exposes property descriptors that contain dots in their names, the escaping is unnecessary. So, next time you find this issue, just remove the square brackets.

The good news is that the issue will be fixed in the next release.

Hope this helps!

Exposing EDM and database server functions to LINQ

Alex published today a description Colin and I wrote on a new feature the team has been working on for LINQ to Entities.

Beyond all technicalities, it is a very simple and attribute-based way of exposing any arbitrary server-side function to LINQ. It goes beyond what LINQ to SQL does with SqlMethods and it leverages our metadata system so that you don't have to specify the full mapping of parameters in the attribute.

The post itself may be a little boring ;), but the scenarios it enables are quite impressive.

Read more here.

Update: I remembered today that Kati Dimitrova and Sheetal Gupta also contributed to the document.

Quick tips for Entity Framework Databinding

One of our customers asked this question yesterday on the Entity Framework forums. There were a few details missing and so I am not completely sure I got the question right. But I think it is about an issue I have heard quite a bit, and so I think it may be useful to share my answer here for others.

Given this simple line of code (i.e. in a WinForms application):

Several things will happening under the hood:

1. Databinding finds that ObjectQuery<T> implements the IListSource interface, then it calls IListSource.GetList() on it, to obtain a binding list (an internal object that implements IBindingList and some other well-known interfaces that databound controls know how to communicate with)
2. GetList() gets the query executed and it copies the contents of the resulting ObjectResult object to the binding list
3. The new binding list is finally passed to the databound control

When a binding list is created this way from a tracked query (i.e. MergeOption in the query is set to anything but NoTracking), there are several other interesting details in the behavior:

1. Changes made to entities in the binding list will get noticed by the state manger. Therefore changes will be saved to the database when SaveChanges is invoked on the tracking ObjectContext
2. Additions of new entities to the context will not result in the new entities being added automatically to a binding list (this is a common expectation). Whether an entity belongs to a binding list is decided at the time the binding list is created. If we wanted to do something different (i.e. have the binding list get new objects added to the context automatically), we would hit some considerable obstacles:
   • The binding list does not remember which filter condition was used in the query.
   • Even if it did, our queries are server queries, expressed in either ESQL or LINQ, but always translated to server queries and then expressed in terms of the current values on the server.
   • Even for LINQ queries, we cannot assume that the same query will have equivalent behavior while querying in-memory objects when compared to the same query translated to the native SQL of the database.
3. As a consequence of #2: You can have multiple binding lists based on the same EntitySet with overlapping or disjoint sets of entities. You can use different queries (or even from the same query, but with different parameters) to get a different set of entities in each binding list.
4. Deletion of entities from the context will result in the deleted entity being removed from all binding lists that contain it. The principle behind this is that the binding list is a window into the "current state of affairs" in the state manager. Even if we don't know if a new entity belongs into a binding list, we do know when it doesn't belong anymore, because it has been deleted.
5. A binding list has its own Add() method you can use. If you could get a reference to one of our binding lists you could “manually” add new entities to the binding list, and the entity you add will also be added to the context automatically, the same as if it had invoked  AddObject() on the context.

All these facts, especially #5, warrant me to give you a couple of tips:

1. If you are interested in getting a reference to the binding list directly (i.e. to use its Add method), you can do something like this:
   var bindingList = ((IListSource)query).GetList() as IBindingList;
   There is a more convenient ToBindingList() extension method included in the EFExtensions you may want to take a look at.
2. If you are not interested in getting the reference to the binding list for yourself, and you are using databinding directly against a query as in the sample at the top of this post, you should know that WinForms will call IListSource.GetList() twice, causing the query to be executed on the server twice! The recommendation then is to bind to the results, rather than the query, using code similar to this:
   grid.DataSource = someQuery.Execute(MergeOption.AppendOnly);
   In this case, since the resulting ObjectResult acts as a foward-only, enumerate-once cursor, its implementation of IListSource.GetList() is different from the one in the query: the call will consume the results (i.e. it will iterate through the whole results) and will cache the binding list in the ObjectResult instance. All subsequent calls to IListSource.GetList() will return the same binding list.

We have a few ideas around things we may improve in future versions. Some of them you may be able to guess from this explanation. But I will save that for another post...

Jarek publishes his excellent Entity Framework POCO Adapter

I have been back from vacation for some time but I haven't had time to post anything (i.e. I was on vacation the day Entity Framework went RTM in .NET 3.5 SP1!).

Finally, something happened that I cannot wait to talk about.

Jarek and I went discussing how persistence ignorant classes could be supported by building a layer on top of the EF v1 API for a long time.

He finally figured out a good way to do it. The result is this sample that wraps Entity Framework APIs and uses simple code-generation to create "adapters" and "proxies" for your POCO classes:

Entity Framework POCO Adapter

You can read Jarek's blog post explaining it all.

We hope people will find it interesting and will send useful feedback that can help us improve the core product.

Sample Entity Framework Provider for Oracle now Available

This new sample builds on top of System.Data.OracleClient and showcases some techniques a provider writer targeting databases different from SQL Server can use.

The code is not meant for production, just a sample directed to provider writers. It has also a few limitations related both to SP1 beta bits and with types not supported in OracleClient.

For more details, read Jarek's post.

You can download the source code from our home page in Code Gallery.

Entity Framework Sample Provider Updated for SP1 Beta

Just to get the news out: The updated version of the Entity Framework Sample Provider that is compatible with .NET 3.5 SP1 Beta is now available in our Code Gallery page. From the description:

The Sample Provider wraps System.Data.SqlClient and demonstrates the new functionality an ADO.NET Provider needs to implement in order to support the ADO.NET Entity Framework

• Provider Manifest
• EDM Mapping for Schema Information
• SQL Generation

Update: for more details on provider API changes since the Beta3 release, you can read Jarek's post here.

EntityDataSource's flattening of complex type properties

I explained a few days ago the rules of wrapping in this blog post. But why do we wrap after all?

Julie asked for some details today in the forums. I think the answer is worth of a blog post.

In ASP.NET there are different ways of specifying which property a databound control binds to: Eval() Bind(), BoundField.DataField, ListControl.DataTextField, etc. In general, they behave differently.

The flattening we did on EntityDataSource is an attempt to make the properties that are exposed by EDM entities available for 2-way databinding in most of those cases.

For instance, for a customer that has a complex property of type Address, we provide a property descriptor for customer.Address, and also for customer.Address.Street, customer.Address.Number, etc.

At runtime, in the case of a control binding to Eval(“Address.Street”) from a customer, Eval will use the property descriptor corresponding to Address, and it will drill down on it to extract the value of the Street property on it.

A grid column of a BoundField derived type with DataField = “Address.Street” will work differently: it will just look for a property descriptor in the data item with a name as “Address.Street”. In fact, EntityDataSource is the first DataSource control that I know off that will provide such a thing.

Bind(“Address.Street”) will work in a similar fashion to Eval() when reading the properties into the data bound control, but will act a little bit more like BoundField when sending back changes to the DataSource.

There are a few cases in which the behavior is not any of the above and hence you end up with a control that cannot have access to a complex type’s properties. You can expect us to work closely with the ASP.NET team in making the experience smoother in future versions. But for the time being, what you can do is create an explicit projection of the properties. For instance, in Entity SQL:

       SELECT c.ContactId, c.Address.Street AS Street 
       FROM   Northwind.Customers AS c

I think it is worthy of mentioning:

• Remember flattening of complex properties only happen under certain conditions (see wrapping).
• We worked very closely with the ASP.NET Dynamic Data in this release, to enable their technology to work EDM through the EntityDataSource. I think it is very worthy of trying.

Hope this helps.

Entity Framework Extensions Project Update

Just a couple of links:

Colin posted a refresh today today that is compatible with .NET 3.5 SP1 Beta and includes some optimizations for the materializer using dynamic methods. Here is his post about it.

EntityDataSource: To wrap or not to wrap

Note: Somehow the <TEntity> generic argument had disappeared from the extension method definition. I am fixing it today after several months. Sorry for the inconvenience!

Fresh from the forums today: A customer asks how to get the real entity object in the RowDataBound event of a GridView.

We made some complex design decisions during the development of the EntityDataSource, and I guess that will give us plenty of material for blogging :)

With the EntityDataSource, we faced the interesting problem of building a bridge between the existing ASP.NET databinding infrastructure and the world of EDM. One of the techniques we use to solve the problem is wrapping entities in smart databinding objects. As Colin explains it in his answer posted to the forums:

Why are you seeing a wrapper instead of the entity? Some of the unique features of the Entity Data Model prevent us from directly binding the entity. For instance, when inserting a Product I also need to insert a relationship to a Category. The wrapper adds the relationship to the entity, basically as a foreign key value.

The wrapper object implements the ICustomTypeDescriptor interface, which makes it work well with databinding, but when you try to get your original entity object, for instance form the arguments of the RowDataBound, you will get a wrapper object instead of the entity object you are expecting.

I don't want to make things much more complicated than they need to be, but I think the generic solution may be useful for some customers.

First of all, here are the rules for wrapping:

• The wrapping mechanism only takes place if you initialize your EntityDataSource using EntitySetName.

• When you instead set CommandText to a query that returns entities (i.e. "SELECT VALUE c FROM Northwind.Customers AS c", then you get normal entities.

• When you instead set CommandText to a query that returns a projection of properties (i.e. "SELECT c.CustomerID, c.CustomerName FROM Northwind.Customers AS c"), then you get a DbDataRecord.

• Finally, if you set the Select property to do a projection (i.e. "it.CustomerID, it.CustomerName", you get DbDataRecord regardless of how you start your query.

If you use the RowDataBound event very often in your code, then, I would suggest having around some code similar to this (thanks David for coming up with this code first):

And this is the usage:

New EntityDataSource Control for ASP.NET

As it was announced today, the EntityDataSource is now part of Entity Framework, and a first beta version of it is available in .NET 3.5 SP1 Beta 1 and Visual Studio SP1 Beta 1.

I have meant to answer Julie's post "Thinking about the EntityDataSource" for several months. But I always thought it was risky to talk about stuff that hadn't been released.

Now that customers can actually smell and touch it, it is about time:

Some history and acknowledgments

EntityDataSource was developed by a small subset of the Entity Framework Team in less than nine months, which is a short period for most software projects at Microsoft. It is amazing to see what this small team has produced.

But bringing it to its current state would have not been possible without the help of several members the ASP.NET team, who were always helpful and devoted their time and knowledge to the project. They helped us not only in making it suit the needs of “classic” ASP.NET developers, but also of ASP.NET Dynamic Data.

Why a new DataSource?

You may have already learned that LinqDataSource is an excellent alternative for displaying the results of any LINQ query (including LINQ to Entities) in ASP.NET, although, to update the database, you need a LINQ to SQL backend. Also, it is possible to write some code and get ObjectDataSource perform 2-way databinding against Entity Framework classes, but that dilutes one of the design goals for DataSources, that it should be possible to implement most common scenarios in a declarative only fashion.

Entity Framework includes a text based query language called EntitySQL, which in v1 is the only way to get access to the full query capabilities of EF. The language is very “composable”, which enables us to create queries using builder methods (similar to LINQ sequence operators, but text based). On the other side, DataSources as all ASP.NET controls can be initialized using string properties in markup.

It is easy to conclude that using EntitySQL, it should be possible to create a DataSource control that does 2-way databinding using only markup.

In fact, this simple markup produces a fully functional CRUD form for the Customers EntitySet:

    <asp:DetailsView ID="CustomerDetailsView"
        runat="server"
        AllowPaging="True"
        AutoGenerateRows="True"
        DataKeyNames="CustomerID"
        DataSourceID="CustomersDataSource">
    </asp:DetailsView>
    <asp:EntityDataSource ID="CustomersDataSource"    
        runat="server"
        ConnectionString="name=NorthwindEntities"
        DefaultContainerName="NorthwindEntities"
        EnableDelete="True"
        EnableInsert="True"
        EnableUpdate="True"
        EntitySetName="Customers">
    </asp:EntityDataSource>

More to follow...

Lazy loading in Entity Framework

Recently, I wrote this little article that got published in the new Insights sidebar in MSDN Magazine. In it, I mention one of the fundamental tenets of ADO.NET: 

*Network roundtrips should not be hidden from the developer*

But guess what... It is not always the case that there is a network (or even a process boundary) between your application and your database. Also, there are many scenarios in which you know that most of your data seldom changes or that you don't care if things changes a bit while your application is running (think of a cache). In those circumstances, implicit lazy loading just makes sense.

We have been sending out the message that you can get implicit lazy loading by changing the standard code generation process in Entity Framework.

My colleague Jarek went far further and created an experimental set of entity classes that completely replaces the default code-generated classes in EF. And his implementation actually includes some very cool ideas that go beyond lazy loading.

Take a look at his post. And you can find the bits (compatible with the just released Visual Studio 2008 SP1 Beta) from our portal in Code Gallery.

Update: Just wanted to add some relevant links to customers asking for lazing loading in EF:

• Jeremy Miller wants lazy loading for Christmas.
• Timothy Khouri does lazy loading.

Colin explains a simple LINQ to Relational materializer

Just a short note about this: You can find his article here. I had the chance to see his presentation before he went to DevConnections in Orlando. Very much recommended stuff!