Entity Framework Design : POCO

Self-Tracking Entities in the Entity Framework

efdesign — Tue, 24 Mar 2009 04:50:52 GMT

Background

One of the biggest pieces of feedback we received from the N-Tier Improvements for Entity Framework post as well as other sources was: “low level APIs are great, but where is the end-to-end architecture for N-tier and the Entity Framework?”. This post outlines some of the additional feedback we’ve received and describes the self-tracking entities architecture that will ship along side of VisualStudio 2010 and .NET Framework 4.0.

When an application goes multi-tier, there are important architectural decisions that the application developer makes around how to communicate between tiers. There are a lot of choices, and the choice depends on a variety of conditions. How rapidly is each tier expected to change? How much control is there over each tier? Are their specific protocol, security, or business policy concerns? The answers to these questions often drive the selection of whether to use data transfer objects (DTOs) or DataSets or something different altogether.

Self-Tracking Entities

Self-tracking entities know how to do their own change tracking regardless of which tier those changes are made on. As an architecture, self-tracking entities falls between DTOs and DataSets and includes some of the benefits of each.

Drawing from DataSet

DataSet on the client tier is very easy to use because there is no need to track changes separately or maintain any extra data structures that include change tracking information. DataSet takes care of serializing state information for each row of data. On the mid tier, applying the changes stored within a DataSet is straightforward. DataSets have also gained popularity because of the number of tools that work with DataSets, and because they are easily bound to many UI/presentation controls. Since it works in so many scenarios, for many applications there is never a need to transform data outside of a DataSet allowing a single paradigm to be used up and down the stack.

However, there are disadvantages to DataSets when used as the communication payload between tiers. The first is the cost of getting the data into a serializable format, and the second is that the DataSet serialization format is generally not very interoperable with other languages that are used to expose services. Another disadvantage of using a DataSet is that you can quickly lose the intent of your service call because so many kinds of things can be included in a DataSet. For example, if the service method is declared as:

DataSet GetCustomer(string id);

It is extra work to ensure that the DataSet that is returned contains only rows of data that pertain to “customers”. It is also more difficult to specify a service contract that says the DataSet is also supposed to return data for each customer’s orders and their order details.

Self-tracking entities share many advantages with DataSet. They also encapsulate change tracking information, which is serialized along with the data contained in the entity. On the mid-tier applying changes from a graph of self-tracking entities to a persistent context is equally straightforward. The Entity Framework will also provide tools for generating self-tracking entities and because these entities are just objects, they can be easily made to work with UI/presentation controls.

Drawing from DTOs

DTOs and SOA are used to give the developer more control over the service contract and payload for tier to tier communication. DTOs themselves do not have behavior, so they are typically very simple classes designed just to provide the needed information to perform a specific service operation. Not only does this provide the opportunity to optimize the wire format (some believe a DTO is all about the wire format), but it makes it possible and easy to capture the intent of each service method. The data contract used with DTOs is typically interoperable which makes it easy to use services that run on different platforms. DTOs also provide a way to separate messaging contracts from the presentation layer, the business logic, and the persistence layer which in many cases creates a maintainable architecture.

There are disadvantages to using DTOs and the primary one is complexity. DTOs are often hand-crafted to include only the specific information that is needed for an operation. When there is a common pattern for mapping DTOs to entity classes, there are some tools available that will do DTO generation and mapping but it is not always possible. With DTOs, it is up to the developer to decide how to do change tracking on each tier (especially the client) which increases the complexity of the presentation layer. The complexity of the service implementation also increases because the developer is responsible for translating the DTO into entities for doing business logic validation, as well as being able to report changes stored in the DTO to the persistence framework.

Not only do self-tracking entities share advantages with DataSets as mentioned above, they also share many advantages with DTOs. Self-tracking entities expose a simple and interoperable wire format and it is clear what kinds of data a service method requires or returns. You can also use a self-tracking entity as part of a message. However, self-tracking entities don’t give quite as much architectural separation as using pure DTOs, but you do gain a less complex solution that requires fewer data transformations. It is important to note that self-tracking entities can be made to be ignorant of any particular persistence framework making them essentially POCO objects. Particular persistence frameworks such as the Entity Framework will have the capabilities to create these entities and interpret the change tracking information when saving changes.

.NET Framework 3.5SP1 Challenge

With the .NET Framework 3.5SP1, this sort of solution was very hard to implement using the Entity Framework because change tracking was always done by a centralized ObjectContext which contains an ObjectStateManager. In particular, “reattaching” to report changes back to the ObjectStateManager was all but impossible without completely shredding your entity graph and applying changes one entity and one relationship at a time. The new API changes that are being added to the Entity Framework in .NET Framework 4.0 are enablers for an easier experience of reporting changes back to the ObjectStateManager, making self-tracking entities (as well as other architectures) easier to build.

Mid-tier experience

Using self-tracking entities on the mid-tier is about working with entity graphs and the Entity Framework. The service contract that is used in the following examples contains two simple methods for retrieving a Customer entity graph and applying updates to that entity graph:

interface ICustomerService
{
Customer GetCustomer(string customerID);
bool UpdateCustomer(Customer customer);
}

The implementation of the GetCustomer service method can be done using an Entity Framework ObjectContext and using LINQ or query builder methods to retrieve the entity you want. In the example below, a query is issued for a particular customer entity and all of the Orders and OrderDetails are included.

public Customer GetCustomer(string customerID)
{
    using (NorthwindEFContext context = new NorthwindEFContext())
    {
        var result = context.Customers.
              Include("Orders.OrderDetails").
              Single(c => c.CustomerID == customerID);
        return result;
    }
}

UpdateCustomer is example of how to save the changes that are made to a graph of self-tracking entities. Similar to DataSet, applying these changes to the persistence layer and saving them should be simple. In the below example, a new Entity Framework API, “ApplyChanges” is used which understands how to interpret the change tracking information that is stored by each entity and how to tell the ObjectContext’s ObjectStateManager about those changes.

public void UpdateCustomer(Customer customer)
{
    using (NorthwindEFContext context = new NorthwindEFContext())
    {
        context.Customers.ApplyChanges(customer);
        context.SaveChanges();
    }
}

Client Experience

The client experience when working with self-tracking entities is similar to how you would manipulate any object graph. You can make changes to scalar or complex properties, add or remove references, and add or remove from collections of related entities. The key part of the experience is that tracking changes is hidden from the client because it is done internally on each entity. There is no ObjectContext and there is no extra state that has to be maintained or passed from client to the tier that does persistence.

In this example, the test first queries for a customer, then deletes one of the orders. The resulting entity graph is sent back to the service tier using the UpdateCustomer method.

public void DeleteObjectsTest()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var customer = client.GetCustomer("ALFKI");
        customer.Orders.First().Delete();
        client.UpdateCustomer(customer);
    }
}

The next test shows how to add a new Order with two OrderDetails to an existing customer. By default, the constructor of a self-tracking entity puts the entity in the “Added” state. There are conveience methods on each self-tracking entity to change this state if needed (Delete() and SetUnchanged()).

public void AddObjectsTest()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var customer = client.GetCustomer("ALFKI");
        customer.Orders.Add(new Order() {
            OrderID = 100,
            OrderDetails = {
                new OrderDetail{
                     ProductID = 3,
                     Quantity = 7
                },
                new OrderDetail{
                     ProductID = 4,
                     Quantity = 8
                },
            }
        });
        client.UpdateCustomer(customer);
    }
}

The final example shows that a self-tracking entity graph can contain any number of changes and those changes can be any combination of adds, modifications, and deletes.

public void ComplexModificationsTest ()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var cust = client.GetCustomer("ALFKI");
        // remove first order - will do cascade in the database
        cust.Orders.Remove(cust.Orders.First());
        // modify one of the orders
        var order = cust.Orders.Last();
        order.RequiredDate = DateTime.Now;
        order.OrderDetails.Add(
            new OrderDetail{
                ProductID = 7,
                Quantity = 3
        });
       // add new order
        cust.Orders.Add(new Order()
        {
            OrderID = 100,
            OrderDate = DateTime.Now,
            ShipCity = "Redmond",
            ShipAddress = "One Microsoft Way",
            ShipRegion = "WA",
            ShipPostalCode = "98052",
            OrderDetails = {
                new OrderDetail{
                    ProductID = 3,
                    Quantity = 7
                },
                new OrderDetail{
                    ProductID = 4,
                    Quantity = 8
                },
            }
        });
        client.UpdateCustomer(cust);
    }
}

Generating Self-Tracking Entities
Building self-tracking entities should be as easy as using them. We plan to ship a T4 template that will do the code generation for creating self-tracking entities from your EDM. This template will show up in the list of available templates when you right click on your model and choose Add New Artifact Generation Item.

Design Notes

Inside a Self-Tracking Entity

There have not been final decisions about what exactly will be included inside of a self-tracking entity, but it is important to track:

The state of the entity, Added, Deleted, Modified, or Unchanged
The original value from reference relationship properties
Adds and removes from collection relationship properties

This information will be included with the data contract of the entity. As part of the code generation of each self-tracking entity, scalar and complex property changes will mark the entity as “dirty” meaning that its state will change from Unchanged to Modified.

Inside ApplyChanges

ApplyChanges is a new API on the ObjectContext and ObjectSet classes that attaches an entity graph and interprets the change tracking information stored in each entity. The design for discovering the change tracking information has not yet been finalized, but there are a couple of options available. One would be to code generate an interface as part of the T4 template for self-tracking entities as well as a version of ApplyChanges that knew about the interface. Another option would be fall back to runtime discovery of the particular properties using a convension and some of the new dynamic capabilities of the framework. The thing we want to avoid is causing the self-tracking entity implementation to have a dependency on any of the Entity Framework assemblies.

The algorithm that ApplyChanges uses is:

1. Attaching the entity graph to the ObjectContext

2. Changing the state of each entity using the ChangeObjectState API.

3. For any reference relationship property, if there is an original value changing the state of the original reference relationship to Deleted and change the state of the current reference relationship to Added. This is done using the ChangeRelationshipState API.

4. For any collection relationship property, use the ChangeRelationshipState API to mark any removed relationships as Deleted and to mark any added relationships as Added.

Summary

We’ve received a lot of feedback and suggestions on how to make developing multi-tier applications easier using the Entity Framework. One of the components of improving this experience is the introduction of an end-to-end architecture around self-tracking entities. We’d like to hear your feedback on this addition to the Entity Framework, and any other comments you have on the matter of multi-tier development using domain models and entities.

Jeff Derstadt,
Dev Lead, Entity Framework Team, Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

N-Tier Improvements for Entity Framework

efdesign — Thu, 20 Nov 2008 08:33:00 GMT

The first version of Entity Framework provides convenient ways to load, manipulate and persist objects and relationships. As with many other O/RMs, Entity Framework has a state manager that tracks every change made. Existing objects are typically loaded first from the database, later modified, and finally the changes are saved back to the store.

Another feature, full graph serialization, makes it very easy for developers to ship around object graphs representing snapshots of the current state of the world, across execution boundaries.

The next version of Entity Framework will also support Persistence Ignorance. Therefore object graphs can now be made of POCO instances, which WCF now also supports.

Nonetheless, there is a task that belongs in any N-Tier application that still requires a great amount of work for developers using EF: decoding messages that represent state changes performed by the client, meant to be processed or persisted by the service.

The major pain point in the first version is the number of intricate steps a program needs to perform in order to setup the state manager in the appropriate state according to the changes encoded in the message. For instance, for any but the simplest scenarios, in order to use the basic graph manipulations APIs in ObjectContext, like AddObject and AttachTo, it is necessary to “shred” the object graphs contained in the message, which destroys important information about relationships between object instances that therefore has to be kept somewhere else (as an illustration of this approach, see Danny Simmons’s EntityBag in project Perseus).

In addition, given that the message typically contains all the necessary information and that concurrency is usually a concern, the program shouldn’t have to re-query the database. Instead, it should be easier to “tell” the state manager about the original state (that came from the database previously) and about what changes were made.

There are a few approaches to N-Tier that are very popular in the industry. Customers using Entity Framework for the first time often carry the expectation that it will support a similar experience:

DataSet

The original ADO.NET data access technology includes the DataSet, a versatile data cache that can be serialized as XML and be used to transport collections of rows and relationships and, also diffgrams representing changes.

DataSets make many N-Tier scenarios extremely easy to implement, and over time, have proven helpful for many customers. In spite of this, they are not appropriate for some scenarios and architectural patterns:

The lack of a widely accepted standard serialization format has hindered the development of implementations that could produce or consume datasets in other platforms.
Diffgrams represent sets of CREATE, UPDATE and DELETE operations of arbitrary depth, which is not adequate for architectural patterns that conceive operations with more constrained semantics (i.e. PlaceOrder, CreateCustomer).
No simple ways to control the application of batch CUD operations contained in diffgrams makes them unsuitable any time there is a trust boundary between the client and the server.
Also, the data-centered perspective of DataSets leads to an anti-pattern known as the “anemic domain model”, which is characterized by a separation between the data aspect and the behavioral aspect of domain objects.

Data Transfer Objects

The approach typically used in the DDD and SOA communities to address multi-tier scenarios is to hand-build a service with well defined operation semantics together with the a conceptual message format, usually composed of data transfer objects (DTOs). Implementing DTOs often also involves writing the logic that translates DTOs back and forth to persistent objects.

Here is a simplified set of rules that characterizes the use of DTOs:

Actual entities are not serialized, just DTOs, which are value-only objects (without behavior) are exposed in the boundaries.
The client does not depend on the types defined on the server. The shapes of the DTOs are the sole contract, and the types used in the client are for the client only.
The context or container is not sent together with the message.
The service decides what to do with the contents of the message. It is not the message who decides (as an extreme counter example, imagine a message that tells the service to delete all the contents of the database).

Those rules are especially important in SOA scenarios, in which the client and the server can be separated by a trust boundary, such as the one that exists between two different companies.

There are other permutations of layered architectures, though, in which the level of decoupling that DTOs provide may not justify the increased complexity, for example:

A multi-tier application in which the client is fully trusted.
A rich internet application in which an infrastructure service is purely used for data persistence (is by design a CRUD-only service) and therefore types exposed do not contain behavior anyway.
Any architecture in which the persistence framework is actually used to provide and manage DTOs and not the actual entities.

REST

ADO.NET Data Services provides a rich framework that developers can easily use to expose any Entity Data Model as a REST-style resource collection. Simple HTTP verbs like GET, PUT, POST and DELETE are used in combination with URIs to represent CRUD operations against the persistence service.

This pattern is suitable for many applications and can easily be combined with service operations with richer semantics in the same application.

ADO.NET Data Services, however, may not be compelling for those customers who want to define their services as operations with richer semantics entirely.

Goals

ADO.NET Data Services goes a long way addressing the needs of customers doing RESTful services, and so there is no need for us to improve Entity Framework experience on that space.

With the N-Tier improvements for Entity Framework, we want to address some of the same problem space as DataSet, but we want to avoid the primary issues with it.

Ideally, we would like to provide building blocks that are appealing for developers building solutions on a wide range of architectures. For instance, we would like to provide fine enough control for DTO proponents, but at the same time reduce the level of pain that those trying to address simpler scenarios experience today.

Design

If we just wanted to provide a DataSet like experience on top of Entity Framework, the most straightforward (although very expensive to test) way would probably be based on full serialization of the state manager. Basically, the internal representation of ObjectStateManager includes collections of objects to be deleted, inserted, and modified, and also relationships to be added and deleted, as well as objects and relationships that should remain unchanged.

Such a model would lead to very fine grain control of the state manager, but you would lose some of the benefits of operating on the domain objects and the higher level APIs. For instance, it would make it extremely easy for the program to set the state manager in a completely invalid state.

An alternative representation for such an experience would be to ship a “change log” containing the history of all operations performed on the client. This representation has the disadvantage of being more verbose than the previous one, but it would make “point in time” rollbacks pretty easy to implement.

Besides these two, there are a few more interesting generic representations for changes in a graph, but in general, they all suffer from the same disadvantage: providing a solution for them does not give the user the level of control that the most complicated scenarios and sophisticated patterns require.

Based on this fact, we decided to adopt the following goal for the design:

Entity Framework won’t define its own unique representation for the set of changes represented in an N-Tier application. Instead, it will provide basic building block APIs that will facilitate the use of a wide range of representations.

This has the desirable consequence that Entity Framework won’t impose a pattern for N-Tier. Both DTO-style and DataSet-like experiences can be built on top of a minimal set of building blocks APIs. It is up to the developer to select the pattern that better suits the application.

Here is a first cut of the new methods that we are proposing to add to the ObjectContext. The new APIs will work with EntityObjects, IPOCO and POCO entities:

public partial class ObjectContext 
{ 
    /// <summary> 
    /// Apply property values to original state of the entity 
    /// </summary> 
    /// <param name="entitySetName">name of EntitySet of root</param> 
    /// <param name="original">entity with original values</param> 
    public void ApplyOriginalValues( 
        string entitySetName, 
        object original) { } 
    
    /// <summary> 
    /// Changes the state of the entity and incident relationships 
    /// </summary> 
    /// <param name="entity">entity to change state of</param> 
    /// <param name="state">new state</param> 
    public void ChangeObjectState( 
        object entity, 
        EntityState state) { } 

    /// <summary> 
    /// Changes state of relationship 
    /// </summary> 
    /// <param name="source">source object of relationship</param> 
    /// <param name="target">target object of relationship</param> 
    /// <param name="relationshipName">name of relationship</param> 
    /// <param name="sourceRole">role of source object</param> 
    /// <param name="targetRole">role of target object </param> 
    /// <param name="state">new state</param> 
    public void ChangeRelationshipState( 
        object source, 
        object target, 
        string relationshipName, 
        string sourceRole, 
        string targetRole, 
        EntityState state) { } 

    /// <summary> 
    /// Changes state of relationship represented by navigation 
    /// property 
    /// </summary> 
    /// <param name="source">source object of relationship</param> 
    /// <param name="target">target object of relationship</param> 
    /// <param name="navigationProperty">navigation property</param> 
    /// <param name="state">new state</param> 
    public void ChangeRelationshipState( 
        object source, 
        object target, 
        string navigationProperty, 
        EntityState state) { } 
    
    /// <summary> 
    /// Changes state of relationships represented by lambda 
    /// expression 
    /// </summary> 
    /// <typeparam name="TSource">type of source entity</typeparam> 
    /// <param name="source">source entity of relationship</param> 
    /// <param name="target">target entity of relationship</param> 
    /// <param name="selector">property selector expression</param> 
    /// <param name="state">new state</param> 
    public void ChangeRelationshipState<TSource>( 
        TSource source, 
        object target, 
        Expression<Func<TSource, object>> selector, 
        EntityState state) { } 
  
} 

ApplyOriginalValues

Similar to the existing ApplyPropertyChanges, but this API keeps the tracked entity intact, only affecting the corresponding original values. This operation only works for entities in Modified, Unchanged or Deleted states. Added entities by definition don’t have original state.

ChangeObjectState

This method will transition the state of a tracked entity passed as argument, but will also have side-effects on incident relationships. In case the new state is modified, it will also mark all properties as modified, regardless of the original and current values.

ChangeRelationshipState

Similar to ChangeObjectState, but sets the relationship between two tracked entities to the provided state. Relationships cannot be in the modified state, but this method can be used to report which relationship should be added, deleted, or unchanged.

The relationship doesn’t need to exist in the context, but the entities do. If the relationship doesn’t exist, it can be created in the new state. For instance, when this code is invoked, a new relationship can be created between customer1 and order1 in the added state:

context.ChangeRelationship( 
    customer1, 
    order1, 
    c=>c.Orders, 
    EntityState.Added); 

The relationship is typically represented by the navigation property but the metadata names of the relationship and roles can also be used in a different overload.

Design notes

All parameters named “entity”, “source” and “target” accept either entities or EntityKeys.
We could choose to deprecate the ApplyPropertyChanges API and replace it with ApplyCurrentValues, which is more consistent name-wise with the new ApplyOriginalValues method.
For convenience, we should add similar APIs in other types like ObjectStateEntry, RelatedEnd and others as appropriate. For instance, the ObjectStateEntry will also have a ChangeState API that will only take the target state parameter.
We could choose to put ChangeObjectState() and the various ChangeRelationshipState() in ObjectStateManager rather than in ObjectContext, since this is a lower level API than everything else currently in the ObjectContext.

Scenario tests

To get a sense of how the new API can be used, we have tried several different approaches and representations. We also have plans to release sample code that will show in more detail different ways to use the new API. In the meanwhile, here are a few simple cases.

General purpose Attach with state resolution delegate

In this scenario between a client and mid-tier, the intention is to allow a client to query for a Customer entity with the Customer’s Order entities. The client then makes a variety of changes to the Customer entity graph including modifying the Customer, and adding or removing Orders. The client would like to send the updated Customer entity graph to the mid-tier so that it can be processed and persisted.

This system will use a simple mechanism to help with change tracking on DTO entities. Each DTO will implement a small interface to help determine the state of the entity using three properties: IsNew, IsModified, and IsDeleted. In this example, original values are not tracked by the client.

public interface IEntityWithChanges 
{ 
    bool IsNew { get; } 
    bool IsModified { get; } 
    bool IsDeleted { get; } 
} 

When the client makes changes to the Customer entity graph, the only requirement in this scenario is that when an entity changes, the appropriate property will return true to help determine the state of the entity.

The mid-tier will expose a service method that implements the logic to update the Customer entity graph. Rather than traverse the Customer, her Orders and OrderLines looking for changes, the service method will use a convention based mechanism that knows how to deal with the IEntityWithChanges interfaces. In this scenario, this mechanism takes the form of an extension method to the ObjectContext that will attach an entity graph to the context, and use a delegate to map each entity that implements IEntityWithChanges to an EntityState so that it can be used with the new ChangeObjectState APIs.

First we can define a method to map from an IEntityWithChanges to an EntityState:

/// <summary> 
/// A mapping from an IEntityWithChanges to an EntityState 
/// </summary> 
public EntityState GetEntityState(object entity) 
{ 
    IEntityWithChanges entityWithChanges = 
       entity as IEntityWithChanges; 
    if (entityWithChanges != null) 
    { 
        if (entityWithChanges.IsNew) 
        { 
            return EntityState.Added; 
        } 
        else if (entityWithChanges.IsModified) 
        { 
            return EntityState.Modified; 
        } 
        else if (entityWithChanges.IsDeleted) 
        { 
            return EntityState.Deleted; 
        } 
    } 
    return EntityState.Unchanged; 
} 

Next we will define the extension method on an ObjectContext that can attach an entity graph and use a mapping delegate to determine the state of each entity in the graph.

/// <summary> 
/// Attach a graph of entities using a supplied mapping function to 
/// determine the entity's state. 
/// In this example, the state of relationships between entities is 
/// determined by the rules of ChangeObjectState: 
/// - if one entity in the relationship is Added, the relationship is 
///   Added 
/// - if an entity is Deleted or Detached, all incident relationships 
///   are Deleted or Detached 
/// </summary> 
/// <param name="entitySetName">name of EntitySet of root</param> 
/// <param name="entityGraph">graph of entities to attach</param> 
/// <param name="entityStateMap">EntityState resolver</param> 
public static void Attach( 
    this ObjectContext context, 
    string entitySetName, 
    object entityGraph, 
    Func<object, EntityState> entityStateMap) 
{ 
    context.AttachTo(entitySetName, entityGraph); 

    // Create a list of unique entities in the graph 
    var allEntities = 
        context 
        .ObjectStateManager 
        .GetObjectStateEntries(EntityState.Unchanged) 
        .Where(entry => !entry.IsRelationship) 
        .Select(entry => entry.Entity); 
        
    // Apply the entityStateMap to each entity 
    foreach (object entity in allEntities) 
    { 
        context.ChangeObjectState(entity, entityStateMap(entity)); 
    } 
} 

Finally we can define our service method that can update a Customer entity graph using the mapping delegate and the Attach extension method.

/// <summary> 
/// A service method to update a Customer's entity graph which 
/// can include modifications to the Customer entity, 
/// new Orders, and removal of Orders 
/// </summary> 
/// <param name="c"></param> 
public void UpdateCustomerOrder(Customer customer) 
{ 
    using (NorthwindEntities context = new NorthwindEntities()) 
    { 
        // Call the Attach extenion method with the mapping delegate 
        // that goes from IEntityWithChanges to EntityState 
        context.Attach("Customers", customer, GetEntityState); 
        context.SaveChanges(); 
    } 
} 

ChangeOrder Service Operation

In this scenario, the mid-tier exposes a method to perform a very specific operation: changing the Customer that owns an Order. The client can set the Order’s Customer to a new Customer or to one that already exists in the store. In either case, the service method uses the old Customer so that it can remove the relationship between the Order and that Customer.

/// <summary> 
/// Updates an Order by changing the relationship between the Order 
/// and a Customer in this case 
/// </summary> 
/// <param name="updatedOrder">order to update</param> 
/// <param name="oldCustomer">previous customer</param> 
public static void ChangeOrder( 
    Order updatedOrder, 
    Customer oldCustomer) 
{ 
    using (NorthwindEntities context = new NorthwindEntities()) 
    { 
        // Tell the context about the entities to persist 
        context.AttachTo("Orders", updatedOrder); 
   
        // Change the state of the new customer 
        // In this example, this is done by the convention: 
        // - Customer.CustomerID is null --> customer is 'Added' 
        // - Customer.CustomerID is not null --> it already exists 
        if(updatedOrder.Customer.CustomerID == "") 
        { 
            // When state of customer changes to 'Added', state 
            // of the relationship between updatedOrder and 
            // pdatedOrder.Customer is also changed to 'Added' 
            context.ChangeObjectState( 
                updatedOrder.Customer, 
                EntityState.Added); 
        } 
        else 
        { 
            if (updatedOrder.Customer.CustomerID != 
                oldCustomer.CustomerID) 
            { 
                // only relationship needs to be marked as 'Added' 
                context.ChangeRelationshipState( 
                    updatedOrder, 
                    updatedOrder.Customer, 
                    o => o.Customer, 
                    EntityState.Added); 

                // Report removal of relationship between 
                // pdatedOrder and oldCustomer 
                context.ChangeRelationshipState( 
                    updatedOrder, 
                    oldCustomer, 
                    "Customer", 
                    EntityState.Deleted); 
             } 
        } 
        
        // Persist the changes to the store 
        context.SaveChanges(); 
    } 
} 

As is always the case, your feedback on this topic is welcome!

Jeff Derstadt
Jaroslaw Kowalski
Diego Vega
and The Object Services Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Discussion about API changes necessary for POCO:

efdesign — Fri, 01 Aug 2008 19:24:00 GMT

Evolving an API to support new requirements, like POCO, while maintaining backward compatibility is challenging.

The following design discussion from members of the Object Services team illustrates some of the issues and hard calls involved.

Have a read, and tell us what you think.

In particular are we missing something, or overstating the importance of something? Let us know...

Anyway over to Diego and Mirek...

POCO API Discussion: Snapshot State Management

What is in a snapshot?

In Entity Framework v1 there is a single way for the state manager to learn about changes in entity instances: the change tracking mechanism is set in such a way that the entity instances themselves notify the state manager of any property change.

This works seamlessly if you use default code-generated classes, and it is also part of the IPOCO contract for developers willing to create their own entity class hierarchies.

For version 2, we are currently working on a snapshot-based change tracking mechanism that removes the requirement from classes to send notifications, and enables us to provide full POCO support.

The basic idea with snapshots is that when you start tracking an entity, a copy of its scalar values is made so that at a later point – typically but not always at the moment of saving changes into the database - you can detect whether anything has changed and needs to be persisted.

The challenge

When we created the v1 API, we made a few assumptions that were totally safe with notification-based change tracking but don’t completely prevail in a snapshot change tracking world.

We now need to choose the right set of adjustments for the API for it to gracefully adapt to the new scenarios we want to support.

Mainline scenario: SaveChanges() Method

In notification-based change tracking, by the time SaveChanges is invoked, the required entity state information is ready to use.

With snapshot thought, a property by property comparison needs to be computed for each tracked entity just to know whether it is unchanged or modified.

Once the snapshot comparison has taken place, SaveChanges can proceed normally.

In fact, assuming the process is triggered implicitly on every call to SaveChanges, a typical unit of work in which the program queries and attaches entities, then modifies and adds new entities, and finally persists the changes to the database, works unmodified with POCO classes:

Category category = context.Categories.First();

category.Name = "some new name"; // modify existing entity

Category newCategory = new Category(); // create a new entity

newCategory.ID = 2;

newCategory.Name = "new category";

context.AddObject("Categories", newCategory); // add a new entity

context.SaveChanges(); // detects all changes before saving

Things get more complicated when a user wants to use lower level APIs that deal with state management.

State Manager Public API

ObjectStateManager and ObjectStateEntry classes comprise the APIs that you need to deal with if you want to either get input from, or customize the behavior of Entity Framework’s state management in your own code.

Typically, you use these APIs if you want to:

Query for objects that are already loaded into memory
Manipulate the state of tracked entities
Validate state transitions or data just before persisting to the database
Etc.

As the name implies, ObjectStateManager is Entity Framework’s state manager object, which maintains state and original values for each tracked entity and performs identity management.

ObjectStateEntries represent entities and relationships tracked in the state manager. ObjectStateEntry functions as a wrapper around each tracked entity or relationship and allows you to get its current state (Unchanged, Modified, Added, Deleted and Detached) as well as the current and original values of its properties.

Needless to say, the primary client for these APIs is the Entity Framework itself.

ObjectStateEntry.State Property

The fundamental issue with snapshot is exemplified by this property.

With notification-based change tracking, the value for the new state is computed on the fly on each notification, and saved to an internal field. Getting the state later only encompasses reading the state from the internal field.

With snapshot, the state manager no longer gets notifications, and so the actual state at any time depends on the state when the entity began being tracked, whatever state transitions happened, and the current contents of the object.

Example: Use ObjectStateEntry to check the current state of the object.

Category category = context.Categories.First();

category.Name = "some new name"; // modify existing entity

ObjectStateEntry entry = context.ObjectStateManager.GetObjectStateEntry(category);

Console.WriteLine("State of the object: " + entry.State);

Question #1: What are the interesting scenarios for using the state management API in POCO scenarios?

Proposed solutions

In above example there are two possible behaviors and it's not obvious for us which one is better:

Alternative 1: Public ObjectStateManager.DetectChanges() Method

In the first alternative, computation of the snapshot comparison for the whole ObjectStateManager is deferred until a new ObjectStateManager method called DetectChanges is invoked. DetectChanges would iterate through all entities tracked by a state manager and would detects changes in entity's scalar values, references and collections using a snapshot comparison in order to compute the actual state for each ObjectStateEntry.

In the example below, the first time ObjectStateEntry.State is accessed, it returns EntityState.Unchanged. In order to get a current state of the "category" entity, we would need to invoke DetectChanges first:

Category category = context.Categories.First();

category.Name = "some new name"; // modify existing entity

ObjectStateEntry entry = context.ObjectStateManager.GetObjectStateEntry(category);

Console.WriteLine("State of the object: " + entry.State); // Displays "Unchanged"

context.ObjectStateManager.DetectChanges();

Console.WriteLine("State of the object: " + entry.State); // Displays "Modified"

DetectChanges would be implicitly invoked from within ObjectContext.SaveChanges.

Pros:

· User knows when detection of changes is performed

· DetectChanges is a method that user would expect to throw exceptions if some constraint is violated

· This alternative requires minimal changes in Entity Framework current API implementation

Cons:

· Since DetectChanges iterates through all the ObjectStateEntries, it is a potentially expensive method

· User has to remember to explicitly call DetectChanges() before using several methods/properties, otherwise, their result will be inaccurate:

o ObjectStateEntry.State

o ObjectStateEntry.GetModifiedProperties()

o ObjectStateManager.GetObjectStateEntries()

· This alternative implies adding a new method to ObjectStateManager

Alternative 2: Private ObjectStateEntry.DetectChanges() Method

In the second alternative, there is no public DetectChanges method. Instead, the computation of the current state of an individual entity or relationship is deferred until state of the entry is accessed.

Pros:

· User doesn't have to remember explicitly calling DetectChanges to get other APIs to work correctly

· Existing API works as expected in positive cases regardless of notification-based or snapshot tracking

· No new public API is added

Cons:

· The following methods now require additional processing to return accurate results in snapshot:

o ObjectStateEntry.State

o ObjectStateEntry.GetModifiedProperties()

o ObjectStateManager.GetObjectStateEntries()

· Existing API works differently in negative cases in notification-based and snapshot tracking:

o some of the existing methods would start throwing exceptions

o would require to introduce a new state of an entry – Invalid state (see below for details)

Question #2: What API pattern is better? Having an explicit method to compute the current state based on the snapshot comparisons or having the state to be computed automatically when accessing the state?

How this affects other APIs

While the State property exemplifies the issue, there are other APIs that would have different behaviors with the two proposed solutions.

ObjectStateEntry.GetModifiedProperties() Method

GetModifiedProperties returns the names of the properties that have been modified in an entity. Similar to the State property, with notification-based change tracking, an internal structure is modified on the fly on each notification. Producing the list later on only encompasses iterating through that structure.

With snapshot, the state manager no longer gets notifications, and at any given point in time, the actual list of modified properties really depends on a comparison between the original value and the current value of each property.

Therefore, for alternative #1, this APIs will potentially return wrong results unless it is invoke immediately after DetectChanges. For alternative #2, the behavior would be always correct.

ObjectStateManager.GetObjectStateEntries()

This is the case in which an implementation detail that was a good idea for notification-based change tracking stops offering performance benefits in snapshot. Internally, ObjectStateManager stores ObjectStateEntires in separate dictionaries depending on their state. But in snapshot, any unchanged or modified entity can become deleted because of a referential integrity constraint, and any unmodified entity can become modified.

In alternative #1, DetectChanges would iterate thought the whole contents of the ObjectSateManager, and thus it would update the internal dictionaries. Once this it done, it becomes safe to do in-memory queries using GetObjectStateEntires the same way it is done today.

In alternative #2, GetObjectStateEntires would need to look in unchanged, modified and deleted storage when asked for deleted entries; also in unchanged and modified storage when asked for modified entities.

Querying with MergeOption.PreserveChanges

In Entity Framework, MergeOptions is a setting that changes the behavior of a query with regards to its effects on the state manager. Of all the possibilities, PreserveChanges requires an entity to be in the Unchanged state in order to overwrite it with values coming from the database. In order for PreserveChanges to work correctly, accurate information on the actual state of entities is needed.

Therefore, with alternative #1, querying with PreserveChanges will not have a correct behavior unless it is done immediately after invoking DetectChanges. For alternative #2, querying with PreserverChanges would behavior correctly at any time.

Referential Integrity constraints

When there is a referential integrity constraint defined in the model, a dependent entity may become deleted if the principal entity or the relationship with the principal entity becomes deleted.

For alternative #1, DetectChanges would also trigger Deletes to propagate downwards throw all referential integrity constraints.

For alternative #2, finding out the current state of an entity that is dependent in a RIC would requires traversing the graph upwards to find if either the principal entity or a relationship has been deleted.

Mixed Mode Change Tracking

At the same time we are working on snapshot change tracking, we are also working on another feature, denominated dynamic proxies. This feature consists of Entity Framework automatically creating derived classes of POCOs that override virtual properties. Overriding properties enables us to inject automatic lazy loading on property getters, and notifications for notification-based change tracking in property setters. This introduces a subtle scenario: when using proxies, it is possible that:

a. Not all properties in the class are declared virtual. The remaining properties need to still be processed using snapshot.

b. Sometimes, non-proxy instances of POCOs and proxy instances of the same type have to coexist in the same ObjectStateManager. For the former, it will have to use snapshot tracking. For thelatter, a combination of snapshot and notifications.

All in all, it becomes clear that the actual state of an entity does not entirely depend on the value of the internal state field, nor on the snapshot comparison, but on a combination of both.

ObjectStateEntry.SetModified()

As with mixed mode change tracking, SetModified() requires a combination of the internal state and the snapshot comparison to return valid results.

Handling of invalid states

When working in notification-based change tracking, Entity Framework throws exceptions as soon it learns that entity key values has been modified. With the default code-generated classes, the exception prevents the change for being accepted.

For alternative #1, DetectChanges can throw exceptions if some model constraint (i.e. key immutability) is violated. It is too late to prevent the value from changing.

Example:

Category category = context.Categories.First();

category.ID = 123; // Modify a key property. This would throw if category wasn't a POCO class.

context.ObjectStateManager.DetectChanges(); // Throws because key property change was detected.

For alternative #2, reading the state of modified properties from an entity with modified keys could throw an exception:

Example:

Category category = context.Categories.First();

category.ID = 123; // Modify a key property. This would throw if category wasn't a POCO class.

ObjectStateEntry entry = context.ObjectStateManager.GetObjectStateEntry(category);

Console.WriteLine("State of the object: " + entry.State);

// Throws because key property change was detected.

Getting an exception thrown here would be unexpected. But there is an alternative design that is to define a new EntityState that indicates that an entity is Invalid. This new state would account for the fact that POCO classes per se do not enforce immutable keys.

Since EntityState is a flag enum, Invalid could be potentially combined with other states.

SaveChanges would still need to throw an exception if any entity in the state manager is invalid.

It would be possible to query the state manager for entities in the Invalid state using GetObjectStateEntries method.

Question #3: Is it better to have an Invalid state for entries or should the state manager just throw exceptions immediately every time it finds a change on a key?

Our questions:

1. What are the interesting scenarios for using the state management API in POCO scenarios?

2. What API pattern is better? Having an explicit method to compute the current state based on the snapshot comparisons or having the state to be computed automatically when accessing the state?

3. Is it better to have an Invalid state for entries or should the state manager just throw exceptions immediately every time it finds a change on a key?

---

We really want to hear your thoughts on the above questions.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Initial POCO Design Screencast

dpblogs — Wed, 25 Jun 2008 04:36:00 GMT

Two guys on the team knocked out this quick screencast of some of the prototype work we've been doing around POCO an the State Manager. It's not a long or in depth screencast, but we're hoping to inspire some additional feedback. If you prefer to download a copy of the video, you can download a .zip here.

Elisa Flasko
Program Manager, Data Programmability

Initial POCO Design 1-Pager

dpblogs — Tue, 24 Jun 2008 22:20:00 GMT

Here is a raw cut and paste for our POCO 1-Pager. We are currently working through the design and have some prototype work going on and we would like to hear your feedback. Note this is the "feature design 1-Pager" not the "implementation design 1-Pager" we are still looking at the details for the design and at this point would love to hear feedback on the experience plus thought around state mgmt in general. We will post additional design notes for this feature after the upcoming team design meeting.

1. Overview

Persistence Ignorance refers to being able to allow the developer to write and test domain objects in a way that is entirely independent of fundamental requirements and assumptions that may be made by the infrastructure service (in this case, the Entity Framework). Such requirements / assumptions may often include:

The need to implement a specific interface (for e.g., IPOCO)
Inheritance from a base class
Providing specific constructors
Object Instantiation/Construction requirements – use a specific factory for instance
The need for metadata or mapping class or property Attributes
The need to use specific relationship mechanisms

This amounts to being able to use Plain Old CLR Objects (POCO) so that a developer can author their domain objects free of all assumptions and requirements imposed by the framework. Using this approach, once the domain objects are ready to their satisfaction, the developer can use these classes with the Entity Framework in order for relational database access and persistence.

2. Context

While the customization of entity types has always been a goal with Entity Framework, v1 imposed several restrictions/requirements in terms of how the entity types had to be built – for instance, at the minimum, a few interfaces had to be implemented at a minimum:

IEntityWithRelationships
IEntityWithChangeTracking

Pure POCO should eliminate the need to implement these interfaces.

In addition, POCO shouldn’t require the use of attributes that are required to map object members to the C-space.

3. Design

When the Entity Framework supports Persistence Ignorance in the form of pure POCO, entity classes that are free of any infrastructure related artifacts can be authored as shown. The following example shows a Customer entity and an Order entity. There is a relationship between Customer and Order – i.e. a Customer can have many Orders.

O-Space to C-Space mapping is done by convention – i.e. the CLR types of the entities below map to the corresponding entities already defined in the Conceptual Model.

    public class Customer
    {
        string _CustomerID;
        string _ContactName;
        string _City;
        List<Order> _Orders;

        public string CustomerID
        {
            get { return _CustomerID; }
            set { _CustomerID = value; }
        }

        public string ContactName
        {
            get { return _ContactName; }
            set { _ContactName = value; }
        }

        public string City
        {
            get { return _City; }
            set { _City = value; }
        }

        public List<Order> Orders
        {
            get { return _Orders; }
            set { _Orders = value; }
        }
    }

    public class Order
    {
        int _OrderID;
        Customer _Customer;
        DateTime _OrderDate;

        public int OrderID
        {
            get { return _OrderID; }
            set { _OrderID = value; }
        }

        public Customer CustomerID
        {
            get { return _CustomerID; }
            set { _CustomerID = value; }
        }

        public DateTime OrderDate
        {
            get { return _OrderDate; }
            set { _OrderDate = value; }
        }
    }

Once the POCO classes are in place, one can program against them as first class entity objects.

For instance, query can be used to materialize POCO objects as shown:

        var query = from c in db.Customers.Include("Orders")
                    where c.City == "London"
                    select c;

ObjectContext can be used for CUD operations much like the usage patterns supported today:

        Customer customer = new Customer();
        customer.City = "London";
        customer.ContactName = "Steve";
        customer.CompanyName = "Acme";

        Order order = new Order();
        order.OrderDate = DateTime.Today;

        customer.Orders.Add(order);

        db.AddObject("Customers", customer);
        db.SaveChanges();

Tim Mallalieu
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.