Entity Framework Design : Entity Framework

Code Only – Further Enhancements

efdesign — Mon, 12 Oct 2009 17:31:00 GMT

We’ve come a long way since the last post on Code-Only. So it’s high time for another update.

We’ve been working really hard on Code-Only revving the design, and spotting missing capabilities and responding to feedback both internal and external etc.

The current plan still holds. Code-Only will not be in .Net 4.0 with the possible exception of the DDL features described in the last post. For that portion, the implementation and requirements are more clear to us, and because making the DDL stuff work requires changes to things already in the .NET framework, and getting provider writers lined up, we are still working hard to get the DDL changes into .NET 4.0. The rest of Code-Only will continue to evolve, and we will ship another preview or two, before getting it rolled into .NET as soon as we can after 4.0 ships.

API Refactoring

As hinted above, we’ve spent a lot of time validating code only against real world scenarios, and thinking about how the customer code hangs together etc. As a results we’ve done some API refactoring.

Mappings are now part of Configurations

In the last version Mappings were derived from Configurations, which had some strange side-effects on the API.

We’ve re-arranged things now so that Configurations hold mappings, which are terminated by assignment to a table.

builder.Entity<Customer>()
   .HasKey(c => c.ID)
   .MapSingleType(c => new {
         cid = c.ID,
         nme = c.Name
      }
   )
   .ToTable(“dbo.custs”);

This maps the Customer entity to the ‘dbo.Custs’ table, the ID property to the cid column, the Name property to the nme column, and registers the ID property as the EntityKey / Primary Key.

As you can see we’ve also added a helper method called Entity<TEntity>() so that you can fluently create configurations and mappings. You might also have noticed that we have removed RegisterKey and replaced it with HasKey which we feel is more inline with our goal of having an intentional API.

TPH mapping syntax no longer violates DRY

The previous syntax for specifying a TPH mapping forced you to repeat mappings for columns that could easily have been inherited.

This was unfortunate because we normally encourage TPH because generally it has the best performance characteristics.

This is where MapHierarchy() and Case() come in.

Imagine if we want to write these classes using TPH:

public class Person{
   public virtual int ID {get;set;}
   public virtual string Firstname {get;set;}
   public virtual string Surname {get;set;}
}
public class Employee: Person
{
   public virtual Employee Manager {get;set;}
   public virtual List<Employee> Reports {get;set;}
}

You would now do it like this:

builder
   .Entity<Person>()
   .HasKey(p => p.ID)
   .MapHierarchy()
   .Case<Person>(
      p => new {
         p.ID,
         p.Firstname,
         p.Surname,
         thisIsADiscriminator = “P”
      }
   )
   .Case<Employee>(
       e => new {
         manager = e.Manager.Id
         thisIsADiscriminator = “E”
       }
   )
   .ToTable(“dbo.People”);

By using Case<> mappings are re-used for derived type, so there is no need to repeat column mappings for ID, Firstname and Surname.

Which means the Employee Case statement is only responsible for mapping properties and references declared by the Employee class, and for specifying a new discriminator value.

The discriminator column and values are there so the Entity Framework can distinguish between rows in the dbo.People table that represent Person objects and those that represent Employee objects.

MapSingleType and MapHierarchy

Because Mappings are now part of the configuration, there is no need to create them independently and assign them.

You now simply MapXXX() methods on the EntityConfiguration.

The available mapping methods are:

MapSingleType(λ)
MapHierarchy(λ)
MapHierarchy().Case<TEntity>(λ).Case<TDerived>(λ).

If you know the Entity Frameworks MSL files these correspond to:

Type
OfType
OfType with an addition requirement for a type discriminator

These mappings can be combined in many interesting ways, but in general this table shows the recommended way to do most common mappings:

Scenario	Code	Notes
No Inheritance	builder.Entity<A>() .MapSingleType(λ) .ToTable(“dbo.A”);	Columns not mapped are not part of the Entity.
TPH	builder.Entity<A>() .MapHierarchy() .Case<A>(λ) .Case<B>(λ) .Case<C>(λ) .ToTable(“dbo.ABC”);	Generally each Case expression only maps properties declared by the current type and the discriminator for that type. But it is possible to override the mapping for inherited properties if required.
TPT	builder.Entity<A>() .MapHierarchy(λ) .ToTable(“dbo.A”); builder.Entity<B>() .MapHierarchy(λ) .ToTable(“dbo.B”); builder.Entity<C>() .MapHierarchy(λ) .ToTable(“dbo.C”);	Each MapHierarchy expression only maps properties declared by the current type, and properties that make up the entity key.
TPC	builder.Entity<A>() .MapSingleType(λ) .ToTable(“dbo.A”); builder.Entity<B>() .MapSingleType(λ) .ToTable(“dbo.B”); builder.Entity<C>() .MapSingleType(λ) .ToTable(“dbo.C”);	Each MapSingleType expression maps all properties, both those declared by the current type and those inherited.

Foreign Keys

In Beta2 of .NET 4.0 we added FK Association Support, so Code-Only needs a way to link a FK property and an Navigation Property together.

Given this class:

public class Product{
   public virtual int ID {get;set;}
   public virtual string Name {get;set;}
   public virtual Category Category {get;set;}
   public virtual int CategoryID {get;set;}
}

You need to be able to tell code-only that Category.ID and CategoryID should have the same value.

You do it like this:

builder.Entity<Product>()
.Relationship(p => p.Category)
.HasConstraint((p,c) => p.CategoryID == c.ID);

This says that c.ID (i.e. p.Category.ID) must equal p.CategoryID, which tells code-only that p.CategoryID is an FK property and p.Category is a navigation property backed by this FK property.

HasConstraint(λ) can also be used in conjunction with Relationship(λ).FromProperty(λ) like this:

builder.Entity<Product>()
   .Relationship(p => p.Category)
   .FromProperty(c => c.Products)
   .HasConstraint((p,c) => p.CategoryID == c.ID);

Which tells code only that Product.Category and Category.Products are inverses and that Product.CategoryID and Product.Category.ID must be the same, which implies Product.CategoryID is an FK property.

Missing Navigation Properties

Sometimes you don’t have a navigation property or FK property in the Entity which would naturally hold the FK in the database. For example imagine this scenario:

public class Product{
public virtual int ID {get;set;}
public virtual string Name {get;set;}
}

public class Category{
   public virtual int ID {get;set;}
   public virtual string Name {get;set;}
   public virtual List<Product> Products {get;set;}
}

Here there is ‘probably’ a one to many relationship between Categories and Products, and that is probably best modeled using an FK in the products table.

If you start mapping the Product entity you need a way to map the FK column, but there is no reference (i.e. Product.Category) and no FK property (i.e. Product.CategoryID) to map.

So we added the ability to create a fake navigation property to help out:

builder.Entity<Product>().MapSingleType(
   p => new {
       pid = p.ID,
       nme = p.Name
       cid = EntityMap.Related<Category>(c => c.Products).ID
   }
).ToTable(“dbo.Products”);

Here the interesting part is EntityMap.Related, it creates a fake navigation property, just so you can use it in the mapping.

The method signature looks like this:

public static TEntity Related<TEntity>(
Expression<Func<TEntity, object>
);

Notice that this function simply returns TEntity, so in the above mapping fragment, we are mapping the ID property of the Category class to the cid column, in the ‘dbo.Products’ table.

Complex Types

Code-Only’s default behavior is to ignore properties that are not recognized as either a primitive type or an EntityType. So to support Complex Types we need a mechanism to register one, like this:

var addressConf = builder.ComplexType<Address>();

This returns a ComplexTypeConfiguration through which you can configure the properties of the ComplexType in the same way you configure the properties of an Entity:

addressConf.Property(a => a.Street).HasMaxLength(100);
addressConf.Property(a => a.Zip).HasMaxLength(10);

Now if an entity is mapped by convention and it has a ComplexType property we will automatically introduce columns for each property of the ComplexType.

If you want to explicitly map the Entity mapping the ComplexType property is pretty simple too:

builder.Entity<Person>().MapSingleType(
   p => new {
      p.ID,
      fn = p.Firstname,
      sn = p.Surname,
      street = p.Address.Street,
      zip = p.Address.Zip
   }
);

Registering Entity Sets

Some of our early CodeOnly adopters wanted to use Code Only without having an specialized ObjectContext, and in theory with is possible:

builder = contextBuilder.Create<ObjectContext>();
builder.Entity<Person>().HasKey(p => p.ID);

But what we found is you have no control over the name of the set generated. In this case we will generate an EntitySet called PersonSet, but what if I the name should be People?

To address this issues we have added this method:

builder.RegisterSet<Person>(“People”);

Not only is this useful for specifying names, it also allows you to intentional specify what EntitySets you need. The alternative would be to encode your intent into calls to Entity<TEntity> and Relationship<TRelated>(..).

Association Mapping

We also need to add support for mapping associations that are not co-located with the Entity. For example take the relationship between Jobs and Candidates. If a candidate can apply for multiple jobs this is a classic example of a many to many relationship.

If you want to take control of how this relationship is mapped you need a way to specify the mapping for the relationship, like this:

builder
.Entity<Job>()
.Relationship(job => job.Applicants)
.FromProperty(candidate => candidate.Applications)
.Map(“dbo.JobApplications”,
           (job,candidate) => new {
               applicantId = candidate.Id,
               jobId = job.Id
            }
       );

This does a number of things, first it indicates that Job.Candidates and Candidate.Applications are opposite ends of the same relationship.

Then the Map() call indicates that the relationship should be stored in the “dbo.JobApplications” join table, with a compound PK made up of the applicantId and jobId columns.

The applicantId holds the candidate.Id, and the jobId holds the job.Id, and in both cases Code Only will emit an FK constraint to ensure referential integrity.

Interesting Column Names

The existing mapping syntax that uses anonymous types for the table and column definitions, limits you to column names that are valid CLR identifiers.

So for example if you need to map to a column with a space in the name you can’t.

To rectify this we added an alternate mapping syntax:

builder.Entity<Product>().MapSingleType(
   p => EntityMap.Row(
      EntityMap.Column(p.ID, “p i d”),
      EntityMap.Column(p.Name),
      EntityMap.Column(p.CategoryId, “c i d”)
   )
).ToTable(“dbo.Products”);

This snippet of code, maps the Product entity to the “dbo.Products” table and the ID property to a ‘p i d’ column, the Name property to a Name column and the CategoryId property to the ‘c i d’ column.

This alternate API has another advantage too: it makes it easier to create mapping expressions programmatically, primarily because you no longer need to create an anonymous type when building the expression. Which is really useful if you want to configure code-only at runtime.

Extracting the EDMX

We’ve also added the ability to get the EDMX that Code Only is producing internally, either via an XmlWriter or as an XDocument:

DbConnection connection = // some code to get a connection.
XDocument document = builder.GetEdmx(connection);

// or using XmlWriter
var swriter = new StringWriter();
var xwriter = new XmlTextWriter(swriter);
builder.WriteEdmx(connection, xwriter);

This is pretty handy if you want to use Code Only to build your model and mappings but then pull them out so you work with the XML and our designer tooling.

Setting the StoreType

When producing the Storage Model Code Only asks the current database provider to return an appropriate store type for the CLR type and facets specified.

But some CLR types and facet combinations can map to multiple possible database types. For example fixed length byte[] can map to both binary and timestamp. So in this situation SqlClient returns binary by default.

But you might need a timestamp:

builder
   .Entity<Order>()
   .Property(o => o.Version)
   .HasStoreType(“timestamp”)
   .IsComputed();

This tells Code Only that the Version property of Order (which is a byte[]) is computed in the database, and is actually a ‘timestamp’ column, which is computed in the database after every insert / update.

DDL Provider Model

In our last post we covered our plans around the DDL provider model, so check that out.

Summary

As you can see Code Only is now looking much more complete.

But it isn’t completely finished yet, we are still working on the rough edges, so as always we are very interested to get your feedback.

In particular are there things you think we should add / rename / simplify?

Alex James
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Extending the Entity Framework Provider Model to support DDL

efdesign — Tue, 06 Oct 2009 21:24:18 GMT

As part of the first previews of Code-Only we shared some code to create a database:

// Create a context using code-only
using (var mycontext = builder.Create(dbConnection))
{
    // Create the database if it doesn’t already exist
    if (!myContext.DatabaseExists())
        myContext.CreateDatabase();
    // Standard EF code goes here.
}

But in the first preview of Code-Only this code only worked against SQL Server.

The problem is that our public Provider Model (i.e. DbProviderServices) has no Database Definition Language or DDL features.

Provider Model Changes

To support DDL features across different databases, we plan to extend DbProviderServices, with DDL specific services.

These services will be accessed through these public methods:

public string CreateDatabaseScript(
string providerManifestToken,
StoreItemCollection storeItemCollection);

public void CreateDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

public bool DatabaseExists(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

public void DeleteDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

Now internally those methods will call through to the following ‘protected virtual’ methods which will do the actual work:

protected virtual string DbCreateDatabaseScript(
string providerManifestToken,
StoreItemCollection storeItemCollection);

protected virtual void DbCreateDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

protected virtual bool DbDatabaseExists(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

protected virtual void DbDeleteDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

And the base implementations of these methods in DbProviderServices will simply throw ProviderIncompatibleExceptions.

Which means the provider writers job will be to override these ‘protected virtual’ methods with an implementation that makes sense for their backend database.

The key is to understand that the StoreItemCollection (aka the SSDL or StorageModel part of the EDMX) represents the intended shape of the database.

This means the provider writer will need to iterate over the EntitySets (tables) and the corresponding EntityTypes (table structures) in the StoreItemCollection and create / drop / script the database and tables as required.

Provider writers will be expected to override these functions so that:

DbCreateDatabaseScript: creates a native text command to create the tables and foreign key constraints defined in the StoreItemCollection. I.e. for SqlClient this would be the contents of a .sql DDL file.
DbCreateDatabase: is similar to DbCreateDatabaseScript except it should actually goes ahead and create the database, tables and foreign key constraints.
DbDatabaseExists: checks to see if the database exists. The SqlClient provider will simply check that the database itself exists, but custom provider writers could get more fancy and check to see if every table / foreign key constraint is found too.
DbDeleteDatabase: should go ahead and delete the database, or if the database server has a single database model (like Oracle) the provider writer should delete just the tables defined in the StoreItemCollection.

Simplifying Wrapping Providers

We are also planning something to simplify writing Wrapping Providers. A wrapping provider is just a provider that wraps an existing provider (i.e. SqlClient) and adds additional services (i.e. Auditing, Logging, Caching etc).

Jarek has some some sample wrapping providers if you are interested.

Today writing a wrapping provider is a little tricky, in fact one ‘protected’ method is impossible to wrap without reflection. So to help we plan to add one public wrapper method:

public DbCommandDefinition CreateCommandDefinition(
DbProviderManifest providerManifest,
DbCommandTree commandTree);

One of the reasons we plan on doing this, is we think people might take a ‘basic’ provider that has no DDL support and wrap it to add DDL support.

End User API

Now so far we’ve been looking at the planned extensions to Provider Services, but Provider Services is a very low level API that few developers will ever program against.

Most people will work directly against the ObjectContext, to which we plan to add these methods:

public void CreateDatabase()

public void DeleteDatabase();

public bool DatabaseExists();

public String CreateDatabaseScript();

This little snippet shows how easy it will be to script, create, check and delete a database:

MyContext ctx = new MyContext();
String sql = ctx.CreateDatabaseScript();
ctx.CreateDatabase();
Assert.True(ctx.DatabaseExists());
ctx.DeleteDatabase();
Assert.False(ctx.DatabaseExists());

As you it could hardly be easier to use.

Summary:

While Code-Only provides the catalyst to add DDL support to the Entity Framework’s Provider model, this feature is about more than just Code-Only.

In fact we think this feature will add significantly to the usability of the Entity Framework.

But as always we are keen to hear what you think.

LINQ to SQL to Entity Framework conversion template

efdesign — Fri, 14 Aug 2009 00:10:00 GMT

Many customers have asked us how to port a LINQ to SQL application to the Entity Framework. In order to make this process easier, we have developed a metadata conversion template.

This template converts LINQ to SQL metadata (.dbml) to metadata compatible with Entity Framework (.edmx). While not a complete solution to the conversion problem, it is a useful first step. The template available for download is still in the development stage, and any feedback you have on the user experience, additional features, or any other aspect of the conversion would be greatly appreciated.

The T4 template is intended to convert simple valid .dbml files to .edmx files. All of the basic elements of the .dbml file are mapped to their .edmx counterparts. For instance, Tables are mapped to EntitySets and Types are mapped to EntityTypes. Additionally, stored procedures (including customized insert, update and delete stored procedures) and associations are also converted.

Downloading the Metadata Conversion Template

Included in the zip file is a .vsix installer for the T4 Conversion Template. Also included is detailed documentation for the template. VS 2010 Beta 1 is required.

Download the Template

Walkthroughs

Two walkthroughs have been written to help guide users through the conversion process. The walkthroughs both take the form of unit tests which can be run against the existing LINQ to SQL metadata and the Entity Framework metadata produced by the conversion template. Each walkthrough contains detailed instructions on how to convert the existing LINQ to SQL project to Entity Framework using the Conversion Template. The first is the Widget Factory walkthrough, which shows a basic conversion. The second (CustomCUD) shows how to convert a LINQ to SQL project which uses custom insert, update and delete stored procedures. The walkthroughs are available individually.

Widget Factory Walkthrough

Custom CUD Walkthrough

Caveats and Known Issues

Because the Conversion Template is still under development, there are a number of open issues and unimplemented features. A more comprehensive list can be found in the documentation included with the template installer.

· Inheritance: Inheritance has not been fully implemented. While the template may work in some basic cases, this aspect of the conversion remains an open issue.

· Independent Associations: Foreign-key associations will not be available until .NET 4.0 Beta 2, so the current version of the template will only work with the "UseIndependentAssociations" flag set to true.

· Property Attributes: not all of the property attributes (such as "Unicode" and "FixedLength") are converted because these are not explicitly present in the .dbml file.

Feedback

We are interested to hear any comments you have about known issues, additional feature requests, suggestions for additional walkthroughs or any other aspect of the conversion process.

Thanks,
Christina Ilvento
Developer,
Entity Framework Team

Code Only Enhancements

efdesign — Tue, 04 Aug 2009 01:11:00 GMT

We've been working hard on Code Only since the first preview.

In the next release you will be able to specify:

Navigation Property Inverses.
Property Facets, like Nullability, MaxLength, Precision etc.
Property to Column mappings
Type to Table(s) mappings
Inheritance strategy
Encapsulate configuration

The rest of this post will drill into each of these features in turn.

Registering NavigationProperty inverses:

You can now Register Inverses, i.e. link one navigation property to another, like this:

builder.RegisterInverse(
(Customer c) => c.Orders,
(Order o) => o.Customer)
);

This code indicates that Customer.Orders is the other end of the Order.Customer relationship. Adding order1 to the customer1.Orders collection, has the same effect as setting the order1.Customer to customer1.

Specifying Property Facets:

You can also specify property facets, i.e. things like Nullability, MaxLength, Precision etc, like this:

var customerConfig = new EntityConfiguration<Customer>();
// We can infer that the ID is the Primary Key,
// but not that it is generated in the database on insert.
customerConfig.ForProperty(c => c.ID)
              .Identity();
customerConfig.ForProperty(c => c.Name)
              .MaxLength(100)
              .NonUnicode();
customerConfig.ForProperty(c => c.Website)
              .MaxLength(200)
              .Nullable()

builder.Configure(customerConfig);

This configures the Customer type so that:

The ID property is an Identity column, i.e. the value is computed by the database when we do an insert.
The Name property has a MaxLength of 100 characters and is NonUnicode i.e. in SqlServer VARCHAR rather than NVARCHAR.
The Website property has a MaxLength of 200 characters and is Nullable.

These facets target the Conceptual Model (i.e. CSDL), and from there flow to the database too (i.e. SSDL).

Encapsulating Facet Configuration

You can create a class to encapsulate all this configuration by deriving from EntityConfiguration<T>.

For example:

public class CustomerConfig: EntityConfiguration<Customer>
{
    public CustomerConfig(){
       ForProperty(c => c.ID)
                 .Identity();
       ForProperty(c => c.Name)
                 .MaxLength(100)
                 .NonUnicode();
       ForProperty(c => c.Website)
                 .MaxLenght(200)
                 .Nullable();
    }
}

We recommend creating classes like this instead of configuring an EntityConfiguration<> because of the encapsulation benefits.

Specifying the Tablename

When you use Configure<T>(..) the EF infers a default mapping, inheritance strategy (TPH) and table name(s) for you.

But if you want to specify the table name you can do this:

var customerConfig = new EntityConfiguration<Customer>();
// configure the facets, as per the above example
...

// register configuration with a particular tablename
builder.Tables[“dbo.Custs”] = customerConfig;

Specifying Mappings:

If you need more control over the mappings (for example to map to an existing database or use corporate naming policies) then you can specify mappings like this:

EntityMap<Customer> customerMap =
    Map.OfType<Customer>(
        c => new {
            cid = c.ID,
            c.Name,
            csite = c.Website
        }
    );

Interpreting a Mapping

This mapping states that the ID property is mapped to the ‘cid’ column, the Name property is stored in the ‘Name’ column and the Website property is mapped to the ‘csite’ column.

Properties not referenced are not persisted (just like properties in a partial class when using the default code generation of EF)

LINQ Comprehension Syntax

You can even specify exactly the same thing using a LINQ comprehension syntax too:

EntityMap<Customer> customerMap =
    from c in Map.OfType<Customer>()
    select new {
            cid = c.ID,
            c.Name,
            csite = c.Website
        };

Specifying Facets with Mapping

Once you’ve configured the mapping you can also specify facets on the map like this:

customerMap.ForProperty(c => c.ID)
           .Identity();
customerMap.ForProperty(c => c.Name)
           .MaxLength(100)
           .NonUnicode();
customerMap.ForProperty(c => c.Website)
           .MaxLenght(200)
           .Nullable();

Specifying the table

The final step is to assign the map to a table.

builder.Tables[“dbo.Custs”] = customerMap;

Now we’ve specified a custom table, mapping and custom facets for our Customer class.

Specifying Inheritance:

The default inheritance strategy used by CodeOnly is Table Per Hierarchy (or TPH).

If however you need a different strategy you need to dive in and configure the mappings.

Imagine if you have three classes you need to map: Vehicle , Car and Boat where both Car and Boat are derived from Vehicle which is abstract.

Table Per Hierarchy (TPH)

If you want to map this using TPH, you could do it like this:

var vehicleMap =
    Map.OfTypeOnly<Vehicle>(
        v => new {
            vid = v.ID,
            v.Name,
            vdesc = v.Description
            v.MaxPassengers,
        }
    ).Union(Map.OfTypeOnly<Car>(
        c => new {
            vid = c.ID,
            c.Name,
            vdesc = c.Description
            c.MaxPassengers,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinder,
            discriminator = “CAR”
        })
    ).Union(Map.OfTypeOnly<Boat>(
        b => new {
            vid = b.ID,
            b.Name,
            vdesc = b.Description
            b.MaxPassengers,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
            discriminator = “BOAT”
        })
    );

builder.Tables[“dbo.vehicles”] = vehicleMap;

In a TPH mapping:

OfTypeOnly() is used to create mapping fragments.
Mapping fragments are then unioned so they can be assigned to one table (TPH uses a single table for the whole hierarchy).
A discriminator column is required for each non-abstract type. The discriminator column can have any name and must have a different constant value (i.e. “CAR”) for each non abstract type in the hierarchy.
When mapping a derived type you must re-map all properties mapped in the base-type(s).

Table Per Type (TPT)

If you want to map the same hierarchy using Table Per Type or TPT, this is how you do it:

builder.Table[“dbo.Vehicles”]=
    Map.OfType<Vehicle>(
        v => new {
            vid = v.ID,
            v.Name,
            vdesc = v.Description
            v.MaxPassengers,
        }
    );

builder.Tables[“dbo.Cars”] =
    Map.OfType<Car>(
        c => new {
            cid = c.ID,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinders,
         }
     );

builder.Tables[“dbo.Boats”] =
    Map.OfType<Boat>(
        b => new {
            bid = b.ID,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
        }
    );

In a TPT mapping:

OfType<>() is used in each mapping 'fragment'
Each mapping 'fragment' is assigned to a different table.
Each mapping 'fragment' only maps properties declared on the current type, except…
Key properties must be mapped in every fragment (this allows the tables participating in the type to be joined).
There are no discriminators

Table Per Class (TPC)

You can also map this hierarchy using TPC like this:

builder.Tables[“dbo.Cars”] =
    Map.OfTypeOnly<Car>(
        c => new {
            cid = c.ID,
            c.Name,
            vdesc = c.Description
            c.MaxPassengers,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinders,
         }
     );

builder.Tables[“dbo.Boats”] =
    Map.OfTypeOnly<Boat>(
        b => new {
            bid = b.ID,
            b.Name,
            description = b.Description
            b.MaxPassengers,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
        }
    );

In a TPC Mapping:

Like TPH we use OfTypeOnly(..)
But unlike TPH each non-abstract type has a its own table. (So there is no table for Vehicles because it is abstract).
Each mapping fragment re-maps every, non-transient, property.
There is no mapping for any abstract classes in the hierarchy.
There are no discriminators

Default Foreign Key Locations:

If we see a reference (i.e. Order.Customer) we assume the multiplicity is 0..1. Meaning the the Foreign Key or FK is nullable.

If we see a collection (i.e. Customer.Orders) we assume the multiplicity is many.

Then when you Register Inverses, we know the multiplicity of both ends of the relationships i.e. in the above example we know that there are 0..1 Customers for an Order and many Orders for a Customer.

So by convention there are 3 main types of relationships, we need to infer the FK location for:

0..1 to many –> by convention we put the FK on the many end. So in the Customer.Orders example the FK is put in the Orders table.

many to many –> there is no option but to introduce a join table

0..1 to 0..1 –> You can configure where the FK lives, but in the absence of configuration we will introduce a join table.

Sometimes however a reference is not 0..1 it is 1, i.e. the FK, wherever it might be, may not be nullable.

This is how you specify that the FK is non-nullable:

var orderConfig = builder.Configure<Order>();
orderConfig.RegisterInverse(o => o.Customer, c => c.Orders);
orderConfig.ForProperty(o => o.Customer).NonNullable();

This tells us that there is exactly 1 Customer per Order and many Orders for a Customer.

The ability to tell us that a reference is NonNullable introduces several new multiplicity combinations, for which we need conventions too:

1 to many –> by convention we put the FK on the many end, and make it non-nullable in the database.

0..1 to 1 –> by convention we put the FK on the 1 end, and make it non-nullable.

1 to 1 –> like 0..1 to 0..1 we can’t decide where to put the FK so by convention we introduce a join table.

Specifying FK Mappings:

So far we’ve created mappings for the properties of entities.

What about Navigation Properties and FKs?

All types of relationships (except many to many) can be modeled without a join table in the database. So we allow you to map FKs as part of an EntityMap like this:

EntityMap<Customer> customerMap =
    from c in Map.OfType<Customer>()
    select new {
            cid = c.ID,
            c.Name,
            csite = c.Website,
            salesPersonFK = c.SalesPerson.ID
        };

customerMap.RegisterInverse(c => c.SalesPerson, s => c.Clients);
builder.Tables[“dbo.Custs”] = customerMap;

This specifies that the Customer.SalesPerson navigation property, and it’s inverse SalesPerson.Customers, are stored in the salesPersonFK column in the dbo.Custs table. Because the fragment maps the salesPersonFK column to the c.SalesPerson.ID, where SalesPerson.ID is the Primary Key (or part of the PrimaryKey) of the related SalesPerson entity, and of course FK’s point to PK’s.

Specifying Join Table Mappings:

In the case of many to many relationships you must have a join table, so absent mapping information we produce a join table by convention.

If however you need more control you can do this:

var blogPostsMap = new AssociationMap<Blog, Post>(
b => b.Posts
).Map(
(b, p) => new {BlogId = b.ID, PostId = p.ID}
);

builder.Tables[“dbo.BlogPosts”] = blogPostsMap;

This says the many to many relationships between Blog and Post is stored in the ‘dbo.BlogPosts’ table which has two columns:

‘BlogId’ which is also an FK pointing to the column to which the ID property of Blog is mapped.
‘PostId’ which is also an FK pointing to the column to which the ID property of Post is mapped.

Entity Splitting:

Code Only can even support more advanced mapping strategies like entity splitting:

builder.Tables[“dbo.Customer”] = Map.OfType<Customer>(
   c => new {
      cid = c.ID,
      c.Name,
      active = c.IsActive
   }
);

builder.Tables[“dbo.CustomerDetails”] = Map.OfType<Customer>(
    c => new {
       cid = c.ID,
       c.Size,
       c.Industry
    }
);

This says that the Customer entity is split across the dbo.Customer and dbo.CustomerDetails tables.

Encapsulating all the Configuration:

You can also write a class that derives from EntityMap<T>, to hold all the mapping, facets etc. For example here a class that holds the configuration for Product

public class ProductMap: EntityMap<Product>
{
    public ProductMap{
         this.Map( p => new {
              pid = p.ID,
              pcode = p.Name,
              cid = p.Category.ID
         });
         this.ForProperty(p => p.ID).Identity();
         this.ForProperty(p => p.Name).MaxLength(100)
             .NonUnicode();
         this.ForProperty(p => p.Category).NonNullable();
         this.RegisterInverse(p => p.Category,
                              c => c.Products);
     }
}

This approach is highly recommended, because configuring the Product type is now trivial:

builder.Tables[“dbo.Products”] = new ProductMap();

Summary:

As you can see we are planning a lot of enhancements which will allow most core scenarios.

What do you think? Do you like the API? Are there things you’d like to see look a little different?

As always we are very keen to hear your feedback.

Alex James
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Code Only

efdesign — Wed, 10 Jun 2009 23:04:00 GMT

There are currently two ways to get Entity Framework apps up and running, we call these Database First and Model First respectively.

Database First has been with us since .NET 3.5 SP1, and involves reverse engineering an Entity Data Model from an existing database.

Model First is new to Visual Studio 2010/.NET 4.0 and allows you to create an Entity Data Model from scratch and then generate a database and mapping for it.

However many developers view their Code as their model.

Ideally these developers just want to write some Domain classes and without ever touching a designer or a piece of XML be able to use those classes with the Entity Framework.
Basically they want to write ‘Code Only’.

We are pleased to announce that we’ll be shipping a preview of “Code Only” on top of the .NET Framework 4.0 Beta 1 (more on that in the coming days).

How it works:

To use Code only you simply write some POCO classes, here is a simple example:

public class Category
{
   public int ID { get; set; }
   public string Name { get; set; }
   public List<Product> Products { get; set; }
}

public class Product
{
   public int ID { get; set; }
   public string Name { get; set; }
   public Category Category { get; set; }
}

Then write a class that derives from ObjectContext that describes the shape of your model and how you want to access your POCO classes, so something like this:

public class MyContext : ObjectContext
{
   public MyContext(EntityConnection conn)
    : base(conn){ }
   public ObjectSet<Category> Categories {
      get {
         return base.CreateObjectSet<Category>();
      }
   }
   public ObjectSet<Product> Products {
      get {
         return base.CreateObjectSet<Product>();
      }
   }
}

NOTE: this persistence / EF aware class could easily be in another assembly so you can keep it separate from your nice pristine domain classes.

At this point you have everything you need from a CLR perspective, but you won’t be able to use the MyContext class without some EF metadata, which is normally stored in the EDMX file.

But remember Code-Only means there isn’t an EDMX file.
So where do we get the metadata?
That is where the ContextBuilder class comes in:

SqlConnection connection = …; // some un-opened SqlConnection
MyContext context = ContextBuilder.Create<MyContext>(connection);

What is happening here is that the ContextBuilder is looking at the Properties on MyContext, inferring a default Conceptual Model, Storage Model and Mapping by convention, it the uses that metadata plus the SqlConnection you passed in to create an EntityConnection, and finally it constructs an instance of MyContext by passing the EntityConnection to the constructor we created previously.

Once you have an instance of your context, we ship several extension methods you can use to automatically create a database script, see if the database exists, drops the database, and/or create the database,. For example this code snippet uses two of those extension methods to create the database if it doesn’t already exist.

if (!context.DatabaseExists())
context.CreateDatabase();

The CreateDatabase call looks at the EF metadata, in particular the Storage Model, aka SSDL, and uses it to produce Database Definition Language (or DDL) which it then executes against against the connection.

Overriding Conventions

In the examples so far everything is done by convention. This is great if you are happy with our conventions. However you may want to override them, for example to register different keys, use different mappings or use a different inheritance strategy etc.

To override the convention instead of creating an ObjectContext directly you create an instance of a ContextBuilder that you can configure.

var contextBuilder = new ContextBuilder<MyContext>();

In the first Feature CTP, Code Only provides the ability to override how the key properties are inferred:

contextBuilder.RegisterKey((Product p) => p.ID);

This code tells the builder that the key of Product is its ID property.

When you’ve finished configuring how you want your model and mappings to work in your code, you create an instance of your context, this time by calling Create on the builder instance you have been configuring:

MyContext context = contextBuilder.Create(connection);

It is that simple.

Over time the features of the ContextBuilder will grow so you will be able to setup custom mappings, mark properties as transient, setup up Facets (e.g. MaxLength), change inheritance strategies, link properties that are the inverse of each other together (e.g. product.Category and category.Products) and more. In fact here is an example of the sort of thing we are aiming to support for the next release of the Feature CTP:

builder[“dbo.prds”] =
from c in builder.OfType<Product>()
select new {
    pcode = p.ID,
    p.Name,
    cat = p.Category.ID
}
);

This snippet:

Maps Product entities to the ‘prds’ table.
Maps Product.ID to a ‘pcode’ column.
Maps Product.Name to a ‘Name’ column.
Maps the FK under the Product.Category relationship to the ‘cat’ column.

Here is another example that does basic TPH inheritance:

builder[“dbo.Cars”] = (
from b in builder.OfTypeOnly<Car>()
select new {
     id = b.CID.AsKey(),
     b.Color,
     b.Manufacturer,
     disc = “C”
   }
).Merge(
from s in builder.OfType<SportsCar>()
select new {
     id = s.CID.AsKey(),
     s.Color,
     s.Manufacturer,
     disc = “S”,
     hp = s.HorsePower,
     tq = s.Torque
}
);

These two examples just scratch the surface of what we are planning on doing, hopefully though they give you a sense of where we are heading.

We are super excited about Code Only. But as of right now our plans to support overriding conventions aren’t finalized, so we’d love to get your feedback, this really is your chance to help us get it right.

Alex James,
Program Manager, Entity Framework Team, Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Foreign Keys in the Entity Framework

efdesign — Mon, 16 Mar 2009 23:47:00 GMT

Background

A number of months ago we asked whether Foreign Keys (FKs) in Conceptual and Object models were important.

The feedback we got indicated that there are a real mix of opinions on the topic. Some people are all for them while others think that FKs pollute the conceptual model.

The fact that our customers were so divided meant we thought it was important to provide options here. Those who want FKs in their Entities should be able to do so, so they can gain all the benefits that having FKs in your Entities undoubtedly provide. On the other hand customers who are concerned that have FKs in their Entities in someway pollutes their model can continue to use the type of associations we had in .NET 3.5 SP1.

In the past we've called the .NET 3.5 SP1 style Associations "first class" associations. Which while accurate somehow implies that associations based on FKs in Entities are "second class". In fact when we talked about this new work recently with our MVPs one of the first questions was "are these second class?", which they most definitely are not.

The fact is these new associations are different but first class nevertheless.

So what's been added?

In .NET 4.0 we will add support for a new type of association called "FK Associations".

The way this works is we will allow you to include your FK columns in your entities as "FK properties", and once you have FK Properties in your entity you can create an "FK Association" that is dependent upon those properties.

Okay so we will have "FK Associations", what are we going to call the older style associations? Obviously saying something like ".NET 3.5 SP1 style Associations" every time isn't ideal. So we are going to call them "Independent Associations".

The term "Independent Association" resonates for us because they are independently mapped, whereas FK Associations need no mapping, simply mapping the Entity(Set) is sufficient.

How do I use an "FK Association"?

The real reason so many customers and partners are asking for "FK Associations" is that they significantly simplify some key coding patterns. So much so that we are convinced that for most people "FK Associations" will be an automatic choice going forward.

Some of the things that are hard with "Independent Associations" are trivially easy using FK Associations. Scenarios as diverse as DataBinding, Dynamic Data, Concurrency Control, ASP.NET MVC Binders, N-Tier etc are all positively impacted.

Let's have a look at some code snippets that will work with "FK Associations":

1) Create a new Product and FK Association to an existing Category by setting the FK Property directly:

using (var context = new Context())
{
    //Create a product and a relationship to a known category by ID
    Product p = new Product
    {
        ID = 1,
        Name = "Bovril",
        CategoryID = 13
    };
    //Add the product (and create the relationship by FK value)
    context.Products.AddObject(p);
    context.SaveChanges();
}

This sort of approach works both for insert and update, and is particular useful for things like databinding, where you often have the new Value of the FK in a grid or something but you don't have the corresponding object, and you don't want to wear the cost of a query to pull back the principal object (i.e. the Category).

2) Create a new Product and a new FK Association to an existing Category by setting the reference instead:

public void Create_new_Product_in_existing_Category_by_reference()
{
    using (var context = new Context())
    {
        //Create a new product and relate to an existing category
        Product p = new Product
        {
            ID = 1,
            Name = "Bovril",
            Category = context.Categories
                        .Single(c => c.Name == "Food")
        };
        // Note: no need to add the product, because relating
        // to an existing category does that automatically.
        // Also notice the use of the Single() query operator
        // this is new to EF in .NET 4.0 too.
        context.SaveChanges();
    }
}

This programming pattern is not new to the Entity Framework, you could do this in .NET 3.5 SP1. I called it out because it shows you can still write the sort of code you write with Independent Associations even when using FK Associations.

3) Update an existing Product without informing the Entity Framework about the original value of the CategoryID (not supported with Independent Associations):

public void Edit(Product editedProduct)
{
    using (var context = new Context())
    {
        // Create a stand-in for the original entity
        // by just using the ID. Of the editedProduct
        // Note: you don't have to provide an existing Category or CategoryID
        context.Products.Attach(
            new Product { ID = editedProduct.ID });

        // Now update with new values including CategoryID
        context.Products.ApplyCurrentValues(editedProduct);
        context.SaveChanges();
    }
}

In this example "editedProduct" is a product which has been edited somewhere, this is exactly the sort of the code you might write in the Edit method of a ASP.NET MVC controller, and is a great improvement over the code you have to write using Independent Associations.

There is much more you can do with FK Associations, but these 3 samples give you a flavor of the sort of coding patterns FK Associations allow.

Keeping FKs and References in Sync

One thing that hasn't been mentioned so far is that the Entity Framework tries to keep related References and FKs in sync as much as possible.

When this synchronization occurs depends upon when the Entity Framework is notified of changes, which depends upon the type of Entity Classes involved: be they POCO with Proxies, POCO without Proxies, IPOCO or EntityObjects etc.

Another post will cover this in more detail.

How do I create an "FK Association"?

There are a number of ways.

One mainline scenario is when using the tools to infer a model from a database. We have added an option to choose whether "FK Associations" or "Independent Associations" are generated by default. The same is true if you use EdmGen.exe (i.e. our command line tool).

The second mainline scenario for creating FK Associations is creating a model from scratch, aka Model First which is new to .NET 4.0.

Here are the steps involved:

Create two Entities (say Product and Category) that look something like this:
Add a property that will be the FK Property: i.e. CategoryID
Create an association between the two EntityTypes with the correct end multiplicities and NavigationProperty names:
Double Click the line between the two Entities, that represents the Association, and add a Referential Integrity Constraint:

This is the step that tells the Entity Framework that the CategoryID is an "FK Property" and that the "CategoryProduct" association is an "FK Association".
You end up with something that looks like this:

And you are done you've set up an FK Association. Notice that this association doesn't need to be mapped you simply need to make sure you map all the properties of both Entities.

Very easy.

Summary

In .NET 4.0 we will add support for FK Properties and FK Associations to the Entity Framework. The existing style Independent Associations will still be possible, but we expect that FK Associations will become most peoples automatic choice because they simplify so many common Entity Framework coding tasks.

As always we'd love to hear your thoughts on this work.

Alex James,
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Customizing Entity Classes in VS 2010

efdesign — Thu, 22 Jan 2009 22:15:00 GMT

When VS 2010 ships it will include some significant improvements to our code generation story for the Entity Framework. The basic idea is to make use of T4 templates for code generation and ship strong integration into the Entity Framework Designer to make the experience of customizing those templates as seamless as possible.

Below Sanjay from our Tools team outlines what will be possible once VS 2010 is released.

Customize the code generated by the Entity Designer with T4 templates

In Visual Studio 2008 SP1, the ADO.NET Entity Designer generates classes from the CSDL portion of the EDMX file using the EntityClassGenerator APIs. Numerous customers have asked us how to customize the code generation for a variety of scenarios including:

Make the generated ObjectContext internal
Make the generated ObjectContext and Entity classes implement a user-defined interface
Add user-defined CLR attributes to generated ObjectContext and generated Entity classes
Influence generated classes based on structural annotations in CSDL
Generate the ObjectContext and Entity classes into separate files
Generate “proxy classes” for the generated classes
Partially or fully change how classes are generated, maybe even generate additional (non code) artifacts in the project
Completely replace entity framework code generation with custom code
Generate POCO classes from the model to use as a starting point in my applications
Generate self-tracking entity classes
· ...

While the SingleFileGenerator techniques described here enable some of these scenarios these techniques are generally difficult to implement and install or are tricky to debug and customize.

In Visual Studio 2010, we plan to give customers the ability to use T4 templates to generate their classes, and are also planning deep integration with the Entity Designer and Visual Studio to provide a great end-to-end experience.

Walkthrough

The example below customizes code generation to make the generated classes implement a user-defined interface (e.g. IValidate).

Preconditions:

User has installed Visual Studio 2010
User is familiar with customizing T4 templates
User has a C# or a VB console project that targets FX4.0 with an EDMX file in the project

Walkthrough:

User: opens EDMX file in the Entity Designer. Note that the generated classes are in a child file of the EDMX file (i.e. in Northwind.Designer.cs)
User: right-clicks on an empty area of the designer surface and chooses “Add New Artifact Generation Item…” from the context menu.
Visual Studio: displays the standard “Add New Item” dialog and shows a filtered list of items for the user to select from. As expected, the “Add New Item” dialog only shows a list of item templates specific to the current project target framework version, project type and language.
User: selects “ADO.NET EntityObject Generator”, specifies a file name and clicks “Add"
Visual Studio: checks out the project from Source Control if necessary
Visual Studio: sets the “Custom Tool” property of the EDMX file to empty, which will cause the existing .designer.cs file to be deleted
Visual Studio: the VS item template adds a new .tt file to the project in the same project directory as the EDMX file
Visual Studio: configures the new .tt file to process the selected EDMX file. The EDMX file to process is a replaceable parameter in the .tt file, and VS updates it to point to the selected EDMX file. Thus, the .tt file now knows which EDMX file to generate code for.

[Screenshot of .tt file open in the Clarius T4 Editor Community edition for VS 2008]
User: double-clicks the .tt file in the project to open it in VS10
User: edits the .tt file in Visual Studio to make every Entity object implement the IValidate interface

[Screenshot of .tt file open in the Clarius T4 Editor Community edition for VS 2008]
User: saves and closes .tt file
Visual Studio: Transforms the .tt file to produce the generated classes as a child file of the .tt file
User: notes that the generated classes (under the .tt file) implement the IValidate interface as expected
User: switches to the EDMX file in the designer and makes some changes (e.g. add a new entity)
User: saves EDMX file
Visual Studio: Transforms all .tt files in the project that reference the EDMX file. As before, the generated classes are child files of the .tt files

Design

The design affects multiple parts of the Entity Framework and the Entity Designer and the key changes are described below:

Entity Framework Runtime

We started by creating C# and VB T4 templates to generate classes from CSDL and EDMX files
Since the T4 engine is installed with Visual Studio and not with the.NET FX runtime, we preprocess the templates as described here and include the compiled code into System.Data.Entity.Design.dll
We added a new class System.Data.Entity.Design.EntityCodeGenerator (in System.Data.Entity.Design.dll ) that generates code using the preprocessed T4 template
Finally, we enhanced a few runtime metadata APIs and added extension methods to load and iterate over models a lot easier. Our T4 templates make heavy use of these new methods.

Entity Designer

We took the C# and VB T4 templates used by the runtime and changed them to be more “VS friendly” as described below.

We made the EDMX file name referenced in the T4 templates a replaceable parameter
We created a wizard (with no GUI) that is launched by Visual Studio when the item is added to the project. The wizard does 2 things:
1. updates the EDMX file name referenced in the T4 templates and
2. sets the “Custom Tool” property of the EDMX file to empty
We rolled up the T4 templates (and wizard) into standard C# and VB Visual Studio item template which are installed with VS 2010; this makes them show up in the “Add New Item” dialog in Visual Studio
We added a new context menu to the designer surface to launch the “Add New Item” dialog and also added code to only show VS item templates whose names start with the prefix ADONETArtifactGenerator_
Our design ensures that user installed VS item templates in %MyDocuments% as well as the VS item templates we ship in box show up in the “Add…New…Item” dialog (per the template name prefix rule above)
We added code to the designer that finds all .tt files related to an EDMX file and automatically transform them when the EDMX file is saved.
Finally, we added a new property called “Process related T4 templates on Save” on the designer surface to let users control whether or not to transform .tt files related to the EDMX file on save.

Multi Targeting considerations

As many of you are aware, VS 2010 allows developers to target FX4.0 as well as FX3.5 (and older runtimes). The T4 templates that we ship in the Visual Studio box generate code that works with the Entity Framework in .NET FX 4.0. This means the VS item templates we ship in the box will not be available for projects that target FX3.5.

However, it is certainly possible (and supported) for users to create new T4 templates (and new VS item templates) that generate code from an EDMX file in projects that target FX3.5 and light up in the same experience. In fact, we might release some new templates ourselves later, on CodeGallery perhaps.

How can 3^rd parties plug in?

Since our design is based upon VS item templates that wrap T4 templates, we have opened the door to let 3^rd party code generators participate in the end-to-end experience. Our contract is straight forward and easy to implement:

Create a (C# or VB) T4 template to generate code as you desire. The T4 templates we ship in the box can be invaluable points of reference as you roll your own.
Make the EDMX reference in your .tt file a replaceable parameter by calling it $edmxInputFile$ like we do in our .tt file.
Wrap the .tt file in a VS item template. This is pretty easy and well documented on MSDN. You can also use the Export Template wizard included Visual Studio. Also specify additional VS item template settings such as target framework, project type, etc
Prefix the name of your .vstemplate file with ADONETArtifactGenerator_ if you want your VS item template to show up in the “Add New Item” dialog when it is launched from the context menu in the Entity Designer.
You can use the wizard we ship in Visual Studio to do parameter replacement in your VS item template by adding the following to your .vstemplate file:

<WizardExtension>
<Assembly>
Microsoft.Data.Entity.Design, Version=10.0.0.0, Culture=neutral,PublicKeyToken=b03f5f7f11d50a3a
</Assembly>
<FullClassName> Microsoft.Data.Entity.Design.VisualStudio.ModelWizard.AddArtifactGeneratorWizard
</FullClassName>
</WizardExtension>

Of course, you can also create your own wizard that does custom processing when the item is added to the project as described here.
Create a VSCONTENT file and zip up your custom item template & related files into a Visual Studio Content Installer (.vsi) file
Upload the .vsi file someplace your customers can download it from
Visual Studio already knows how to install .vsi files and everything you need is already available in Visual Studio.

A few key takeaways:

This design lets the EF team release new T4 templates outside the usual Visual Studio and .NET FX ship cycles and the experience to consume them is exactly the same.
There are several excellent resources on the web that describe advanced T4 template scenarios and our design enables them. In fact, we expect users to leverage these resources while creating new T4 templates or customizing the ones we ship. For example, code in a custom T4 template can use the EnvDTE APIs to access the project system and generate outputs to multiple files.
Over time, we expect to blog about some of these advanced scenarios and also work with MVPs, industry experts and Patterns & Practices to provide guidance to customers in a manner similar to the Guidance Automation Toolkit.
Custom T4 templates can generate anything you want, not just code. For example, we have an internal prototype of a T4 template that iterates over a model and generates an HTML file with a nicely formatted report of the entities and associations in the model
VS item templates can add multiple files to your project (more details here). This means your custom item templates can include all kinds of files (e.g. workflow files, multiple T4 files, pre-canned code files, etc). For example, you can easily imagine a 3^rd party VS item template that adds 2 .tt files in your project that process the same EDMX file but generate different kinds of outputs.

We hope this gives you an overview of our design and intent of this feature and we would love to hear your comments.

Sanjay Nagamangalam,
Lead PM, ADO.NET Entity Designer

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Model Defined Functions

efdesign — Wed, 07 Jan 2009 23:32:00 GMT

Today the Entity Framework, and more specifically the Entity Data Model, have a limited notion of Functions.

We are currently restricted to Function Imports that allow stored procedures to be invoked, and Canonical / Store Functions for database independent and database specific functions respectively.

Now however we want to support functions defined, not just declared, in the EDM (aka. the CSDL).

An example would be:

<Function Name="GetAge" ReturnType="Edm.Int32">
      <Parameter Name="Person" Type="Model.Person" />
      <DefiningExpression>
            Edm.DiffYears(Edm.CurrentDateTime(), Person.Birthday)
      </DefiningExpression>
</Function>

Here are some things to notice:

The DefiningExpression is eSQL.
The function can have zero or more parameters.
The Function must have a return type.
Function Parameters are referenced directly by Name in the DefiningExpression: meaning there is no parameter denoting prefix like @. This means you must be careful to choose parameter names that don't collide with identifiers you need to use in the rest of eSQL expression.
Unlike functions in SSDL, functions in CSDL only support In bound Parameters (i.e Mode="In") because otherwise they become non-composable.
For this reason the Mode of a parameter cannot be set in CSDL (it is always Mode="In").
Functions are declared as Global Items and are declared within the <Schema> element. As such there identity is made up of the Schema's namespace and the function name.
The function parameters and return type can be any of the following:
- A scalar type or collection of scalar types.
- An entity type or collection of entity types.
- A complex type or collection of complex types.
- A row type or collection of row types (See below).
- A ref type or collection of ref types.
Functions with a DefiningExpression do not require mapping, since the eSQL expression is composed out of eSQL fragments that are already mapped.
Functions without a DefiningExpression are simply declarations. Today the Entity Framework doesn't complain when loading a CSDL with such a function, but you can't invoke it. In the future these functions might be used to support mapping Table Value Functions in the store to functions in the Conceptual Model.
Since it is trivial in eSQL to create arbitrary un-named types, imagine a projection that projects 3 of the properties from an Entity, we now need a mechanism for defining these "RowTypes" inline, so that they can be used when defining Function Parameters and ReturnTypes. For example:

<Parameter Name="Coordinate">
   <RowType>
      <Property Name="X" Type="int" Nullable="false"/>
      <Property Name="Y" Type="int" Nullable="false"/>
      <Property Name="Z" Type="int" Nullable="false"/>
   </RowType>
</Parameter>
Since eSQL is primarily set based, we also need a way of defining parameters and return types that are collections of RowTypes:

<Parameter Name="Coordinates">
   <CollectionType>
      <RowType>
         <Property Name="X" Type="int" Nullable="false"/>
         <Property Name="Y" Type="int" Nullable="false"/>
         <Property Name="Z" Type="int" Nullable="false"/>
      </RowType>
   </CollectionType>
</Parameter>

Using the Function via eSQL:

It is trivial to use the function via eSQL. For example:

SELECT Namespace.GetAge(p)
FROM Container.People AS P
WHERE P.Firstname = ‘Jim’

Here we get Jim's age, assuming of course there is only one Jim!

It is also possible to compose functions together, you must simply ensure return types and target parameter types are the same (in the case of named types) or structurally equivalent (in the case of row types or collections of row types).

Things get a little trickier when you are dealing with functions that return sets, for example imagine a function that returns someone's friends, used in conjunction with the GetAge function:

SELECT VALUE (F)
FROM Container.People AS P
CROSS APPLY Namespace.GetFriends(P) AS F
WHERE Namespace.GetAge(P) > 21

Here we get all the friends of people older than 21.

As you can see to do this sort of thing you need a crash course in eSQL.

Using the Function via LINQ:

It is also possible to use these functions in LINQ, but this does require a extra step to create an appropriate stub function in the CLR.

This solution is based on the techniques described here, and involves creating a Stub function in the CLR language of your choice, and annotating it something like this:

[EdmFunction("Namespace", "GetAge")]
public static int GetAge(Person p)
{
throw new NotSupportedException(…);
}

The Entity Framework uses the signature of the function and the EdmFunction attribute to map calls to this function when encountered to the appropriate Model Defined Function.

Once you have this stub it is then trivial to use it in a LINQ query like this:

var peopleOver21 =
     from p in ctx.People
     where GetAge(p) < 21
     select p;

Indeed if you are familiar with LINQ you will probably find composing functions together a lot easier too:

var friendOfPeopleOver21 =
     from p in ctx.People
     from f in GetFriends(p)
     where GetAge(p) < 21
     select f;

Notice that the CLR functions don't need to be directly callable, in the example above the CLR stub throws an exception if called directly.

However the existence of the stub allows you to create LINQ expressions that compile correctly, and then at runtime, when used in a LINQ to Entities query, the function call is simply translated by the entity framework into a query that runs in the database.

Summary

As you can see Model Defined Function's are very powerful, and this post has barely scratched the surface of possibilities they open up.

The Entity Framework team would love to hear your comments.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Pluralization

efdesign — Tue, 02 Dec 2008 03:20:00 GMT

Unfortunately in the current version of the Entity Framework, which ships in .NET 3.5 SP1, we don't make any attempt to Singularize or Pluralize names when reverse engineering a model from the database.

So if, for example, your database has a table called Orders you will get an EntityType called Orders too, which clearly doesn't make for the most read-able code:

Orders order = new Orders(); //? why not just Order ?

LINQ to SQL does a better job here, because it ships with simple but effective pluralization and singuralization services, that for the most part produces names you would expect.

Unfortunately the lack of a similar feature in the Entity Framework, means that generally the first thing you do after you've reverse engineered a model with the Entity Framework is fix-up all the silly names.

Now from the perspective of someone who delivers plenty of Entity Framework demos, this is quite embarrassing!

But of course the real problem here is that it is time consuming and error prone for all but the most trivial models.

Solution:

In the next version we have added basic pluralization support. This support only really handles English, and here's why:

Creating language and culture aware services is very hard, and we thought it would be better to spend our time working on the core of the Entity Framework.
These sort of services are very general and as such we think that they belong elsewhere, maybe inside Windows itself?
We have made the PluralizationService class public so you are free to write your own.

Our default English language pluralization is then used whenever you generate a model from a database or visa-versa (see model first) to help produce appropriate names.

And for command line users familiar with EdmGen.exe there is a new /pluralize switch also.

Basic Pluralization Rules:

So how does the Entity Framework use this pluralization service?

Well it calls the service to:

Singularize EntityType names
Singularize NavigationProperty names that point to 0-1 entities
Pluralize EntitySet names
Pluralize NavigationProperty names that point to 0-* entities

It turns out that if you do this, you get the model you are looking for most of the time.

The Pluralization APIs:

So far we've talked about default behavior, if however you need more control, for example if you want to specify custom pluralization rules or a create a completely custom service, you have to drop down to the API level.

Everything revolves around a newly added public PluralizationService class that looks something like this:

public abstract class PluralizationService
{
public static PluralizationService CreateService(CultureInfo culture);

public abstract string Pluralize(string word);
public abstract string Singularize(string word);
}

As you can see classes that derive from PluralizationService can singularize and pluralize words.

You can also ask the PluralizationService to create a concrete PluralizationService for you based on Culture (as mentioned previously only "en" based cultures are supported out of the box).

Here is a little code that gets hold of the built-in English Language pluralization services, configures it so "Child" will pluralize to "Children" and then checks that it pluralizes correctly.

PluralizationService pluralizationService =
PluralizationService.CreateService(
new CultureInfo("en-US"));

ICustomPluralizationMapping mapping =
pluralizationService as ICustomPluralizationMapping;

if (mapping != null) // it shouldn't be but just checking
{
//Specifying the child pluralizes as children
mapping.Add("Child", "Children");
}

Debug.Assert(pluralizationService.Pluralize("Child") == "Children");

You can use your pluralization service in Model Generation (i.e. the step that produces CSDL and MSL from SSDL) by handing your pluralization service to the EntitySchemaModelGenerator:

//Create an EDM from SSDL generator
EntityModelSchemaGenerator generator =
    new EntityModelSchemaGenerator(
        storageModel,
        "MyNamespace",
        "MyContainer",
        pluralizationService);

//Generate CSDL and MSL (in memory)
generator.GenerateMetadata();

The resulting CSDL will have been pluralized/singularized as described in the earlier rules using the provided pluralization service.

See this for more information on the EntityModelSchemaGenerator.

Summary

So as you can see we've added simple pluralization capabilities, akin to those found in LINQ to SQL, that will reduce the amount of work required to create meaningful models.

And to top it off, the solution we've created is easy to customize and public so you can probably use this for other things too!

As always we are very keen to hear what you think.

Alex James
Program Manager
Microsoft

UPDATE: A number of people have asked whether pluralization can be turned off, because the post doesn't make this clear. The answer is absolutely, pluralization is definitely optional, in Visual Studio this is as simple as checking/unchecking a checkbox in the wizard, and for EDMGen.exe you have to opt-in for pluralization with the /pluralize commandline switch.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

N-Tier Improvements for Entity Framework

efdesign — Thu, 20 Nov 2008 08:33:00 GMT

The first version of Entity Framework provides convenient ways to load, manipulate and persist objects and relationships. As with many other O/RMs, Entity Framework has a state manager that tracks every change made. Existing objects are typically loaded first from the database, later modified, and finally the changes are saved back to the store.

Another feature, full graph serialization, makes it very easy for developers to ship around object graphs representing snapshots of the current state of the world, across execution boundaries.

The next version of Entity Framework will also support Persistence Ignorance. Therefore object graphs can now be made of POCO instances, which WCF now also supports.

Nonetheless, there is a task that belongs in any N-Tier application that still requires a great amount of work for developers using EF: decoding messages that represent state changes performed by the client, meant to be processed or persisted by the service.

The major pain point in the first version is the number of intricate steps a program needs to perform in order to setup the state manager in the appropriate state according to the changes encoded in the message. For instance, for any but the simplest scenarios, in order to use the basic graph manipulations APIs in ObjectContext, like AddObject and AttachTo, it is necessary to “shred” the object graphs contained in the message, which destroys important information about relationships between object instances that therefore has to be kept somewhere else (as an illustration of this approach, see Danny Simmons’s EntityBag in project Perseus).

In addition, given that the message typically contains all the necessary information and that concurrency is usually a concern, the program shouldn’t have to re-query the database. Instead, it should be easier to “tell” the state manager about the original state (that came from the database previously) and about what changes were made.

There are a few approaches to N-Tier that are very popular in the industry. Customers using Entity Framework for the first time often carry the expectation that it will support a similar experience:

DataSet

The original ADO.NET data access technology includes the DataSet, a versatile data cache that can be serialized as XML and be used to transport collections of rows and relationships and, also diffgrams representing changes.

DataSets make many N-Tier scenarios extremely easy to implement, and over time, have proven helpful for many customers. In spite of this, they are not appropriate for some scenarios and architectural patterns:

The lack of a widely accepted standard serialization format has hindered the development of implementations that could produce or consume datasets in other platforms.
Diffgrams represent sets of CREATE, UPDATE and DELETE operations of arbitrary depth, which is not adequate for architectural patterns that conceive operations with more constrained semantics (i.e. PlaceOrder, CreateCustomer).
No simple ways to control the application of batch CUD operations contained in diffgrams makes them unsuitable any time there is a trust boundary between the client and the server.
Also, the data-centered perspective of DataSets leads to an anti-pattern known as the “anemic domain model”, which is characterized by a separation between the data aspect and the behavioral aspect of domain objects.

Data Transfer Objects

The approach typically used in the DDD and SOA communities to address multi-tier scenarios is to hand-build a service with well defined operation semantics together with the a conceptual message format, usually composed of data transfer objects (DTOs). Implementing DTOs often also involves writing the logic that translates DTOs back and forth to persistent objects.

Here is a simplified set of rules that characterizes the use of DTOs:

Actual entities are not serialized, just DTOs, which are value-only objects (without behavior) are exposed in the boundaries.
The client does not depend on the types defined on the server. The shapes of the DTOs are the sole contract, and the types used in the client are for the client only.
The context or container is not sent together with the message.
The service decides what to do with the contents of the message. It is not the message who decides (as an extreme counter example, imagine a message that tells the service to delete all the contents of the database).

Those rules are especially important in SOA scenarios, in which the client and the server can be separated by a trust boundary, such as the one that exists between two different companies.

There are other permutations of layered architectures, though, in which the level of decoupling that DTOs provide may not justify the increased complexity, for example:

A multi-tier application in which the client is fully trusted.
A rich internet application in which an infrastructure service is purely used for data persistence (is by design a CRUD-only service) and therefore types exposed do not contain behavior anyway.
Any architecture in which the persistence framework is actually used to provide and manage DTOs and not the actual entities.

REST

ADO.NET Data Services provides a rich framework that developers can easily use to expose any Entity Data Model as a REST-style resource collection. Simple HTTP verbs like GET, PUT, POST and DELETE are used in combination with URIs to represent CRUD operations against the persistence service.

This pattern is suitable for many applications and can easily be combined with service operations with richer semantics in the same application.

ADO.NET Data Services, however, may not be compelling for those customers who want to define their services as operations with richer semantics entirely.

Goals

ADO.NET Data Services goes a long way addressing the needs of customers doing RESTful services, and so there is no need for us to improve Entity Framework experience on that space.

With the N-Tier improvements for Entity Framework, we want to address some of the same problem space as DataSet, but we want to avoid the primary issues with it.

Ideally, we would like to provide building blocks that are appealing for developers building solutions on a wide range of architectures. For instance, we would like to provide fine enough control for DTO proponents, but at the same time reduce the level of pain that those trying to address simpler scenarios experience today.

Design

If we just wanted to provide a DataSet like experience on top of Entity Framework, the most straightforward (although very expensive to test) way would probably be based on full serialization of the state manager. Basically, the internal representation of ObjectStateManager includes collections of objects to be deleted, inserted, and modified, and also relationships to be added and deleted, as well as objects and relationships that should remain unchanged.

Such a model would lead to very fine grain control of the state manager, but you would lose some of the benefits of operating on the domain objects and the higher level APIs. For instance, it would make it extremely easy for the program to set the state manager in a completely invalid state.

An alternative representation for such an experience would be to ship a “change log” containing the history of all operations performed on the client. This representation has the disadvantage of being more verbose than the previous one, but it would make “point in time” rollbacks pretty easy to implement.

Besides these two, there are a few more interesting generic representations for changes in a graph, but in general, they all suffer from the same disadvantage: providing a solution for them does not give the user the level of control that the most complicated scenarios and sophisticated patterns require.

Based on this fact, we decided to adopt the following goal for the design:

Entity Framework won’t define its own unique representation for the set of changes represented in an N-Tier application. Instead, it will provide basic building block APIs that will facilitate the use of a wide range of representations.

This has the desirable consequence that Entity Framework won’t impose a pattern for N-Tier. Both DTO-style and DataSet-like experiences can be built on top of a minimal set of building blocks APIs. It is up to the developer to select the pattern that better suits the application.

Here is a first cut of the new methods that we are proposing to add to the ObjectContext. The new APIs will work with EntityObjects, IPOCO and POCO entities:

public partial class ObjectContext 
{ 
    /// <summary> 
    /// Apply property values to original state of the entity 
    /// </summary> 
    /// <param name="entitySetName">name of EntitySet of root</param> 
    /// <param name="original">entity with original values</param> 
    public void ApplyOriginalValues( 
        string entitySetName, 
        object original) { } 
    
    /// <summary> 
    /// Changes the state of the entity and incident relationships 
    /// </summary> 
    /// <param name="entity">entity to change state of</param> 
    /// <param name="state">new state</param> 
    public void ChangeObjectState( 
        object entity, 
        EntityState state) { } 

    /// <summary> 
    /// Changes state of relationship 
    /// </summary> 
    /// <param name="source">source object of relationship</param> 
    /// <param name="target">target object of relationship</param> 
    /// <param name="relationshipName">name of relationship</param> 
    /// <param name="sourceRole">role of source object</param> 
    /// <param name="targetRole">role of target object </param> 
    /// <param name="state">new state</param> 
    public void ChangeRelationshipState( 
        object source, 
        object target, 
        string relationshipName, 
        string sourceRole, 
        string targetRole, 
        EntityState state) { } 

    /// <summary> 
    /// Changes state of relationship represented by navigation 
    /// property 
    /// </summary> 
    /// <param name="source">source object of relationship</param> 
    /// <param name="target">target object of relationship</param> 
    /// <param name="navigationProperty">navigation property</param> 
    /// <param name="state">new state</param> 
    public void ChangeRelationshipState( 
        object source, 
        object target, 
        string navigationProperty, 
        EntityState state) { } 
    
    /// <summary> 
    /// Changes state of relationships represented by lambda 
    /// expression 
    /// </summary> 
    /// <typeparam name="TSource">type of source entity</typeparam> 
    /// <param name="source">source entity of relationship</param> 
    /// <param name="target">target entity of relationship</param> 
    /// <param name="selector">property selector expression</param> 
    /// <param name="state">new state</param> 
    public void ChangeRelationshipState<TSource>( 
        TSource source, 
        object target, 
        Expression<Func<TSource, object>> selector, 
        EntityState state) { } 
  
} 

ApplyOriginalValues

Similar to the existing ApplyPropertyChanges, but this API keeps the tracked entity intact, only affecting the corresponding original values. This operation only works for entities in Modified, Unchanged or Deleted states. Added entities by definition don’t have original state.

ChangeObjectState

This method will transition the state of a tracked entity passed as argument, but will also have side-effects on incident relationships. In case the new state is modified, it will also mark all properties as modified, regardless of the original and current values.

ChangeRelationshipState

Similar to ChangeObjectState, but sets the relationship between two tracked entities to the provided state. Relationships cannot be in the modified state, but this method can be used to report which relationship should be added, deleted, or unchanged.

The relationship doesn’t need to exist in the context, but the entities do. If the relationship doesn’t exist, it can be created in the new state. For instance, when this code is invoked, a new relationship can be created between customer1 and order1 in the added state:

context.ChangeRelationship( 
    customer1, 
    order1, 
    c=>c.Orders, 
    EntityState.Added); 

The relationship is typically represented by the navigation property but the metadata names of the relationship and roles can also be used in a different overload.

Design notes

All parameters named “entity”, “source” and “target” accept either entities or EntityKeys.
We could choose to deprecate the ApplyPropertyChanges API and replace it with ApplyCurrentValues, which is more consistent name-wise with the new ApplyOriginalValues method.
For convenience, we should add similar APIs in other types like ObjectStateEntry, RelatedEnd and others as appropriate. For instance, the ObjectStateEntry will also have a ChangeState API that will only take the target state parameter.
We could choose to put ChangeObjectState() and the various ChangeRelationshipState() in ObjectStateManager rather than in ObjectContext, since this is a lower level API than everything else currently in the ObjectContext.

Scenario tests

To get a sense of how the new API can be used, we have tried several different approaches and representations. We also have plans to release sample code that will show in more detail different ways to use the new API. In the meanwhile, here are a few simple cases.

General purpose Attach with state resolution delegate

In this scenario between a client and mid-tier, the intention is to allow a client to query for a Customer entity with the Customer’s Order entities. The client then makes a variety of changes to the Customer entity graph including modifying the Customer, and adding or removing Orders. The client would like to send the updated Customer entity graph to the mid-tier so that it can be processed and persisted.

This system will use a simple mechanism to help with change tracking on DTO entities. Each DTO will implement a small interface to help determine the state of the entity using three properties: IsNew, IsModified, and IsDeleted. In this example, original values are not tracked by the client.

public interface IEntityWithChanges 
{ 
    bool IsNew { get; } 
    bool IsModified { get; } 
    bool IsDeleted { get; } 
} 

When the client makes changes to the Customer entity graph, the only requirement in this scenario is that when an entity changes, the appropriate property will return true to help determine the state of the entity.

The mid-tier will expose a service method that implements the logic to update the Customer entity graph. Rather than traverse the Customer, her Orders and OrderLines looking for changes, the service method will use a convention based mechanism that knows how to deal with the IEntityWithChanges interfaces. In this scenario, this mechanism takes the form of an extension method to the ObjectContext that will attach an entity graph to the context, and use a delegate to map each entity that implements IEntityWithChanges to an EntityState so that it can be used with the new ChangeObjectState APIs.

First we can define a method to map from an IEntityWithChanges to an EntityState:

/// <summary> 
/// A mapping from an IEntityWithChanges to an EntityState 
/// </summary> 
public EntityState GetEntityState(object entity) 
{ 
    IEntityWithChanges entityWithChanges = 
       entity as IEntityWithChanges; 
    if (entityWithChanges != null) 
    { 
        if (entityWithChanges.IsNew) 
        { 
            return EntityState.Added; 
        } 
        else if (entityWithChanges.IsModified) 
        { 
            return EntityState.Modified; 
        } 
        else if (entityWithChanges.IsDeleted) 
        { 
            return EntityState.Deleted; 
        } 
    } 
    return EntityState.Unchanged; 
} 

Next we will define the extension method on an ObjectContext that can attach an entity graph and use a mapping delegate to determine the state of each entity in the graph.

/// <summary> 
/// Attach a graph of entities using a supplied mapping function to 
/// determine the entity's state. 
/// In this example, the state of relationships between entities is 
/// determined by the rules of ChangeObjectState: 
/// - if one entity in the relationship is Added, the relationship is 
///   Added 
/// - if an entity is Deleted or Detached, all incident relationships 
///   are Deleted or Detached 
/// </summary> 
/// <param name="entitySetName">name of EntitySet of root</param> 
/// <param name="entityGraph">graph of entities to attach</param> 
/// <param name="entityStateMap">EntityState resolver</param> 
public static void Attach( 
    this ObjectContext context, 
    string entitySetName, 
    object entityGraph, 
    Func<object, EntityState> entityStateMap) 
{ 
    context.AttachTo(entitySetName, entityGraph); 

    // Create a list of unique entities in the graph 
    var allEntities = 
        context 
        .ObjectStateManager 
        .GetObjectStateEntries(EntityState.Unchanged) 
        .Where(entry => !entry.IsRelationship) 
        .Select(entry => entry.Entity); 
        
    // Apply the entityStateMap to each entity 
    foreach (object entity in allEntities) 
    { 
        context.ChangeObjectState(entity, entityStateMap(entity)); 
    } 
} 

Finally we can define our service method that can update a Customer entity graph using the mapping delegate and the Attach extension method.

/// <summary> 
/// A service method to update a Customer's entity graph which 
/// can include modifications to the Customer entity, 
/// new Orders, and removal of Orders 
/// </summary> 
/// <param name="c"></param> 
public void UpdateCustomerOrder(Customer customer) 
{ 
    using (NorthwindEntities context = new NorthwindEntities()) 
    { 
        // Call the Attach extenion method with the mapping delegate 
        // that goes from IEntityWithChanges to EntityState 
        context.Attach("Customers", customer, GetEntityState); 
        context.SaveChanges(); 
    } 
} 

ChangeOrder Service Operation

In this scenario, the mid-tier exposes a method to perform a very specific operation: changing the Customer that owns an Order. The client can set the Order’s Customer to a new Customer or to one that already exists in the store. In either case, the service method uses the old Customer so that it can remove the relationship between the Order and that Customer.

/// <summary> 
/// Updates an Order by changing the relationship between the Order 
/// and a Customer in this case 
/// </summary> 
/// <param name="updatedOrder">order to update</param> 
/// <param name="oldCustomer">previous customer</param> 
public static void ChangeOrder( 
    Order updatedOrder, 
    Customer oldCustomer) 
{ 
    using (NorthwindEntities context = new NorthwindEntities()) 
    { 
        // Tell the context about the entities to persist 
        context.AttachTo("Orders", updatedOrder); 
   
        // Change the state of the new customer 
        // In this example, this is done by the convention: 
        // - Customer.CustomerID is null --> customer is 'Added' 
        // - Customer.CustomerID is not null --> it already exists 
        if(updatedOrder.Customer.CustomerID == "") 
        { 
            // When state of customer changes to 'Added', state 
            // of the relationship between updatedOrder and 
            // pdatedOrder.Customer is also changed to 'Added' 
            context.ChangeObjectState( 
                updatedOrder.Customer, 
                EntityState.Added); 
        } 
        else 
        { 
            if (updatedOrder.Customer.CustomerID != 
                oldCustomer.CustomerID) 
            { 
                // only relationship needs to be marked as 'Added' 
                context.ChangeRelationshipState( 
                    updatedOrder, 
                    updatedOrder.Customer, 
                    o => o.Customer, 
                    EntityState.Added); 

                // Report removal of relationship between 
                // pdatedOrder and oldCustomer 
                context.ChangeRelationshipState( 
                    updatedOrder, 
                    oldCustomer, 
                    "Customer", 
                    EntityState.Deleted); 
             } 
        } 
        
        // Persist the changes to the store 
        context.SaveChanges(); 
    } 
} 

As is always the case, your feedback on this topic is welcome!

Jeff Derstadt
Jaroslaw Kowalski
Diego Vega
and The Object Services Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Foreign Keys in the Conceptual and Object Models

efdesign — Mon, 27 Oct 2008 19:48:00 GMT

If you are reading this, you have probably heard by now about the so called impedance mismatch between the relational world and the object world – and there are a number of concepts in the relational database that don’t translate easily to corresponding concepts supported by the object oriented paradigm. One of these factors that is particularly interesting (and often controversial) is the concept of Foreign Keys and whether or not they belong in your conceptual/object model.

FKs are used to represent relationships in the database – but with objects, the natural way to represent relationships is through real references between objects. So as an object relational mapping platform, should a product support one or the other, or both?

I think most will agree that there is tremendous value in supporting relationships in the model as first class references between objects.

What are some of the issues with supporting both foreign keys and references in the same model? We could take a real world example i.e. LINQ to SQL and its foreign key support to look at some of the benefits as well as the downsides to having foreign keys.

Foreign Key Support in LINQ to SQL

LINQ to SQL takes the simplistic approach of making foreign keys available to you as a scalar property in your entities. In essence, LINQ to SQL allows you to write code dealing with relationships like so:

Product chai = db.Products.Where(p => p.ProductName == "Chai").Single(); 
chai.CategoryID = 1; 
db.SubmitChanges(); 

This is possible in LINQ to SQL mainly because foreign keys are somewhat central to the way relationships are implemented and supported.

This particular example achieves something quite powerful, however - here you have essentially changed the category that the product Chai belonged to without ever having to query and materialize the new category that you associated Chai with.

However, LINQ to SQL also allows you to do this:

Product chai = db.Products.Where(p => p.ProductName == "Chai").Single(); 
Category beverages = db.Categories.Where(c => c.CategoryName == "Beverages").Single(); 
chai.Category = beverages; 

We have been considering for a while about whether or not we want to include support to allow you to expose foreign keys in the model. Let’s see what you gain / lose by having foreign keys in the model.

One of the fundamental issues is that the simple foreign key concept that works so well in the relational model isn’t sufficient to represent relationships in the conceptual model. In the Entity Framework, a relationship is a first class concept that must be mapped to any set of columns in the database – and the important thing here is that these columns don’t have to represent relationships on the database by the means of FKs and constraints. Another point to note is that in the Entity Framework, relationships are always comprised of two ends, and are bi-directional unlike foreign keys.

What does all of this do for the model? This opens up possibilities, such as Referential Integrity constraints that are represented in the EDM even though they may not necessarily be present on the relational model in the store. Bi-directional nature of the relationships makes it possible for you to navigate in both directions along a relationship in a way that is generally not possible with a simple FK.

Foreign Keys are not without value, however. An interesting challenge for us is to figure out how to bring the best aspects of Foreign Keys into the Entity Framework without compromising the richness and flexibility that relationships bring to the table. Let’s look at the upside and the downsides of plain relational foreign keys:

Benefits of Foreign Keys

Keeps it simple (for the simple cases) and allows you to deal with relationship like you deal with them in the database
Technically, you can update relationships without having both ends loaded/materialized. This is however in reality not always interesting since you will likely load both ends but this feature is definitely useful.

Disadvantages of Foreign Keys in the Model

It is a part of the impedance mismatch problem.
It doesn’t allow the concepts that you would expect from relationships in objects (easily getting from one end to the other) for instance.
Having foreign keys as well as object references for relationship navigation presents the problem of two different artifacts representing relationships – this introduces complexity and now you have to make sure that you keep these two in sync.

So where do you stand? Do you like foreign keys or do you think they are evil? Would you like to see Foreign Keys exposed in the model, and if they are available in your object model, would it be sufficient if they were read-only?

Let us know!

Faisal Mohamood
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

EDM and Store functions exposed in LINQ

efdesign — Wed, 08 Oct 2008 11:37:00 GMT

In this post Colin Meek and Diego Vega delve into some enhancements we are planning for LINQ to Entities, anyway over to them...

Entity Framework v1 customers preferring to write their queries using LINQ often hit a limitation on the range of functions and query patterns supported in LINQ to Entities. For some of those customers, having to resort to Entity SQL, or even to Entity SQL builder methods, feels awkward and reduces the appeal of Entity Framework.

There are two things we want to do in order to address this in future versions:

Expand the range of patterns and standard BCL methods we recognize in LINQ expressions.
Provide an extensibility mechanism that people can use to map arbitrary CLR methods to appropriate server and EDM functions.

This blog post expands on the second approach:

It is actually possible for us to improve our LINQ implementation so that all functions defined in the EDM and in the store, and even user defined functions, can be mapped to CLR methods with homologous signatures.

Design

Problem space

There are multiple dimensions to the problem space we want to address:

Functions can be defined in either the conceptual or the storage space
Functions can be defined in either the manifest, or just declared in the model
Functions can be mapped to either static CLR methods or to instance methods on the ObjectContext
This feature specifically targets composable functions

How it looks like: EdmFunctionAttribute

The basis of the extensibility mechanism is a new method-level attribute that carry function mapping information. Here is the basic signature of the attribute’s constructor:

public EdmFunctionAttribute(string namespaceName, string functionName)

The namespaceName parameter indicates the namespace for the function in metadata (i.e. “EDM” or “SQLSERVER”, or other store provider namespace). The functionName parameter takes the name of the function itself.

The following example could be product code or customer code applying the attribute on an extension method (it could be a regular static function) in order to map it to the standard deviation SQL Server function:

public static class SqlFunctions       
{       
    [EdmFunction("SqlServer", "stdev")]       
    public static double? StandardDeviation(this IEnumerable<int?> source)       
    {     
        throw EntityUtil.NotSupported(       
            System.Data.Entity.Strings.ELinq_EdmFunctionDirectCall);       
    }       
} 

Notice that while this method can’t be called directly it can be used in a query like this:

var query =  
    from p in context.Products       
    where !p.Discontinued       
    group p by p.Category into g       
    select g.Select(each => each.ReorderLevel).StandardDeviation(); 

The following example shows how the canonical DiffYear function is mapped:

public static class EntityFunctions        
{       
    [EdmFunction("EDM", "DiffYears")]       
    public static Int32? DiffYears(DateTime? arg1, DateTime? arg2)       
    {       
        throw EntityUtil.NotSupported(System.Data.Entity.Strings.ELinq_EdmFunctionDirectCall);  
    }       
} 

Usage is:

var query =     
    from p in context.Products     
    where EntityFunctions.DiffYears(DateTime.Today, p.CreationDate) < 5     
    select p; 

The following example shows how a user defined function defined in SQL Server can be mapped:

public static class MyCustomFunctions      
{     
    [EdmFunction("SqlServer", "MyFunction")]     
    public static Int32? MyFunction(string myArg)     
    {     
        throw new NotSupportedException("Direct calls not supported");  
    }     
} 

Convention based function name

We can establish that by convention the name of the CLR function defines the value of the functionName parameter. That makes the functionName parameter in the EdmFunctionAttribute optional.

EdmFunctionNamespaceAttribute

To avoid having to always specify the namespaceName for each function, we define a new class-level attribute named EdmFunctionNamespaceAttribute that would define the namespace mapping globally for a given class:

public EdmFunctionNamespaceAttribute(string namespaceName)

Using EdmFunctionNamespaceAttribute and the convention based constructor:

[EdmFunctionNamespace("EDM")]     
public static class EdmMethods      
{     
    [EdmFunction]     
    public static Int32? DiffYears(DateTime? arg1, DateTime? arg2)     
    {     
        throw EntityUtil.NotSupported(     
            System.Data.Entity.Strings.ELinq_EdmFunctionDirectCall);  
    }     
}

How it works

When a method with the EdmFunction attribute is detected within a LINQ query expression, its treatment is identical to that of a function within an Entity-SQL query. Overload resolution is performed with respect to the EDM types (not CLR types) of the function arguments. Ambiguous overloads, missing functions or lack of overloads result in an exception. In addition, the return type of the method must be validated. If the CLR return type does not have an implicit cast to the appropriate EDM type, the translation will fail.

Instance methods on the ObjectContext will be supported as well. This allows the method to bootstrap itself and trigger direct evaluation, as in the following example (definition of the method and sample query):

public static class MyObjectContext : ObjectContext     
{     
    // Method definition     
    [EdmFunction("edm", "floor")]     
    public double? Floor(double? value)     
    {  
        return this.QueryProvider.Execute<double?>(Expression.Call(     
            Expression.Constant(this),     
            (MethodInfo)MethodInfo.GetCurrentMethod(),     
            Expression.Constant(value, typeof(double?))));     
    }     
}     
    
…     
    
// evaluated in the store!     
context.Floor(0.1);     

Without the ObjectContext, the function cannot reach the store! To support this style of bootstrapping, the context needs to expose the LINQ query provider. For this reason, we now expose a “QueryProvider” property on the ObjectContext. This provider includes the necessary surface to construct or execute a query given a LINQ expression.

public class ObjectContext     
{     
    public IQueryProvider QueryProvider { get; }     
} 

If such a method is encountered inline in another query, then we must validate that the instance argument (MethodCallExpression.Object) is the correct context, but the instance is otherwise ignored:

// positive     
var q1 = from p in context.Products select context.Floor(p.Price);     
    
// negative     
var q2 = from p in context.Products select context2.Floor(p.Price); 

A function proxy can sometimes bootstrap itself without an explicit context, e.g. when an input argument is itself an IQueryable:

public static class SqlFunctions     
{     
    [EdmFunction("SqlServer", "stdev")]     
    public static double? StandardDeviation(this IQueryable<int?> source)     
    {     
        return source.Provider.Execute<double?>(Expression.Call(     
            (MethodInfo)MethodInfo.GetCurrentMethod(),     
            Expression.Constant(source)));     
    }     
} 

Nullability considerations

Particularly for functions taking collections, we will need to provide overloads for nullable and non-nullable elements. We don’t want to require awkward constructions like:

var query = (from p in products select (int?)p.ReorderLevel).StandardDeviation(); 

Tool for Generating the Functions

We created a simple internal tool that generates the classes that represent all the EDM canonical function and the SQL Server store functions. The tool will take the function definitions from Metadata and generate the appropriate function stubs/implementations.

The tool will be outside the product and will be run on demand. We expect to make a version of this tool available for provider writers together with the provider samples.

Naming

The methods will be in the following classes:

Namespace	Class name
System.Data.Objects	EntityFunctions
System.Data.Objects.SqlClient	SqlFunctions

Note: The equivalent class in LINQ to SQL is System.Data.Linq.SqlClient.SqlMethods.

The method names will correspond to the name of the EDM/SQL function they represent. The argument names will correspond to the argument names of the EDM/SQL functions as retrieved by the metadata.

The recommendation for provider writers will be to include a similar static class in a namespace of the following form:

System.Data.Objects.[Standard provider namespace].[Standard provider prefix]Functions

Overloads and Implementation

Non-aggregate Functions

For each non-aggregate function we create an overload with all inputs type as nullable of the CLR equivalent of their EDM primitive type, and the return type nullable of the CLR equivalent of their EDM primitive type.

The implementation of the functions (what gets executed if the function is invoked outside an expression tree) will be to throw a NotSupportedException.

Example:

public static class EntityFunctions      
{     
    [EdmFunction("EDM", "DiffYears")]     
    public static Int32? DiffYears(DateTime? arg1, DateTime? arg2)     
    {     
        throw EntityUtil.NotSupported(System.Data.Entity.Strings.ELinq_EdmFunctionDirectCall);  
    }     
}      
      
public static class SqlFunctions      
{     
    [EdmFunction("SqlServer", "DiffYears")]     
    public static Int32? DiffYears(DateTime? arg1, DateTime? arg2)     
    {     
        throw EntityUtil.NotSupported(System.Data.Entity.Strings.ELinq_EdmFunctionDirectCall);  
    }     
} 

Aggregate Functions

For each aggregate function we will provide two overloads, one with IEnumerable<Nullable<T>> and another one with IEnumerable<T>, where T is the CLR equivalent of the EDM primitive type of the input. The implementations of these will check whether the input is IQueryable in which case it will implement the self-bootstrapping.

Example:

[EdmFunction("EDM", "VARP")]     
public static double? VarP(IEnumerable<int> arg1)     
{     
    ObjectQuery<int> objectQuerySource = source as ObjectQuery<int>;     
    if (objectQuerySource != null)     
    {     
        return ((IQueryable)objectQuerySource).Provider.Execute<double?>(Expression.Call(     
            (MethodInfo)MethodInfo.GetCurrentMethod(),     
        Expression.Constant(source)));     
    }     
    throw EntityUtil.NotSupported(System.Data.Entity.Strings.ELinq_EdmFunctionDirectCall);  
}     
    
[EdmFunction("EDM", "VARP")]     
public static double? VarP(IEnumerable<int?> arg1)     
{     
    ObjectQuery<int?> objectQuerySource = source as ObjectQuery<int?>;     
    if (objectQuerySource != null)     
    {     
        return ((IQueryable)objectQuerySource).Provider.Execute<double?>(Expression.Call(     
            (MethodInfo)MethodInfo.GetCurrentMethod(),     
        Expression.Constant(source)));     
    }     
    throw EntityUtil.NotSupported(System.Data.Entity.Strings.ELinq_EdmFunctionDirectCall);  
}     

The Entity Framework team would love to hear your comments.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Model First

efdesign — Wed, 10 Sep 2008 23:55:00 GMT

One of the most painful omissions from the Entity Framework V1 was Model First, which basically means creating a conceptual 'model first' and then deriving a storage model, database and mappings from that.

People ask for this scenario all the time in the forums.

Well Noam, a Program Manager on the Entity Framework Tools team, outlines what we are considering:

Generating Databases from Models

The next release of the Entity Framework will include the ability to generate database schemas from your model. The main entry point into this feature is via the Designer context menu, to which we will add a new option called “Create Database from Model”.

(Note: User interface elements in this walkthrough are not final user interfaces, as the design is still under review, so please provide feedback...)

Selecting this option will bring up the following warning:

We would like users to understand that this feature will regenerate all SSDL and all MSL from scratch.

If you have started from an empty model, you will be asked to specify the target database. This screen is identical to the one shown when reverse engineering a model from a database. The implication here is that you will need an available server and database, which the system will use to determine what “flavor” of DDL to generate.

Once you have selected the target database, you will be presented with a summary screen which will provide a preview of the DDL that will be generated, as well as a tree-view of the objects.

The tree view:

The DDL view:

The DDL in the above screenshot is there merely as a graphic used to show how what the window will look like and is not intended to be representative of the DDL that will be generated by the system when you are actually creating your database (more about this soon). The DDL will however be read-only: Since it is generated by a template, editing of the results should either be done in the template, or in a separate DDL file which will not get regenerated.

Two options will be available:

- Save the DDL (off by default). This option will add the DDL as a dependent file under your EDMX file.

- Deploy the DDL (on by default). This option will deploy the DDL to the specified target database.

The generated DDL will not migrate data or schema - by default your database will be recreated from scratch.

Out of the box, we will support the Table-per-Type mapping strategy, meaning that we will create a table for each of your types and subtypes. For example, for a model like this…

…the following schema will be generated:

Of interest here is that PK-to-PK constraint between the Customer and Persons table. This helps enforce the inheritance relationship and the creation of foreign key columns to represent the various associations. In addition, the engine will create clustered keys on primary keys, and indexes on foreign keys that represent associations.

Under the Hood

The model first process is implemented using a Windows Workflow Foundation workflow that looks like this:

Here is what the stages do:

Stage

Purpose

Stage

Purpose

CSDLtoSSDL

Creates the mappings (MSL) and database store model (SSDL) in the EDMX file.

SSDLtoDBSchema

Converts the SSDL to the format used by the Microsoft.Data.Schema APIs. This format is used as a “universal” database description format and includes physical information not present in the Entity Frameworks store model, such as indexes.

GenerateDDL

Uses the Microsoft.Data.Schema APIs to convert the universal format to store-specific DDL. In this release, we will support a minimum of SQL 2005 and SQL 2008. A provider model is in place, however, and we hope to add support for additional databases.

SuspendToConfirm

This activity pauses the workflow to allow the wizard to display the DDL.

DeployToDatabase

Deploys the DDL to the target database.

OutputDDL

Writes the DDL to the file system.

These stages are expressed in a XAML file which will be placed underneath your EDMX, to allow for customization: You can add your own steps or replace ones we have created with steps that you write.

Templates

Several of the steps above make use of templates to provide an additional point of control for users, and this is where we expect most of the customization to happen. These templates use the T4 Engine that is included in Visual Studio. We are currently working on making these templates as simple as possible by providing a set of supporting APIs that provide metadata collections that are designed for artifact generation – for example, a collection of all inherited properties for a type, or a collection of both inherited and defined properties. There are three templates:

Template

Purpose

Template

Purpose

CSDL to SSDL

Creates the SSDL for the target database.

CSDL to MSL

Creates the MSL for mapping the CSDL to the generated SSDL.

SSDL to DBSchema

Creates the physical database description, including elements such as foreign keys and indexes.

So, for example, if you need all table names to start with “_tbl”, you would modify the SSDL generation template to give all tables the appropriate name, and also modify the MSL template to provide the appropriate mappings. The DBSchema template will not need to be modified as it will automatically pick up the changes made to the SSDL which is its input.

As another example, if you wish to change the mapping strategy from Table-per-Type to Table-per-Hierarchy, you would need to change this same pair of templates.

Finally, if you needed to support a database for which no Microsoft.Data.Schema provider is available, you could replace the GenerateDDL step with your own template-driven activity which could them transform the schema “manually” to your store’s DDL format.

We hope this gives you enough information to understand the design and intent of this feature.

We would love to hear your comments.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Structural Annotations - One Pager

efdesign — Tue, 12 Aug 2008 22:50:00 GMT

In V1 of the Entity Framework it is possible to annotate a schema using attributes declared in another XSD.

However XML attributes are a very limited form of annotation. It would be better if we could annotate using full elements.

This is what we are calling Structural Annotations.

This feature will allow both customers and partners like Reporting Services to modify the model so that it includes information important to them which can’t be captured in vanilla EDM format.

1 EDM extensions

While it should be possible to annotate any level in the XML hierarchy, there are however some restrictions.

The general rule is you can annotation any CSDL / SSDL element that has a corresponding MetadataItem which in practice means everything except <Using>, <Schema>, <Key> and <PropertyRef> elements.

These annotations should be named, so that they can be accessed using the same API as today.

i.e. something like this:

1.1 Key points to notice:

In the above example both the CLR and RS namespaces must have been declared somewhere. Probably at the root of the CSDL somewhere, i.e. something like this:
<Schema xmlns:RS="http://schemas.microsoft.com/RS/2006" xmlns:CLR=http://schemas.microsoft.com/net/3.5…
Alternatively the namespace could be in lined in the annotation:
<Security xmlns="http://schemas.microsoft.com/RS/2006"> …
</Security>
In both cases the unique identity of the annotation is in the form {namespace}:{elementname}. So for the RS Security annotation example, irrespective of how the RS namespace is introduced the identity would be:
http://schemas.microsoft.com/RS/2006:Security
Structural annotations should always follow all other sub-elements i.e. when structurally annotating an <EntityType> element the annotation element should follow all <Key> <Property> and <NavigationProperty> elements.
Attribute based annotations (as supported in V1 and used for things like CodeGen) are scoped to the same annotation identity namespace. Hence care should be taken to verify that annotations using either approach don’t have colliding identities.
It should be possible to have more than one named “Structural Annotation” per CSDL (or SSDL) element. Indeed element names can collide so long as the combination of Namespace + Element Name is unique for a particular element (or more specifically each MetadataItem). This means for instance you can’t have two <RS:Security> elements under any one MetadataItem.

1.2 Positive and Negative Cases:

This:

…would be invalid, because of the identity collision, likewise this would also be invalid:

where-as this is fine:

… since all the above annotations have unique identities.

1.3 What can an <Annotation> contain?

A structural annotation is simply an XML element. As such it can be considered a root of an XML document that can contain any valid XML structures. These structures are simply ignored by the Entity Framework and Metadata APIs.

1.4 What elements can be annotated?

If an element in the CSDL or SSDL has a corresponding MetadataItem in the Metadata API that element should support structural annotations. Since <Using>, <Schema>, <Key> and <PropertyRef> elements have no corresponding MetadataItem(s) they don’t support structural annotations.

Notice while TypeUsage(s) have no CSDL representation today,they are MetadataItem(s) so they could in theory be annotated in the future. For example if a public mutable metadata API is produced.

2 Metadata API Changes:

In this example we are using the Entity Frameworks Metadata API to get the EdmType for an EntityType called Content. From that we then get the MetadataProperty called http://schemas.microsoft.com/RS/2006:Security:

EdmType contentType = metadataWorkspace.GetEdmType(“MyNamespace.Content”);
if (contentType.MetadataProperties.Contains(
"http://schemas.microsoft.com/RS/2006:Security")
{
    MetadataProperty annotationProperty = contentType.MetadataProperties[ "http://schemas.microsoft.com/RS/2006:Security"];
    object annotationValue = annotationProperty.Value;
    //Do something...
}

Notice that the way you access structural annotations is the same as the way you access attribute annotations, i.e. by name/identity, in the MetadataProperties collection.

This of course means that their names can’t collide, name collisions should be picked up when metadata is loaded from persistent formats (like CSDL) or when annotations are created by hand in the any future mutable metadata API.

2.1 MetadataProperty TypeUsage

When creating a MetadataProperty to hold the XElement for the structural annotation, we need to assign a TypeUsage too. For V1 style attribute annotations, the TypeUsage we chose was based on Edm.String. Ideally we should use Edm.XML etc, but since this is not currently available, we will use Edm.String in the meantime. We have filled a bug to change this if/when [j1] XML becomes an EDM type.

3 What is the MetadataProperty.Value?

The Value of the MetadataProperty will be a XElement for structural annotations originating from CSDL.

So if you know that the annotation is a structural annotation you can access it like this:

MetadataProperty annotationProperty =
contentType.MetadataProperties["CLR:Attributes"];

XElement myAnnotation = annotationProperty.Value as XElement;

3.1 XElement is rooted at the <Annotation> element

The XElement returned will be the XElement containing everything including the structural annotation itself.

3.2 XElement is mutable

There is nothing to stop the consumer of the Metadata API from modifying the XElement since it is simply a reference, to an instance of a type the Entity Framework doesn’t control.

We could address this by cloning on access to make things a little safer. But this negatively affects the mainline scenario, namely where a customer uses annotations purely for read, and is therefore not required.

The important thing to understand is that should a user modify the XElement all bets are off. We should provide guidance both via API hints:

/// <summary>
/// ....
/// <remarks>
/// Should this return a reference type like an XElement, you should consider
/// this immutable, no guarantees are made about behavior if this reference
/// type is modified in anyway.
/// </remarks>
/// </summary>
public object Value { get {

And via appropriate documentation.

3.3 No universal annotation format

Rather than try to invent a new universal data format for annotations, we decided to be pragmatic.

So we are limiting ourselves to XML for structural annotations loaded from CSDL (an XML format).

However since the MetadataProperty.Value is of type object, in a world where we have an alternative EDM representation (like in memory using mutable metadata for instance) we could easily assign something other than an XElement to the MetadataProperty.Value, like for example a xaml Serializable CLR type instance.

4 Out of Scope

The current plan is to simply support annotation in the CSDL & SSDL. Meaning annotating MSL is currently out of scope.

If we uncover meaningful important scenarios for MSL annotations, we will undertake that work separately.

Incidentally supporting structural annotations in MSL touches a completely separate code path, and as a result will require roughly the same amount of work. Additionally in order to make undertaking this work meaningful we would first need to expose a public mapping metadata API.

....

As always we are keen to hear any feedback you have on this item.

Do you think you will find a use for Structured Annotations?

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Discussion about API changes necessary for POCO:

efdesign — Fri, 01 Aug 2008 19:24:00 GMT

Evolving an API to support new requirements, like POCO, while maintaining backward compatibility is challenging.

The following design discussion from members of the Object Services team illustrates some of the issues and hard calls involved.

Have a read, and tell us what you think.

In particular are we missing something, or overstating the importance of something? Let us know...

Anyway over to Diego and Mirek...

POCO API Discussion: Snapshot State Management

What is in a snapshot?

In Entity Framework v1 there is a single way for the state manager to learn about changes in entity instances: the change tracking mechanism is set in such a way that the entity instances themselves notify the state manager of any property change.

This works seamlessly if you use default code-generated classes, and it is also part of the IPOCO contract for developers willing to create their own entity class hierarchies.

For version 2, we are currently working on a snapshot-based change tracking mechanism that removes the requirement from classes to send notifications, and enables us to provide full POCO support.

The basic idea with snapshots is that when you start tracking an entity, a copy of its scalar values is made so that at a later point – typically but not always at the moment of saving changes into the database - you can detect whether anything has changed and needs to be persisted.

The challenge

When we created the v1 API, we made a few assumptions that were totally safe with notification-based change tracking but don’t completely prevail in a snapshot change tracking world.

We now need to choose the right set of adjustments for the API for it to gracefully adapt to the new scenarios we want to support.

Mainline scenario: SaveChanges() Method

In notification-based change tracking, by the time SaveChanges is invoked, the required entity state information is ready to use.

With snapshot thought, a property by property comparison needs to be computed for each tracked entity just to know whether it is unchanged or modified.

Once the snapshot comparison has taken place, SaveChanges can proceed normally.

In fact, assuming the process is triggered implicitly on every call to SaveChanges, a typical unit of work in which the program queries and attaches entities, then modifies and adds new entities, and finally persists the changes to the database, works unmodified with POCO classes:

Category category = context.Categories.First();

category.Name = "some new name"; // modify existing entity

Category newCategory = new Category(); // create a new entity

newCategory.ID = 2;

newCategory.Name = "new category";

context.AddObject("Categories", newCategory); // add a new entity

context.SaveChanges(); // detects all changes before saving

Things get more complicated when a user wants to use lower level APIs that deal with state management.

State Manager Public API

ObjectStateManager and ObjectStateEntry classes comprise the APIs that you need to deal with if you want to either get input from, or customize the behavior of Entity Framework’s state management in your own code.

Typically, you use these APIs if you want to:

Query for objects that are already loaded into memory
Manipulate the state of tracked entities
Validate state transitions or data just before persisting to the database
Etc.

As the name implies, ObjectStateManager is Entity Framework’s state manager object, which maintains state and original values for each tracked entity and performs identity management.

ObjectStateEntries represent entities and relationships tracked in the state manager. ObjectStateEntry functions as a wrapper around each tracked entity or relationship and allows you to get its current state (Unchanged, Modified, Added, Deleted and Detached) as well as the current and original values of its properties.

Needless to say, the primary client for these APIs is the Entity Framework itself.

ObjectStateEntry.State Property

The fundamental issue with snapshot is exemplified by this property.

With notification-based change tracking, the value for the new state is computed on the fly on each notification, and saved to an internal field. Getting the state later only encompasses reading the state from the internal field.

With snapshot, the state manager no longer gets notifications, and so the actual state at any time depends on the state when the entity began being tracked, whatever state transitions happened, and the current contents of the object.

Example: Use ObjectStateEntry to check the current state of the object.

Category category = context.Categories.First();

category.Name = "some new name"; // modify existing entity

ObjectStateEntry entry = context.ObjectStateManager.GetObjectStateEntry(category);

Console.WriteLine("State of the object: " + entry.State);

Question #1: What are the interesting scenarios for using the state management API in POCO scenarios?

Proposed solutions

In above example there are two possible behaviors and it's not obvious for us which one is better:

Alternative 1: Public ObjectStateManager.DetectChanges() Method

In the first alternative, computation of the snapshot comparison for the whole ObjectStateManager is deferred until a new ObjectStateManager method called DetectChanges is invoked. DetectChanges would iterate through all entities tracked by a state manager and would detects changes in entity's scalar values, references and collections using a snapshot comparison in order to compute the actual state for each ObjectStateEntry.

In the example below, the first time ObjectStateEntry.State is accessed, it returns EntityState.Unchanged. In order to get a current state of the "category" entity, we would need to invoke DetectChanges first:

Category category = context.Categories.First();

category.Name = "some new name"; // modify existing entity

ObjectStateEntry entry = context.ObjectStateManager.GetObjectStateEntry(category);

Console.WriteLine("State of the object: " + entry.State); // Displays "Unchanged"

context.ObjectStateManager.DetectChanges();

Console.WriteLine("State of the object: " + entry.State); // Displays "Modified"

DetectChanges would be implicitly invoked from within ObjectContext.SaveChanges.

Pros:

· User knows when detection of changes is performed

· DetectChanges is a method that user would expect to throw exceptions if some constraint is violated

· This alternative requires minimal changes in Entity Framework current API implementation

Cons:

· Since DetectChanges iterates through all the ObjectStateEntries, it is a potentially expensive method

· User has to remember to explicitly call DetectChanges() before using several methods/properties, otherwise, their result will be inaccurate:

o ObjectStateEntry.State

o ObjectStateEntry.GetModifiedProperties()

o ObjectStateManager.GetObjectStateEntries()

· This alternative implies adding a new method to ObjectStateManager

Alternative 2: Private ObjectStateEntry.DetectChanges() Method

In the second alternative, there is no public DetectChanges method. Instead, the computation of the current state of an individual entity or relationship is deferred until state of the entry is accessed.

Pros:

· User doesn't have to remember explicitly calling DetectChanges to get other APIs to work correctly

· Existing API works as expected in positive cases regardless of notification-based or snapshot tracking

· No new public API is added

Cons:

· The following methods now require additional processing to return accurate results in snapshot:

o ObjectStateEntry.State

o ObjectStateEntry.GetModifiedProperties()

o ObjectStateManager.GetObjectStateEntries()

· Existing API works differently in negative cases in notification-based and snapshot tracking:

o some of the existing methods would start throwing exceptions

o would require to introduce a new state of an entry – Invalid state (see below for details)

Question #2: What API pattern is better? Having an explicit method to compute the current state based on the snapshot comparisons or having the state to be computed automatically when accessing the state?

How this affects other APIs

While the State property exemplifies the issue, there are other APIs that would have different behaviors with the two proposed solutions.

ObjectStateEntry.GetModifiedProperties() Method

GetModifiedProperties returns the names of the properties that have been modified in an entity. Similar to the State property, with notification-based change tracking, an internal structure is modified on the fly on each notification. Producing the list later on only encompasses iterating through that structure.

With snapshot, the state manager no longer gets notifications, and at any given point in time, the actual list of modified properties really depends on a comparison between the original value and the current value of each property.

Therefore, for alternative #1, this APIs will potentially return wrong results unless it is invoke immediately after DetectChanges. For alternative #2, the behavior would be always correct.

ObjectStateManager.GetObjectStateEntries()

This is the case in which an implementation detail that was a good idea for notification-based change tracking stops offering performance benefits in snapshot. Internally, ObjectStateManager stores ObjectStateEntires in separate dictionaries depending on their state. But in snapshot, any unchanged or modified entity can become deleted because of a referential integrity constraint, and any unmodified entity can become modified.

In alternative #1, DetectChanges would iterate thought the whole contents of the ObjectSateManager, and thus it would update the internal dictionaries. Once this it done, it becomes safe to do in-memory queries using GetObjectStateEntires the same way it is done today.

In alternative #2, GetObjectStateEntires would need to look in unchanged, modified and deleted storage when asked for deleted entries; also in unchanged and modified storage when asked for modified entities.

Querying with MergeOption.PreserveChanges

In Entity Framework, MergeOptions is a setting that changes the behavior of a query with regards to its effects on the state manager. Of all the possibilities, PreserveChanges requires an entity to be in the Unchanged state in order to overwrite it with values coming from the database. In order for PreserveChanges to work correctly, accurate information on the actual state of entities is needed.

Therefore, with alternative #1, querying with PreserveChanges will not have a correct behavior unless it is done immediately after invoking DetectChanges. For alternative #2, querying with PreserverChanges would behavior correctly at any time.

Referential Integrity constraints

When there is a referential integrity constraint defined in the model, a dependent entity may become deleted if the principal entity or the relationship with the principal entity becomes deleted.

For alternative #1, DetectChanges would also trigger Deletes to propagate downwards throw all referential integrity constraints.

For alternative #2, finding out the current state of an entity that is dependent in a RIC would requires traversing the graph upwards to find if either the principal entity or a relationship has been deleted.

Mixed Mode Change Tracking

At the same time we are working on snapshot change tracking, we are also working on another feature, denominated dynamic proxies. This feature consists of Entity Framework automatically creating derived classes of POCOs that override virtual properties. Overriding properties enables us to inject automatic lazy loading on property getters, and notifications for notification-based change tracking in property setters. This introduces a subtle scenario: when using proxies, it is possible that:

a. Not all properties in the class are declared virtual. The remaining properties need to still be processed using snapshot.

b. Sometimes, non-proxy instances of POCOs and proxy instances of the same type have to coexist in the same ObjectStateManager. For the former, it will have to use snapshot tracking. For thelatter, a combination of snapshot and notifications.

All in all, it becomes clear that the actual state of an entity does not entirely depend on the value of the internal state field, nor on the snapshot comparison, but on a combination of both.

ObjectStateEntry.SetModified()

As with mixed mode change tracking, SetModified() requires a combination of the internal state and the snapshot comparison to return valid results.

Handling of invalid states

When working in notification-based change tracking, Entity Framework throws exceptions as soon it learns that entity key values has been modified. With the default code-generated classes, the exception prevents the change for being accepted.

For alternative #1, DetectChanges can throw exceptions if some model constraint (i.e. key immutability) is violated. It is too late to prevent the value from changing.

Example:

Category category = context.Categories.First();

category.ID = 123; // Modify a key property. This would throw if category wasn't a POCO class.

context.ObjectStateManager.DetectChanges(); // Throws because key property change was detected.

For alternative #2, reading the state of modified properties from an entity with modified keys could throw an exception:

Example:

Category category = context.Categories.First();

category.ID = 123; // Modify a key property. This would throw if category wasn't a POCO class.

ObjectStateEntry entry = context.ObjectStateManager.GetObjectStateEntry(category);

Console.WriteLine("State of the object: " + entry.State);

// Throws because key property change was detected.

Getting an exception thrown here would be unexpected. But there is an alternative design that is to define a new EntityState that indicates that an entity is Invalid. This new state would account for the fact that POCO classes per se do not enforce immutable keys.

Since EntityState is a flag enum, Invalid could be potentially combined with other states.

SaveChanges would still need to throw an exception if any entity in the state manager is invalid.

It would be possible to query the state manager for entities in the Invalid state using GetObjectStateEntries method.

Question #3: Is it better to have an Invalid state for entries or should the state manager just throw exceptions immediately every time it finds a change on a key?

Our questions:

1. What are the interesting scenarios for using the state management API in POCO scenarios?

2. What API pattern is better? Having an explicit method to compute the current state based on the snapshot comparisons or having the state to be computed automatically when accessing the state?

3. Is it better to have an Invalid state for entries or should the state manager just throw exceptions immediately every time it finds a change on a key?

---

We really want to hear your thoughts on the above questions.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Entity Framework Design : Entity Framework

Code Only – Further Enhancements

API Refactoring

Mappings are now part of Configurations

TPH mapping syntax no longer violates DRY

MapSingleType and MapHierarchy

Foreign Keys

Missing Navigation Properties

Complex Types

Registering Entity Sets

Association Mapping

Interesting Column Names

Extracting the EDMX

Setting the StoreType

DDL Provider Model

Summary

Extending the Entity Framework Provider Model to support DDL

Provider Model Changes

Simplifying Wrapping Providers

End User API

Summary:

LINQ to SQL to Entity Framework conversion template

Downloading the Metadata Conversion Template

Walkthroughs

Caveats and Known Issues

Feedback

Code Only Enhancements

Registering NavigationProperty inverses:

Specifying Property Facets:

Encapsulating Facet Configuration

Specifying the Tablename

Specifying Mappings:

Interpreting a Mapping

LINQ Comprehension Syntax

Specifying Facets with Mapping

Specifying the table

Specifying Inheritance:

Table Per Hierarchy (TPH)

Table Per Type (TPT)

Table Per Class (TPC)

Default Foreign Key Locations:

Specifying FK Mappings:

Specifying Join Table Mappings:

Entity Splitting:

Encapsulating all the Configuration:

Summary:

Code Only

How it works:

Overriding Conventions

Foreign Keys in the Entity Framework

Background

So what's been added?

How do I use an "FK Association"?

Keeping FKs and References in Sync

How do I create an "FK Association"?

Summary

Customizing Entity Classes in VS 2010

Customize the code generated by the Entity Designer with T4 templates

Walkthrough

Preconditions:

Walkthrough:

Design

Entity Framework Runtime

Entity Designer

Multi Targeting considerations

How can 3rd parties plug in?

A few key takeaways:

Model Defined Functions

Using the Function via eSQL:

Using the Function via LINQ:

Summary

Pluralization

N-Tier Improvements for Entity Framework

DataSet

Data Transfer Objects

REST

Goals

Design

ApplyOriginalValues

ChangeObjectState

How can 3^rd parties plug in?