We’ve come a long way since the last post on Code-Only. So it’s high time for another update.

We’ve been working really hard on Code-Only revving the design, and spotting missing capabilities and responding to feedback both internal and external etc. 

The current plan still holds.  Code-Only will not be in .Net 4.0 with the possible exception of the DDL features described in the last post.  For that portion, the implementation and requirements are more clear to us, and because making the DDL stuff work requires changes to things already in the .NET framework, and getting provider writers lined up, we are still working hard to get the DDL changes into .NET 4.0.  The rest of Code-Only will continue to evolve, and we will ship another preview or two, before getting it rolled into .NET as soon as we can after 4.0 ships.

API Refactoring

As hinted above, we’ve spent a lot of time validating code only against real world scenarios, and thinking about how the customer code hangs together etc. As a results we’ve done some API refactoring.

Mappings are now part of Configurations

In the last version Mappings were derived from Configurations, which had some strange side-effects on the API.

We’ve re-arranged things now so that Configurations hold mappings, which are terminated by assignment to a table.

builder.Entity<Customer>()
   .HasKey(c => c.ID)
   .MapSingleType(c => new {
         cid = c.ID,
         nme = c.Name
      }
   )
   .ToTable(“dbo.custs”);

This maps the Customer entity to the ‘dbo.Custs’ table, the ID property to the cid column, the Name property to the nme column, and registers the ID property as the EntityKey / Primary Key.

As you can see we’ve also added a helper method called Entity<TEntity>() so that you can fluently create configurations and mappings. You might also have noticed that we have removed RegisterKey and replaced it with HasKey which we feel is more inline with our goal of having an intentional API.

TPH mapping syntax no longer violates DRY

The previous syntax for specifying a TPH mapping forced you to repeat mappings for columns that could easily have been inherited.

This was unfortunate because we normally encourage TPH because generally it has the best performance characteristics.

This is where MapHierarchy() and Case() come in.

Imagine if we want to write these classes using TPH:

public class Person{
   public virtual int ID {get;set;}
   public virtual string Firstname {get;set;}
   public virtual string Surname {get;set;}
}
public class Employee: Person
{
   public virtual Employee Manager {get;set;}
   public virtual List<Employee> Reports {get;set;}
}

You would now do it like this:

builder
   .Entity<Person>()
   .HasKey(p => p.ID)
   .MapHierarchy()
   .Case<Person>(
      p => new {
         p.ID,
         p.Firstname, 
         p.Surname,
         thisIsADiscriminator = “P”
      }
   )
   .Case<Employee>(
       e => new {
         manager = e.Manager.Id
         thisIsADiscriminator = “E”
       }
   )
   .ToTable(“dbo.People”);

By using Case<> mappings are re-used for derived type, so there is no need to repeat column mappings for ID, Firstname and Surname.

Which means the Employee Case statement is only responsible for mapping properties and references declared by the Employee class, and for specifying a new discriminator value.

The discriminator column and values are there so the Entity Framework can distinguish between rows in the dbo.People table that represent Person objects and those that represent Employee objects.

MapSingleType and MapHierarchy

Because Mappings are now part of the configuration, there is no need to create them independently and assign them.

You now simply MapXXX() methods on the EntityConfiguration.

The available mapping methods are:

• MapSingleType(λ)
• MapHierarchy(λ)
• MapHierarchy().Case<TEntity>(λ).Case<TDerived>(λ).

If you know the Entity Frameworks MSL files these correspond to:

• Type
• OfType
• OfType with an addition requirement for a type discriminator

These mappings can be combined in many interesting ways, but in general this table shows the recommended way to do most common mappings:

Foreign Keys

In Beta2 of .NET 4.0 we added FK Association Support, so Code-Only needs a way to link a FK property and an Navigation Property together.

Given this class:

public class Product{
   public virtual int ID {get;set;}
   public virtual string Name {get;set;}
   public virtual Category Category {get;set;}
   public virtual int CategoryID {get;set;}
}

You need to be able to tell code-only that Category.ID and CategoryID should have the same value.

You do it like this:

builder.Entity<Product>()
   .Relationship(p => p.Category)
   .HasConstraint((p,c) => p.CategoryID == c.ID);

This says that c.ID (i.e. p.Category.ID) must equal p.CategoryID, which tells code-only that p.CategoryID is an FK property and p.Category is a navigation property backed by this FK property.

HasConstraint(λ) can also be used in conjunction with Relationship(λ).FromProperty(λ) like this:

builder.Entity<Product>()
   .Relationship(p => p.Category)
   .FromProperty(c => c.Products)
   .HasConstraint((p,c) => p.CategoryID == c.ID);

Which tells code only that Product.Category and Category.Products are inverses and that Product.CategoryID and Product.Category.ID must be the same, which implies Product.CategoryID is an FK property.

Missing Navigation Properties

Sometimes you don’t have a navigation property or FK property in the Entity which would naturally hold the FK in the database. For example imagine this scenario:

public class Product{
   public virtual int ID {get;set;}
   public virtual string Name {get;set;}
}

public class Category{
   public virtual int ID {get;set;}
   public virtual string Name {get;set;}
   public virtual List<Product> Products {get;set;}
}

Here there is ‘probably’ a one to many relationship between Categories and Products, and that is probably best modeled using an FK in the products table.

If you start mapping the Product entity you need a way to map the FK column, but there is no reference (i.e. Product.Category) and no FK property (i.e. Product.CategoryID) to map.

So we added the ability to create a fake navigation property to help out:

builder.Entity<Product>().MapSingleType(
   p => new {
       pid = p.ID,
       nme = p.Name
       cid = EntityMap.Related<Category>(c => c.Products).ID
   }
).ToTable(“dbo.Products”);

Here the interesting part is EntityMap.Related, it creates a fake navigation property, just so you can use it in the mapping.

The method signature looks like this:

public static TEntity Related<TEntity>(
    Expression<Func<TEntity, object>
);

Notice that this function simply returns TEntity, so in the above mapping fragment, we are mapping the ID property of the Category class to the cid column, in the ‘dbo.Products’ table.

Complex Types

Code-Only’s default behavior is to ignore properties that are not recognized as either a primitive type or an EntityType. So to support Complex Types we need a mechanism to register one, like this:

var addressConf = builder.ComplexType<Address>();

This returns a ComplexTypeConfiguration through which you can configure the properties of the ComplexType in the same way you configure the properties of an Entity:

addressConf.Property(a => a.Street).HasMaxLength(100);
addressConf.Property(a => a.Zip).HasMaxLength(10);

Now if an entity is mapped by convention and it has a ComplexType property we will automatically introduce columns for each property of the ComplexType.

If you want to explicitly map the Entity mapping the ComplexType property is pretty simple too:

builder.Entity<Person>().MapSingleType(
   p => new {
      p.ID,
      fn = p.Firstname,
      sn = p.Surname,
      street = p.Address.Street,
      zip = p.Address.Zip
   }
);

Registering Entity Sets

Some of our early CodeOnly adopters wanted to use Code Only without having an specialized ObjectContext, and in theory with is possible:

builder = contextBuilder.Create<ObjectContext>();
builder.Entity<Person>().HasKey(p => p.ID);

But what we found is you have no control over the name of the set generated. In this case we will generate an EntitySet called PersonSet, but what if I the name should be People?

To address this issues we have added this method:

builder.RegisterSet<Person>(“People”);

Not only is this useful for specifying names,  it also allows you to intentional specify what EntitySets you need. The alternative would be to encode your intent into calls to Entity<TEntity> and Relationship<TRelated>(..).

Association Mapping

We also need to add support for mapping associations that are not co-located with the Entity. For example take the relationship between Jobs and Candidates. If a candidate can apply for multiple jobs this is a classic example of a many to many relationship.

If you want to take control of how this relationship is mapped you need a way to specify the mapping for the relationship, like this:

builder
  .Entity<Job>()
  .Relationship(job => job.Applicants)
  .FromProperty(candidate => candidate.Applications)
  .Map(“dbo.JobApplications”,
           (job,candidate) => new {
               applicantId = candidate.Id,
               jobId = job.Id
            }
       );

This does a number of things, first it indicates that Job.Candidates and Candidate.Applications are opposite ends of the same relationship.

Then the Map() call indicates that the relationship should be stored in the “dbo.JobApplications” join table, with a compound PK made up of the applicantId and jobId columns.

The applicantId holds the candidate.Id, and the jobId holds the job.Id, and in both cases Code Only will emit an FK constraint to ensure referential integrity. 

Interesting Column Names

The existing mapping syntax that uses anonymous types for the table and column definitions, limits you to column names that are valid CLR identifiers.

So for example if you need to map to a column with a space in the name you can’t.

To rectify this we added an alternate mapping syntax:

builder.Entity<Product>().MapSingleType(
   p => EntityMap.Row(
      EntityMap.Column(p.ID, “p i d”),
      EntityMap.Column(p.Name),
      EntityMap.Column(p.CategoryId, “c i d”)
   )
).ToTable(“dbo.Products”);

This snippet of code, maps the Product entity to the “dbo.Products” table and the ID property to a ‘p i d’ column, the Name property to a Name column and the CategoryId property to the ‘c i d’ column.

This alternate API has another advantage too: it makes it easier to create mapping expressions programmatically, primarily because you no longer need to create an anonymous type when building the expression. Which is really useful if you want to configure code-only at runtime.

Extracting the EDMX

We’ve also added the ability to get the EDMX that Code Only is producing internally, either via an XmlWriter or as an XDocument:

DbConnection connection = // some code to get a connection.
XDocument document = builder.GetEdmx(connection);

// or using XmlWriter
var swriter = new StringWriter();
var xwriter = new XmlTextWriter(swriter);
builder.WriteEdmx(connection, xwriter);

This is pretty handy if you want to use Code Only to build your model and mappings but then pull them out so you work with the XML and our designer tooling.

Setting the StoreType

When producing the Storage Model Code Only asks the current database provider to return an appropriate store type for the CLR type and facets specified.

But some CLR types and facet combinations can map to multiple possible database types. For example fixed length byte[] can map to both binary and timestamp. So in this situation SqlClient returns binary by default.

But you might need a timestamp:

builder
   .Entity<Order>()
   .Property(o => o.Version) 
   .HasStoreType(“timestamp”)
   .IsComputed();

This tells Code Only that the Version property of Order (which is a byte[]) is computed in the database, and is actually a ‘timestamp’ column, which is computed in the database after every insert / update.

DDL Provider Model

In our last post we covered our plans around the DDL provider model, so check that out.

Summary

As you can see Code Only is now looking much more complete.

But it isn’t completely finished yet, we are still working on the rough edges, so as always we are very interested to get your feedback.

In particular are there things you think we should add / rename / simplify?

Alex James
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

As part of the first previews of Code-Only we shared some code to create a database:

// Create a context using code-only
using (var mycontext = builder.Create(dbConnection))
{
    // Create the database if it doesn’t already exist
    if (!myContext.DatabaseExists())
        myContext.CreateDatabase();
    // Standard EF code goes here.
}

But in the first preview of Code-Only this code only worked against SQL Server.

The problem is that our public Provider Model (i.e. DbProviderServices) has no Database Definition Language or DDL features.

Provider Model Changes

To support DDL features across different databases, we plan to extend DbProviderServices, with DDL specific services.

These services will be accessed through these public methods:

public string CreateDatabaseScript(
   string providerManifestToken,
   StoreItemCollection storeItemCollection);

public void CreateDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

public bool DatabaseExists(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

public void DeleteDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

Now internally those methods will call through to the following ‘protected virtual’ methods which will do the actual work:

protected virtual string DbCreateDatabaseScript(
    string providerManifestToken,
    StoreItemCollection storeItemCollection);

protected virtual void DbCreateDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

protected virtual bool DbDatabaseExists(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

protected virtual void DbDeleteDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

And the base implementations of these methods in DbProviderServices will simply throw ProviderIncompatibleExceptions.

Which means the provider writers job will be to override these ‘protected virtual’ methods with an implementation that makes sense for their backend database.

The key is to understand that the StoreItemCollection (aka the SSDL or StorageModel part of the EDMX) represents the intended shape of the database.

This means the provider writer will need to iterate over the EntitySets (tables) and the corresponding EntityTypes (table structures) in the StoreItemCollection and create / drop / script the database and tables as required.

Provider writers will be expected to override these functions so that:

• DbCreateDatabaseScript: creates a native text command to create the tables and foreign key constraints defined in the StoreItemCollection. I.e. for SqlClient this would be the contents of a .sql DDL file.
• DbCreateDatabase: is similar to DbCreateDatabaseScript except it should actually goes ahead and create the database, tables and foreign key constraints.
• DbDatabaseExists: checks to see if the database exists. The SqlClient provider will simply check that the database itself exists, but custom provider writers could get more fancy and check to see if every table / foreign key constraint is found too.
• DbDeleteDatabase: should go ahead and delete the database, or if the database server has a single database model (like Oracle) the provider writer should delete just the tables defined in the StoreItemCollection.

Simplifying Wrapping Providers

We are also planning something to simplify writing Wrapping Providers. A wrapping provider is just a provider that wraps an existing provider (i.e. SqlClient) and adds additional services (i.e. Auditing, Logging, Caching etc).

Jarek has some some sample wrapping providers if you are interested.

Today writing a wrapping provider is a little tricky, in fact one ‘protected’ method is impossible to wrap without reflection. So to help we plan to add one public wrapper method:

public DbCommandDefinition CreateCommandDefinition(
   DbProviderManifest providerManifest,
   DbCommandTree commandTree);

One of the reasons we plan on doing this, is we think people might take a ‘basic’ provider that has no DDL support and wrap it to add DDL support.

End User API

Now so far we’ve been looking at the planned extensions to Provider Services, but Provider Services is a very low level API that few developers will ever program against.

Most people will work directly against the ObjectContext, to which we plan to add these methods:

public void CreateDatabase()

public void DeleteDatabase();

public bool DatabaseExists();

public String CreateDatabaseScript();

This little snippet shows how easy it will be to script, create, check and delete a database:

MyContext ctx = new MyContext();
String sql = ctx.CreateDatabaseScript();
ctx.CreateDatabase();
Assert.True(ctx.DatabaseExists());
ctx.DeleteDatabase();
Assert.False(ctx.DatabaseExists());

As you it could hardly be easier to use.

Summary:

While Code-Only provides the catalyst to add DDL support to the Entity Framework’s Provider model, this feature is about more than just Code-Only.

In fact we think this feature will add significantly to the usability of the Entity Framework.

But as always we are keen to hear what you think.

Alex James
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Many customers have asked us how to port a LINQ to SQL application to the Entity Framework. In order to make this process easier, we have developed a metadata conversion template.

This template converts LINQ to SQL metadata (.dbml) to metadata compatible with Entity Framework (.edmx). While not a complete solution to the conversion problem, it is a useful first step. The template available for download is still in the development stage, and any feedback you have on the user experience, additional features, or any other aspect of the conversion would be greatly appreciated.

The T4 template is intended to convert simple valid .dbml files to .edmx files. All of the basic elements of the .dbml file are mapped to their .edmx counterparts. For instance, Tables are mapped to EntitySets and Types are mapped to EntityTypes. Additionally, stored procedures (including customized insert, update and delete stored procedures) and associations are also converted.

Downloading the Metadata Conversion Template

Included in the zip file is a .vsix installer for the T4 Conversion Template. Also included is detailed documentation for the template. VS 2010 Beta 1 is required.

Download the Template

Walkthroughs

Two walkthroughs have been written to help guide users through the conversion process. The walkthroughs both take the form of unit tests which can be run against the existing LINQ to SQL metadata and the Entity Framework metadata produced by the conversion template. Each walkthrough contains detailed instructions on how to convert the existing LINQ to SQL project to Entity Framework using the Conversion Template. The first is the Widget Factory walkthrough, which shows a basic conversion. The second (CustomCUD) shows how to convert a LINQ to SQL project which uses custom insert, update and delete stored procedures. The walkthroughs are available individually.

Widget Factory Walkthrough

Custom CUD Walkthrough

Caveats and Known Issues

Because the Conversion Template is still under development, there are a number of open issues and unimplemented features. A more comprehensive list can be found in the documentation included with the template installer.

· Inheritance: Inheritance has not been fully implemented. While the template may work in some basic cases, this aspect of the conversion remains an open issue.

· Independent Associations: Foreign-key associations will not be available until .NET 4.0 Beta 2, so the current version of the template will only work with the "UseIndependentAssociations" flag set to true.

· Property Attributes: not all of the property attributes (such as "Unicode" and "FixedLength") are converted because these are not explicitly present in the .dbml file.

Feedback

We are interested to hear any comments you have about known issues, additional feature requests, suggestions for additional walkthroughs or any other aspect of the conversion process.

Thanks,
Christina Ilvento
Developer,
Entity Framework Team

We've been working hard on Code Only since the first preview.

1. Navigation Property Inverses.
2. Property Facets, like Nullability, MaxLength, Precision etc.
3. Property to Column mappings
4. Type to Table(s) mappings
5. Inheritance strategy
6. Encapsulate configuration

The rest of this post will drill into each of these features in turn.

Registering NavigationProperty inverses:

You can now Register Inverses, i.e. link one navigation property to another, like this:

builder.RegisterInverse(
       (Customer c) => c.Orders,
       (Order o) => o.Customer)
);

This code indicates that Customer.Orders is the other end of the Order.Customer relationship. Adding order1 to the customer1.Orders collection, has the same effect as setting the order1.Customer to customer1.

Specifying Property Facets:

You can also specify property facets, i.e. things like Nullability, MaxLength, Precision etc, like this:

var customerConfig = new EntityConfiguration<Customer>();
// We can infer that the ID is the Primary Key,
// but not that it is generated in the database on insert.
customerConfig.ForProperty(c => c.ID)
              .Identity();
customerConfig.ForProperty(c => c.Name)
              .MaxLength(100)
              .NonUnicode();
customerConfig.ForProperty(c => c.Website)
              .MaxLength(200)
              .Nullable()

builder.Configure(customerConfig);

This configures the Customer type so that:

• The ID property is an Identity column, i.e. the value is computed by the database when we do an insert.
• The Name property has a MaxLength of 100 characters and is NonUnicode i.e. in SqlServer VARCHAR rather than NVARCHAR.
• The Website property has a MaxLength of 200 characters and is Nullable.

These facets target the Conceptual Model (i.e. CSDL), and from there flow to the database too (i.e. SSDL).

Encapsulating Facet Configuration

You can create a class to encapsulate all this configuration by deriving from EntityConfiguration<T>.

For example:

public class CustomerConfig: EntityConfiguration<Customer>
{
    public CustomerConfig(){
       ForProperty(c => c.ID)
                 .Identity();
       ForProperty(c => c.Name)
                 .MaxLength(100)
                 .NonUnicode();
       ForProperty(c => c.Website)
                 .MaxLenght(200)
                 .Nullable();
    }
}

We recommend creating classes like this instead of configuring an EntityConfiguration<> because of the encapsulation benefits.

Specifying the Tablename

When you use Configure<T>(..) the EF infers a default mapping, inheritance strategy (TPH) and table name(s) for you.

But if you want to specify the table name you can do this:

var customerConfig = new EntityConfiguration<Customer>();
// configure the facets, as per the above example
...

// register configuration with a particular tablename
builder.Tables[“dbo.Custs”] = customerConfig;

Specifying Mappings:

If you need more control over the mappings (for example to map to an existing database or use corporate naming policies) then you can specify mappings like this:

EntityMap<Customer> customerMap = 
    Map.OfType<Customer>(
        c => new {
            cid = c.ID,
            c.Name,
            csite = c.Website
        }
    );

Interpreting a Mapping

This mapping states that the ID property is mapped to the ‘cid’ column, the Name property is stored in the ‘Name’ column and the Website property is mapped to the ‘csite’ column.

Properties not referenced are not persisted (just like properties in a partial class when using the default code generation of EF)

LINQ Comprehension Syntax

You can even specify exactly the same thing using a LINQ comprehension syntax too:

EntityMap<Customer> customerMap =
    from c in Map.OfType<Customer>()       
    select new {
            cid = c.ID,
            c.Name,
            csite = c.Website
        };

Specifying Facets with Mapping

Once you’ve configured the mapping you can also specify facets on the map like this:

customerMap.ForProperty(c => c.ID)
           .Identity();
customerMap.ForProperty(c => c.Name)
           .MaxLength(100)
           .NonUnicode();
customerMap.ForProperty(c => c.Website)
           .MaxLenght(200)
           .Nullable();

Specifying the table

The final step is to assign the map to a table.

builder.Tables[“dbo.Custs”] = customerMap;

Now we’ve specified a custom table, mapping and custom facets for our Customer class.

Specifying Inheritance:

The default inheritance strategy used by CodeOnly is Table Per Hierarchy (or TPH).

If however you need a different strategy you need to dive in and configure the mappings.

Imagine if you have three classes you need to map: Vehicle , Car and Boat where both Car and Boat are derived from Vehicle which is abstract.

Table Per Hierarchy (TPH)

If you want to map this using TPH, you could do it like this:

var vehicleMap = 
    Map.OfTypeOnly<Vehicle>(
        v => new {
            vid = v.ID,
            v.Name,
            vdesc = v.Description
            v.MaxPassengers,
        }
    ).Union(Map.OfTypeOnly<Car>(
        c => new { 
            vid = c.ID, 
            c.Name,
            vdesc = c.Description 
            c.MaxPassengers,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinder,
            discriminator = “CAR”
        })
    ).Union(Map.OfTypeOnly<Boat>(
        b => new { 
            vid = b.ID, 
            b.Name,
            vdesc = b.Description 
            b.MaxPassengers,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
            discriminator = “BOAT”
        })
    );

builder.Tables[“dbo.vehicles”] = vehicleMap;

In a TPH mapping:

1. OfTypeOnly() is used to create mapping fragments.
2. Mapping fragments are then unioned so they can be assigned to one table (TPH uses a single table for the whole hierarchy).
3. A discriminator column is required for each non-abstract type. The discriminator column can have any name and must have a different constant value (i.e. “CAR”) for each non abstract type in the hierarchy.
4. When mapping a derived type you must re-map all properties mapped in the base-type(s).

Table Per Type (TPT)

If you want to map the same hierarchy using Table Per Type or TPT, this is how you do it:

builder.Table[“dbo.Vehicles”]= 
    Map.OfType<Vehicle>(
        v => new {
            vid = v.ID,
            v.Name,
            vdesc = v.Description
            v.MaxPassengers,
        }
    );

builder.Tables[“dbo.Cars”] = 
    Map.OfType<Car>(
        c => new {
            cid = c.ID,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinders,
         }
     );

builder.Tables[“dbo.Boats”] = 
    Map.OfType<Boat>(
        b => new {
            bid = b.ID,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
        }
    );

In a TPT mapping:

1. OfType<>() is used in each mapping 'fragment'
2. Each mapping 'fragment' is assigned to a different table.
3. Each mapping 'fragment' only maps properties declared on the current type, except…
4. Key properties must be mapped in every fragment (this allows the tables participating in the type to be joined).
5. There are no discriminators

Table Per Class (TPC)

You can also map this hierarchy using TPC like this:

builder.Tables[“dbo.Cars”] = 
    Map.OfTypeOnly<Car>(
        c => new {
            cid = c.ID,
            c.Name,
            vdesc = c.Description
            c.MaxPassengers,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinders,
         }
     );

builder.Tables[“dbo.Boats”] = 
    Map.OfTypeOnly<Boat>(
        b => new {
            bid = b.ID,
            b.Name,
            description = b.Description
            b.MaxPassengers,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
        }
    );

In a TPC Mapping:

1. Like TPH we use OfTypeOnly(..)
2. But unlike TPH each non-abstract type has a its own table. (So there is no table for Vehicles because it is abstract).
3. Each mapping fragment re-maps every, non-transient, property.
4. There is no mapping for any abstract classes in the hierarchy.
5. There are no discriminators

Default Foreign Key Locations:

If we see a reference (i.e. Order.Customer) we assume the multiplicity is 0..1. Meaning the the Foreign Key or FK is nullable.

If we see a collection (i.e. Customer.Orders) we assume the multiplicity is many.

Then when you Register Inverses, we know the multiplicity of both ends of the relationships i.e. in the above example we know that there are 0..1 Customers for an Order and many Orders for a Customer.

So by convention there are 3 main types of relationships, we need to infer the FK location for:

0..1 to many –> by convention we put the FK on the many end. So in the Customer.Orders example the FK is put in the Orders table. 

many to many –> there is no option but to introduce a join table

0..1 to 0..1 –> You can configure where the FK lives, but in the absence of configuration we will introduce a join table.

Sometimes however a reference is not 0..1 it is 1, i.e. the FK, wherever it might be, may not be nullable.

This is how you specify that the FK is non-nullable:

var orderConfig = builder.Configure<Order>();
orderConfig.RegisterInverse(o => o.Customer, c => c.Orders);
orderConfig.ForProperty(o => o.Customer).NonNullable();

This tells us that there is exactly 1 Customer per Order and many Orders for a Customer.

The ability to tell us that a reference is NonNullable introduces several new multiplicity combinations, for which we need conventions too:

1 to many –> by convention we put the FK on the many end, and make it non-nullable in the database.

0..1 to 1 –> by convention we put the FK on the 1 end, and make it non-nullable.

1 to 1 –> like 0..1 to 0..1 we can’t decide where to put the FK so by convention we introduce a join table.

Specifying FK Mappings:

So far we’ve created mappings for the properties of entities.

What about Navigation Properties and FKs?

All types of relationships (except many to many) can be modeled without a join table in the database. So we allow you to map FKs as part of an EntityMap like this:

EntityMap<Customer> customerMap =
    from c in Map.OfType<Customer>()       
    select new {
            cid = c.ID,
            c.Name,
            csite = c.Website,
            salesPersonFK = c.SalesPerson.ID
        };

customerMap.RegisterInverse(c => c.SalesPerson, s => c.Clients);
builder.Tables[“dbo.Custs”] = customerMap;

This specifies that the Customer.SalesPerson navigation property, and it’s inverse SalesPerson.Customers, are stored in the salesPersonFK column in the dbo.Custs table. Because the fragment maps the salesPersonFK column to the c.SalesPerson.ID, where SalesPerson.ID is the Primary Key (or part of the PrimaryKey) of the related SalesPerson entity, and of course FK’s point to PK’s.

Specifying Join Table Mappings:

In the case of many to many relationships you must have a join table, so absent mapping information we produce a join table by convention.

If however you need more control you can do this:

var blogPostsMap = new AssociationMap<Blog, Post>(
     b => b.Posts
).Map(
    (b, p) => new {BlogId = b.ID, PostId = p.ID}
);

builder.Tables[“dbo.BlogPosts”] = blogPostsMap;

This says the many to many relationships between Blog and Post is stored in the ‘dbo.BlogPosts’ table which has two columns:

• ‘BlogId’ which is also an FK pointing to the column to which the ID property of Blog is mapped.
• ‘PostId’ which is also an FK pointing to the column to which the ID property of Post is mapped.

Entity Splitting:

Code Only can even support more advanced mapping strategies like entity splitting:

builder.Tables[“dbo.Customer”] = Map.OfType<Customer>(
   c => new {
      cid = c.ID,
      c.Name,
      active = c.IsActive
   }
);

builder.Tables[“dbo.CustomerDetails”] = Map.OfType<Customer>(
    c => new {
       cid = c.ID,
       c.Size,
       c.Industry
    }
);

This says that the Customer entity is split across the dbo.Customer and dbo.CustomerDetails tables.

Encapsulating all the Configuration:

You can also write a class that derives from EntityMap<T>, to hold all the mapping, facets etc. For example here a class that holds the configuration for Product

public class ProductMap: EntityMap<Product>
{
    public ProductMap{
         this.Map( p => new {
              pid = p.ID,
              pcode = p.Name,
              cid = p.Category.ID
         });
         this.ForProperty(p => p.ID).Identity();
         this.ForProperty(p => p.Name).MaxLength(100)
             .NonUnicode();
         this.ForProperty(p => p.Category).NonNullable();
         this.RegisterInverse(p => p.Category,
                              c => c.Products);   
     }
}

This approach is highly recommended, because configuring the Product type is now trivial:

builder.Tables[“dbo.Products”] = new ProductMap();

Summary:

As you can see we are planning a lot of enhancements which will allow most core scenarios.

What do you think? Do you like the API? Are there things you’d like to see look a little different?

As always we are very keen to hear your feedback.

Alex James
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

There are currently two ways to get Entity Framework apps up and running, we call these Database First and Model First respectively.

Database First has been with us since .NET 3.5 SP1, and involves reverse engineering an Entity Data Model from an existing database.

Model First is new to Visual Studio 2010/.NET 4.0 and allows you to create an Entity Data Model from scratch and then generate a database and mapping for it.

However many developers view their Code as their model.

Ideally these developers just want to write some Domain classes and without ever touching a designer or a piece of XML be able to use those classes with the Entity Framework.
Basically they want to write ‘Code Only’.

We are pleased to announce that we’ll be shipping a preview of “Code Only” on top of the .NET Framework 4.0 Beta 1 (more on that in the coming days).

How it works:

To use Code only you simply write some POCO classes, here is a simple example:

public class Category
{
   public int ID { get; set; }
   public string Name { get; set; }
   public List<Product> Products { get; set; }
}

public class Product
{
   public int ID { get; set; }
   public string Name { get; set; }
   public Category Category { get; set; }
}

Then write a class that derives from ObjectContext that describes the shape of your model and how you want to access your POCO classes, so something like this:

public class MyContext : ObjectContext
{
   public MyContext(EntityConnection conn)
    : base(conn){ } 
   public ObjectSet<Category> Categories {
      get {
         return base.CreateObjectSet<Category>();
      }
   }
   public ObjectSet<Product> Products {
      get {
         return base.CreateObjectSet<Product>();
      }
   }
}

NOTE: this persistence / EF aware class could easily be in another assembly so you can keep it separate from your nice pristine domain classes.

At this point you have everything you need from a CLR perspective, but you won’t be able to use the MyContext class without some EF metadata, which is normally stored in the EDMX file.

But remember Code-Only means there isn’t an EDMX file.
So where do we get the metadata?
That is where the ContextBuilder class comes in:

SqlConnection connection = …; // some un-opened SqlConnection
MyContext context = ContextBuilder.Create<MyContext>(connection);

What is happening here is that the ContextBuilder is looking at the Properties on MyContext, inferring a default Conceptual Model, Storage Model and Mapping by convention, it the uses that metadata plus the SqlConnection you passed in to create an EntityConnection, and finally it constructs an instance of MyContext by passing the EntityConnection to the constructor we created previously.

Once you have an instance of your context, we ship several extension methods you can use to automatically create a database script, see if the database exists, drops the database, and/or create the database,. For example this code snippet uses two of those extension methods to create the database if it doesn’t already exist.

if (!context.DatabaseExists())
    context.CreateDatabase();

The CreateDatabase call looks at the EF metadata, in particular the Storage Model, aka SSDL, and uses it to produce Database Definition Language (or DDL) which it then executes against against the connection.

Overriding Conventions

In the examples so far everything is done by convention. This is great if you are happy with our conventions. However you may want to override them, for example to register different keys, use different mappings or use a different inheritance strategy etc.

To override the convention instead of creating an ObjectContext directly you create an instance of a ContextBuilder that you can configure.

var contextBuilder = new ContextBuilder<MyContext>();

In the first Feature CTP, Code Only provides the ability to override how the key properties are inferred:

contextBuilder.RegisterKey((Product p) => p.ID);

This code tells the builder that the key of Product is its ID property.

When you’ve finished configuring how you want your model and mappings to work in your code, you create an instance of your context, this time by calling Create on the builder instance you have been configuring:

MyContext context = contextBuilder.Create(connection);

It is that simple.

Over time the features of the ContextBuilder will grow so you will be able to setup custom mappings, mark properties as transient, setup up Facets (e.g. MaxLength), change inheritance strategies, link properties that are the inverse of each other together (e.g. product.Category and category.Products) and more. In fact here is an example of the sort of thing we are aiming to support for the next release of the Feature CTP:

builder[“dbo.prds”] = 
  from c in builder.OfType<Product>()
  select new {
    pcode = p.ID,
    p.Name,
    cat = p.Category.ID
  }
);

This snippet:

• Maps Product entities to the ‘prds’ table.
• Maps Product.ID to a ‘pcode’ column.
• Maps Product.Name to a ‘Name’ column.
• Maps the FK under the Product.Category relationship to the ‘cat’ column.

Here is another example that does basic TPH inheritance:

builder[“dbo.Cars”] = (
  from b in builder.OfTypeOnly<Car>()
  select new {
     id = b.CID.AsKey(),
     b.Color,
     b.Manufacturer,
     disc = “C”
   }
).Merge(
  from s in builder.OfType<SportsCar>()
  select new {
     id = s.CID.AsKey(), 
     s.Color, 
     s.Manufacturer, 
     disc = “S”, 
     hp = s.HorsePower, 
     tq = s.Torque
  }
);

These two examples just scratch the surface of what we are planning on doing, hopefully though they give you a sense of where we are heading.

We are super excited about Code Only. But as of right now our plans to support overriding conventions aren’t finalized, so we’d love to get your feedback, this really is your chance to help us get it right.

Alex James,
Program Manager, Entity Framework Team, Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Background

When an application goes multi-tier, there are important architectural decisions that the application developer makes around how to communicate between tiers. There are a lot of choices, and the choice depends on a variety of conditions. How rapidly is each tier expected to change? How much control is there over each tier? Are their specific protocol, security, or business policy concerns? The answers to these questions often drive the selection of whether to use data transfer objects (DTOs) or DataSets or something different altogether.

Self-Tracking Entities

Drawing from DataSet

However, there are disadvantages to DataSets when used as the communication payload between tiers. The first is the cost of getting the data into a serializable format, and the second is that the DataSet serialization format is generally not very interoperable with other languages that are used to expose services. Another disadvantage of using a DataSet is that you can quickly lose the intent of your service call because so many kinds of things can be included in a DataSet. For example, if the service method is declared as:

DataSet GetCustomer(string id);

Self-tracking entities share many advantages with DataSet. They also encapsulate change tracking information, which is serialized along with the data contained in the entity. On the mid-tier applying changes from a graph of self-tracking entities to a persistent context is equally straightforward. The Entity Framework will also provide tools for generating self-tracking entities and because these entities are just objects, they can be easily made to work with UI/presentation controls.

Drawing from DTOs

There are disadvantages to using DTOs and the primary one is complexity. DTOs are often hand-crafted to include only the specific information that is needed for an operation. When there is a common pattern for mapping DTOs to entity classes, there are some tools available that will do DTO generation and mapping but it is not always possible. With DTOs, it is up to the developer to decide how to do change tracking on each tier (especially the client) which increases the complexity of the presentation layer. The complexity of the service implementation also increases because the developer is responsible for translating the DTO into entities for doing business logic validation, as well as being able to report changes stored in the DTO to the persistence framework.

Not only do self-tracking entities share advantages with DataSets as mentioned above, they also share many advantages with DTOs. Self-tracking entities expose a simple and interoperable wire format and it is clear what kinds of data a service method requires or returns. You can also use a self-tracking entity as part of a message. However, self-tracking entities don’t give quite as much architectural separation as using pure DTOs, but you do gain a less complex solution that requires fewer data transformations. It is important to note that self-tracking entities can be made to be ignorant of any particular persistence framework making them essentially POCO objects. Particular persistence frameworks such as the Entity Framework will have the capabilities to create these entities and interpret the change tracking information when saving changes.

.NET Framework 3.5SP1 Challenge

Mid-tier experience

interface ICustomerService
{
    Customer GetCustomer(string customerID);
    bool UpdateCustomer(Customer customer);
}

public Customer GetCustomer(string customerID)
{
    using (NorthwindEFContext context = new NorthwindEFContext())
    {
        var result = context.Customers.
              Include("Orders.OrderDetails").
              Single(c => c.CustomerID == customerID);
        return result;
    }
}

public void UpdateCustomer(Customer customer)
{
    using (NorthwindEFContext context = new NorthwindEFContext())
    {
        context.Customers.ApplyChanges(customer);
        context.SaveChanges();
    }
}

Client Experience

In this example, the test first queries for a customer, then deletes one of the orders. The resulting entity graph is sent back to the service tier using  the UpdateCustomer method.

public void DeleteObjectsTest()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var customer = client.GetCustomer("ALFKI");
        customer.Orders.First().Delete();
        client.UpdateCustomer(customer);
    }
}

public void AddObjectsTest()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var customer = client.GetCustomer("ALFKI");
        customer.Orders.Add(new Order() { 
            OrderID = 100,
            OrderDetails = {
                new OrderDetail{
                     ProductID = 3,
                     Quantity = 7
                },
                new OrderDetail{
                     ProductID = 4,
                     Quantity = 8
                },
            }
        });
        client.UpdateCustomer(customer);
    }
}

The final example shows that a self-tracking entity graph  can contain any number of changes and those changes can be any combination of adds, modifications, and deletes.

public void ComplexModificationsTest ()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var cust = client.GetCustomer("ALFKI");
        // remove first order - will do cascade in the database
        cust.Orders.Remove(cust.Orders.First());
        // modify one of the orders
        var order = cust.Orders.Last();
        order.RequiredDate = DateTime.Now;
        order.OrderDetails.Add(
            new OrderDetail{
                ProductID = 7, 
                Quantity = 3 
        }); 
       // add new order
        cust.Orders.Add(new Order()
        {
            OrderID = 100,
            OrderDate = DateTime.Now,
            ShipCity = "Redmond",
            ShipAddress = "One Microsoft Way",
            ShipRegion = "WA",
            ShipPostalCode = "98052",
            OrderDetails = {
                new OrderDetail{
                    ProductID = 3,
                    Quantity = 7
                },
                new OrderDetail{ 
                    ProductID = 4,
                    Quantity = 8
                },
            }
        });
        client.UpdateCustomer(cust);
    }
}

Design Notes

Inside a Self-Tracking Entity

There have not been final decisions about what exactly will be included inside of a self-tracking entity, but it is important to track:

• The state of the entity, Added, Deleted, Modified, or Unchanged
• The original value from reference relationship properties
• Adds and removes from collection relationship properties

This information will be included with the data contract of the entity. As part of the code generation of each self-tracking entity, scalar and complex property changes will mark the entity as “dirty” meaning that its state will change from Unchanged to Modified.

Inside ApplyChanges

ApplyChanges is a new API on the ObjectContext and ObjectSet classes that attaches an entity graph and interprets the change tracking information stored in each entity. The design  for discovering the change tracking information has not yet been finalized, but there are a couple of options available. One would be to code generate an interface as part of the T4 template for self-tracking entities as well as a version of ApplyChanges that knew about the interface. Another option would be fall back to runtime discovery of the particular properties using a convension and some of the new dynamic capabilities of the framework. The thing we want to avoid is causing the self-tracking entity implementation to have a dependency on any of the Entity Framework assemblies.

The algorithm that ApplyChanges uses is:

1.       Attaching the entity graph to the ObjectContext

2.       Changing the state of each entity using the ChangeObjectState API.

3.       For any reference relationship property, if there is an original value changing the state of the original reference relationship to Deleted and change the state of the current reference relationship to Added. This is done using the ChangeRelationshipState API.

4.       For any collection relationship property, use the ChangeRelationshipState API to mark any removed relationships as Deleted and to mark any added relationships as Added.

Summary

We’ve received a lot of feedback and suggestions on how to make developing multi-tier applications easier using the Entity Framework. One of the components of improving this experience is the introduction of an end-to-end architecture around self-tracking entities. We’d like to hear your feedback on this addition to the Entity Framework, and any other comments you have on the matter of multi-tier development using domain models and entities.

Jeff Derstadt,
Dev Lead, Entity Framework Team, Microsoft

Background

A number of months ago we asked whether Foreign Keys (FKs) in Conceptual and Object models were important.

The feedback we got indicated that there are a real mix of opinions on the topic. Some people are all for them while others think that FKs pollute the conceptual model.

The fact that our customers were so divided meant we thought it was important to provide options here. Those who want FKs in their Entities should be able to do so, so they can gain all the benefits that having FKs in your Entities undoubtedly provide. On the other hand customers who are concerned that have FKs in their Entities in someway pollutes their model can continue to use the type of associations we had in .NET 3.5 SP1.

In the past we've called the .NET 3.5 SP1 style Associations "first class" associations. Which while accurate somehow implies that associations based on FKs in Entities are "second class". In fact when we talked about this new work recently with our MVPs one of the first questions was "are these second class?", which they most definitely are not.

The fact is these new associations are different but first class nevertheless.

So what's been added?

In .NET 4.0 we will add support for a new type of association called "FK Associations".

The way this works is we will allow you to include your FK columns in your entities as "FK properties", and once you have FK Properties in your entity you can create an "FK Association" that is dependent upon those properties.

Okay so we will have "FK Associations", what are we going to call the older style associations? Obviously saying something like ".NET 3.5 SP1 style Associations" every time isn't ideal. So we are going to call them "Independent Associations".

The term "Independent Association" resonates for us because they are independently mapped, whereas FK Associations need no mapping, simply mapping the Entity(Set) is sufficient.

How do I use an "FK Association"?

The real reason so many customers and partners are asking for "FK Associations" is that they significantly simplify some key coding patterns. So much so that we are convinced that for most people "FK Associations" will be an automatic choice going forward.

Some of the things that are hard with "Independent Associations" are trivially easy using FK Associations. Scenarios as diverse as DataBinding, Dynamic Data, Concurrency Control,  ASP.NET MVC Binders, N-Tier etc are all positively impacted.

Let's have a look at some code snippets that will work with "FK Associations":

1) Create a new Product and FK Association to an existing Category by setting the FK Property directly:

using (var context = new Context())
{
    //Create a product and a relationship to a known category by ID
    Product p = new Product
    {
        ID = 1,
        Name = "Bovril",
        CategoryID = 13
    }; 
    //Add the product (and create the relationship by FK value)
    context.Products.AddObject(p);
    context.SaveChanges();
}

This sort of approach works both for insert and update, and is particular useful for things like databinding, where you often have the new Value of the FK in a grid or something but you don't have the corresponding object, and you don't want to wear the cost of a query to pull back the principal object (i.e. the Category).

2) Create a new Product and a new FK Association to an existing Category by setting the reference instead:

public void Create_new_Product_in_existing_Category_by_reference()
{
    using (var context = new Context())
    {
        //Create a new product and relate to an existing category
        Product p = new Product
        {
            ID = 1,
            Name = "Bovril",
            Category = context.Categories
                        .Single(c => c.Name == "Food")
        };
        // Note: no need to add the product, because relating  
        // to an existing category does that automatically.
        // Also notice the use of the Single() query operator 
        // this is new to EF in .NET 4.0 too.
        context.SaveChanges();
    }
}

This programming pattern is not new to the Entity Framework, you could do this in .NET 3.5 SP1. I called it out because it shows you can still write the sort of code you write with Independent Associations even when using FK Associations.

3) Update an existing Product without informing the Entity Framework about the original value of the CategoryID (not supported with Independent Associations):

public void Edit(Product editedProduct)
{
    using (var context = new Context())
    {
        // Create a stand-in for the original entity
        // by just using the ID. Of the editedProduct
        // Note: you don't have to provide an existing Category or CategoryID
        context.Products.Attach(
            new Product { ID = editedProduct.ID });

        // Now update with new values including CategoryID
        context.Products.ApplyCurrentValues(editedProduct);
        context.SaveChanges();
    }
}

In this example "editedProduct" is a product which has been edited somewhere, this is exactly the sort of the code you might write in the Edit method of a ASP.NET MVC controller, and is a great improvement over the code you have to write using Independent Associations.

There is much more you can do with FK Associations, but these 3 samples give you a flavor of the sort of coding patterns FK Associations allow.

Keeping FKs and References in Sync

One thing that hasn't been mentioned so far is that the Entity Framework tries to keep related References and FKs in sync as much as possible.

When this synchronization occurs depends upon when the Entity Framework is notified of changes, which depends upon the type of Entity Classes involved: be they POCO with Proxies, POCO without Proxies, IPOCO or EntityObjects etc.

Another post will cover this in more detail.

How do I create an "FK Association"?

There are a number of ways.

One mainline scenario is when using the tools to infer a model from a database. We have added an option to choose whether "FK Associations" or "Independent Associations" are generated by default. The same is true if you use EdmGen.exe (i.e. our command line tool).

The second mainline scenario for creating FK Associations is creating a model from scratch, aka Model First which is new to .NET 4.0.

Here are the steps involved:

1. Create two Entities (say Product and Category) that look something like this:
   
   
2. Add a property that will be the FK Property: i.e. CategoryID
   
   
3. Create an association between the two EntityTypes with the correct end multiplicities and NavigationProperty names:
   
   
   
4. Double Click the line between the two Entities, that represents the Association, and add a Referential Integrity Constraint:
   
   
   This is the step that tells the Entity Framework that the CategoryID is an "FK Property" and that the "CategoryProduct" association is an "FK Association".
   
   
5. You end up with something that looks like this:
   
    

And you are done you've set up an FK Association. Notice that this association doesn't need to be mapped you simply need to make sure you map all the properties of both Entities.

Very easy.

Summary

In .NET 4.0 we will add support for FK Properties and FK Associations to the Entity Framework. The existing style Independent Associations will still be possible, but we expect that FK Associations will become most peoples automatic choice because they simplify so many common Entity Framework coding tasks.

As always we'd love to hear your thoughts on this work.

Alex James,
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

When VS 2010 ships it will include some significant improvements to our code generation story for the Entity Framework. The basic idea is to make use of T4 templates for code generation and ship strong integration into the Entity Framework Designer to make the experience of customizing those templates as seamless as possible.

Below Sanjay from our Tools team outlines what will be possible once VS 2010 is released.

Customize the code generated by the Entity Designer with T4 templates

In Visual Studio 2008 SP1, the ADO.NET Entity Designer generates classes from the CSDL portion of the EDMX file using the EntityClassGenerator APIs. Numerous customers have asked us how to customize the code generation for a variety of scenarios including:

• Make the generated ObjectContext internal
• Make the generated ObjectContext and Entity classes implement a user-defined interface
• Add user-defined CLR attributes to generated ObjectContext and generated Entity classes
• Influence generated classes based on structural annotations in CSDL
• Generate the ObjectContext and Entity classes into separate files
• Generate “proxy classes” for the generated classes
• Partially or fully change how classes are generated, maybe even generate additional (non code) artifacts in the project
• Completely replace entity framework code generation with custom code
• Generate POCO classes from the model to use as a starting point in my applications
• Generate self-tracking entity classes
• · ...

While the SingleFileGenerator techniques described here enable some of these scenarios these techniques are generally difficult to implement and install or are tricky to debug and customize.

In Visual Studio 2010, we plan to give customers the ability to use T4 templates to generate their classes, and are also planning deep integration with the Entity Designer and Visual Studio to provide a great end-to-end experience.

Walkthrough

The example below customizes code generation to make the generated classes implement a user-defined interface (e.g. IValidate).

Preconditions:

• User has installed Visual Studio 2010
• User is familiar with customizing T4 templates
• User has a C# or a VB console project that targets FX4.0 with an EDMX file in the project

Walkthrough:

1. User: opens EDMX file in the Entity Designer. Note that the generated classes are in a child file of the EDMX file (i.e. in Northwind.Designer.cs)
   
   
   
2. User: right-clicks on an empty area of the designer surface and chooses “Add New Artifact Generation Item…” from the context menu.
   
   
   
3. Visual Studio: displays the standard “Add New Item” dialog and shows a filtered list of items for the user to select from. As expected, the “Add New Item” dialog only shows a list of item templates specific to the current project target framework version, project type and language.
   
   
   
4. User: selects “ADO.NET EntityObject Generator”, specifies a file name and clicks “Add"
5. Visual Studio: checks out the project from Source Control if necessary
6. Visual Studio: sets the “Custom Tool” property of the EDMX file to empty, which will cause the existing .designer.cs file to be deleted
7. Visual Studio: the VS item template adds a new .tt file to the project in the same project directory as the EDMX file
   
   
   
8. Visual Studio: configures the new .tt file to process the selected EDMX file. The EDMX file to process is a replaceable parameter in the .tt file, and VS updates it to point to the selected EDMX file. Thus, the .tt file now knows which EDMX file to generate code for.
   
   
   [Screenshot of .tt file open in the Clarius T4 Editor Community edition for VS 2008]
   
9. User: double-clicks the .tt file in the project to open it in VS10
10. User: edits the .tt file in Visual Studio to make every Entity object implement the IValidate interface
    
    
    [Screenshot of .tt file open in the Clarius T4 Editor Community edition for VS 2008]
    
11. User: saves and closes .tt file
12. Visual Studio: Transforms the .tt file to produce the generated classes as a child file of the .tt file
13. User: notes that the generated classes (under the .tt file) implement the IValidate interface as expected
14. User: switches to the EDMX file in the designer and makes some changes (e.g. add a new entity)
15. User: saves EDMX file
16. Visual Studio: Transforms all .tt files in the project that reference the EDMX file. As before, the generated classes are child files of the .tt files

Design

The design affects multiple parts of the Entity Framework and the Entity Designer and the key changes are described below:

Entity Framework Runtime

1. We started by creating C# and VB T4 templates to generate classes from CSDL and EDMX files
2. Since the T4 engine is installed with Visual Studio and not with the.NET FX runtime, we preprocess the templates as described here and include the compiled code into System.Data.Entity.Design.dll
3. We added a new class System.Data.Entity.Design.EntityCodeGenerator (in System.Data.Entity.Design.dll ) that generates code using the preprocessed T4 template
4. Finally, we enhanced a few runtime metadata APIs and added extension methods to load and iterate over models a lot easier. Our T4 templates make heavy use of these new methods.

Entity Designer

We took the C# and VB T4 templates used by the runtime and changed them to be more “VS friendly” as described below.

1. We made the EDMX file name referenced in the T4 templates a replaceable parameter
2. We created a wizard (with no GUI) that is launched by Visual Studio when the item is added to the project. The wizard does 2 things:
   1. updates the EDMX file name referenced in the T4 templates and
   2. sets the “Custom Tool” property of the EDMX file to empty
3. We rolled up the T4 templates (and wizard) into standard C# and VB Visual Studio item template which are installed with VS 2010; this makes them show up in the “Add New Item” dialog in Visual Studio
4. We added a new context menu to the designer surface to launch the “Add New Item” dialog and also added code to only show VS item templates whose names start with the prefix ADONETArtifactGenerator_
5. Our design ensures that user installed VS item templates in %MyDocuments% as well as the VS item templates we ship in box show up in the “Add…New…Item” dialog (per the template name prefix rule above)
6. We added code to the designer that finds all .tt files related to an EDMX file and automatically transform them when the EDMX file is saved.
7. Finally, we added a new property called “Process related T4 templates on Save” on the designer surface to let users control whether or not to transform .tt files related to the EDMX file on save.

Multi Targeting considerations

As many of you are aware, VS 2010 allows developers to target FX4.0 as well as FX3.5 (and older runtimes). The T4 templates that we ship in the Visual Studio box generate code that works with the Entity Framework in .NET FX 4.0. This means the VS item templates we ship in the box will not be available for projects that target FX3.5.

However, it is certainly possible (and supported) for users to create new T4 templates (and new VS item templates) that generate code from an EDMX file in projects that target FX3.5 and light up in the same experience. In fact, we might release some new templates ourselves later, on CodeGallery perhaps.

How can 3^rd parties plug in?

Since our design is based upon VS item templates that wrap T4 templates, we have opened the door to let 3^rd party code generators participate in the end-to-end experience. Our contract is straight forward and easy to implement:

1. Create a (C# or VB) T4 template to generate code as you desire. The T4 templates we ship in the box can be invaluable points of reference as you roll your own.
2. Make the EDMX reference in your .tt file a replaceable parameter by calling it $edmxInputFile$ like we do in our .tt file.
3. Wrap the .tt file in a VS item template. This is pretty easy and well documented on MSDN. You can also use the Export Template wizard included Visual Studio. Also specify additional VS item template settings such as target framework, project type, etc
4. Prefix the name of your .vstemplate file with ADONETArtifactGenerator_ if you want your VS item template to show up in the “Add New Item” dialog when it is launched from the context menu in the Entity Designer.
5. You can use the wizard we ship in Visual Studio to do parameter replacement in your VS item template by adding the following to your .vstemplate file:
   
   <WizardExtension>
     <Assembly>
       Microsoft.Data.Entity.Design, Version=10.0.0.0, Culture=neutral,PublicKeyToken=b03f5f7f11d50a3a
     </Assembly>
     <FullClassName>    Microsoft.Data.Entity.Design.VisualStudio.ModelWizard.AddArtifactGeneratorWizard 
     </FullClassName>
   </WizardExtension>
   
   Of course, you can also create your own wizard that does custom processing when the item is added to the project as described here.
6. Create a VSCONTENT file and zip up your custom item template & related files into a Visual Studio Content Installer (.vsi) file
7. Upload the .vsi file someplace your customers can download it from
8. Visual Studio already knows how to install .vsi files and everything you need is already available in Visual Studio.

A few key takeaways:

• This design lets the EF team release new T4 templates outside the usual Visual Studio and .NET FX ship cycles and the experience to consume them is exactly the same.
• There are several excellent resources on the web that describe advanced T4 template scenarios and our design enables them. In fact, we expect users to leverage these resources while creating new T4 templates or customizing the ones we ship. For example, code in a custom T4 template can use the EnvDTE APIs to access the project system and generate outputs to multiple files.
• Over time, we expect to blog about some of these advanced scenarios and also work with MVPs, industry experts and Patterns & Practices to provide guidance to customers in a manner similar to the Guidance Automation Toolkit.
• Custom T4 templates can generate anything you want, not just code. For example, we have an internal prototype of a T4 template that iterates over a model and generates an HTML file with a nicely formatted report of the entities and associations in the model
• VS item templates can add multiple files to your project (more details here). This means your custom item templates can include all kinds of files (e.g. workflow files, multiple T4 files, pre-canned code files, etc). For example, you can easily imagine a 3^rd party VS item template that adds 2 .tt files in your project that process the same EDMX file but generate different kinds of outputs.

We hope this gives you an overview of our design and intent of this feature and we would love to hear your comments.

Sanjay Nagamangalam,
Lead PM, ADO.NET Entity Designer

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

A while back I wrote a post that introduced the concept of Computed Properties. Since that time we’ve had a number of conversations with both customers and internal partners and we’ve had some new ideas, that have changed our thinking somewhat. 

• First off we decided to implement Model Defined Functions, which are significantly more powerful and flexible than Computed Properties, and provide overlapping capabilities.
• Computed Properties on the other hand, would never have been structurally part of the entity, which lead to some tough choices and probable customer confusion:
• Do you always pay the cost of materializing the Computed Property even when it is not required?
• What about if there are lots of Computed Properties on the Entity?
• Today in order to materialize an Entity and it's properties, computed or otherwise, the Entity Framework needs access to setters. Unfortunately the existence of the Setter, sends an erroneous message that Computed Properties can be altered and persisted.
• Upon further investigation we discovered that Computed Properties didn't actually address the requirements of Reporting Services, which was one of the original drivers for Computed Properties. Our original design called for computed properties to operate over an instance of an Entity, and it turned out that Reporting Services needed computed properties to operate over both an instance and a set of instances. Model Defined Functions on the other hand are flexible enough to handle both of these scenarios.
• In the future it may be possible to gain all the benefits of computed properties without any of the above issues, by extending our integration of Model Defined Functions with LINQ so that the [EdmFunction] attribute can be applied to CLR properties too.
  
  If we did this the first parameter of the specified Model Defined Functions could be mapped to be the CLR type containing the property itself.  This would allow a Computed Property, backed by a Model Defined Function, to be used in a targeted way in LINQ queries, without any of the illusions that this might somehow be set-able or structurally part of the entity.

Given all these points we started to think Computed Properties just weren’t as important.

We think our best course of actions in the short term is 1) to focus on supporting Model Defined Functions and 2) ask for your feedback on the relative importance of Computed Properties given the points listed above.

So what do you think?

Do Model Defined Functions address most of the scenarios for Computed Properties?

Particularly if in the future we can treat CLR properties as stubs for Model Defined Functions?

As always we are very keen to hear what you think.

Alex James
Program Manager
Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

NB: For more background on the origins of [EdmFunction] check out this post.

Today the Entity Framework, and more specifically the Entity Data Model, have a limited notion of Functions.

We are currently restricted to Function Imports that allow stored procedures to be invoked, and Canonical / Store Functions for database independent and database specific functions respectively.

Now however we want to support functions defined, not just declared, in the EDM (aka. the CSDL).

An example would be:

<Function Name="GetAge" ReturnType="Edm.Int32">
      <Parameter Name="Person" Type="Model.Person" />
      <DefiningExpression> 
            Edm.DiffYears(Edm.CurrentDateTime(), Person.Birthday)
      </DefiningExpression>
</Function>

Here are some things to notice:

• The DefiningExpression is eSQL.
• The function can have zero or more parameters.
• The Function must have a return type.
• Function Parameters are referenced directly by Name in the DefiningExpression: meaning there is no parameter denoting prefix like @. This means you must be careful to choose parameter names that don't collide with identifiers you need to use in the rest of eSQL expression.
• Unlike functions in SSDL, functions in CSDL only support In bound Parameters (i.e Mode="In") because otherwise they become non-composable.
• For this reason the Mode of a parameter cannot be set in CSDL (it is always Mode="In").
• Functions are declared as Global Items and are declared within the <Schema> element. As such there identity is made up of the Schema's namespace and the function name.
• The function parameters and return type can be any of the following:
  • A scalar type or collection of scalar types.
  • An entity type or collection of entity types.
  • A complex type or collection of complex types.
  • A row type or collection of row types (See below).
  • A ref type or collection of ref types.
• Functions with a DefiningExpression do not require mapping, since the eSQL expression is composed out of eSQL fragments that are already mapped.
• Functions without a DefiningExpression are simply declarations. Today the Entity Framework doesn't complain when loading a CSDL with such a function, but you can't invoke it. In the future these functions might be used to support mapping Table Value Functions in the store to functions in the Conceptual Model.
• Since it is trivial in eSQL to create arbitrary un-named types, imagine a projection that projects 3 of the properties from an Entity, we now need a mechanism for defining these "RowTypes" inline, so that they can be used when defining Function Parameters and ReturnTypes. For example:
  
  
  <Parameter Name="Coordinate">
     <RowType>
        <Property Name="X" Type="int" Nullable="false"/>
        <Property Name="Y" Type="int" Nullable="false"/>
        <Property Name="Z" Type="int" Nullable="false"/>
     </RowType>
  </Parameter>
  
• Since eSQL is primarily set based, we also need a way of defining parameters and return types that are collections of RowTypes:
  
  
  <Parameter Name="Coordinates">
     <CollectionType>
        <RowType>
           <Property Name="X" Type="int" Nullable="false"/>
           <Property Name="Y" Type="int" Nullable="false"/>
           <Property Name="Z" Type="int" Nullable="false"/>
        </RowType>
     </CollectionType>
  </Parameter>

Using the Function via eSQL:

It is trivial to use the function via eSQL. For example:

SELECT Namespace.GetAge(p)
FROM Container.People AS P
WHERE P.Firstname = ‘Jim’

Here we get Jim's age, assuming of course there is only one Jim!

It is also possible to compose functions together, you must simply ensure return types and target parameter types are the same (in the case of named types) or structurally equivalent (in the case of row types or collections of row types).

Things get a little trickier when you are dealing with functions that return sets, for example imagine a function that returns someone's friends, used in conjunction with the GetAge function:

SELECT VALUE (F)
FROM Container.People AS P
CROSS APPLY Namespace.GetFriends(P) AS F
WHERE Namespace.GetAge(P) > 21

Here we get all the friends of people older than 21.

As you can see to do this sort of thing you need a crash course in eSQL.

Using the Function via LINQ:

It is also possible to use these functions in LINQ, but this does require a extra step to create an appropriate stub function in the CLR.

This solution is based on the techniques described here, and involves creating a Stub function in the CLR language of your choice, and annotating it something like this:

[EdmFunction("Namespace", "GetAge")]
public static int GetAge(Person p)
{ 
    throw new NotSupportedException(…);
}

The Entity Framework uses the signature of the function and the EdmFunction attribute to map calls to this function when encountered to the appropriate Model Defined Function.

Once you have this stub it is then trivial to use it in a LINQ query like this:

var peopleOver21 =
     from p in ctx.People
     where GetAge(p) < 21
     select p;

Indeed if you are familiar with LINQ you will probably find composing functions together a lot easier too:

Unfortunately in the current version of the Entity Framework, which ships in .NET 3.5 SP1, we don't make any attempt to Singularize or Pluralize names when reverse engineering a model from the database.

So if, for example, your database has a table called Orders you will get an EntityType called Orders too, which clearly doesn't make for the most read-able code:

LINQ to SQL does a better job here, because it ships with simple but effective pluralization and singuralization services, that for the most part produces names you would expect.

Unfortunately the lack of a similar feature in the Entity Framework, means that generally the first thing you do after you've reverse engineered a model with the Entity Framework is fix-up all the silly names.

Now from the perspective of someone who delivers plenty of Entity Framework demos, this is quite embarrassing! 

But of course the real problem here is that it is time consuming and error prone for all but the most trivial models.

Solution:

In the next version we have added basic pluralization support. This support only really handles English, and here's why:

1. Creating language and culture aware services is very hard, and we thought it would be better to spend our time working on the core of the Entity Framework.
2. These sort of services are very general and as such we think that they belong elsewhere, maybe inside Windows itself?
3. We have made the PluralizationService class public so you are free to write your own.

Our default English language pluralization is then used whenever you generate a model from a database or visa-versa (see model first) to help produce appropriate names.

And for command line users familiar with EdmGen.exe there is a new /pluralize switch also.

Basic Pluralization Rules:

So how does the Entity Framework use this pluralization service?

Well it calls the service to:

• Singularize EntityType names
• Singularize NavigationProperty names that point to 0-1 entities
• Pluralize EntitySet names
• Pluralize NavigationProperty names that point to 0-* entities

It turns out that if you do this, you get the model you are looking for most of the time.

The Pluralization APIs:

So far we've talked about default behavior, if however you need more control, for example if you want to specify custom pluralization rules or a create a completely custom service, you have to drop down to the API level.

Everything revolves around a newly added public PluralizationService class that looks something like this:

You can also ask the PluralizationService to create a concrete PluralizationService for you based on Culture (as mentioned previously only "en" based cultures are supported out of the box).

Here is a little code that gets hold of the built-in English Language pluralization services, configures it so "Child" will pluralize to "Children" and then checks that it pluralizes correctly.

You can use your pluralization service in Model Generation (i.e. the step that produces CSDL and MSL from SSDL) by handing your pluralization service to the EntitySchemaModelGenerator:

The resulting CSDL will have been pluralized/singularized as described in the earlier rules using the provided pluralization service.

See this for more information on the EntityModelSchemaGenerator.

Summary

So as you can see we've added simple pluralization capabilities, akin to those found in LINQ to SQL, that will reduce the amount of work required to create meaningful models.

And to top it off, the solution we've created is easy to customize and public so you can probably use this for other things too!

As always we are very keen to hear what you think.

Alex James
Program Manager
Microsoft

UPDATE: A number of people have asked whether pluralization can be turned off, because the post doesn't make this clear. The answer is absolutely, pluralization is definitely optional, in Visual Studio this is as simple as checking/unchecking a checkbox in the wizard, and for EDMGen.exe you have to opt-in for pluralization with the /pluralize commandline switch. 

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

The first version of Entity Framework provides convenient ways to load, manipulate and persist objects and relationships. As with many other O/RMs, Entity Framework has a state manager that tracks every change made. Existing objects are typically loaded first from the database, later modified, and finally the changes are saved back to the store.

Another feature, full graph serialization, makes it very easy for developers to ship around object graphs representing snapshots of the current state of the world, across execution boundaries.

The next version of Entity Framework will also support Persistence Ignorance. Therefore object graphs can now be made of POCO instances, which WCF now also supports.

Nonetheless, there is a task that belongs in any N-Tier application that still requires a great amount of work for developers using EF:  decoding messages that represent state changes performed by the client, meant to be processed or persisted by the service.

The major pain point in the first version is the number of intricate steps a program needs to perform in order to setup the state manager in the appropriate state according to the changes encoded in the message. For instance, for any but the simplest scenarios, in order to use the basic graph manipulations APIs in ObjectContext, like AddObject and AttachTo, it is necessary to “shred” the object graphs contained in the message, which destroys important information about relationships between object instances that therefore has to be kept somewhere else (as an illustration of this approach, see Danny Simmons’s EntityBag in project Perseus).

In addition, given that the message typically contains all the necessary information and that concurrency is usually a concern, the program shouldn’t have to re-query the database. Instead, it should be easier to “tell” the state manager about the original state (that came from the database previously) and about what changes were made.

There are a few approaches to N-Tier that are very popular in the industry. Customers using Entity Framework for the first time often carry the expectation that it will support a similar experience:

DataSet

The original ADO.NET data access technology includes the DataSet, a versatile data cache that can be serialized as XML and be used to transport collections of rows and relationships and, also diffgrams representing changes.

DataSets make many N-Tier scenarios extremely easy to implement, and over time, have proven helpful for many customers. In spite of this, they are not appropriate for some scenarios and architectural patterns:

1. The lack of a widely accepted standard serialization format has hindered the development of implementations that could produce or consume datasets in other platforms.

2. Diffgrams represent sets of CREATE, UPDATE and DELETE operations of arbitrary depth, which is not adequate for architectural patterns that conceive operations with more constrained semantics (i.e. PlaceOrder, CreateCustomer).

3. No simple ways to control the application of batch CUD operations contained in diffgrams makes them unsuitable any time there is a trust boundary between the client and the server.

4. Also, the data-centered perspective of DataSets leads to an anti-pattern known as the “anemic domain model”, which is characterized by a separation between the data aspect and the behavioral aspect of domain objects.

Data Transfer Objects

The approach typically used in the DDD and SOA communities to address  multi-tier scenarios is to hand-build a service with well defined operation semantics together with the a conceptual message format, usually composed of data transfer objects (DTOs).  Implementing DTOs often also involves writing the logic that translates DTOs back and forth to persistent objects.

Here is a simplified set of rules that characterizes the use of DTOs:

1. Actual entities are not serialized, just DTOs, which are value-only objects (without behavior) are exposed in the boundaries.

2. The client does not depend on the types defined on the server. The shapes of the DTOs are the sole contract, and the types used in the client are for the client only.

3. The context or container is not sent together with the message.

4. The service decides what to do with the contents of the message. It is not the message who decides (as an extreme counter example, imagine a message that tells the service to delete all the contents of the database).

Those rules are especially important in SOA scenarios, in which the client and the server can be separated by a trust boundary, such as the one that exists between two different companies.

There are other permutations of layered architectures, though, in which the level of decoupling that DTOs provide may not justify the increased complexity, for example:

1. A multi-tier application in which the client is fully trusted.

2. A rich internet application in which an infrastructure service is purely used for data persistence (is by design a CRUD-only service) and therefore types exposed do not contain behavior anyway.

3. Any architecture in which the persistence framework is actually used to provide and manage DTOs and not the actual entities.

REST

ADO.NET Data Services provides a rich framework that developers can easily use to expose any Entity Data Model as a REST-style resource collection. Simple HTTP verbs like GET, PUT, POST and DELETE are used in combination with URIs to represent CRUD operations against the persistence service.

This pattern is suitable for many applications and can easily be combined with service operations with richer semantics in the same application.

ADO.NET Data Services, however, may not be compelling for those customers who want to define their services as operations with richer semantics entirely.

Goals

ADO.NET Data Services goes a long way addressing the needs of customers doing RESTful services, and so there is no need for us to improve Entity Framework experience on that space.

With the N-Tier improvements for Entity Framework, we want to address some of the same problem space as DataSet, but we want to avoid the primary issues with it.

Ideally, we would like to provide building blocks that are appealing for developers building solutions on a wide range of architectures. For instance, we would like to provide fine enough control for DTO proponents, but at the same time reduce the level of pain that those trying to address simpler scenarios experience today.

Design

If we just wanted to provide a DataSet like experience on top of Entity Framework, the most straightforward (although very expensive to test) way would probably be based on full serialization of the state manager. Basically, the internal representation of ObjectStateManager includes collections of objects to be deleted, inserted, and modified, and also relationships to be added and deleted, as well as objects and relationships that should remain unchanged.

Such a model would lead to very fine grain control of the state manager, but you would lose some of the benefits of operating on the domain objects and the higher level APIs. For instance, it would make it extremely easy for the program to set the state manager in a completely invalid state.

An alternative representation for such an experience would be to ship a “change log” containing the history of all operations performed on the client. This representation has the disadvantage of being more verbose than the previous one, but it would make “point in time” rollbacks pretty easy to implement.

Besides these two, there are a few more interesting generic representations for changes in a graph, but in general, they all suffer from the same disadvantage: providing a solution for them does not give the user the level of control that the most complicated scenarios and sophisticated patterns require.

Based on this fact, we decided to adopt the following goal for the design:

Entity Framework won’t define its own unique representation for the set of changes represented in an N-Tier application. Instead, it will provide basic building block APIs that will facilitate the use of a wide range of representations.

This has the desirable consequence that Entity Framework won’t impose a pattern for N-Tier. Both DTO-style and DataSet-like experiences can be built on top of a minimal set of building blocks APIs. It is up to the developer to select the pattern that better suits the application.

Here is a first cut of the new methods that we are proposing to add to the ObjectContext. The new APIs will work with EntityObjects, IPOCO and POCO entities:

ApplyOriginalValues

Similar to the existing ApplyPropertyChanges, but this API keeps the tracked entity intact, only affecting the corresponding original values. This operation only works for entities in Modified, Unchanged or Deleted states. Added entities by definition don’t have original state.

ChangeObjectState

This method will transition the state of a tracked entity passed as argument, but will also have side-effects on incident relationships. In case the new state is modified, it will also mark all properties as modified, regardless of the original and current values.

ChangeRelationshipState

Similar to ChangeObjectState, but sets the relationship between two tracked entities to the provided state. Relationships cannot be in the modified state, but this method can be used to report which relationship should be added, deleted, or unchanged.

The relationship doesn’t need to exist in the context, but the entities do. If the relationship doesn’t exist, it can be created in the new state. For instance, when this code is invoked, a new relationship can be created between customer1 and order1 in the added state:

The relationship is typically represented by the navigation property but the metadata names of the relationship and roles can also be used in a different overload.

Design notes

• All parameters named “entity”, “source” and “target” accept either entities or EntityKeys.

• We could choose to deprecate the ApplyPropertyChanges API and replace it with ApplyCurrentValues, which is more consistent name-wise with the new ApplyOriginalValues method.

• For convenience, we should add similar APIs in other types like ObjectStateEntry, RelatedEnd and others as appropriate. For instance, the ObjectStateEntry will also have a ChangeState  API that will only take the target state parameter.

• We could choose to put ChangeObjectState() and the various ChangeRelationshipState() in ObjectStateManager rather than in ObjectContext, since this is a lower level API than everything else currently in the ObjectContext.

Scenario tests

To get a sense of how the new API can be used, we have tried several different approaches and representations. We also have plans to release sample code that will show in more detail different ways to use the new API. In the meanwhile, here are a few simple cases.

General purpose Attach with state resolution delegate

In this scenario between a client and mid-tier, the intention is to allow a client to query for a Customer entity with the Customer’s Order entities. The client then makes a variety of changes to the Customer entity graph including modifying the Customer, and adding or removing Orders.  The client would like to send the updated Customer entity graph to the mid-tier so that it can be processed and persisted.

This system will use a simple mechanism to help with change tracking on DTO entities. Each DTO will implement a small interface to help determine the state of the entity using three properties: IsNew, IsModified, and IsDeleted. In this example, original values are not tracked by the client.

When the client makes changes to the Customer entity graph, the only requirement in this scenario is that when an entity changes, the appropriate property will return true to help determine the state of the entity.

The mid-tier will expose a service method that implements the logic to update the Customer entity graph. Rather than traverse the Customer, her Orders and OrderLines looking for changes, the service method will use a convention based mechanism that knows how to deal with the IEntityWithChanges interfaces. In this scenario, this mechanism takes the form of an extension method to the ObjectContext that will attach an entity graph to the context, and use a delegate to map each entity that implements IEntityWithChanges to an EntityState so that it can be used with the new ChangeObjectState APIs.

First we can define a method to map from an IEntityWithChanges to an EntityState:

Next we will define the extension method on an ObjectContext that can attach an entity graph and use a mapping delegate to determine the state of each entity in the graph.

Finally we can define our service method that can update a Customer entity graph using the mapping delegate and the Attach extension method.

ChangeOrder Service Operation

In this scenario, the mid-tier exposes a method to perform a very specific operation: changing the Customer that owns an Order. The client can set the Order’s Customer to a new Customer or to one that already exists in the store. In either case, the service method uses the old Customer so that it can remove the relationship between the Order and that Customer.

As is always the case, your feedback on this topic is welcome!

Jeff Derstadt
Jaroslaw Kowalski
Diego Vega
and The Object Services Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

If you are reading this, you have probably heard by now about the so called impedance mismatch between the relational world and the object world – and there are a number of concepts in the relational database that don’t translate easily to corresponding concepts supported by the object oriented paradigm. One of these factors that is particularly interesting (and often controversial) is the concept of Foreign Keys and whether or not they belong in your conceptual/object model.

FKs are used to represent relationships in the database – but with objects, the natural way to represent relationships is through real references between objects. So as an object relational mapping platform, should a product support one or the other, or both?

I think most will agree that there is tremendous value in supporting relationships in the model as first class references between objects.

What are some of the issues with supporting both foreign keys and references in the same model? We could take a real world example i.e. LINQ to SQL and its foreign key support to look at some of the benefits as well as the downsides to having foreign keys.

Foreign Key Support in LINQ to SQL

LINQ to SQL takes the simplistic approach of making foreign keys available to you as a scalar property in your entities. In essence, LINQ to SQL allows you to write code dealing with relationships like so:

This is possible in LINQ to SQL mainly because foreign keys are somewhat central to the way relationships are implemented and supported.

This particular example achieves something quite powerful, however - here you have essentially changed the category that the product Chai belonged to without ever having to query and materialize the new category that you associated Chai with.

However, LINQ to SQL also allows you to do this:

We have been considering for a while about whether or not we want to include support to allow you to expose foreign keys in the model. Let’s see what you gain / lose by having foreign keys in the model.

One of the fundamental issues is that the simple foreign key concept that works so well in the relational model isn’t sufficient to represent relationships in the conceptual model. In the Entity Framework, a relationship is a first class concept that must be mapped to any set of columns in the database – and the important thing here is that these columns don’t have to represent relationships on the database by the means of FKs and constraints. Another point to note is that in the Entity Framework, relationships are always comprised of two ends, and are bi-directional unlike foreign keys.

What does all of this do for the model?  This opens up possibilities, such as Referential Integrity constraints that are represented in the EDM even though they may not necessarily be present on the relational model in the store. Bi-directional nature of the relationships makes it possible for you to navigate in both directions along a relationship in a way that is generally not possible with a simple FK.

Foreign Keys are not without value, however. An interesting challenge for us is to figure out how to bring the best aspects of Foreign Keys into the Entity Framework without compromising the richness and flexibility that relationships bring to the table.  Let’s look at the upside and the downsides of plain relational foreign keys:

Benefits of Foreign Keys

1. Keeps it simple (for the simple cases)  and allows you to deal with relationship like you deal with them in the database
2. Technically, you can update relationships without having both ends loaded/materialized. This is however in reality not always interesting since you will likely load both ends but this feature is definitely useful.

Disadvantages of Foreign Keys in the Model

1. It is a part of the impedance mismatch problem.
2. It doesn’t allow the concepts that you would expect from relationships in objects (easily getting from one end to the other) for instance.
3. Having foreign keys as well as object references for relationship navigation presents the problem of two different artifacts representing relationships – this introduces complexity and now you have to make sure that you keep these two in sync.

So where do you stand? Do you like foreign keys or do you think they are evil? Would you like to see Foreign Keys exposed in the model, and if they are available in your object model, would it be sufficient if they were read-only?

Let us know!

Faisal Mohamood 
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

In this post Colin Meek and Diego Vega delve into some enhancements we are planning for LINQ to Entities, anyway over to them...

Entity Framework v1 customers preferring to write their queries using LINQ often hit a limitation on the range of functions and query patterns supported in LINQ to Entities. For some of those customers, having to resort to Entity SQL, or even to Entity SQL builder methods, feels awkward and reduces the appeal of Entity Framework.

There are two things we want to do in order to address this in future versions:

• Expand the range of patterns and standard BCL methods we recognize in LINQ expressions.

• Provide an extensibility mechanism that people can use to map arbitrary CLR methods to appropriate server and EDM functions.

This blog post expands on the second approach:

It is actually possible for us to improve our LINQ implementation so that all functions defined in the EDM and in the store, and even user defined functions, can be mapped to CLR methods with homologous signatures.

Design

Problem space

There are multiple dimensions to the problem space we want to address:

• Functions can be defined in either the conceptual or the storage space

• Functions can be defined in either the manifest, or just declared in the model

• Functions can be mapped to either static CLR methods or to instance methods on the ObjectContext

• This feature specifically targets composable functions

How it looks like: EdmFunctionAttribute

The basis of the extensibility mechanism is a new method-level attribute that carry function mapping information. Here is the basic signature of the attribute’s constructor:

The namespaceName parameter indicates the namespace for the function in metadata (i.e. “EDM” or “SQLSERVER”, or other store provider namespace). The functionName parameter takes the name of the function itself.

The following example could be product code or customer code applying the attribute on an extension method (it could be a regular static function) in order to map it to the standard deviation SQL Server function:

Notice that while this method can’t be called directly it can be used in a query like this:

The following example shows how the canonical DiffYear function is mapped:

Usage is:

The following example shows how a user defined function defined in SQL Server can be mapped:

Convention based function name

We can establish that by convention the name of the CLR function defines the value of the functionName parameter. That makes the functionName parameter in the EdmFunctionAttribute optional.

EdmFunctionNamespaceAttribute

To avoid having to always specify the namespaceName for each function, we define a new class-level attribute named EdmFunctionNamespaceAttribute that would define the namespace mapping globally for a given class:

Using EdmFunctionNamespaceAttribute and the convention based constructor:

How it works

When a method with the EdmFunction attribute is detected within a LINQ query expression, its treatment is identical to that of a function within an Entity-SQL query. Overload resolution is performed with respect to the EDM types (not CLR types) of the function arguments. Ambiguous overloads, missing functions or lack of overloads result in an exception. In addition, the return type of the method must be validated. If the CLR return type does not have an implicit cast to the appropriate EDM type, the translation will fail.

Instance methods on the ObjectContext will be supported as well. This allows the method to bootstrap itself and trigger direct evaluation, as in the following example (definition of the method and sample query):

Without the ObjectContext, the function cannot reach the store! To support this style of bootstrapping, the context needs to expose the LINQ query provider. For this reason, we now expose a “QueryProvider” property on the ObjectContext. This provider includes the necessary surface to construct or execute a query given a LINQ expression.

If such a method is encountered inline in another query, then we must validate that the instance argument (MethodCallExpression.Object) is the correct context, but the instance is otherwise ignored:

A function proxy can sometimes bootstrap itself without an explicit context, e.g. when an input argument is itself an IQueryable:

Nullability considerations

Particularly for functions taking collections, we will need to provide overloads for nullable and non-nullable elements. We don’t want to require awkward constructions like:

Tool for Generating the Functions

We created a simple internal tool that generates the classes that represent all the EDM canonical function and the SQL Server store functions. The tool will take the function definitions from Metadata and generate the appropriate function stubs/implementations.

The tool will be outside the product and will be run on demand. We expect to make a version of this tool available for provider writers together with the provider samples.

Naming

The methods will be in the following classes:

Note: The equivalent class in LINQ to SQL is System.Data.Linq.SqlClient.SqlMethods.

The method names will correspond to the name of the EDM/SQL function they represent. The argument names will correspond to the argument names of the EDM/SQL functions as retrieved by the metadata.

The recommendation for provider writers will be to include a similar static class in a namespace of the following form:

System.Data.Objects.[Standard provider namespace].[Standard provider prefix]Functions

Overloads and Implementation

Non-aggregate Functions

For each non-aggregate function we create an overload with all inputs type as nullable of the CLR equivalent of their EDM primitive type, and the return type nullable of the CLR equivalent of their EDM primitive type.

The implementation of the functions (what gets executed if the function is invoked outside an expression tree) will be to throw a NotSupportedException.

Example:

Aggregate Functions

For each aggregate function we will provide two overloads, one with IEnumerable<Nullable<T>> and another one with IEnumerable<T>, where T is the CLR equivalent of the EDM primitive type of the input. The implementations of these will check whether the input is IQueryable in which case it will implement the self-bootstrapping.

Example:

The Entity Framework team would love to hear your comments.

Alex James 
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

One of the most painful omissions from the Entity Framework V1 was Model First, which basically means creating a conceptual 'model first' and then deriving a storage model, database and mappings from that.

People ask for this scenario all the time in the forums.

Well Noam, a Program Manager on the Entity Framework Tools team, outlines what we are considering:

Generating Databases from Models

The next release of the Entity Framework will include the ability to generate database schemas from your model. The main entry point into this feature is via the Designer context menu, to which we will add a new option called “Create Database from Model”.

(Note: User interface elements in this walkthrough are not final user interfaces, as the design is still under review, so please provide feedback...)

Selecting this option will bring up the following warning:

We would like users to understand that this feature will regenerate all SSDL and all MSL from scratch.

If you have started from an empty model, you will be asked to specify the target database. This screen is identical to the one shown when reverse engineering a model from a database. The implication here is that you will need an available server and database, which the system will use to determine what “flavor” of DDL to generate.

Once you have selected the target database, you will be presented with a summary screen which will provide a preview of the DDL that will be generated, as well as a tree-view of the objects.

The tree view:

The DDL view:

The DDL in the above screenshot is there merely as a graphic used to show how what the window will look like and is not intended to be representative of the DDL that will be generated by the system when you are actually creating your database (more about this soon). The DDL will however be read-only: Since it is generated by a template, editing of the results should either be done in the template, or in a separate DDL file which will not get regenerated.

Two options will be available:

- Save the DDL (off by default). This option will add the DDL as a dependent file under your EDMX file.

- Deploy the DDL (on by default). This option will deploy the DDL to the specified target database.

The generated DDL will not migrate data or schema - by default your database will be recreated from scratch.

Out of the box, we will support the Table-per-Type mapping strategy, meaning that we will create a table for each of your types and subtypes. For example, for a model like this…

…the following schema will be generated:

Of interest here is that PK-to-PK constraint between the Customer and Persons table. This helps enforce the inheritance relationship and the creation of foreign key columns to represent the various associations. In addition, the engine will create clustered keys on primary keys, and indexes on foreign keys that represent associations.

Under the Hood

The model first process is implemented using a Windows Workflow Foundation workflow that looks like this:

Here is what the stages do:

Stage

Purpose

These stages are expressed in a XAML file which will be placed underneath your EDMX, to allow for customization: You can add your own steps or replace ones we have created with steps that you write.

Templates

Several of the steps above make use of templates to provide an additional point of control for users, and this is where we expect most of the customization to happen. These templates use the T4 Engine that is included in Visual Studio. We are currently working on making these templates as simple as possible by providing a set of supporting APIs that provide metadata collections that are designed for artifact generation – for example, a collection of all inherited properties for a type, or a collection of both inherited and defined properties. There are three templates:

Template

Purpose

So, for example, if you need all table names to start with “_tbl”, you would modify the SSDL generation template to give all tables the appropriate name, and also modify the MSL template to provide the appropriate mappings. The DBSchema template will not need to be modified as it will automatically pick up the changes made to the SSDL which is its input.

As another example, if you wish to change the mapping strategy from Table-per-Type to Table-per-Hierarchy, you would need to change this same pair of templates.

Finally, if you needed to support a database for which no Microsoft.Data.Schema provider is available, you could replace the GenerateDDL step with your own template-driven activity which could them transform the schema “manually” to your store’s DDL format.

We hope this gives you enough information to understand the design and intent of this feature.

We would love to hear your comments.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.