Entity Framework Design

Entity Framework Design Blog Retired

dpblogs — Tue, 05 Jun 2012 17:33:00 GMT

Over the last year we’ve seen a marked increase in traffic on the ADO.NET blog, which has encouraged us to use it for all our announcements and discussions. Today we are officially retiring the Entity Framework Design Blog, although we remain committed to the same transparent design process that was its original purpose. Stay tuned to the ADO.NET blog to learn more about new features and to help us shape the future of Entity Framework.

Entity Framework Team

Enums in the Entity Designer

dpblogs — Fri, 12 Aug 2011 22:48:00 GMT

One of the most highly-requested features for the Entity Framework is first-class support for Enums, and we are happy to provide support for this feature in the Entity Designer in the June 2011 CTP recently released. If you haven’t checked it out already, find the bits here:

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=26660

We’d like to give you a look at the experience of developing with Enum types in the Entity Designer. If you haven’t yet, take a look at what is going on under the hood with Enums in the Entity Framework in the blog post here:

http://blogs.msdn.com/b/efdesign/archive/2011/06/29/enumeration-support-in-entity-framework.aspx

and you can follow an example of using Enums in the Walkthrough here:

http://blogs.msdn.com/b/adonet/archive/2011/06/30/walkthrough-enums-june-ctp.aspx

While the Walkthrough will take you briefly through the Entity Designer Enums experience, we want to make sure we show what is available. We’d also like you to use this as a way to give direct feedback to the Entity Designer team on the user experience of working with Enums. We are already working on improvements to the experience shipped in the CTP, so please let us know what you think!

Experience

Similar to how Complex Types are represented in the Entity Designer, the primary location to work with Enums is via the Model Browser.

There are two major points to note about using Enums in your Entity Data Model:

1. Enum Types are not supported as shapes on the Entity Designer diagram surface

2. Enum Types are not created in your model via Database-First actions

a. When you create an EDM from an existing database, Enums are not defined in your model.

b. Update Model from Database will preserve your declaration of Enum types, but again, will not detect Enum constructs from your database.

Please let us know your feedback on the importance of these items. For example, how valuable is it to you to be able to convert reference entity types imported from Database-First into enum types?

Enum Type Dialog

Name: Name of Enum Type

Underlying Type: Valid underlying types for enum types are Int16, Int32, Int64, Byte, SByte.

IsFlags: When checked, denotes that this enum type is used as a bit field, and applies the Flags attribute to the definition of the Enum Type.

Member Name and Value (optional): Type or edit the members for the enum type and define optional values. Values must adhere to the underlying type of the enum.

Errors: Errors with your Enum Type definition will appear in the dialog in red, accompanied by a tooltip explaining what is invalid. Common errors are:

- Not a valid enum member value: A Value entered that is invalid according to the underlying type

- Member is duplicated: Multiple Members entered with the same name

- Not a valid name for an enum member: A Member name entered with unsupported characters

Errors must be fixed before the Enum Type can be created or updated. We do have known issues with our error UI that we are fixing as I write! Feedback is more than welcome as to what is most helpful to you in the dialog.

Entry Points

Create new Enum Type and Edit existing Enum Type:

Context Menu on Enum Types folder or existing Enum node in Model Browser

Convert Property to Enum:

Context menu on a Property in an Entity Type

Converting a property to an Enum type will create a new Enum Type. To use an existing Enum Type for a property, select the property and you will find existing Enum Types available in the Property Window in the Type drop down.

Use existing Enum Type:

Type in Property Window

Feedback

As always, please let us know what is most valuable to you for using Enum Types in the Entity Designer. What would help you further about the existing support in CTP1? What would you need to see for Enum support in the Designer for it to be the most valuable to your development?

Feedback is always appreciated!

Thank you,

Sarah McDevitt

Program Manager

Auto-Compiled LINQ Queries (Entity Framework June 2011 CTP)

dpblogs — Thu, 30 Jun 2011 07:22:00 GMT

When you write a LINQ to Entities query today, the Entity Framework walks over the expression tree generated by the C#/VB compiler and translates (or compiles) that into SQL.

Compiling the expression tree into SQL involves some overhead though, particularly for more complex queries. In order to avoid having to pay this performance penalty every time the LINQ query is executed, the CompiledQuery class was introduced. This allows you to pay the overhead of compilation just once, and gives you back a delegate that points directly at the compiled version of the query in the Entity Framework cache.

static readonly Func<NorthwindEntities, IQueryable<string>> query =

CompiledQuery.Compile<NorthwindEntities, IQueryable<string>>(

db => from c in db.Customers

where c.Country == "USA"

select c.CompanyName);

Each subsequent time you execute the query it’s now much faster.

Auto-Compiled LINQ to Entities Queries

The Entity Framework June 2011 CTP supports a new feature called Auto-Compiled LINQ Queries. Now every LINQ to Entities query that you execute automatically gets compiled and placed in EF’s query cache. Each additional time you run the query, EF will find it in its query cache and won’t have to go through the whole compilation process again.

This also provides a boost to queries issued using WCF Data Services, as it uses LINQ under the covers.

How Does it Work?

The Entity Framework will walk the nodes in the expression tree and create a hash which becomes the key used to place it in the query cache. If it does not find the query in the cache then it will go ahead and compile it and store the compiled query in the cache for subsequent use. Each subsequent time, the hash will be calculated and find the compiled query in the tree, thus saving the compilation overhead.

Turning off Query Caching

A new property on ObjectContext.ContextOptions has been introduced that allows you to control the default behavior of query compilation. By default this property is true, but you can set it to false to disable auto-compilation of LINQ queries:

db.ContextOptions.DefaultQueryPlanCachingSetting = false;

Note: If you’re using DbContext and need to get the ObjectContext instance, just cast to IObjectContextAdapter:

((IObjectContextAdapter)db).ObjectContext.ContextOptions.DefaultQueryPlanCachingSetting = false;

The value of this property gets propagated to the EnablePlanCaching property on each newly-created ObjectQuery for the ObjectContext. Note: There is a known limitation in the June 2011 CTP: while DefaultQueryPlanCachingSetting is propagated to the EnablePlanCaching setting of each new query, the EnablePlanCaching setting is not propagated when using LINQ operators.

Performance Guidance

Obviously when it comes to performance there’s no one-size-fits-all approach. The size/complexity of your application and queries will greatly influence how much of a performance boost your application will get from this. It’s important to measure your specific application with a good profiler.

It is worth noting that using the CompiledQuery class is still the fastest approach, as the delegate that you get back does not need to calculate a hash of the expression tree. Auto-compiled LINQ queries should be almost as fast though, and provide the benefit of not having to manually use the CompiledQuery class.

Feedback

We’d love to get your feedback on this feature and hear if it’s helping in your application. Please download the CTP today and let us know what you think!

Entity Framework Team

Enumeration Support in Entity Framework

dpblogs — Thu, 30 Jun 2011 00:15:00 GMT

We recently shipped Entity Framework 4.1, and it has been exciting to see the reception the new Code First and DbContext APIs have been getting in the developer community. What is also exciting is to be able to start talking more about other features we have been working on, like table-valued functions, unique constraints and spatial types support.

At present, enums support is by far the most requested improvement for Entity Framework. We know it because we get asked about it each time we meet with a customer. We also see questions every week in the comments of our blogs, in our forums, in tweets like this, and likewise in our own feedback site, in which enums currently has more than 1,600 votes. That is about 50% more votes than Code First!

After having had to explain several times why it takes so long for us to add enums support to EF, we are happy to say that we plan to do it in the next version after 4.1.

The goal of this blog post is to describe the design decisions we have made so far and to gather your feedback so that we can refine the design and calibrate the direction we have taken.

Understanding Enums (and why they are so important)

Unlike other things we get to work on, enumerations are an existing programming language feature. Our primary goal is to make sure the feature works as seamlessly as possible with the EF in Object/Relational Mapping scenarios.

At the same time we have to focus on incorporating the concept of enums into the core of the metadata system underlying the EF, the Entity Data Model (EDM), in a manner that serves other key pieces of the EDM ecosystem, such as OData and WCF Data Services.

In order to achieve these goals we had to first recognize what enums are (and what they aren’t) from the language design perspective, look at how they are commonly used within the realm of data oriented applications, understand requirement gaps, prioritize scenarios and come up with a design that can satisfy the most important among them.

A glimpse at enumeration support in the .NET programing languages

The design of enums support in .NET programming languages like C#, VB and F# is amazingly simple. Yet from the perspective of modeling a specific domain, enums unquestionably provide great expressiveness.

This becomes apparent when looking at this straightforward example in C#:

public enum ShippingMethod

{

Express,

Priority,

Ground

}

[Flags]

public enum ContentType : byte

{

Liquid = 1,

Perishable = 2,

Edible = 4

}

public class Package

{

public int Id { get; set; }

public string Description { get; set; }

public ShippingMethod ShippingMethod { get; set; }

public ContentType? ContentType { get; set; }

public string Reference { get; set; }

}

Given this definition of Package, ShippingMethod and ContentType, the code necessary to change the ShippingMethod and ContentType of a package can be as simple as this:

juiceBox.ShippingMethod = ShippingMethod.Ground;

juiceBox.ContentType = ContentType.Liquid | ContentType.Perishable | ContentType.Edible;

Structural definition

In C# enums can be structurally described like this:

The type declaration of an enum contains the name of the type and a list of zero or more named constants referred to as enum members
Each enum type also has an underlying type which is an non-nullable integral primitive type (Int32 if not specified in the declaration)
Each enum member consists of a member name and a member value
The set of values that an enum member can take is the valid range for the underlying type
The value of an enum member is optional in the type declaration. If the value is not specified, it will be zero for the first member and one more than the value of the previous member for subsequent members
Multiple members with different names can have the same value
Member names are case sensitive and must be unique within the enum type declaration

Instances of enum types present specific characteristics:

Any enum value can be explicitly casted to its underlying type
All possible values of an enum type can be obtained by casting any valid value of its underlying type to the enum type (i.e. valid enum values are not limited to the declared members of the enum type)
However, casting directly between two different enum types is not allowed
The default value for any enum type is always zero, casted to the enum type, even if there is no declared member in the enum type with a value of zero
Enum instances are equal-comparable and order-comparable against any other instance of the same enum type

The following comparisons are supported for enums in C#:

Equality comparison between enums: ==, !=
Order comparison between enums: <, >, <=, >=

The following operators are supported in C# and return an enum of the same type:

Simple arithmetic between enums and the underlying type: +, -, ++, --, +=, -=
Bitwise operators between enums of the same type: ^, &, |, ~

The definition of enums in VB is very similar to C#, although enum types in VB are required to have at least one member, and member names are case insensitive. Operator rules over enums are also different in VB: most arithmetic operators can be applied but the result of the operation will be of the underlying type instead of being an enumeration. In fact, only bitwise operations between two enum instances will result in another enum of the same type.

Independently of the language, the following library methods on System.Enum are particularly useful:

HasFlag: Used to verify whether an instance of an enum type contains a specific value (see more on enums that can contain multiple values in the next section).
Parse and TryParse: Converts a valid string representation to an enum instance.
ToString: Converts an instance of an enum to a string representation.

How enums are used in practice

As simple as enums are, there is a lot of richness in how people think about them and how they are used in real world applications. At a high level, the common usages of enums are characterized by the following notions:

Enum types in .NET allow grouping a set of named numeric constants associated to the same concepts into a type:

Like regular constants, enums are a much more manageable alternative to sprinkling magic numbers across your code.
The ability to group sets of related constants and strong typing enable useful design-time services such as Intellisense and provide compile-time checks that help keep your code organized.

Enum types in .NET always have an “underlying” integer type that determines the range of values that an instance can store:

Fundamental aspects of using enums, such as the behavior of comparison, default values, arithmetic and bitwise operators depend on this underlying representation.
Although this correlation to integers is fundamental to the design, it is often possible to use Enums without paying much attention to it.
This is all consistent with the design of Enumerations in the C language, but in .NET enums are more type safe (e.g. you cannot assign directly a value of the underlying type to an enum variable without casting it explicitly).

In some cases, a variable of an enum type is used to simultaneously store a combination of multiple values:

This is implemented by treating the underlying integer representation as a “bit field”.
In .NET, it is possible to express the intent of using an enum type in this way by adding the FlagsAttribute to the declaration of the type, and then by declaring each member to have an integer value that represents a different bit position:

[Flags]

public enum ContentType : byte

{

Liquid = 1,

Perishable = 2,

Edible = 4

}

Enumerations that can represent multiple values simultaneously are a powerful concept, but the implementation in .NET is extremely simple: the programing languages handle all enum types in the same way, e.g. bitwise operators can be applied to enums regardless of the presence of FlagsAttribute (in fact, the attribute’s sole impact in the behavior of the Base Class Library is in the conversion of an enum value to a string). In other words, there is no strong contract in using FlagsAttribute, and it is up to the developer to use only enums that were meant for multi-value in such a way.

Some tend to think of enums types as a way to model constrained categorizations, which assumes variables of an enum type can be guaranteed to only hold instances of its declared members:

Although enums are strongly typed in .NET, there is currently no way in .NET to express the intent to constrain the values to be only of the declared members. In fact, barring WCF serialization, nothing in .NET will constraint the value of an enum instance in this way.
That said, this is evidently a very useful thing to do, in particular in cases in which an enum is part of a public contract of a service, or in data centered applications, in which these constraints will often be backed by a database constraint.
Other languages outside of .NET, such as Java, deviate from the C design of enums based on integer underlying types to instead focus on constraining enums instances to only store declared values.

Use of enums in data oriented applications

Without native support for enums in the current version of Entity Framework, customers have come up with different ways to achieve similar functionality. The workarounds often require writing lots of additional code, and in the end, the experience is not as seamless as we would like it to be.

Incidentally, a common workaround, which consists of creating additional entity types for each of the types you would like to define as enums, can help illustrate the actual role of enums in data oriented applications.

Starting with the previous example, we would create additional entity types for ShippingMethod and ContentType, associating them to Package and then turning the ShippingMethod and ContentTypes properties into navigation properties:

public class ShippingMethod

{

public int Id { get; set; }

public string Name { get; set; }

}

public class ContentType

{

public int Id { get; set; }

public string Name { get; set; }

public virtual ICollection<Package> Packages { get; set; }

}

public class Package

{

public int Id { get; set; }

public virtual ShippingMethod ShippingMethod { get; set; }

public virtual ICollection<ContentType> ContentTypes { get; set; }

}

Given this definition for Package, ShippingMethod and ContentType, the code for changing the ShippingMethod and ContentType of a package would be similar to this:

var ground = context.ShippingMethods.Single(c => c.Name == "Ground");

var liquid = context.ContentTypes.Single(c => c.Name == "Liquid");

var perishable = context.ContentTypes.Single(c => c.Name == "Perishable");

var edible = context.ContentTypes.Single(c => c.Name == "Edible");

juice.ShippingMethod = ground;

juice.ContentTypes.Clear();

juice.ContentTypes.Add(liquid);

juice.ContentTypes.Add(perishable);

juice.ContentTypes.Add(edible);

The solution based on entities is conceptually equivalent to one based on enums: in both cases, the list of all possible ShippingMethods and ContentTypes are used as “reference data” for Package.

However, there are noticeable differences:

Additional tables and foreign key constraints need to exist in the database and be mapped to the entities.
In order to establish an association, you need to first retrieve an instance of the related entity, e.g. by executing a query (this part can be made easier by adding an extra foreign key property to the entity, so you can establish an association by just assigning the value of the corresponding key to it, but the value of the key would likely become a magic number in your code).
Many-to-many associations are necessary to achieve similar results to “flags” enums, such as Package.ContentTypes. Many-to-many associations require twice as many tables and foreign key constraints in the database as regular one-to-many associations.
While instances of an entity type can have multiple properties, members of an enum type are in general constrained to only have a name and a value.
In general, an application will only be able to reason about enum members that existed at compile time. On the other hand, a solution based on entities can more easily evolve after the application has been written and compiled, because all the data exist in the database and can be modified at any time.

In summary:

The main role of enumeration types in data oriented applications is to serve as simple lookup lists for reference data
Enums require significantly less ceremony than using Entities
Because they are hardcoded and are constrained to only have names and values they are less broadly applicable

Enums support in Entity Framework and Entity Data Model

Taking into account the design of Enumerations in the .NET programing languages, the typical usage and the role of Enums in data oriented applications we explained above, we have decided to align the structural definition of Enums in the Entity Data Model with their definition in C#.

This means that the declaration of Enums in EDM will have the same elements that they have in C#, including the optionally specified underlying type, case sensitive member names, optional values, etc., and that the same rules are used to choose values not specified, e.g. the default underlying type is also Int32 and the implicit value of members is the previous member plus one, starting with zero for the first member.

This alignment in the design makes satisfying the requirements of O/RM scenarios for our customers much easier to use. It also enables all the richness of enums in the language, including such things as support for multi-valued (flags) enums in LINQ to Entities queries.

When it comes to other consumers and producers of the EDM, like OData, the fact that other languages might have significantly different definitions of enums becomes relevant. In particular, Java 1.5 introduces first-class language support for the “typesafe enum pattern” previously used in Java programs to substitute missing support for enums. Java enums remove the idea of integral underlying types and instead implements enums that are completely constrained to the declared members.

We recognize that there is a going to be a mismatch between the EDM semantics for enums and Java, but we believe the mismatch can be mitigated by the specific library implementations.

Requirements for Enums support in EF

The following is an exhaustive list of the things that we believe developers using EF should be able to do with enumerations:

Models, Metadata and Mapping

Users creating an Entity Data Model should be able to define enum types (optionally specifying their underlying integer type) and their members (optionally specifying member values explicitly).

The following XML fragment is an example of how the definition of a simple enum type looks in CSDL:

</EnumType>

The corresponding enum type in C# looks like this:

public enum ShippingMethod

{

Express,

Priority,

Ground

}

Notice that the two type definitions above omit underlying type and member values. Since EDM will use the same rules for implicit underlying type and the assignment of unspecified member values as .NET programming languages, the two types above will unambiguously match, i.e. the underlying type for both is a 32 bit integer and the member values are 0, 1 and 2 for Express, Priority and Ground respectively.

Flags enums

When declaring an enum type in EDM, users should also be able to specify the intent to use this type as a “flags” enums:

</EnumType>

The only effect of this attribute in the EnumType element is to drive code generation to include the FlagsAttribute in the enum type definition. For instance, this is the equivalent code in C#:

[Flags]

public enum ContentType : byte

{

Liquid = 1,

Perishable = 2,

Edible = 4

}

Nullable enums

Once an enum type has been defined, it should be possible to declare nullable or non-nullable properties of this type in entities or complex types.

Example:

<Key>

</Key>

…

</EntityType>

Enums as keys

In addition, properties of enum types can participate in the definition of primary keys, unique constraints and foreign keys, as well as take part in concurrency control checks, and have declared default values.

Mapping

Properties of enum types support all the same mapping scenarios as properties of the underlying type:

An enum property in EDM can be mapped to any of the store integral types supported by EF: Byte, Int16, Int32 and Int64
Entities with enum properties can be mapped using stored procedures and query views
For object to conceptual mapping, both attribute-based (most commonly used in EntityObject code generated entities) and convention (most commonly used for POCO types) based mapping are available.

Note: We expect that customers will use enums in many cases in which previously they would have used entities with reference data. Unlike entities, enum types are not mapped to tables in the database. Users are free to have tables and foreign key constraints, or check constraints in the database that enforce enum values. However Entity Framework will not reason about these constraints at runtime, so it is up to the user to keep rows in those tables and enum members in sync.

New metadata APIs

The metadata API has been updated to be able to reason about enum types. The following classes, EnumType and EnumMember are used to represent enums in the EDM layer:

System.Data.Metadata.Edm.EnumType class represents an enum type
System.Data.Metadata.Edm.EnumMember class represents an enum member
New overloads of MetadataWorkspace.GetObjectSpaceType(), MetadataWorkspace.GetEdmSpaceType() and ObjectItemCollection.GetClrType() have been added that allow navigating the mapping between enums the EDM layer and the object layer.

Code First APIs

Code First’s property mapping fluent APIs have been extended to support enum types.

Example:

public class ShippingContext : DbContext

{

public DbSet<Package> Packages { get; set; }

protected override void OnModelCreating(DbModelBuilder modelBuilder)

{

modelBuilder.Entity<Package>().Property(p => p.ShippingMethod).HasColumnName("sm");

}

Query

All types of Entity Framework queries (including EntityClient queries and object layer queries written in LINQ or Entity SQL) can refer to enums. For instance, the following operations are allowed:

Projection of enum properties, constants and parameters
Use of enum properties, constants and parameters in filtering predicates
Use of enums as parameters and in the return type of Functions
Use of enums as parameters and in the return type of pass-through SQL APIs such as ExecuteStoreQuery, ExecuteStoreCommand, SqlQuery, etc.
Cast explicitly between enum instances and the underlying type of the enum (or types that support explicit cast from underlying type of the enum) in queries
Use of equality and order comparisons: ==, !=, <, >, <=, >=
Use of arithmetic and bitwise operators in LINQ queries: +, -, ^, &, |, ~ (the exact behavior and result type depends on the .NET language being used)

Note: Increment and decrement operators such as ++, --, += and -= have side effects and therefore cannot be supported in queries.

Example:

var groundPackages =

from p in context.Packages

where p.ShippingMethod == ShippingMethod.Ground

select p;

Update

Entity Framework entities and complex types that contain enum properties can be inserted, updated and deleted. In addition, enum properties can be store generated and used for concurrency control.

Example:

var juiceBox = context.Packages.Where(p => p.Name == "Juice box");

juiceBox.ShippingMethod = ShippingMethod.Ground;

juiceBox.ContentType = ContentType.Liquid | ContentType.Perishable | ContentType.Edible;

context.SaveChanges();

Designer support

The Entity Designer has been extended to support the definition of enum types. An upcoming blog post will contain more details about designer support.

Note: Reverse engineering a database into a model (usually called “database first”) never introduces enums into the model automatically. Users need to manually specify which properties represent enum types.

Code generation

Code generation, including EdmGen.exe, EntityObject template, Self-Tracking Entities template and DbContext template have been updated to generate enum type definitions and properties of enum types.

Limitations

There are several features we considered that will not be included in the upcoming release of EF:

ToString() and Parse()/TryParse() won’t be supported in queries

Translation into store queries would require very complex expressions to achieve fidelity with the behavior of these methods in the CLR.

Mapping Enums to strings and arbitrary types

We consider the general ability to perform property type transformations in the mapping to be an important feature, but it wasn’t a priority this time around. Barring type transformations the most natural thing to do for enums is to support mapping to integer numeric types.

Using enum properties as mapping discriminators is not supported

The Entity Framework runtime supports the use of Boolean properties as mapping discriminators, i.e. the value of the property in an entity is used to decide in what table in the store that entity instance should stored. We could consider adding the same capability for enum properties, but this wasn’t a priority in this release.

Feedback requested

As always, your opinions on all aspects of this design are welcome. However, there are specific areas in which we are seeking your help in deciding where to take the design. Please, take some time to answer them in the comments.

Checked or Strict enums

As we explained before, constraining the value of an enum to the declared members is one of the common notions of enums, however in .NET we don’t provide a way to express this intent. One thing we could consider doing is to enable this in EF:

When declaring an enum property, users should also be able to declare the intent to constrain the values of the property. They could do this using adding a new “Checked” facet to properties.

Example:

<Key>

</Key>

…

</EntityType>

The only practical effect of the Checked attribute in the Property element of an enum type would be to enable the DbContext API to perform data validation in the property. DbContext performs data validation automatically by default on SaveChanges and can also perform it on demand. For an enum property that has this attribute, if the property value does not correspond to a declared member (or to a valid combination if the enum type is “flags”) validation will fail and SaveChanges will abort.

The equivalent behavior could be obtained when using Code First by adding a new validation data annotation or using a fluent API.

Example:

public class Package

{

public int Id { get; set; }

public string Description { get; set; }

[Checked]

public ShippingMethod ShippingMethod { get; set; }

[Checked]

public ContentType? ContentType { get; set; }

public string Reference { get; set; }

}

Note: The data validation feature in DbContext is explained in more details here and here.

Question: How important is it to validate enum values in this way for you? Would you give up any of the other features explained in this post for it?

HasFlag support

.NET 4.0 introduced this new method in System.Enum that makes it possible to detect whether an instance of a “flags” enum contains a value without having to resort to or think about bitwise operators. We could consider adding support for this method on LINQ to Entities.

Example:

var perishableLiquids =

from p in context.Packages

where p.ContentType.HasFlag(ContentType.Perishable | ContentType.Liquid)

select p;

This is equivalent to:

var perishableLiquids =

from p in context.Packages

where p.ContentType & (ContentType.Perishable | ContentType.Liquid) ==

(ContentType.Perishable | ContentType.Liquid)

select p;

Question: How important is it to be able to use HasFlag in LINQ queries for you? Would you give up any of the other features explained in this post for it?

Options to disable code generation for specific types

Enum types are primarily a code construct and therefore their existence is commonly independent of an EF model. If you try to use an enum type that is already declared with your code-generated entities it can become very difficult because EF code generation will always generate a new enum type for the type defined in the EDM model.

We could consider improving this by defining an annotation that you could easily put on specific types (enum, entity or complex type) in your model that would signal EF code generation templates to ignore that type.

Question: How important is it to you to be able to skip generating specific types from your model? Would you give up any of the other features explained in this post for it?

Thanks,

Diego Vega
Program Manager
Entity Framework Team

Spatial Types in the Entity Designer

dpblogs — Thu, 12 May 2011 17:22:00 GMT

One of the highly-anticipated features coming in the next version of Entity Framework is Spatial support. We’d like to give you a look at the experience of developing with Spatial types in the Entity Designer. If you haven’t already, take a look at what is going on under the hood with Spatial types in the Entity Framework in the blog post here.

We’d like to show you what is coming your way for Spatial types in the Entity Designer and of course, let us know what you think and give us feedback on the experience!

Goals

The goals for supporting Spatial types in the Entity Designer align with Entity Framework design goals for Spatial as well:

First class support for Spatial types in the Entity Designer
Model-first and Database-first support through the Entity Designer

Spatial Types

Geometry
Geography

Experience

Since the Entity Framework adds Spatial types as primitive EDM types, we treat them exactly as you are used to using primitive types in the Entity Designer.

Add Property to Entity Type
Set Data Type as Geography through the Property Window

Restrictions

Spatial types cannot be used as entity keys, in associations, or as discriminators. We give design-time validation errors if you attempt to use one of those invalid patterns.

Attempt to set property of type Geometry as entity key
Validation error in Error List

Feedback

As always, please let us know what is most valuable to you in the Entity Designer for using Spatial data types. Feedback is always appreciated!

Thank you,

Sarah McDevitt

Program Manager

Spatial Types in the Entity Framework

dpblogs — Wed, 04 May 2011 21:37:00 GMT

One of the highly-anticipated features coming in the next version of Entity Framework is Spatial support. The team has been hard at work designing a compelling story for Spatial, and we would love to get your feedback on it. In this post we will cover:

The basics of SQL Server’s spatial support
Design goals
Walkthrough of design: CLR types, Metadata, Usage, WCF Serialization, Limitations
Questions (we want to hear from you!)

This entry will not cover the tools experience as we want to focus on what is happening under the hood. We will blog about the tools design at a later time. For now, be sure that we plan on shipping Spatial with Tools support.

The Basics of Spatial
For those of you that are new to Spatial, let’s briefly describe what it’s all about: There are two basic spatial types called Geometry and Geography which allow you to perform geospatial calculations. An example of a geospatial calculation is figuring out the distance between Seattle and New York, or calculating the size (i.e. area) of a state, where the state has been described as a polygon.

Geometry and Geography are a bit different. Geometry deals with planar data. Geometry is well documented under the Open Geospatial Consortium (OGC) Specification. Geography deals with ellipsoidal data, taking in mind the curvature of the earth when performing any calculations. SQL introduced spatial support for these two types in SQL Server 2008. The SQL implementation supports all of the standard functions outlined in the OGC spec.

Programming Experience in SQL
The query below returns all the stores within half a mile of the address of a person whose ID is equal to 2. In the where clause we multiply the result of STDistance by 0.00062 to convert meters to miles.

DECLARE @dist sys.geography

SET @dist = (SELECT p.Address

FROM dbo.People as p

WHERE p.PersonID = 2)

SELECT [Store].[Name],

[Store].[Location].Lat,

[Store].[Location].Long

FROM [dbo].[Stores] AS [Store]

WHERE (([Store].[Location].STDistance(@dist)) * cast(0.00062 as float(53))) <= .5

In the sample below, we change the person’s address to a different coordinate. Note that STGeomFromText takes two parameters: the point, and a Spatial Reference Identifier (SRID). The value of 4326 maps to the WGS 84 which is the standard coordinate system for the Earth.

update [dbo].[People]

set [Address] = geography::STGeomFromText('POINT(-122.206834 57.611421)', 4326)

where [PersonID] = 2

Note that when listing a point, the longitude is listed before the latitude.

Design

Goals
The goals for spatial are the following:

To provide a first class support for spatial in EDM
Rich Programming experience in LINQ and Entity SQL against Spatial types
Code-first, Model-first, and Database-first support
Tooling (not covered in this blog)

We have introduced two new primitive EDM Types called Geometry and Geography. This allows us to have spatial-typed properties in our Entities and Complex Types. As with every other primitive type in EDM, Geometry and Geography will be associated with CLR types. In this case, we have created two new types named DBGeometry and DBGeography which allows us to provide a first-class programming experience against these types in LINQ and Entity SQL.

One can describe these types in the CSDL in a straightforward fashion:

<Key>

</Key>

</EntityType>

Representing in SSDL is very simple as well:

<Key>

</Key>

</EntityType>

Be aware that spatial types cannot be used as entity keys, cannot be used in relationships, and cannot be used as discriminators.

Usage
Here are some scenarios and corresponding queries showing how simple it is to write spatial queries in LINQ:

Query Scenario	Example
Simple Select of Entities with Spatial columns	// Store is an entity type with Location as a Geography type var stores = from s in db.Stores select s;
Simple Select of Spatial values	var storeLocations = from s in db.Stores select s.Location;
Query with filter, using static constructors	var store = from s in db.Stores where s.Location == DbGeography.Parse(”POINT(-122.206834 47.611421)”) select s;
Query with filter using local variable	var store = from s in db.Stores where s.Location == loc select s;
Query involving Spatial methods	var distances = from s in db.Stores select s.Location.Distance(anotherLocation);
Find all the stores within a mile of a given address	var person = db.People.Single(p => p.PersonID == 2); var stores = from s in db.Stores where s.Location.Distance(person.Address) * .00062 <= 1 select new Location { Name = s.Name, Latitude = s.Location.Latitude, Longitude = s.Location.Longitude };

Have in mind that spatial types are immutable, so they can’t be modified after creation. Here is how to create a new location of type DbGeography:

s.Location = DbGeography.Parse("POINT(-122.206834 47.611421)");

db.SaveChanges();

Spatial Functions

Our Spatial implementation relies on the underlying database implementation of any of the spatial functions such as Distance, Intersects, and others. To make this work, we have created the most common functions as canonical functions on EDM. As a result, Entity Framework will defer the execution of the function to the server.

Client-side Behavior
DbGeometry and DbGeography internally use one of two implementations of DbSpatialServices for client side behavior which we will make available:

One implementation relies on Microsoft.SQLServer.Types.SQLGeography and Microsoft.SQLServer.Types.SQLGeometry being available to the client. If these two namespaces are available, then we delegate all spatial operations down to the SQL assemblies. Note that this implementation introduces a dependency.

Another implementation provides limited services such as serialization and deserialization, but does not allow performing non-trivial spatial operations. We create these whenever you explicitly create a spatial value or deserialize one from a web service.

DataContract Serialization

Our implementation provides a simple wrapper that is serializable, which allows spatial types to be used in multi-tier applications. To provide maximum interoperability, we have created a class called DbWellKnownSpatialValue which contains the SRID, WellKnownText (WKT), and WellKnownBinary (WKB) values. We will serialize SRID, WKT and WKB.

Questions

We want to hear from you. As we work through this design, it is vital to hear what you think about our decisions, and that you chime in with your own ideas. Here are a few questions, please take some time to answer them in the comments:

In order to have client-side spatial functionality, EF relies on a spatial implementation supplied by the provider. As a default, EF uses the SQL Spatial implementation. Do you foresee this being a problematic for Hosted applications which may or may not have access to a spatial implementation?
Do you feel as though you will need richer capabilities on the client side?
In addition to Geometry and Geography types, do you need to have constrained types like Point, Polygon?
Do you foresee using heterogeneous columns of spatial values in your application?

Spatial data is ubiquitous now thanks to the widespread use of GPS-enabled mobile devices. We are very excited about bringing spatial type support to the Entity Framework. We encourage you to leave your thoughts on this design below.

Cheers,
Pedro Ardila
Program Manager – Entity Framework Team

Unique Constraints in the Entity Framework

dpblogs — Thu, 10 Mar 2011 01:44:00 GMT

Update: this feature has been postponed and will not be included in Entity Framework 5.

Unique Constraints in the Entity Framework

We’ve been busy working on enabling unique constraints in the Entity Framework. Unique Constraints allow you to model candidate keys (also known as alternate keys, or natural keys). This post covers the design of the feature so that we can get some early thoughts and opinions on what you’d like to see in this feature.

Note: Tooling and the design time experience is an essential part of any Entity Framework feature that we do. However, the motivation of the Design blog is to take a look under the hood of the features as we build them, and therefore we cover XML metadata and code as opposed to what the design time experience will look like. We will have subsequent posts that cover the designer experience.

Background

Being able to model and use Unique Constraints is one of the top requests we hear from the community using Entity Framework. Like a primary key, a unique constraint is a set of properties on an entity whose values identify the entity as unique. Examples of unique constraints might be Social Security Number, Employee Number, Serial Number, etc. Unique constraints are often used to represent a natural and meaningful key, when a generated key is used as a primary key.

Most databases support unique constraints today. Foreign key constraints can generally be based on any unique constraint (as opposed to only on the Primary Key) on the principal side.

A key difference between a unique constraint and an entity key is that null values are allowed in columns that participate in the unique constraint.

Consider the following example. Here, I have an Employee entity type which includes an EmployeeId, as well as a Name and a Social Security Number.

The EmployeeId is the entity key, and it serves the purpose of uniquely identifying each Employee entity. However, I also have a SocialSecurityNumber property that can be used to uniquely identify the entity. This is a unique constriant, and today in the Entity Framework there is no way to specify that this is unique field.

This requirement itself is actually pretty self-explanatory, and we might as well say that what’s needed is a way to represent this “uniqueness” in the metadata that describes the entity data model. However, the fact that you can make assumptions about the uniqueness of a given property allows you to take advantage of that fact in interesting ways. For example, DDL generation could leverage this information to generate the corrsponding database constructs .

Taking this example further, let’s say I have the following entity data model.

In this case, I now have a few associations that are based on the EmployeeId. What I might actually want in this case is to build one of the associations using EmployeeId as the foreign key, but build the other association using the SocialSecurityNumber as the foreign key value – i.e. in this particular example, it might make sense to build my data model like this instead:

So what I’ve done here is that I’ve used the unique constriant on SocialSecurityNumber as the basis for the association between TaxWithholding and Employee. In this particular case, the SocialSecurityNumber property seems to be more appropriate and pertinent information defining the TaxWitholding entity type. It is also the perfect for acting as a “foreign key” to the parent Employee. Because I have a first class association based on this constraint, I am also able to take advantage of the richness in the conceptual model – like being able to navigate the association using a Navigation Property.

Unique Constraints in the Model

Let’s take a look at CSDL metadata for Employee and TaxWithholding types, and the association between them:

<Key>

</Key>

</UniqueConstraint>

<NavigationProperty Name="EmployeeDetail"

Relationship="UniqueModel.EmployeeEmployeeDetail"

FromRole="Employee" ToRole="EmployeeDetail" />

<NavigationProperty Name="TaxWithholding"

Relationship="UniqueModel.EmployeeTaxWithholding"

FromRole="Employee" ToRole="TaxWithholding" />

</EntityType>

<Key>

</Key>

<NavigationProperty Name="Employee"

Relationship="UniqueModel.EmployeeTaxWithholding"

FromRole="TaxWithholding" ToRole="Employee" />

</EntityType>

<End Type="Model.Employee"

Role="Employee"

Multiplicity="1" />

<End Type="Model.TaxWithholding"

Role="TaxWithholding"

Multiplicity="0..1" />

</Principal>

</Dependent>

</ReferentialConstraint>

</Association>

Foreign Key Associations may be based on a Unique Constraint on the principal end. The ReferentialConstraint element above shows exactly this scenario.

In order to represent unique constraints as they are defined in the store / database, Store Metadata (SSDL) also includes the ability to specify unique constraints as they are defined on the database schema. Here’s the corresponding SSDL for the above example:

<Key>

</Key>

</UniqueConstraint>

</EntityType>

<Key>

</Key>

</EntityType>

<End Role="TaxWithholding"

Type="Store.TaxWithholdings" Multiplicity="0..1" />

</Principal>

</Dependent>

</ReferentialConstraint>

</Association>

The details specified in SSDL are very similar to what is specified in the CSDL. A Unique Constraint in CSDL can map to a Unique constraint on the SSDL implicitly, based on how the columns are mapped in the MSL metadata. Nothing additional needs to be specified.

Here are some more details with regards to how Unique constraints can be specified:

In many ways unique constraints are similar to entity keys:
- Unique constraints will be supported on Entity Type
- More than one scalar property may compose a Unique constraint.
- Server generated values can participate in a unique constraint
- Navigation Properties may not participate in a unique constraint.
- Associations can be based on unique constraints
However, unlike Entity Keys:
- Unique constraints have names to identify them. These names should be unique within a type hierarchy.
- Nullable properties may participate in unique constraints
A given property can participate in more than one unique constraint.
Properties participating in unique constraint may be mutable
Unique constraints may include properties from a base type

Support for OnUpdate cascading behavior

Unlike Primary Keys, Entity Framework will support mutability on properties that participate in a unique constraint.

Along with that, we plan on supporting OnUpdate cascading behavior. This is something that is supported by databases. It simply allows you to specify that when the principal end of the relationship changes, the dependent end should be automatically updated accordingly as well.

Cascading is only handled for entities and relationships that are loaded into the object state manager. If you have OnUpdate Cascade behavior specified in the model, it will most certainly also make sense to make sure that the database itself has the same OnUpdate Cascade specification so that the database will manage the referential integrity of your data. This is equivalent to the OnDelete Cascade behavior supported today for associations based on primary keys.

This is how you can specify OnUpdate Cascade behavior:

<End Role="TaxWithholding"

Type="Store.TaxWithholdings" Multiplicity="0..1" />

</End>

</Principal>

</Dependent>

</ReferentialConstraint>

</Association>

The default Action for OnUpdate is “None”.

What mapping scenarios are not supported?

We are still contemplating exactly what set of capabilities might not make it into the next release with regards to the unique constraint support– therefore your feedback on this will be of significant value to us.

As it stands right now, the following scenarios will *NOT* be supported:

Split Entities based on Unique Constraints – this is the scenario where a single entity is split across multiple tables on the database, where the split/join is based on a Unique Constraint instead of Primary Key.
Inheritance Support with Unique Constraints – Support mapping inheritance based on unique constraints as opposed to primary key
Independent Association based on Unique Constraints – This is the association type that was present in Entity Framework 3.5SP1 (as opposed to the Foreign Key Associations that we released in Entity Framework 4.0). In theory it is possible to enable Independent associations to store the foreign key values based on unique constraints but we are considering not supporting it because of the popularity of Foreign Key Associations as the standard way of representing associations.

We would like to know more about whether you need the following capabilities:

Associations based on unique constraints – Do you see the need to form associations using unique constraints where you are associating to a unique constraint on the principal end instead of the primary key on that entity? Is this something you see in existing/legacy databases?
Mutability of Unique Constraints – Mutability of unique constraints are important. However, if you are associating to a unique constraint on the principal end, how important is mutability in this scenario?

Once again, the purpose of this post is to find out how you’d like to see the feature built – so please feel free to tell us what your own preferences would be around this feature.

Please let us know what you think!

Faisal Mohamood – Program Manager, Entity Framework

Kati Iceva – Software Design Engineer, Entity Framework

Table-Valued Function Support

dpblogs — Sat, 22 Jan 2011 01:07:00 GMT

One of the key features coming in the next release of Entity Framework is Table-Valued Function support. It is a very popular customer request and we have been working diligently to design a solution that we hope you find simple, yet useful. This design intends to make TVFs first-class citizens on Entity Framework, by allowing a user to map TVFs to both Entities and Complex types.

This article will explore the design of the feature in the framework, so we will focus on the metadata within the EDMX as opposed to the designer experience in Visual Studio. We will cover that experience in a later post. Other topics we will cover in the future include QueryViews and other complex Function Mapping scenarios.

With that said, here are the specific areas we will cover now:

· A basic overview of TVFs; pros and cons vs. Stored Procedures and Views

· How to call a TVF Directly

· Mapping a TVF to a Collection of Complex type

· Mapping a TVF to a Collection of Entity type and performing CRUD on the Entity

We’d love to hear your thoughts, so please feel free to comment and make suggestions about any part of this design.

What Are TVFs?

TVFs are User Defined Functions (UDFs) which live in the target database and whose return type is a table. UDFs are constructs made up of one or more T-SQL statements. In the context of a T-SQL query, TVFs can be used anywhere a View or a Table would be used. Although there is some overlap between TVFs, Views, and Stored Procedures, there are key advantages which make TVFs appealing in certain scenarios. Let’s look at how TVFs stack up against Views and Stored Procedures:

TVFs vs. Views
TVFs and Views are similar in that they are both composable. This means that you can use the function—or the view—anywhere within the body of a T-SQL query. Their major differences are, first, that SQL Views can only have one SELECT statement, whereas TVFs can have multiple statements as well as procedural code. This makes it possible to design more complex logic with TVFs than with Views. Additionally, TVFs can take parameters, whereas Views cannot.

TVFs vs. Stored Procedures
TVFs and stored procedures may both contain procedural code, but unlike stored procedures, TVFs can be used anywhere in a query. This allows us to both compose queries using TVFs and filter the result set of a TVF. One advantage of stored procedures over TVFs is that stored procedures are capable of returning multiple result sets.

Background: TVF declaration and usage in SQL

TVFs allow you to write complex statements and queries, all in the same place. In this post we will use relatively simple TVFs in order to keep complexity low. Please read this article to learn more about TVFs in SQL.

Throughout the post we will use these two TVFs to showcase functionality:

· GetCategories() returns all of the entries in the Categories table.

CREATE FUNCTION [dbo].[GetCategories]()

RETURNS TABLE

RETURN

SELECT [CategoryID]

,[CategoryName]

,[Description]

,[Picture]

FROM [dbo].[Categories]

· GetDetailsForOrder(int) returns the Order Details for a given order for Id equal to Oid.

10.

11.

CREATE FUNCTION [dbo].[GetDetailsForOrder]

(@Oid INT)

RETURNS TABLE

RETURN

SELECT [OrderID]

,[ProductID]

,[UnitPrice]

,[Quantity]

,[Discount]

FROM [dbo].[OrderDetails]

WHERE OrderID = @Oid

Typical usage of a TVF in T-SQL as follows:

SELECT d.ProductID, d.Quantity

FROM dbo.GetDetailsForOrder(10248) AS d

This simple statement returns the category ID and Category Name from our TVF.

ProductID Quantity

----------- --------

11 12

42 10

72 5

Note that we were able to call the TVF in the FROM clause of our T-SQL query, as we would a table. Also note that we were able to pass in parameters, and that the result of the query was a table. This combination of factors, as well as the possibility to have procedural code within your function, is what makes TVFs appealing.

Why Should You Use TVFs in EF?

If you already use TVFs, the advantages of having EF support are obvious: You will now be able to use your TVFs as the source for materializing entity types and complex types via queries in EF – giving you the ability to use LINQ and Entity SQL against them. If you don’t already use TVFs, think about the potential we get from creating Entity or Complex type that is backed by a function instead of an entity. TVF support enables interesting scenarios such as Full-Text search since in SQL, it is implemented using TVFs.

Using TVFs in Entity Framework

Now that we’ve established what a TVF is, let’s see what artifacts we need in order to use one in an EF app:

· Metadata describing the store function and its return type.

· A function stub is required to call the function from a LINQ query. The stub is not required for Entity SQL queries.

These two items are all we need if we want to call the TVF directly in a LINQ or Entity SQL query. However, to truly take advantage TVFs in EF (reusability, change tracking, CUD), we also need to describe the mapping of the function to a Collection of Entity Type or Collection of Complex Type.

TVFs as Functions in SSDL returning RowType

Storage Layer
In order to make the TVF visible to Entity Framework, we must declare the store function in the storage layer (SSDL). We must specify the function name, whether or not it is composable, and the schema it belongs to. Up until version 4 of EF, the Function node was only capable of returning a scalar. Our new design required us to make two changes to this node: First, we can now set IsComposable to true, and therefore we are able to compose functions. Second, the return type is no longer a property of Function, but instead a child node in which we can describe the rich return types produced by TVFs.

Here is how we declare the store function called GetCategories in our SSDL:

10.

11.

12.

13.

</RowType>

</CollectionType>

</ReturnType>

</Function>

Here we specify the ReturnType of the Function. In this case it will be a collection of RowTypes.

Object Layer
We need a function stub to refer to the store function from a LINQ query. The result of calling the function stub is a set of unstructured rows. See this design blog to learn more about function stubs. Here is what the stub will look like:

[EdmFunction("NorthwindModel.Store", "GetCategories")]

public IQueryable<DbDataRecord> GetCategoriesDirect()

{

return this.CreateQuery<DbDataRecord>("[NorthwindModel.Store].[GetCategories]()");

}

This stub will usually reside within your strongly typed ObjectContext class. Here are a few things to take note of:

· The stub returns a collection of DbDataRecord. These records aren’t richly typed and therefore only provide us with limited functionality.

· The EdmFunction attribute is the store’s namespace since we are calling a store function.

· Our function stub contains a meaningful body, as opposed to an exception (see here for an example). This ‘bootstrapping’ allows us to call the function in the FROM clause of our LINQ query. Here is exactly how we would call our TVF:

using (var context = new NorthwindEntities())

{

//Call GetCategories directly using Entity LINQ

var query = from c in context.GetCategoriesDirect()

select c;

foreach (var r in query)

Console.WriteLine("[{0}] {1}", r.GetInt32(0), r.GetString(1));

}

The stub is not required when using Entity SQL for our queries. Here is how to call the same TVF in Entity SQL:

10.

using (var context = new NorthwindEntities())

{

//Call GetCategories directly using Entity SQL

string query = @"Select Category.CategoryId, Category.CategoryName

From NorthwindModel.Store.getCategories() as Category";

var categories = context.CreateQuery<DbDataRecord>(query);

foreach (DbDataRecord category in categories)

Console.WriteLine("[{0}]: {1}", category.GetInt32(0), category.GetString(1));

}

The functionality above is useful, but only to a limited extent since unstructured data records are read only.

Mapping TVF to Complex Types and Entity Types

Function Mapping is a new feature which allows us to map the TVF’s results to a collection of Complex Type as well as a collection of Entity Type. By mapping to an Entity Type, we gain full CRUD functionality against our entities. The first step is to specify the function mapping in the mapping layer.

Result Type Mapping

Below we map a function in the conceptual layer to a function in the storage layer with an explicit return type mapping. Explicit mapping is supported for conceptual functions returning collection of Complex Type or a collection of Entity Type. Using explicit mapping, we describe how each property in the conceptual function will map to a property in the store function. We could use implicit mapping, but only for conceptual functions returning a scalar or a collection of Rowtype with primitive properties.

Collection of Complex Type

A TVF’s result set can map to a collection Complex Type. This is the default behavior when using Model First or EDMGen. Let’s explore this mapping using GetDetailsForOrder:

Storage Layer

10.

11.

12.

13.

14.

15.

16.

</RowType>

</CollectionType>

</ReturnType>

</Function>

Mapping Layer

To tie the function to a collection of Complex Types, we need to create a mapping between our store function and a conceptual function. We do that in the following way:

10.

11.

12.

13.

14.

15.

16.

<FunctionMapping StorageFunction="NorthwindModel.Store.GetDetailsForOrder"

CdmFunction="NorthwindModel.GetDetailsForOrder">

</Parameters>

</ComplexTypeMapping>

</ResultMapping>

</FunctionMapping>

The explicit mapping above describes how to map the columns returned by the TVF to our Complex Type’s properties. It also describes the parameters. In this iteration we will only support parameters of Primitive type.

Conceptual Layer

Finally we need a conceptual function whose return type is a collection of Complex Types. The Complex type and the function look like the following:

10.

11.

12.

13.

</ComplexType>

…

<Function Name="GetDetailsForOrder"

ReturnType="Collection(NorthwindModel.GetDetailsForOrder_Result)">

</Function>

In the code above we declared a complex type then we created a function which returns a collection of such type. EDMGen uses “<FunctionMame>_result” as the Complex Type’s name by default when generating metadata.

Object Layer

Once these are in place we can use call TVF using Entity SQL. However, we need a new function stub to use the TVF in a LINQ query:

[EdmFunction("NorthwindModel", "GetDetailsForOrder")]

public IQueryable<GetDetailsForOrder_Result> GetDetailsForOrder(int Oid)

{

return this.CreateQuery<GetDetailsForOrder_Result>("NorthwindModel.GetDetailsForOrder(@Oid)",

new ObjectParameter("Oid", Oid));

}

The noteworthy differences from our first stub are that this one returns a collection of Complex Type, rather than a collection of unstructured records. Also, note that the first parameter in the EdmFunction attribute is the conceptual function, as opposed to the store function.

Usage
Let’s have a look at the basic usage of our function in LINQ and Entity SQL.

10.

using (var context = new NorthwindEntities())

{

int OrderId = 10248;

//GetDetailsForOrder using LINQ

var OrderDetails = from O in context.GetDetailsForOrder(OrderId)

select O;

foreach (var od in OrderDetails)

Console.WriteLine("[{0}] {1}", od.OrderID, od.ProductID);

}

10.

11.

12.

13.

14.

using (var context = new NorthwindEntities())

{

int OrderId = 10248;

//GetDetailsForOrder using Entity SQL

string queryString = @"Select VALUE OrderDetail

From NorthwindModel.GetDetailsForOrder(@Oid) as OrderDetail";

ObjectQuery<GetDetailsForOrder_Result> OrderDetailsQuery =

new ObjectQuery<GetDetailsForOrder_Result>(queryString, context);

OrderDetailsQuery.Parameters.Add(new ObjectParameter("Oid", OrderId));

foreach (var result in OrderDetailsQuery)

Console.WriteLine("[{0}] {1}", result.OrderID, result.ProductID);

}

Both samples do the same. They call GetDetailsForOrder with an OrderId of 10248 and then iterate over the result.

Collection of Entity Type

We can also make our TVF return entities. We get all of the functionality we would expect from entities, such as reading, writing, and navigation provided by using NavigationProperties. EDMGen does not map to Entities by default, but we intend to provide tooling support to create such mapping without having to edit the EDMX.

Storage Layer
The store function definition remains the same as described above.

Mapping Layer
Now we need to describe the mapping between the store function and the conceptual function.

10.

11.

12.

13.

14.

15.

16.

<FunctionMapping StorageFunction="NorthwindModel.Store.GetDetailsForOrder"

CdmFunction="NorthwindModel.GetDetailsForOrderAsEntity" >

</Parameters>

</EntityTypeMapping>

</ResultMapping>

</FunctionMapping>

This mapping describes the TVF’s parameters and maps the function’s result to a collection of Entity types named OrderDetails. Each ScalarProperty node has two properties, Name and ColumnName. Name refers to the EntityType and ColumnName refers to the Store function. Function Mapping supports Table-Per-Hierarchy inheritance, much like in FunctionImportMapping. This topic will be covered in a later post.

Conceptual Layer
When you map a function to a collection of Entity types, you get the following in the CSDL:

<Function Name="GetDetailsForOrderAsEntity" ReturnType="Collection(NorthwindModel.OrderDetails)"

EntityContainer="NorthwindEntities" EntitySet="OrderDetails">

</Function>

This metadata is similar to our previous example. However, when mapping to Entity types, we must list two additional properties in our Function node: an EntityContainer and an EntitySet. These two values must refer to existing artifacts in the conceptual layer.

Object Layer
Here is our function stub for a TVF’s result set mapped to a collection of Entity Types:

[EdmFunction("NorthwindModel", "GetDetailsForOrderAsEntity")]

public IQueryable<OrderDetails> GetDetailsForOrderAsEntity(int Oid)

{

return this.CreateQuery<OrderDetails>("[NorthwindModel].[GetDetailsForOrderAsEntity](@Oid)",

new ObjectParameter("Oid", Oid));

}

Usage
With our mapping in place, we can begin using our Entities in the usual fashion. In the following example we will call our TVF and do some simple update to the resulting Entities:

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

using (var context = new NorthwindEntities())

{

// Use GetDetailsForOrderAsEntity with LINQ and update entities using ObjectContext

var OrderDetails = from OrderDetail in context.GetDetailsForOrderAsEntity(10248)

where OrderDetail.Quantity > 1

select OrderDetail;

foreach (var item in OrderDetails)

{

Console.WriteLine("[{0}] {1} – Original Quantity={2}",

item.OrderID, item.ProductID, item.Quantity);

item.Quantity = 1;

}

context.SaveChanges();

OrderDetails = from OrderDetail in context.GetDetailsForOrderAsEntity(10248)

select OrderDetail;

Console.WriteLine("New Values");

foreach (var item in OrderDetails)

Console.WriteLine("[{0}] {1} – New Quantity={2}",

item.OrderID, item.ProductID, item.Quantity);

}

In this example we get the products in Order #10248 with a quantity greater than 1. Then we change each quantity to 1. The important thing to note about this scenario is that we were able to query for a specific set of order details –those with a quantity greater than 1—using a TVF. Then we edited the resulting entities to which the TVFs result set was mapped.

Wrap-Up

In this article we have covered:

· Background on TVFs in SQL

· Calling a TVF directly from a LINQ or Entity SQL query

· Metadata for mapping a collection of Complex Type to a TVF, and subsequent usage

· Metadata for mapping a collection of Entity Type to a TVF, and subsequent usage including CUD

TVF support will be a key component of Entity Framework’s next release. Our goal is to provider first-class support for TVFs by allowing users to map the function’s result set to Entity types and Complex types. We look forward to getting your feedback, so feel free to make any comments or ask questions about this design.

Pedro Ardila
Program Manager
ADO.Net Entity Framework Team

Code First Database Evolution (aka Migrations)

dpblogs — Fri, 22 Oct 2010 21:42:00 GMT

Back in July we released the latest preview of Code First and we’ve been receiving a lot of feedback that is helping us shape this feature. One of the top asks we are hearing is for a solution that will evolve the database schema as your object model changes over time, often referred to as migrations.

Our team is looking at a few different options in this area and we really want to get your feedback on one of the options we are considering that takes a somewhat different approach to most existing migration solutions.

We’ve put together a screen cast to walk you through a storyboard style mock-up we built that shows this solution being used to build and deploy an application. We really want your feedback to help us shape and evaluate this option, you can provide feedback using the comment functionality on this post.

This is the first time our team has used a storyboard screencast to share proposed designs so we’d also love to hear your feedback on whether this is an effective way to share our ideas.

ADO.NET Entity Framework Team

(HD option is available in lower right corner of player while the screencast is playing)

Entity Designer Improvements Preview

dpblogs — Mon, 11 Oct 2010 23:13:00 GMT

The team has been quietly working on a number of improvements to the entity designer that we would like to show off and get feedback on. This particular set of features is targeted at making modeling more productive and scalable, and models themselves more comprehensible.

We’ll start with the small, fun things, and build up to a grand finale.

1. Association Highlighting

Let’s start with this. Consider the following model:

We would like to rename the navigation properties to something meaningful, as well as the foreign keys. But which navigation property is related to which foreign key? And to which association connector?

So, now you can select an association or navigation property and find out. Here, we select the “Address.Person” navigation property, and the designer highlights the two entity types involved in the association, the association connector, the navigation property on the other side, as well as the foreign keys that hold the navigation information.

This makes it easier for us to rename things to:

2. Property Reordering

Imagine that you want to add a “Name” property to the “Property” type above. You do this and end up with a type that looks like this:

But of course, you want this new property to be the second one in the type, below “Id”. Well, now you can select this property and, for example, press Alt + Home, which will move that property to the top of the type, then on Alt + Down Arrow to move it down one slot. Here is a screenshot of the type, along with the new “Move Properties” menu:

You can now select multiple properties at the same time and move them. For example:

3. Entity Shape Coloring

Imagine, now, that our model has gotten more complex, and viewing it all in one window renders areas of the diagram hard to distinguish:

Entity shape coloring allows us to visually divide a single diagram into multiple meaningful areas by color coding them. You can select one or more entities and in their property sheet, change their color, which lets us do something like this:

4. Multiple Diagrams!

But what if you just want to look at subject areas within a diagram? Say, the product catalog entities, or the sales types. In that case, you can select any number of entities and move them into a new diagram by right-clicking and selecting the new “Move to new Diagram” menu item. If we do this to our “Property” type, we’ll see it appear in a new diagram. And, if we make the model browser tool window visible, we’ll see this:

Note that we now have a new “Diagrams” folder in the model browser, and that the new diagram opens in its own tab, allowing you to have multiple windows open at once against the same model. This diagram is a little sparse. Let’s bring into it any types related to “Property” by right-clicking it and selecting “Include Related”:

Now, color the “Person” type green, then bring its related type, and see how they pick up the original type’s color:

(Note that this models is not a real-world one and only intended for visualization purposes.)

Some additional notes:

- You can also drag and drop associations, types, and entity sets from the model browser.

- You can cut or copy and paste objects from one diagram to another.

- Deleting objects from a diagram will no longer delete them from the model, but only from the diagram.

- You can delete objects from the model by using Shift+Del, right clicking and selecting “Delete from Model”, or deleting them from the model browser.

- Diagrams are stored in a child file of the edmx for new edmx files. For backwards compatibility, models created in previous versions of visual studio will keep the diagram information in the EDMX file itself. However, you can move the diagram information from the EDMX to a child file by right clicking and selecting the “Move Diagrams to Separate File” context menu.

We look forward to your feedback!

Noam

Data Binding with DbContext

dpblogs — Wed, 08 Sep 2010 20:20:00 GMT

The information in this post is out of date.

Visit msdn.com/data/ef for the latest information on current and past releases of EF.

For WPF Data Binding see http://msdn.com/data/jj574514

For WinForms Data Binding see http://msdn.com/data/jj682076

We recently announced the release of Feature CTP4 for the ADO.Net Entity Framework (EF). CTP4 contains a preview of new features that we are considering adding to the core framework and would like community feedback on. CTP4 builds on top of the existing Entity Framework 4 (EF4) which shipped as part of .NET Framework 4.0 and Visual Studio 2010.

CTP4 contains the first preview of a set of Productivity Improvements for EF that provide a cleaner and simpler API surface designed to allow developers to accomplish the same tasks with less code and fewer concepts. The Productivity Improvement effort is described in more detail in a recent Design Blog post.

Since the release of CTP4 we have been working to enable more scenarios for developers using DbContext & DbSet, one of the scenarios we have been looking at is Data Binding in WPF and WinForms. The purpose of this post is to share the direction we are heading and provide the chance for feedback on our design. The features discussed in this post are not included in CTP4.

Requirements

Before we cover the current design let’s take a quick look at the requirements we used to drive the design:

The high level requirement is to “enable two way data binding to a set of objects retrieved from a DbContext”

The set of objects bound to a control will all be of the same base type
(i.e. all belong to the same DbSet<T>)
Binding should display Unchanged, Modified and Added objects of the given type
- Deleted objects should not be displayed
Data binding features in WPF and WinForms (such as sorting) should be supported natively
Data binding should be two-way
- When the UI adds an object to the data source it should be added to the DbContext
- When the UI removes an object from the data source it should deleted from the DbContext
- When a new object enters the DbContext it should appear in the UI
- When an object is deleted from the DbContext it should be removed from the UI

WPF Experience

Let’s say we have a WPF window that displays a list of Employees and has a “Save” button:

Using some new additions to the DbSet<T> API we could implement the code behind our window as follows:

namespace DatabindingSample
{
    public partial class CustomerWindow : Window
    {
        private EmployeeContext context = new EmployeeContext();

        public CustomerWindow()
        {
            InitializeComponent();
        }

        private void Window_Loaded(object sender, RoutedEventArgs e)
        {
            context.Employees.Load();
            this.DataContext = context.Employees.Local;
        }

        private void Save_Click(object sender, RoutedEventArgs e)
        {
            this.context.SaveChanges();
        }

        private void Window_Closing(object sender, CancelEventArgs e)
        {
            this.context.Dispose();
        }
    }
}

DbSet<TEntity>.Local

This new property will give you an ObservableCollection<TEntity> that contains all Unchanged, Modified and Added objects that are currently tracked by the DbContext for the given DbSet. As new objects enter the DbSet (through queries, DbSet.Add/Attach, etc.) they will appear in the ObservableCollection. When an object is deleted from the DbSet it will also be removed from the ObservableCollection. Adding or Removing from the ObservableCollection will also perform the corresponding Add/Remove on the DbSet. Because WPF natively supports binding to an ObservableCollection there is no additional code required to have two way data binding with full support for WPF sorting, filtering etc.

Load Extension Method

Because DbSet<TEntity>.Local gives us objects that are currently tracked by the DbContext we need a way to get existing objects from the database loaded into memory. Load is a new extension method on IQueryable that will cause the results of the query to be iterated, in EF this equates to materializing the results as objects and adding them to the DbContext in the Unchanged state. Of course you aren’t restricted to using the Load method to bring data into memory for databinding, any existing operations that cause objects to be materialized will also work (such as iterating results, ToList, ToArray, etc.).

Filtering

Because Load is an extension on IQueryable it can be called directly on a DbSet to load all items into memory or it can be used after LINQ operators. For example we could re-write our WPF code to only load Employees with an IsActive flag set to true:

private void Window_Loaded(object sender, RoutedEventArgs e)
{
context.Employees.Where(e => e.IsActive).Load();
this.DataContext = context.Employees.Local;
}

The above example only works if we only ever bring active Employees into the context, but if other parts of the application need to bring inactive Employees into memory using the same context then we need a better way to filter. If you are using WPF then this functionality is already built in:

private void Window_Loaded(object sender, RoutedEventArgs e)
{
     context.Employees.Load();

     var view = new ListCollectionView(context.Employees.Local);
     view.Filter = e => ((Employee)e).IsActive;
     this.DataContext = view;
}

Outside of WPF there are many other databinding frameworks (such as Continuous LINQ) that solve this problem.

WinForms Experience

The WinForms experience is very similar to WPF except that we need a collection that implements IBindingList rather than an ObservableCollection. We unfortunately can’t cater for both technologies with the same collection type because WPF will use the IBindingList interface if it is present and this gives a degraded databinding experience. Therefore we require one additional step to convert our ObservableCollection<TEntity> to something that WinForms will happily bind to. Using a similar form as the WPF example but this time in WinForms:

Our code behind looks very similar to the WPF code except that we call ToBindingList to convert our local list to a WinForms friendly equivalent:

namespace DatabindingSample
{
    public partial class CustomerForm : Form
    {
        private EmployeeContext context = new EmployeeContext();

        public CustomerForm()
        {
            InitializeComponent();
        }

        private void CustomerForm_Load(object sender, EventArgs e)
        {
            context.Employees.Load();
            this.employeeBindingSource.DataSource = context.Employees.Local.ToBindingList();
        }

        private void saveButton_Click(object sender, EventArgs e)
        {
            this.context.SaveChanges();
        }

        private void CustomerForm_FormClosing(object sender, FormClosingEventArgs e)
        {
            this.context.Dispose();
        }
    }
}

Binding to Queries

One obvious question might be “Why not just bind directly to the results of a query?”. There are a couple of reasons we decided not to pursue this approach. LINQ to Entities queries pull results directly from the database which means they do not include objects that are in the context in an added state and may also include objects that have been marked as deleted, this is obviously not ideal when databinding as the UI won’t display new objects and may show deleted objects. Binding directly to a query would also cause a query to be issued to the database every time the UI iterates the collection, meaning if the UI needed to refresh it would result in all your data being pulled from the database again.

Summary

In this post we covered our plans to support databinding to the contents of a DbSet<T>, including existing data that is loaded from the database and new objects that haven’t been persisted to the database yet. We saw that two way databinding is supported in both WPF and WinForms. We also covered the new Load extension method that is used to materialize the results of a LINQ query into object instances prior to databinding. We’d like to hear any feedback you have on the proposed patterns or API surface.

Rowan Miller
Program Manager
ADO.Net Entity Framework Team

Productivity Improvements for the Entity Framework

dpblogs — Tue, 22 Jun 2010 00:46:00 GMT

Background

We’ve been hearing a lot of good feedback on the recently released update to the Entity Framework in .NET 4. This release marks a significant advancement from the first release that shipped with .NET 3.5 SP1. I’m not going to spend time here talking about what’s new, but you can check here to see for yourself.

With all that said, there are still a number of things we can do to simplify the process of writing data access code with the Entity Framework. We’ve been paying attention to the most common patterns that we see developers using with the EF and have been brewing up a set of improvements to the Entity Framework designed to allow developers to accomplish the same tasks with less code and fewer concepts.

These improvements provide a cleaner and simpler API surface that focuses your attention on the most common scenarios but still allows you to drill down to more advanced functionality when it’s needed. We hope you will enjoy this simpler experience, but we should be quick to assure you that this is NOT a new data access technology. These improvements are built on the same technology for mapping, LINQ, providers and every other part of the Entity Framework. Think of this as a fast path to writing data access code using conventions over configuration, better tuned APIs and other techniques intended to reduce development time when using the EF.

At this stage we have worked through what we think the core API and functionality should look like and would like your feedback. There are still some capabilities such as data binding and concurrency resolution which will cause the API to evolve as we continue the design process, but the main concepts are in place, so it is a good time for feedback.

Introducing DbContext & DbSet

At the heart of the Entity Framework Productivity Improvements are two new types, DbContext and DbSet<TEntity>. DbContext is a simplified alternative to ObjectContext and is the primary object for interacting with a database using a specific model. DbSet<TEntity> is a simplified alternative to ObjectSet<TEntity> and is used to perform CRUD operations against a specific type from the model. These new types can be used regardless of whether you created your model using the Entity Designer or code.

The obvious question is ‘Why not just simplify ObjectContext and ObjectSet<T>?’ We are opting to introduce these new types in order to, on the one hand, preserve full backward compatibility with existing EF applications and continue to address all of the advanced scenarios that are possible given the EF’s existing flexibility, while on the other hand streamlining the experience of using the EF and tuning it for the most common cases. We believe it is critical that the EF programming experience improve in some fundamental ways, and at the same time we are absolutely committed to our existing customers. Establishing a collaborative relationship between the existing types and the new types allows us to achieve both requirements. Also, there are easy ways to get to ObjectContext and ObjectSet<T> from DbContext and DbSet in case you want more control for a particular task.

One point we want to be very clear about is that these new types are not replacing any existing types; they are a simplified alternative that build on the existing types, and as we add features to the Entity Framework they will always be available in ObjectContext/ObjectSet, and they will also be available in DbContext/DbSet if appropriate.

We’ll drill into the API surface later, but first let’s take a look at the coding experience using these new types.

Code First Experience

DbContext provides a simplified Code First pattern that requires less code and takes care of some common concerns such as model caching, database provisioning, schema creation and connection management. This simplified pattern uses a number of conventions to take care of these tasks and allows tweaking or overriding of this behavior when required. Let’s start off by using these conventions to write the code needed to build a console application that performs data access using a model:

A Visual Basic version of these code samples is available here.

using System.Collections.Generic;

using System.Data.Entity;

namespace MyDataAccessDemo

{

class Program

{

static void Main(string[] args)

{

using (var context = new ProductContext())

{

var food = new Category { CategoryId = "FOOD" };

context.Categories.Add(food);

var cheese = new Product { Name = "Cheese" };

cheese.Category = context.Categories.Find("FOOD");

context.Products.Add(cheese);

context.SaveChanges();

}

public class ProductContext : DbContext

{

public DbSet<Product> Products { get; set; }

public DbSet<Category> Categories { get; set; }

}

public class Product

{

public int ProductId { get; set; }

public string Name { get; set; }

public Category Category { get; set; }

}

public class Category

{

public string CategoryId { get; set; }

public string Name { get; set; }

public ICollection<Product> Products { get; set; }

}

That is 100% of the code you would write to get this program running. No separate model definition, XML metadata, config file or anything else is required. Conventions are used to fill in all of this information. Obviously there is quite a bit going on under the covers so let’s take a closer look at some of the things DbContext is doing automatically.

Model Discovery

During construction we scan the derived context for DbSet properties and include the types in the model. Model Discovery uses the existing Code First functionality so the new default conventions we recently blogged about are processed during discovery. You can opt out of set discovery by specifying an attribute on the set properties that should be ignored.

Of course there will be times when you want to further describe a model or change what was detected by convention. For example say you have a Book entity whose ISBN property is the primary key, this won’t be detected by convention. There are two options here; you can use data annotations to annotate the property in your class definition:

public class Book

{

[Key]

public string ISBN { get; set; }

public string Title { get; set; }

}

Alternatively DbContext includes a virtual method that can be overridden to use the Code First fluent API on ModelBuilder to further configure the model:

public class ProductContext : DbContext

{

public DbSet<Book> Books { get; set; }

protected override void OnModelCreating(ModelBuilder modelBuilder)

{

modelBuilder.Entity<Book>().HasKey(b => b.ISBN);

}

Model Caching

There is some cost involved in discovering the model, processing Data Annotations and applying fluent API configuration. To avoid incurring this cost every time a derived DbContext is instantiated the model is cached during the first initialization. The cached model is then re-used each time the same derived context is constructed in the same AppDomain. Model caching can be turned off by setting the CacheForContextType property on ModelBuilder to ‘false’ in the OnModelCreating method.

DbSet Initialization

You’ll notice in the sample that we didn’t assign a value to either of the DbSet properties on the derived context. During construction DbContext will scan the derived context for DbSet properties and, if they expose a public setter, will construct a new DbSet and assign it to the property. You can opt out of set initialization by specifying an attribute on the set properties that should not be initialized.

You can also create DbSet instances using the DbContext.Set<TEntity>() method if you don’t want to expose public setters for the DbSet properties.

Database Provisioning

By default the database is created and provisioned using SqlClient against localhost\SQLEXPRESS and has the same name as the derived context. This convention is configurable and is controlled by an AppDomain setting that can be tweaked or replaced. You can tweak the default SqlClient convention to connect to a different database, replace it with a SqlCe convention that we include or define your own convention by implementing the IDbConnectionFactory interface.

public interface IDbConnectionFactory

{

DbConnection CreateConnection(string databaseName);

}

The active IDbConnectionFactory can be retrieved or set via the static property, Database.DefaultConnectionFactory.

DbContext also includes a constructor that accepts a string to control the value that is passed to the convention, the SqlClient and SqlCE factories allow you to specify either a database name or the entire connection string.

Before calling the convention, DbContext will check in the app/web.config file for a connection string with the same name as your context (or the string value if you used the constructor that specifies a string). If there is a matching entry, we will use that rather than calling the convention. Because connection string entries also include provider information this allows you to target multiple providers in one application.

Finally, if you want full control over your connections there is a constructor on DbContext that accepts a DbConnection.

Database First and Model First Experience

In the latest version of the Entity Framework that shipped with .Net Framework 4.0 and Visual Studio 2010 we introduced T4 based code generation. T4 allows you to customize the code that is generated based on a model you have defined using the designer in either the Database First or Model First approach. The default template generates a derived ObjectContext with an ObjectSet<T> for each entity set in your model.

The productivity improvements will also include a template that generates a derived DbContext with a DbSet<TEntity> for each entity set in your model. This allows Model First and Database First developers to make use of the simplified API surface described in the next section.

API Surface

DbContext is the starting point for interacting with your model. Compared to ObjectContext it has a greatly reduced number of methods and properties that are exposed at the root level. The aim is to expose only the most commonly used methods on DbContext and have the ability to drill down to more advanced APIs. One example of this is the Database property that exposes database related APIs. We will add a couple more members as we work through adding the rest of the advanced functionality but we want to keep it as minimal as possible. In most cases you would work with a context that derives from DbContext and exposes strongly typed DbSet properties for the types in your model.

public class DbContext : IDisposable

{

public DbContext(DbModel model, string nameOrConnectionString);

public DbContext(DbModel model, DbConnection existingConnection);
public DbContext(ObjectContext objectContext);

protected DbContext();

protected DbContext(string nameOrConnectionString);

protected DbContext(DbConnection existingConnection);

public Database Database { get; }

protected ObjectContext ObjectContext { get; }

protected virtual void OnModelCreating(ModelBuilder modelBuilder);

public virtual int SaveChanges();

public DbSet<TEntity> Set<TEntity>() where TEntity : class;

public void Dispose();

}

DbModel Constructors

These constructors can be used with Database First, Model First and Code First development. They are used by our T4 template for Database First and Model First Development and can also be used in Code First scenarios where a model is built externally using ModelBuilder.

Code First previously contained a ContextBuilder type which we have now split into two components, ModelBuilder and DbModel. ModelBuilder is mutable and exposes the fluent API for defining your model. ModelBuilder creates an immutable DbModel type that can be used to construct an ObjectContext or DbContext. DbModel can also be constructed from a Database First or Model First approach where an edmx file is generated.

ObjectContext Constructor

If you have an existing code base that uses ObjectContext and want to make use of the alternate surface of DbContext in some parts of your code then this constructor allows you to wrap the ObjectContext with the simpler surface.

Protected Constructors

The three protected constructors can be exposed or used within a derived context where you want to make use of the simplified Code First experience which was explained in the ‘Code First Experience’ section. They are protected because the model discovery, database provisioning and model caching mechanisms rely on having a derived context.

Database

The Database property exposes an instance of the new Database type that contains methods for interacting with the underlying database.

public class Database

{

public DbConnection Connection { get; }

public void Create ();

public bool CreateIfNotExists();

public bool Exists();

public static bool Exists(string nameOrConnectionString);

public static bool Exists(DbConnection existingConnection);

public void Delete ();

public static void Delete (string nameOrConnectionString);

public static void Delete (DbConnection existingConnection);

public bool DeleteIfExists();

public static bool DeleteIfExists(string nameOrConnectionString);

public static bool DeleteIfExists(DbConnection existingConnection);

}

ObjectContext

DbContext uses an ObjectContext internally and we make this available as a protected property just in case you ever need to drop down to the lower level API. You can use or expose the required functionality from a derived DbContext.

OnModelCreating

This protected member can be overridden when defining a derived context in CodeFirst development and allows you tweak the shape of your model that was detected by convention.

SaveChanges

This method exposes the same functionality as SaveChanges on ObjectContext and persists all pending changes to the database. It represents the Unit of Work for the context.

Set<TEntity>

This method will create a DbSet<TEntity> for an entity type that is part of your model, similar to CreateObjectSet<T> on ObjectContext.

DbSet represents a collection of a given entity type in your model, similar to ObjectSet<T> except that this new type also supports derived types whereas ObjectSet<T> only supported base types.

public class DbSet<TEntity> : IDbSet<TEntity>, IQueryable<TEntity> where TEntity : class

{

public void Add(TEntity entity);

public void Remove(TEntity entity);

public void Attach(TEntity entity);

public TEntity Find(params object[] keyValues);

}

Add, Remove & Attach

Add, Remove and Attach are similar to AddObject, DeleteObject and Attach on ObjectSet<T>. We’ve renamed Add and Remove to keep DbSet<TEntity> consistent with other sets in the framework.

Add and Attach can now be called on objects that are already tracked by the context, Add will ensure the object is in the added state and Attach the unchanged state. This is helpful in N-Tier and Web scenarios where you have a detached graph containing both existing and new entities that need to be hooked up to the context.

For example assume you have a new product that is linked to an existing category, neither instance is attached to a context (they may have been returned from a distributed client via a web service call or built from a post back in a web application). Because Add and Attach are graph operations, calling Add on the product will also Add the category. Previously you would need to drop down to lower level APIs (ObjectStateManager.ChangeObjectState) to mark the category as unchanged. Now however this can be achieved by calling Attach:

public void AddProduct(Product product)

{

using (var context = new ProductContext())

{

context.Products.Add(product);

context.Categories.Attach(product.Category);

context.SaveChanges();

}

Find

The new Find method will locate an object with the supplied primary key value(s). Find initially checks the in-memory objects before querying the database and is capable of finding added entities that haven’t been persisted to the store yet. If find doesn’t locate an entity with the matching key it returns null.

IDbSet<TEntity>

DbSet<TEntity> implements IDbSet<TEntity> to facilitate building testable applications.

public interface IDbSet<TEntity> : IQueryable<TEntity> where TEntity : class

{

void Add(TEntity entity);

void Attach(TEntity entity);

void Remove(TEntity entity);

TEntity Find(params object[] keyValues);

}

IDbSet<TEntity> allows you to define an interface that can be implemented by your derived context, similar to the example shown below. This allows the context to be replaced with an in-memory test double for testing. DbContext will still perform set discovery and initialization, described in the ‘Code First Development’ section, for properties typed as IDbSet<TEntity>.

public interface IProductContext

{

IDbSet<Product> Products { get; }

IDbSet<Category> Categories { get; }

}

public class ProductContext : DbContext, IProductContext

{

public IDbSet<Product> Products { get; set; }

public IDbSet<Category> Categories { get; set; }

}

Summary

These improvements are designed to provide a cleaner and simpler API surface that allows you to achieve common scenarios with ease while also allowing you to drill down to more advanced functionality when required. The improvements build on top of the existing Entity Framework components using conventions over configuration and a simplified API surface.

We’d like to hear any feedback you have on the proposed functionality and API surface.

Rowan Miller
Program Manager
ADO.NET Entity Framework Team

Conventions for Code First

dpblogs — Tue, 01 Jun 2010 20:03:00 GMT

The latest preview of Code First allows you to describe a model using C# or VB.Net classes. The basic shape of the model is detected by convention and then a fluent API can be used to further refine your model.

We recently posted about our plans to support Data Annotations as another way to further describe your model. We are now looking at extending and improving the conventions that initially infer the shape of the model. This post describes the conventions we are planning to include.

Conventions are designed to provide a starting point for a model. Data Annotations or the fluent API can then be used to further describe the model or change what was detected by convention. Precedence is given to configuration via the fluent API followed by Data Annotations then convention.

Primary Key

Previously Code First would infer that a property is a primary key if the property is called ‘Id’ or ‘<class name>Id’. The only change to this convention is that once primary key properties are detected if their type is ‘int’, ‘long’ or ‘short’, they are registered as identity columns in the database by default. Primary key detection is not case sensitive.

Relationship Inverses

When defining a relationship between two types it is common to include a navigation property on both types, such as in the following example:

public class Product

{

public int ProductId { get; set; }

public string Name { get; set; }

public Category Category { get; set; }

}

public class Category

{

public int CategoryId { get; set; }

public string Name { get; set; }

public ICollection<Product> Products { get; set; }

}

Previously Code First would create two separate relationships between Product and Category but will now infer that Product.Category and Category.Products represent different ends of the same relationship. Inverse detection will occur when both types involved in the relationship define one and only one navigation property (reference to or collection) to each other. If one of the types in a relationship defines two or more navigation properties that reference the other type then inverse detection will not occur and the relationships will need to be manually configured using Data Annotations or the fluent API.

Foreign Keys

Building on the previous convention it is also common to include a foreign key property on the dependent end of a relationship, in this case BookReview.SubjectISBN:

public class BookReview

{

public int Id { get; set; }

public Book Subject { get; set; }

public string SubjectISBN { get; set; }

}

public class Book

{

[Key]

public string ISBN { get; set; }

public string Name { get; set; }

public ICollection<BookReview> Reviews { get; set; }

}

Code First will now infer that any property named ‘<navigation property name><primary key property name>’ (i.e. SubjectISBN), ‘<principal class name><primary key property name>’ (i.e. BookISBN) or ‘<primary key property name>’ (i.e. ISBN), with the same data type as the primary key, represents a foreign key for the relationship. If multiple matches are found then precedence is given in the order listed above. Foreign key detection will not be case sensitive.

When a foreign key property is detected Code First will also infer the multiplicity of the relationship based on the nullability of the foreign key, if the property is nullable then the relationship is registered as optional, otherwise the relationship is registered as required, and cascade delete is turned on. The multiplicity and cascade delete behavior detected by convention can be overridden using the fluent API.

Type Discovery

Previously Code First would only include types that were declared in object sets on your derived context or manually registered through the fluent API. Given the following example Product would have been included in your model, but Category would not:

public class ProductContext : ObjectContext

{

public ProductContext(EntityConnection connection)

: base(connection)

{ }

public ObjectSet<Product> Products

{

get { return base.CreateObjectSet<Product>(); }

}

public class Product

{

public int Id { get; set; }

public string Name { get; set; }

public Category Category { get; set; }

}

public class Category

{

public int Id { get; set; }

public string Name { get; set; }

public ICollection<Product> Products { get; set; }

}

Code First will now recognize that Product has a property that references Category and automatically include Category in your model. Reachability is recursive, so if Category then referenced another unregistered type, this would also be included in the model. Reachability will also follow references to types defined in another assembly. The convention may include types that do not belong in the model; these can be removed using the StoreIgnore Data Annotation or via the fluent API as follows:

var builder = new ContextBuilder<ProductContext>();

builder.Ignore<Category>();

Complex Type Discovery

Building on the reachability convention, if Code First discovers a class definition where a primary key cannot be inferred, and no primary key is registered through Data Annotations or the fluent API, then the type will be automatically registered as a complex type. Complex type detection also requires that the type does not have properties that reference entity types and is not referenced from a collection property on another type. Given the following class definitions Code First would infer that Name is a complex type because it has no primary key:

public class Person

{

public int PersonId { get; set; }

public Name Name { get; set; }

}

public class Name

{

public string Title { get; set; }

public string FirstName { get; set; }

public string LastName { get; set; }

}

Summary

Code First will provide an extended set of default conventions to determine the shape of a model. These conventions can be overridden using Data Annotations which in turn can be overridden via the fluent API.

We’d like to hear any feedback you have on the set of conventions as well as the rules associated with each convention.

Rowan Miller
Program Manager
ADO.Net Entity Framework Team

Announcing the release of Entity Framework 4

dpblogs — Mon, 12 Apr 2010 16:17:55 GMT

The first release of Entity Framework shipped in .NET Framework 3.5 SP1 and Visual Studio 2008. Since then, we have been focused on incorporating your feedback and enabling the scenarios that you wanted us to work on.

The feedback we have received through the betas and release candidate has been instrumental in helping us to get to where we are today; so thank you very much for being a part of the development process.

Today, we are happy to announce the release of Entity Framework 4 as part of the release of Visual Studio 2010 and .NET Framework 4.

Entity Framework 4 builds on Entity Framework 3.5 in the following areas:

Development Approach / Design-time Improvements

Model-First Development: In addition to the Database-First approach you had in EF 3.5, Entity Data Model designer in Visual Studio 2010 includes Model-First capability to allow you to start from a model, and generate database based on that model. Read more about this feature here.

Customizable Code-Generation: EF4 leverages the T4 code generation templating engine in Visual Studio. You can now write your own templates that specify exactly how you want code generation to happen; or you can modify the built-in templates such as the Entity Object Code Generator or Self Tracking Entities Code Generator templates. Read more about this feature here. We have also shipped a POCO Template as a separate download available on the Visual Studio Extensions Gallery. Read more about this POCO Template here.

Design-time support for Complex Types (with refactoring): Entity Data Model Designer in VS 2010 lets you define complex types, view complex types in the model explorer tree and refactor existing properties into complex type.

Pluralization: There is now Singularization/Pluralization support such that reverse engineering existing databases will result in models that have more meaningful names for entity types, entity sets and relationships. Read more about this feature here.

Multi-Targeting Support: Entity Data Model Designer and Entity Framework fully support Multi-Targeting capabilities of VS 2010. You can use the designer to continue building your EF 3.5 applications or move forward to EF4. We take care of upgrading / downgrading you to the right EDMX versions and such so that you can use a single IDE to build apps using EF, regardless of the framework version you are targeting.

Architectural Concerns & Patterns

Persistence Ignorance & POCO: EF4 allows developers to use their pure domain classes without needing to implement interfaces that deal with persistence concerns. Dynamic proxies are supported for lazy loading and efficient change tracking. Read more about this feature here.

Self-Tracking Entities & N-Tier support: Visual Studio 2010 includes code-generation templates for Self Tracking Entities that allow you to easily build N-Tier applications. There are also new APIs in the product that offer you more control in N-Tier scenarios. Read more about this feature here.

Application Patterns and Testability: EF4 includes an additional interface to help you to write testable code when using the framework. In addition, we have made sure that use of patterns such as Repository and UnitOfWork are possible.

Data Access Features

Foreign Key Associations: EF4 includes a new type of association called Foreign Key Association. FK associations allow you to include FK properties in your model and use those as a basis for relationship between entities. Independent Associations from EF 3.5 are still supported but FK Associations will help simplify a few scenarios such as data binding. You can now include foreign keys in your model should you choose to do so. Read more about this feature here.

Lazy Loading: There is now support for automatic Lazy loading. All new projects created using the Entity Designer in Visual Studio 2010 will also have automatic lazy loading enabled by default. Upgraded projects will require this to be explicitly enabled, so that the behavior of your existing projects is preserved unless you explicitly choose to take advantage of this functionality.

More LINQ Operators in LINQ to Entities: More LINQ operators are supported by EF4 such as Contains, Single, SingleOrDefault and DefaultIfEmpty.

CreateDatabase and DDL Provider Services: EF4 includes APIs on ObjectContext that allow you to do database creation based on a model. Provider writers can implement DDL generation capability in their providers that can take advantage of this capability. Read more about this feature here.

Model Defined Functions LINQ support: Model Defined Functions allow you to define composable functions in your model using Entity SQL. You can also expose these functions such that they can be used from LINQ. Read more about this feature here.

EntityDataSource support for QueryExtender, POCO and FKs: EntityDataSource control now includes support for ASP.NET QueryExtender and POCO entities. QueryExtender is a new addition to ASP.NET, which allows you to have more control over the data retrieval query of a Data Source while leveraging LINQ capabilities of EF.

ObjectMaterialized event: Now you can write logic that is executed immediately after an object has been materialized.

Generated SQL improvements for better performance and readability: EF4 includes a lot of optimizations to the queries generated by EF such as removal of unnecessary joins, better translations of certain functions, removal of unneeded levels of nesting, and more. Read more about this here.

Ad-hoc native query support: EF4 supports more ad-hoc query patterns that allow you to run a store query or a command directly. Read more about this feature here.

There are many other smaller improvements we haven’t covered here. We hope that you find these features useful. Be sure to let us know what you think of EF4, and what you’d like to see us do next.

Entity Framework Team

Data Annotations in the Entity Framework and Code First

dpblogs — Wed, 31 Mar 2010 05:14:20 GMT

Data annotation attributes were introduced in .NET 3.5 as a way to add validation to classes used by ASP.NET applications. Since that time, RIA Services has begun using data annotations and they are now part of Silverlight. Code First allows you to build an EDM Entity Framework model using code (C# or VB.NET) and is now in its third CTP.

Feedback we’ve received from these CTPs shows demand for Code First to read data annotation attributes from class and property definitions to configure aspects such as maximum string length, which properties are keys, or the table name in which to store a type of entity.

This post outlines which existing data annotation attributes we’ll adopt and which new attributes we are proposing.

Using Data Annotations

When building a model using Code First, you start by writing a set of entity classes. For example:

public class Book
{
    public string ISBN { get; set; }
    public string Title { get; set; }
    public string AuthorSSN { get; set; }
    public Person Author { get; set; }
}

public class Person
{
    public string SSN { get; set; }
    public string Name { get; set; }
    public ICollection<Book> Books { get; set; }
}

When using this class model with Code First in CTP3, you would need to use the fluent API to describe which properties are keys, which relationships exist, and the length of each string property if you wanted these to be constrained. These are very common tasks and we’d like users of Code First to be able to make these kinds of configuration without having to use the fluent API to describe the model. Using data annotation attributes, the above model characteristics can be made declaratively as part of the class definition:

public class Book
{
    [Key]
    public string ISBN { get; set; }

    [StringLength(256)]
    public string Title { get; set; }

    public string AuthorSSN { get; set; }

    [RelatedTo(RelatedProperty=“Books”, Key=”AuthorSSN”, RelatedKey=”SSN”)]
    public Person Author { get; set; }
}

public class Person
{
    [Key]
    public string SSN { get; set; }

    [StringLength(512)]
    public string Name { get; set; }

    [RelatedTo(RelatedProperty=”Author”)]
    public ICollection<Book> Books { get; set; }
}

Existing Data Annotation Attributes

There are several existing data annotation attributes in the System.ComponentModel.DataAnnotations namespace that can be used by Code First. These are:

KeyAttribute

KeyAttribute is used to specify that a property/column is part of the primary key of the entity and applies to scalar properties only.

StringLengthAttribute

StringLengthAttribute is used to specify the maximum length of a string. As there is no minimum length in Code First or EF, the “MinimumLength” property on the attribute would be ignored. This attribute applies only to properties that are of type string.

In .NET 4.0, a negative MaximumValue is not allowed. but in the next version of the framework the recommendation is to change this behavior so that a value of ‘-1’ means “max”. This isn’t ideal, so another idea we are considering is to make a MaxStringLengthAttribute that derives from StringLengthAttribute. The MaxStringLengthAttribute wouldn’t take parameters, but would just assume the maximum allowable value.

ConcurrencyCheckAttribute

ConcurrencyCheckAttribute is used to specify that a property/column has a concurrency mode of “fixed” in the EDM model. A fixed concurrency mode means that this property is part of the concurrency check of the entity during save operations and applies to scalar properties only.

RequiredAttribute

RequiredAttribute is used to specify that a property/column is non-nullable and applies to scalar, complex, and navigation properties:

Scalar properties: means this property is non-nullable (even if the CLR type says it is nullable).
Complex properties: the attribute is not allowed here so we would throw an exception
Navigation properties
- Collection property: the attribute is not allowed here so we would throw an exception
- Reference property: this means the backing FK property is non-null, and that the end of the association is cardinality “1” (instead of 0..1).

TimestampAttribute

The TimestampAttribute is used to specify that a byte[] property/column has a concurrency mode of “fixed” in the model and that it should be treated as a timestamp column on the store model (non-nullable byte[] in the CLR type). This attribute applies to scalar properties of type byte[] only. and only one TimestampAttribute can be present on an entity as this is what is allowed on most database platforms.

DataMemberAttribute

The DataMemberAttribute, while not strictly part of the data annotation collection, can be used to help specify the ordinal of primary keys. This is important on entities where there is a composite primary key and you want to make an assertion about the order the keys are specified on the database or are used in your application. This attribute applies only to scalar properties only that are part of the key. Ordinals do not need to be consecutive, they only specify the relative order of the key properties in metadata.

Ideally we’d like to be able to specify the key order as part of the KeyAttribute via an “Ordinal” property, but we can’t make changes to KeyAttribute until the next release of the .NET framework.

New Data Annotation Attributes

While the existing data annotation attributes provide many useful options for defining your model, there are still several common cases where we’d like to be able to make model changes with annotation attributes. Our goal with data annotation support in Code First is not to provide another way to completely specify a model, but to provide a mechanism that addresses the most common kinds of configurations that developers need to influence their model.

This is an area where it would be great to get early feedback so we can incorporate your ideas into our data annotation plan - below are descriptions of the annotation attributes we are proposing for Code First.

RelatedToAttribute

The RelatedToAttribute is used for two things:

To assign bi-directionality to certain relationships where the built-in convention doesn’t detect it
To associate a navigation property with its foreign key(s)

The RelatedToAttribute is placed on navigation properties and can be present on zero, one, or both ends of a relationship. There are a few parameters on the RelatedToAttribute:

A property to optionally specify the RelatedProperty on the target type.
This is to support bi directionality in relationships (Book.Author is related to Person.Books, for example).
A property to optionally specify the keys on this entity that constrain the relationship.
This can be either the primary key(s) if the attribute is on a navigation property on the principal end of a relationship, or the foreign key(s) if the attribute is on a navigation property on the dependent end of a relationship.
A property to optionally specify the keys on the related entity that constrain the relationship.
This can be either the primary key(s) if the attribute is on a navigation property on the dependent end of a relationship, or the foreign key(s) if the attribute is on a navigation property on the principal end of a relationship.
A property to optionally specify cascade delete behavior.
When this property is set to true and the principal entity is deleted, the dependent end of the relationship will also be deleted.

In the example at the beginning of the blog post, the RelatedToAttribute was used to related the Book.Author and Person.Books navigation properties so that all of Book.AuthorSSN, Book.Author, and Person.Books represent the same relationship in the model.

We also considered using the existing AssociationAttribute to specify relationships. however this attribute assumes that there already is a backing model and requires a relationship name and required key properties which are not needed when describing your model with Code First.

LengthAttribute

LengthAttribute can be used to specify the maximum length of an array type, such as a byte[] and applies only to properties. A value of ‘-1’ means “max” and all other negative values are not allowed. There is overlap here with the StringLengthAttribute: when the LengthAttribute is specified on a property of type string, the semantics will be the same as StringLengthAttribute.

StoreGeneratedAttribute

The StoreGeneratedAttribute can be used to specify that a property’s value is generated on the database store and the only store generated pattern supported in the first version is “identity”. This attribute applies to properties only and has a single optional nullable boolean property Identity. A null value for Identity means that no decision should be made for Identity: the convention or an explicit API call will be made to turn the identity pattern on or off.

StoreNameAttribute

The StoreNameAttribute can be used to specify a name for a store/database item. This attribute has different meanings depending on the kind of thing this attribute applies to:

If the attribute is on a context class, it means “database name”.
If the attribute is on an entity class, it means “table name”.
If the attribute is on a property in an entity class, it means “column name”.

StoreIgnoreAttribute

StoreIgnoreAttribute can be used to specify that a CLR type or property should not be included when reading type or property metadata from a CLR type. This attribute can apply to classes, which means they should not be considered as an EntityType or ComplexType object but can also apply to properties, which means that property should not be part of the EntityType or ComplexType. We haven’t finalized the name of this attribute yet, so if you have suggestions, please let us know.

StoreInlineAttribute

StoreInlineAttribute can be used to specify that a CLR type should be considered as a ComplexType when reading type metadata from CLR types. Complex types can be thought of as a class without an identity/keys, or a logic grouping of properties (such as an Address or PhoneNumber) and applies only to class definitions. ComplexTypes are stored inline with the entity. We may choose not to include this attribute as the convention we are thinking of adding to Code First should detect most cases where a complex type is being used.

Other Possibilities

There may be other kinds of model definition that you’d like to be able to do using data annotation attributes. Some other candidates are:

Being able to specify whether a string is unicode or not
Being able to specify whether a type is variable or fixed length
Having data annotation attributes put CSDL annotations in your model that other systems (such as OData) can read. An example of this is the RangeAttribute.

Let us know if any of these options sound interesting or if there is something else you’d like us to consider.

Summary

Code First will offer a few different ways of configuring a model: purely by built-in conventions, using data annotation attributes to make certain kinds of model changes, or by using a fluent API to fully customize the model. To provide more flexibility over the existing data annotation attributes, we are adding new attributes that Code First can use.

We’d like to hear your feedback on the set of attributes that are available and how Code First will use them.

Jeff Derstadt
Entity Framework Team

POCO Template Code Generation Options

dpblogs — Thu, 11 Mar 2010 02:23:00 GMT

We recently released an updated version of the POCO template that works with Visual Studio 2010 RC on the Visual Studio Gallery. In this post we will delve into the various possible code generation options for future releases of the POCO template. We hope that you will read the various options presented here and give us feedback about the most compelling features for you.

1. Template Versus Generate Once experience

Template Experience (current design)

This experience is the same as what we currently have, where Text transform (.tt) files are added to the project either from the EDM Designer surface or the Solution Explorer. The generated code files are nested under the .tt files and are added to the project. This is useful in the following scenarios –

· Your model is evolving, so you expect to regenerate your entities often

· You have patterns of changes you want to make to your entities. This includes additional code you want included in each of your entities or modifying parts of the auto-generated code.

An example of a patterned change would be to introduce a data value constraint check in the property setters of certain data types. More information about customizing T4 templates can be found in this blog post.

Generate once experience

This user experience would mean that you could generate entity classes from the ADO.NET Entity Data Model (.edmx file) by right clicking on it from the Solution Explorer and selecting “Generate Entities”. Instead of template files being added to your project, the resulting code files for the entities would be added to your project. In future if the model changes, the entity classes will not be regenerated. These are the scenarios where this experience would be useful –

· You are working with a legacy database and don’t plan to evolve your model frequently. In this case it is only required to generate the entities once.

· After generating once, you expect to customize your entities with business logic or code that is specific to each entity type, and not a pattern of changes.

In these scenarios generate once is a seamless, VS integrated way of creating entity classes. The code generation would still use T4 templates behind the scenes. We can potentially support additional features in this scenario like customizing the generated code by choosing which features you want (details follow in the next sections). We would take your input through a VS dialog and feed it to the templates to generate the code your application requires.

2. Full featured versus Basic POCOs

These are the features that can be added to the basic POCO entities -

Proxies – For change tracking and lazy loading
Fixup logic in entities – Ensures that the object graph remains connected even when EF is not present
WCF Serialization support – Adds DataContractAttribute, DataMemberAttribute and KnownTypeAttributes to the entities and its members
Data binding support – Entities implement INotifyPropertyChanged interface
Data annotations based on model facets – Adds data validation by specifying attributes based on model facets like StringLengthAttribute (MaxLength facet), RequiredAttribute (Nullable facet), KeyAttribute(Key element), etc.

Proxies

Proxies add functionality to the basic POCO entities while maintaining the simplicity of POCO classes. They are dynamically generated entity types which derive from the basic POCO entity types and notify the object state manager about changes to entities. Thus they obviate the need to maintain snapshots of the original property values and the snapshot comparisons on SaveChanges. They also enable lazy loading which enables automatic loading of the related entity/entities when a navigation property is dereferenced. More details on POCO proxies can be found here.

Fixup

A fixup method is written for every navigation property on an entity and is called from the setter of the navigation property whenever its value changes. Its purpose is to ensure that each end of a bidirectional relationship stays in sync with the other. For example, in a one-to-many relationship between Cutomer and Order, whenever Order.Customer is set, the fixup method ensures that the Order is in the Customer’s Orders collection. It also keeps the corresponding foreign key property viz. Order.CustomerID in sync with the new Customer’s primary key (ID) value. This logic can be useful if the POCO entities are used independently of the EF stack, like for writing tests against them which don’t hit the database. Fixup ensures that the object graph is connected in the same way as you would expect while using them with EF. Fixup methods are a bit complex to write and hence it is useful to have them auto-generated if you are planning on using the entities in an EF independent scenario.

We will explore five interesting combinations of these features -

1. Basic POCO without Fixup - In this case the POCO classes would be very simple as both fixup logic and proxies are removed. Cons of this approach are - (a) it uses snapshot mechanism to infer the UPDATE commands sent to the database server. It cannot perform notification-based change tracking which provides performance improvement in some scenarios. (b) Writing EF-independent tests can be inconvenient since fixup logic is not included and the object graph may be disconnected (or inconsistent) until DetectChanges is called.

public class Order
{
    public int OrderID { get; set; }
    public string CustomerID { get; set; }
    // Navigation Properties
    public Customer Customer { get; set; }
}

2. Basic POCO with Fixup – Similar to (1) proxies are removed. However fixup logic is added to the entities so that the navigation properties and foreign keys are kept in sync even when EF is not present. Like (1) snapshot mechanism is used to detect property and relationship changes and performance could be improved in some scenarios by using proxies. The main disadvantage of this option as compared to (1) is that the code is more complex due to presence of fixup logic.

public class Order
{
    public int OrderID { get; set; }

    public string CustomerID
    {
        get { return _customerID; }
        set
        {
            try
            {
                _settingFK = true;
                if (_customerID != value)
                {
                    if (Customer != null 
                        && Customer.CustomerID != value)
                    {
                        Customer = null;
                    }
                    _customerID = value;
                }
            }
            finally
            {
                _settingFK = false;
            }
        }
    }
    private string _customerID;
    
    // Navigation Properties
    public Customer Customer
    {
        get { return _customer; }
        set
        {
            if (!ReferenceEquals(_customer, value))
            {
                var previousValue = _customer;
                _customer = value;
                FixupCustomer(previousValue);
            }
        }
    }
    private Customer _customer;

    // Association Fixup
    private bool _settingFK = false;
    private void FixupCustomer(Customer previousValue)
    {
        if (previousValue != null 
            && previousValue.Orders.Contains(this))
        {
            previousValue.Orders.Remove(this);
        }

        if (Customer != null)
        {
            if (!Customer.Orders.Contains(this))
            {
                Customer.Orders.Add(this);
            }
            if (CustomerID != Customer.CustomerID)
            {
                CustomerID = Customer.CustomerID;
            }
        }
        else if (!_settingFK)
        {
            CustomerID = null;
        }
    }
}

3. Change Tracking Proxies without Fixup – This marks all the properties of the entities as “virtual” which are then internally overridden by the EF to inform the state manager about changes to the entity. The change tracking POCO entities are capable of doing their own fixup, so that code can be removed in this version. However you would have fixup if the entities are used with EF and not have it if they are used independently of EF like for testing them without hitting the database. Thus they would exhibit different behaviors depending on the scenario. Pros are of this approach are the performance improvement on SaveChanges and the simplicity of the code. There are no known issues with this combination, except for serialization.

public class Order
{
    public virtual int OrderID { get; set; }
    public virtual string CustomerID { get; set; }
    // Navigation Properties
    public virtual Customer Customer { get; set; }
}

4. Change Tracking Proxies with Fixup – This has the advantage of performance improvement on SaveChanges. Bidirectional navigation properties and foreign keys are kept in sync even when EF is not present. Cons of this combination are that (a) it increases the complexity of the POCO entities due to presence of the fixup logic. (b) We used this combination for the Visual Studio 2010 RC POCO templates release and noted a few issues as documented in the post. We are working on fixing those issues.

public partial class Order
{
    public virtual int OrderID { get; set; }

    public virtual string CustomerID
    {
        get { return _customerID; }
        set
        {
            try
            {
                _settingFK = true;
                if (_customerID != value)
                {
                    if (Customer != null 
                        && Customer.CustomerID != value)
                    {
                        Customer = null;
                    }
                    _customerID = value;
                }
            }
            finally
            {
                _settingFK = false;
            }
        }
    }
    private string _customerID;
    
    // Navigation Properties
    public virtual Customer Customer
    {
        get { return _customer; }
        set
        {
            if (!ReferenceEquals(_customer, value))
            {
                var previousValue = _customer;
                _customer = value;
                FixupCustomer(previousValue);
            }
        }
    }
    private Customer _customer;

    // Association Fixup
    private bool _settingFK = false;
    private void FixupCustomer(Customer previousValue)
    {
        if (previousValue != null 
            && previousValue.Orders.Contains(this))
        {
            previousValue.Orders.Remove(this);
        }

        if (Customer != null)
        {
            if (!Customer.Orders.Contains(this))
            {
                Customer.Orders.Add(this);
            }
            if (CustomerID != Customer.CustomerID)
            {
                CustomerID = Customer.CustomerID;
            }
        }
        else if (!_settingFK)
        {
            CustomerID = null;
        }
    }
}

5. Lazy Loading Proxies with Fixup – Here you get lazy loading functionality for your entities by marking only the navigation properties as virtual. Pros are that it provides automatic loading of related entities when the navigation properties are dereferenced. Bidirectional navigation properties and foreign keys are kept in sync even when EF is not present. There are no known issues for this combination. Cons are that (a) performance could be improved in some scenarios by using notification-based mechanism for SaveChanges. (b) Also code is complex due to presence of fixup logic.

public partial class Order
{
    public int OrderID { get; set; }

    public string CustomerID
    {
        get { return _customerID; }
        set
        {
            try
            {
                _settingFK = true;
                if (_customerID != value)
                {
                    if (Customer != null 
                        && Customer.CustomerID != value)
                    {
                        Customer = null;
                    }
                    _customerID = value;
                }
            }
            finally
            {
                _settingFK = false;
            }
        }
    }
    private string _customerID;
    
    // Navigation Properties
    public virtual Customer Customer
    {
        get { return _customer; }
        set
        {
            if (!ReferenceEquals(_customer, value))
            {
                var previousValue = _customer;
                _customer = value;
                FixupCustomer(previousValue);
            }
        }
    }
    private Customer _customer;

    // Association Fixup
    private bool _settingFK = false;
    private void FixupCustomer(Customer previousValue)
    {
        if (previousValue != null 
            && previousValue.Orders.Contains(this))
        {
            previousValue.Orders.Remove(this);
        }

        if (Customer != null)
        {
            if (!Customer.Orders.Contains(this))
            {
                Customer.Orders.Add(this);
            }
            if (CustomerID != Customer.CustomerID)
            {
                CustomerID = Customer.CustomerID;
            }
        }
        else if (!_settingFK)
        {
            CustomerID = null;
        }
    }
}

Data Binding, WCF Serialization and Data Annotations

We could potentially add data binding support, WCF serialization support and data annotations based on model facets to the generated POCO classes.

3. Fake ObjectContext

POCO entities offer independence from the EF stack so that the logic in the entities can be tested without hitting the database. This offers advantages like the tests run faster and the database doesn’t have to be reset to its initial state after each test run. However to help write a completely EF independent test, you would also need a fake repository or a fake ObjectContext. Currently the typed ObjectContext generated from the POCO template derives from ObjectContext which is an EF class and thus is dependent on the Entity Framework. More information on using a fake ObjectContext for testability can be found in Jonathan’s post here. Let us see what generating fakes would mean for an application containing a typed ObjectContext named BloggingEntities,

1. It would create the interface IBloggingEntities. The interface basically contains property getters for all the IObjectSets contained in this ObjectContext and the SaveChanges method. The interface for ObjectSet<T>, IObjectSet<T> is a public interface in EF.

2. It would generate the classes FakeBloggingEntities and FakeObjectSet<T> which implement the interfaces IBloggingEntities and IObjectSet<T>. FakeObjectSet<T> uses an in-memory data structure like HashSet<T> to store the entities. The properties on FakeBloggingEntities return FakeObjectSets instead of real ObjectSets.

3. The typed ObjectContext, BloggingEntities would derive from ObjectContext as well as IBloggingEntities and would contain the additional property members defined by IBloggingEntities.

4. It could generate a repository class, BlogRepository, which serves as a thin mapper between the entities and the store. The store would be represented by a member of type IBloggingEntities. So you could substitute the store with an in-memory data structure that implements IBloggingEntities, like FakeBloggingEntities. The repository class would serve as a starting point that you could customize as needed.

The tests would run against the repository, BlogRepository which would work on fake ObjectContext and fake ObjectSets and thus not hit the database. The full implementation of these classes can be found in the same post.

Summary

In this post we have explored pros and cons of the various code generation options for POCO template like generate-once and template experience, full featured versus basic POCO classes and creation of fake ObjectContext for testability of your application. We would love to hear your feedback about which options are the most compelling for you and if there is anything else you believe should be added to the POCO template.

Thanks,
Sheetal Gupta
Developer
Entity Framework Team

Code Only – Further Enhancements

efdesign — Mon, 12 Oct 2009 17:31:00 GMT

We’ve come a long way since the last post on Code-Only. So it’s high time for another update.

We’ve been working really hard on Code-Only revving the design, and spotting missing capabilities and responding to feedback both internal and external etc.

The current plan still holds. Code-Only will not be in .Net 4.0 with the possible exception of the DDL features described in the last post. For that portion, the implementation and requirements are more clear to us, and because making the DDL stuff work requires changes to things already in the .NET framework, and getting provider writers lined up, we are still working hard to get the DDL changes into .NET 4.0. The rest of Code-Only will continue to evolve, and we will ship another preview or two, before getting it rolled into .NET as soon as we can after 4.0 ships.

API Refactoring

As hinted above, we’ve spent a lot of time validating code only against real world scenarios, and thinking about how the customer code hangs together etc. As a results we’ve done some API refactoring.

Mappings are now part of Configurations

In the last version Mappings were derived from Configurations, which had some strange side-effects on the API.

We’ve re-arranged things now so that Configurations hold mappings, which are terminated by assignment to a table.

builder.Entity<Customer>()
   .HasKey(c => c.ID)
   .MapSingleType(c => new {
         cid = c.ID,
         nme = c.Name
      }
   )
   .ToTable(“dbo.custs”);

This maps the Customer entity to the ‘dbo.Custs’ table, the ID property to the cid column, the Name property to the nme column, and registers the ID property as the EntityKey / Primary Key.

As you can see we’ve also added a helper method called Entity<TEntity>() so that you can fluently create configurations and mappings. You might also have noticed that we have removed RegisterKey and replaced it with HasKey which we feel is more inline with our goal of having an intentional API.

TPH mapping syntax no longer violates DRY

The previous syntax for specifying a TPH mapping forced you to repeat mappings for columns that could easily have been inherited.

This was unfortunate because we normally encourage TPH because generally it has the best performance characteristics.

This is where MapHierarchy() and Case() come in.

Imagine if we want to write these classes using TPH:

public class Person{
   public virtual int ID {get;set;}
   public virtual string Firstname {get;set;}
   public virtual string Surname {get;set;}
}
public class Employee: Person
{
   public virtual Employee Manager {get;set;}
   public virtual List<Employee> Reports {get;set;}
}

You would now do it like this:

builder
   .Entity<Person>()
   .HasKey(p => p.ID)
   .MapHierarchy()
   .Case<Person>(
      p => new {
         p.ID,
         p.Firstname,
         p.Surname,
         thisIsADiscriminator = “P”
      }
   )
   .Case<Employee>(
       e => new {
         manager = e.Manager.Id
         thisIsADiscriminator = “E”
       }
   )
   .ToTable(“dbo.People”);

By using Case<> mappings are re-used for derived type, so there is no need to repeat column mappings for ID, Firstname and Surname.

Which means the Employee Case statement is only responsible for mapping properties and references declared by the Employee class, and for specifying a new discriminator value.

The discriminator column and values are there so the Entity Framework can distinguish between rows in the dbo.People table that represent Person objects and those that represent Employee objects.

MapSingleType and MapHierarchy

Because Mappings are now part of the configuration, there is no need to create them independently and assign them.

You now simply MapXXX() methods on the EntityConfiguration.

The available mapping methods are:

MapSingleType(λ)
MapHierarchy(λ)
MapHierarchy().Case<TEntity>(λ).Case<TDerived>(λ).

If you know the Entity Frameworks MSL files these correspond to:

Type
OfType
OfType with an addition requirement for a type discriminator

These mappings can be combined in many interesting ways, but in general this table shows the recommended way to do most common mappings:

Scenario	Code	Notes
No Inheritance	builder.Entity<A>() .MapSingleType(λ) .ToTable(“dbo.A”);	Columns not mapped are not part of the Entity.
TPH	builder.Entity<A>() .MapHierarchy() .Case<A>(λ) .Case<B>(λ) .Case<C>(λ) .ToTable(“dbo.ABC”);	Generally each Case expression only maps properties declared by the current type and the discriminator for that type. But it is possible to override the mapping for inherited properties if required.
TPT	builder.Entity<A>() .MapHierarchy(λ) .ToTable(“dbo.A”); builder.Entity<B>() .MapHierarchy(λ) .ToTable(“dbo.B”); builder.Entity<C>() .MapHierarchy(λ) .ToTable(“dbo.C”);	Each MapHierarchy expression only maps properties declared by the current type, and properties that make up the entity key.
TPC	builder.Entity<A>() .MapSingleType(λ) .ToTable(“dbo.A”); builder.Entity<B>() .MapSingleType(λ) .ToTable(“dbo.B”); builder.Entity<C>() .MapSingleType(λ) .ToTable(“dbo.C”);	Each MapSingleType expression maps all properties, both those declared by the current type and those inherited.

Foreign Keys

In Beta2 of .NET 4.0 we added FK Association Support, so Code-Only needs a way to link a FK property and an Navigation Property together.

Given this class:

public class Product{
   public virtual int ID {get;set;}
   public virtual string Name {get;set;}
   public virtual Category Category {get;set;}
   public virtual int CategoryID {get;set;}
}

You need to be able to tell code-only that Category.ID and CategoryID should have the same value.

You do it like this:

builder.Entity<Product>()
.Relationship(p => p.Category)
.HasConstraint((p,c) => p.CategoryID == c.ID);

This says that c.ID (i.e. p.Category.ID) must equal p.CategoryID, which tells code-only that p.CategoryID is an FK property and p.Category is a navigation property backed by this FK property.

HasConstraint(λ) can also be used in conjunction with Relationship(λ).FromProperty(λ) like this:

builder.Entity<Product>()
   .Relationship(p => p.Category)
   .FromProperty(c => c.Products)
   .HasConstraint((p,c) => p.CategoryID == c.ID);

Which tells code only that Product.Category and Category.Products are inverses and that Product.CategoryID and Product.Category.ID must be the same, which implies Product.CategoryID is an FK property.

Missing Navigation Properties

Sometimes you don’t have a navigation property or FK property in the Entity which would naturally hold the FK in the database. For example imagine this scenario:

public class Product{
public virtual int ID {get;set;}
public virtual string Name {get;set;}
}

public class Category{
   public virtual int ID {get;set;}
   public virtual string Name {get;set;}
   public virtual List<Product> Products {get;set;}
}

Here there is ‘probably’ a one to many relationship between Categories and Products, and that is probably best modeled using an FK in the products table.

If you start mapping the Product entity you need a way to map the FK column, but there is no reference (i.e. Product.Category) and no FK property (i.e. Product.CategoryID) to map.

So we added the ability to create a fake navigation property to help out:

builder.Entity<Product>().MapSingleType(
   p => new {
       pid = p.ID,
       nme = p.Name
       cid = EntityMap.Related<Category>(c => c.Products).ID
   }
).ToTable(“dbo.Products”);

Here the interesting part is EntityMap.Related, it creates a fake navigation property, just so you can use it in the mapping.

The method signature looks like this:

public static TEntity Related<TEntity>(
Expression<Func<TEntity, object>
);

Notice that this function simply returns TEntity, so in the above mapping fragment, we are mapping the ID property of the Category class to the cid column, in the ‘dbo.Products’ table.

Complex Types

Code-Only’s default behavior is to ignore properties that are not recognized as either a primitive type or an EntityType. So to support Complex Types we need a mechanism to register one, like this:

var addressConf = builder.ComplexType<Address>();

This returns a ComplexTypeConfiguration through which you can configure the properties of the ComplexType in the same way you configure the properties of an Entity:

addressConf.Property(a => a.Street).HasMaxLength(100);
addressConf.Property(a => a.Zip).HasMaxLength(10);

Now if an entity is mapped by convention and it has a ComplexType property we will automatically introduce columns for each property of the ComplexType.

If you want to explicitly map the Entity mapping the ComplexType property is pretty simple too:

builder.Entity<Person>().MapSingleType(
   p => new {
      p.ID,
      fn = p.Firstname,
      sn = p.Surname,
      street = p.Address.Street,
      zip = p.Address.Zip
   }
);

Registering Entity Sets

Some of our early CodeOnly adopters wanted to use Code Only without having an specialized ObjectContext, and in theory with is possible:

builder = contextBuilder.Create<ObjectContext>();
builder.Entity<Person>().HasKey(p => p.ID);

But what we found is you have no control over the name of the set generated. In this case we will generate an EntitySet called PersonSet, but what if I the name should be People?

To address this issues we have added this method:

builder.RegisterSet<Person>(“People”);

Not only is this useful for specifying names, it also allows you to intentional specify what EntitySets you need. The alternative would be to encode your intent into calls to Entity<TEntity> and Relationship<TRelated>(..).

Association Mapping

We also need to add support for mapping associations that are not co-located with the Entity. For example take the relationship between Jobs and Candidates. If a candidate can apply for multiple jobs this is a classic example of a many to many relationship.

If you want to take control of how this relationship is mapped you need a way to specify the mapping for the relationship, like this:

builder
.Entity<Job>()
.Relationship(job => job.Applicants)
.FromProperty(candidate => candidate.Applications)
.Map(“dbo.JobApplications”,
           (job,candidate) => new {
               applicantId = candidate.Id,
               jobId = job.Id
            }
       );

This does a number of things, first it indicates that Job.Candidates and Candidate.Applications are opposite ends of the same relationship.

Then the Map() call indicates that the relationship should be stored in the “dbo.JobApplications” join table, with a compound PK made up of the applicantId and jobId columns.

The applicantId holds the candidate.Id, and the jobId holds the job.Id, and in both cases Code Only will emit an FK constraint to ensure referential integrity.

Interesting Column Names

The existing mapping syntax that uses anonymous types for the table and column definitions, limits you to column names that are valid CLR identifiers.

So for example if you need to map to a column with a space in the name you can’t.

To rectify this we added an alternate mapping syntax:

builder.Entity<Product>().MapSingleType(
   p => EntityMap.Row(
      EntityMap.Column(p.ID, “p i d”),
      EntityMap.Column(p.Name),
      EntityMap.Column(p.CategoryId, “c i d”)
   )
).ToTable(“dbo.Products”);

This snippet of code, maps the Product entity to the “dbo.Products” table and the ID property to a ‘p i d’ column, the Name property to a Name column and the CategoryId property to the ‘c i d’ column.

This alternate API has another advantage too: it makes it easier to create mapping expressions programmatically, primarily because you no longer need to create an anonymous type when building the expression. Which is really useful if you want to configure code-only at runtime.

Extracting the EDMX

We’ve also added the ability to get the EDMX that Code Only is producing internally, either via an XmlWriter or as an XDocument:

DbConnection connection = // some code to get a connection.
XDocument document = builder.GetEdmx(connection);

// or using XmlWriter
var swriter = new StringWriter();
var xwriter = new XmlTextWriter(swriter);
builder.WriteEdmx(connection, xwriter);

This is pretty handy if you want to use Code Only to build your model and mappings but then pull them out so you work with the XML and our designer tooling.

Setting the StoreType

When producing the Storage Model Code Only asks the current database provider to return an appropriate store type for the CLR type and facets specified.

But some CLR types and facet combinations can map to multiple possible database types. For example fixed length byte[] can map to both binary and timestamp. So in this situation SqlClient returns binary by default.

But you might need a timestamp:

builder
   .Entity<Order>()
   .Property(o => o.Version)
   .HasStoreType(“timestamp”)
   .IsComputed();

This tells Code Only that the Version property of Order (which is a byte[]) is computed in the database, and is actually a ‘timestamp’ column, which is computed in the database after every insert / update.

DDL Provider Model

In our last post we covered our plans around the DDL provider model, so check that out.

Summary

As you can see Code Only is now looking much more complete.

But it isn’t completely finished yet, we are still working on the rough edges, so as always we are very interested to get your feedback.

In particular are there things you think we should add / rename / simplify?

Alex James
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Extending the Entity Framework Provider Model to support DDL

efdesign — Tue, 06 Oct 2009 21:24:18 GMT

As part of the first previews of Code-Only we shared some code to create a database:

// Create a context using code-only
using (var mycontext = builder.Create(dbConnection))
{
    // Create the database if it doesn’t already exist
    if (!myContext.DatabaseExists())
        myContext.CreateDatabase();
    // Standard EF code goes here.
}

But in the first preview of Code-Only this code only worked against SQL Server.

The problem is that our public Provider Model (i.e. DbProviderServices) has no Database Definition Language or DDL features.

Provider Model Changes

To support DDL features across different databases, we plan to extend DbProviderServices, with DDL specific services.

These services will be accessed through these public methods:

public string CreateDatabaseScript(
string providerManifestToken,
StoreItemCollection storeItemCollection);

public void CreateDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

public bool DatabaseExists(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

public void DeleteDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

Now internally those methods will call through to the following ‘protected virtual’ methods which will do the actual work:

protected virtual string DbCreateDatabaseScript(
string providerManifestToken,
StoreItemCollection storeItemCollection);

protected virtual void DbCreateDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

protected virtual bool DbDatabaseExists(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

protected virtual void DbDeleteDatabase(
   DbConnection connection,
   int? commandTimeout,
   StoreItemCollection storeItemCollection);

And the base implementations of these methods in DbProviderServices will simply throw ProviderIncompatibleExceptions.

Which means the provider writers job will be to override these ‘protected virtual’ methods with an implementation that makes sense for their backend database.

The key is to understand that the StoreItemCollection (aka the SSDL or StorageModel part of the EDMX) represents the intended shape of the database.

This means the provider writer will need to iterate over the EntitySets (tables) and the corresponding EntityTypes (table structures) in the StoreItemCollection and create / drop / script the database and tables as required.

Provider writers will be expected to override these functions so that:

DbCreateDatabaseScript: creates a native text command to create the tables and foreign key constraints defined in the StoreItemCollection. I.e. for SqlClient this would be the contents of a .sql DDL file.
DbCreateDatabase: is similar to DbCreateDatabaseScript except it should actually goes ahead and create the database, tables and foreign key constraints.
DbDatabaseExists: checks to see if the database exists. The SqlClient provider will simply check that the database itself exists, but custom provider writers could get more fancy and check to see if every table / foreign key constraint is found too.
DbDeleteDatabase: should go ahead and delete the database, or if the database server has a single database model (like Oracle) the provider writer should delete just the tables defined in the StoreItemCollection.

Simplifying Wrapping Providers

We are also planning something to simplify writing Wrapping Providers. A wrapping provider is just a provider that wraps an existing provider (i.e. SqlClient) and adds additional services (i.e. Auditing, Logging, Caching etc).

Jarek has some some sample wrapping providers if you are interested.

Today writing a wrapping provider is a little tricky, in fact one ‘protected’ method is impossible to wrap without reflection. So to help we plan to add one public wrapper method:

public DbCommandDefinition CreateCommandDefinition(
DbProviderManifest providerManifest,
DbCommandTree commandTree);

One of the reasons we plan on doing this, is we think people might take a ‘basic’ provider that has no DDL support and wrap it to add DDL support.

End User API

Now so far we’ve been looking at the planned extensions to Provider Services, but Provider Services is a very low level API that few developers will ever program against.

Most people will work directly against the ObjectContext, to which we plan to add these methods:

public void CreateDatabase()

public void DeleteDatabase();

public bool DatabaseExists();

public String CreateDatabaseScript();

This little snippet shows how easy it will be to script, create, check and delete a database:

MyContext ctx = new MyContext();
String sql = ctx.CreateDatabaseScript();
ctx.CreateDatabase();
Assert.True(ctx.DatabaseExists());
ctx.DeleteDatabase();
Assert.False(ctx.DatabaseExists());

As you it could hardly be easier to use.

Summary:

While Code-Only provides the catalyst to add DDL support to the Entity Framework’s Provider model, this feature is about more than just Code-Only.

In fact we think this feature will add significantly to the usability of the Entity Framework.

But as always we are keen to hear what you think.

LINQ to SQL to Entity Framework conversion template

efdesign — Fri, 14 Aug 2009 00:10:00 GMT

Many customers have asked us how to port a LINQ to SQL application to the Entity Framework. In order to make this process easier, we have developed a metadata conversion template.

This template converts LINQ to SQL metadata (.dbml) to metadata compatible with Entity Framework (.edmx). While not a complete solution to the conversion problem, it is a useful first step. The template available for download is still in the development stage, and any feedback you have on the user experience, additional features, or any other aspect of the conversion would be greatly appreciated.

The T4 template is intended to convert simple valid .dbml files to .edmx files. All of the basic elements of the .dbml file are mapped to their .edmx counterparts. For instance, Tables are mapped to EntitySets and Types are mapped to EntityTypes. Additionally, stored procedures (including customized insert, update and delete stored procedures) and associations are also converted.

Downloading the Metadata Conversion Template

Included in the zip file is a .vsix installer for the T4 Conversion Template. Also included is detailed documentation for the template. VS 2010 Beta 1 is required.

Download the Template

Walkthroughs

Two walkthroughs have been written to help guide users through the conversion process. The walkthroughs both take the form of unit tests which can be run against the existing LINQ to SQL metadata and the Entity Framework metadata produced by the conversion template. Each walkthrough contains detailed instructions on how to convert the existing LINQ to SQL project to Entity Framework using the Conversion Template. The first is the Widget Factory walkthrough, which shows a basic conversion. The second (CustomCUD) shows how to convert a LINQ to SQL project which uses custom insert, update and delete stored procedures. The walkthroughs are available individually.

Widget Factory Walkthrough

Custom CUD Walkthrough

Caveats and Known Issues

Because the Conversion Template is still under development, there are a number of open issues and unimplemented features. A more comprehensive list can be found in the documentation included with the template installer.

· Inheritance: Inheritance has not been fully implemented. While the template may work in some basic cases, this aspect of the conversion remains an open issue.

· Independent Associations: Foreign-key associations will not be available until .NET 4.0 Beta 2, so the current version of the template will only work with the "UseIndependentAssociations" flag set to true.

· Property Attributes: not all of the property attributes (such as "Unicode" and "FixedLength") are converted because these are not explicitly present in the .dbml file.

Feedback

We are interested to hear any comments you have about known issues, additional feature requests, suggestions for additional walkthroughs or any other aspect of the conversion process.

Thanks,
Christina Ilvento
Developer,
Entity Framework Team

Code Only Enhancements

efdesign — Tue, 04 Aug 2009 01:11:00 GMT

We've been working hard on Code Only since the first preview.

In the next release you will be able to specify:

Navigation Property Inverses.
Property Facets, like Nullability, MaxLength, Precision etc.
Property to Column mappings
Type to Table(s) mappings
Inheritance strategy
Encapsulate configuration

The rest of this post will drill into each of these features in turn.

Registering NavigationProperty inverses:

You can now Register Inverses, i.e. link one navigation property to another, like this:

builder.RegisterInverse(
(Customer c) => c.Orders,
(Order o) => o.Customer)
);

This code indicates that Customer.Orders is the other end of the Order.Customer relationship. Adding order1 to the customer1.Orders collection, has the same effect as setting the order1.Customer to customer1.

Specifying Property Facets:

You can also specify property facets, i.e. things like Nullability, MaxLength, Precision etc, like this:

var customerConfig = new EntityConfiguration<Customer>();
// We can infer that the ID is the Primary Key,
// but not that it is generated in the database on insert.
customerConfig.ForProperty(c => c.ID)
              .Identity();
customerConfig.ForProperty(c => c.Name)
              .MaxLength(100)
              .NonUnicode();
customerConfig.ForProperty(c => c.Website)
              .MaxLength(200)
              .Nullable()

builder.Configure(customerConfig);

This configures the Customer type so that:

The ID property is an Identity column, i.e. the value is computed by the database when we do an insert.
The Name property has a MaxLength of 100 characters and is NonUnicode i.e. in SqlServer VARCHAR rather than NVARCHAR.
The Website property has a MaxLength of 200 characters and is Nullable.

These facets target the Conceptual Model (i.e. CSDL), and from there flow to the database too (i.e. SSDL).

Encapsulating Facet Configuration

You can create a class to encapsulate all this configuration by deriving from EntityConfiguration<T>.

For example:

public class CustomerConfig: EntityConfiguration<Customer>
{
    public CustomerConfig(){
       ForProperty(c => c.ID)
                 .Identity();
       ForProperty(c => c.Name)
                 .MaxLength(100)
                 .NonUnicode();
       ForProperty(c => c.Website)
                 .MaxLenght(200)
                 .Nullable();
    }
}

We recommend creating classes like this instead of configuring an EntityConfiguration<> because of the encapsulation benefits.

Specifying the Tablename

When you use Configure<T>(..) the EF infers a default mapping, inheritance strategy (TPH) and table name(s) for you.

But if you want to specify the table name you can do this:

var customerConfig = new EntityConfiguration<Customer>();
// configure the facets, as per the above example
...

// register configuration with a particular tablename
builder.Tables[“dbo.Custs”] = customerConfig;

Specifying Mappings:

If you need more control over the mappings (for example to map to an existing database or use corporate naming policies) then you can specify mappings like this:

EntityMap<Customer> customerMap =
    Map.OfType<Customer>(
        c => new {
            cid = c.ID,
            c.Name,
            csite = c.Website
        }
    );

Interpreting a Mapping

This mapping states that the ID property is mapped to the ‘cid’ column, the Name property is stored in the ‘Name’ column and the Website property is mapped to the ‘csite’ column.

Properties not referenced are not persisted (just like properties in a partial class when using the default code generation of EF)

LINQ Comprehension Syntax

You can even specify exactly the same thing using a LINQ comprehension syntax too:

EntityMap<Customer> customerMap =
    from c in Map.OfType<Customer>()
    select new {
            cid = c.ID,
            c.Name,
            csite = c.Website
        };

Specifying Facets with Mapping

Once you’ve configured the mapping you can also specify facets on the map like this:

customerMap.ForProperty(c => c.ID)
           .Identity();
customerMap.ForProperty(c => c.Name)
           .MaxLength(100)
           .NonUnicode();
customerMap.ForProperty(c => c.Website)
           .MaxLenght(200)
           .Nullable();

Specifying the table

The final step is to assign the map to a table.

builder.Tables[“dbo.Custs”] = customerMap;

Now we’ve specified a custom table, mapping and custom facets for our Customer class.

Specifying Inheritance:

The default inheritance strategy used by CodeOnly is Table Per Hierarchy (or TPH).

If however you need a different strategy you need to dive in and configure the mappings.

Imagine if you have three classes you need to map: Vehicle , Car and Boat where both Car and Boat are derived from Vehicle which is abstract.

Table Per Hierarchy (TPH)

If you want to map this using TPH, you could do it like this:

var vehicleMap =
    Map.OfTypeOnly<Vehicle>(
        v => new {
            vid = v.ID,
            v.Name,
            vdesc = v.Description
            v.MaxPassengers,
        }
    ).Union(Map.OfTypeOnly<Car>(
        c => new {
            vid = c.ID,
            c.Name,
            vdesc = c.Description
            c.MaxPassengers,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinder,
            discriminator = “CAR”
        })
    ).Union(Map.OfTypeOnly<Boat>(
        b => new {
            vid = b.ID,
            b.Name,
            vdesc = b.Description
            b.MaxPassengers,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
            discriminator = “BOAT”
        })
    );

builder.Tables[“dbo.vehicles”] = vehicleMap;

In a TPH mapping:

OfTypeOnly() is used to create mapping fragments.
Mapping fragments are then unioned so they can be assigned to one table (TPH uses a single table for the whole hierarchy).
A discriminator column is required for each non-abstract type. The discriminator column can have any name and must have a different constant value (i.e. “CAR”) for each non abstract type in the hierarchy.
When mapping a derived type you must re-map all properties mapped in the base-type(s).

Table Per Type (TPT)

If you want to map the same hierarchy using Table Per Type or TPT, this is how you do it:

builder.Table[“dbo.Vehicles”]=
    Map.OfType<Vehicle>(
        v => new {
            vid = v.ID,
            v.Name,
            vdesc = v.Description
            v.MaxPassengers,
        }
    );

builder.Tables[“dbo.Cars”] =
    Map.OfType<Car>(
        c => new {
            cid = c.ID,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinders,
         }
     );

builder.Tables[“dbo.Boats”] =
    Map.OfType<Boat>(
        b => new {
            bid = b.ID,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
        }
    );

In a TPT mapping:

OfType<>() is used in each mapping 'fragment'
Each mapping 'fragment' is assigned to a different table.
Each mapping 'fragment' only maps properties declared on the current type, except…
Key properties must be mapped in every fragment (this allows the tables participating in the type to be joined).
There are no discriminators

Table Per Class (TPC)

You can also map this hierarchy using TPC like this:

builder.Tables[“dbo.Cars”] =
    Map.OfTypeOnly<Car>(
        c => new {
            cid = c.ID,
            c.Name,
            vdesc = c.Description
            c.MaxPassengers,
            trans = c.Transmission,
            tspd = c.Topspeed,
            ccty = c.EngineCapacity,
            ncyld = c.NoCylinders,
         }
     );

builder.Tables[“dbo.Boats”] =
    Map.OfTypeOnly<Boat>(
        b => new {
            bid = b.ID,
            b.Name,
            description = b.Description
            b.MaxPassengers,
            lng = b.Length,
            b.HasSail,
            b.HasEngine
        }
    );

In a TPC Mapping:

Like TPH we use OfTypeOnly(..)
But unlike TPH each non-abstract type has a its own table. (So there is no table for Vehicles because it is abstract).
Each mapping fragment re-maps every, non-transient, property.
There is no mapping for any abstract classes in the hierarchy.
There are no discriminators

Default Foreign Key Locations:

If we see a reference (i.e. Order.Customer) we assume the multiplicity is 0..1. Meaning the the Foreign Key or FK is nullable.

If we see a collection (i.e. Customer.Orders) we assume the multiplicity is many.

Then when you Register Inverses, we know the multiplicity of both ends of the relationships i.e. in the above example we know that there are 0..1 Customers for an Order and many Orders for a Customer.

So by convention there are 3 main types of relationships, we need to infer the FK location for:

0..1 to many –> by convention we put the FK on the many end. So in the Customer.Orders example the FK is put in the Orders table.

many to many –> there is no option but to introduce a join table

0..1 to 0..1 –> You can configure where the FK lives, but in the absence of configuration we will introduce a join table.

Sometimes however a reference is not 0..1 it is 1, i.e. the FK, wherever it might be, may not be nullable.

This is how you specify that the FK is non-nullable:

var orderConfig = builder.Configure<Order>();
orderConfig.RegisterInverse(o => o.Customer, c => c.Orders);
orderConfig.ForProperty(o => o.Customer).NonNullable();

This tells us that there is exactly 1 Customer per Order and many Orders for a Customer.

The ability to tell us that a reference is NonNullable introduces several new multiplicity combinations, for which we need conventions too:

1 to many –> by convention we put the FK on the many end, and make it non-nullable in the database.

0..1 to 1 –> by convention we put the FK on the 1 end, and make it non-nullable.

1 to 1 –> like 0..1 to 0..1 we can’t decide where to put the FK so by convention we introduce a join table.

Specifying FK Mappings:

So far we’ve created mappings for the properties of entities.

What about Navigation Properties and FKs?

All types of relationships (except many to many) can be modeled without a join table in the database. So we allow you to map FKs as part of an EntityMap like this:

EntityMap<Customer> customerMap =
    from c in Map.OfType<Customer>()
    select new {
            cid = c.ID,
            c.Name,
            csite = c.Website,
            salesPersonFK = c.SalesPerson.ID
        };

customerMap.RegisterInverse(c => c.SalesPerson, s => c.Clients);
builder.Tables[“dbo.Custs”] = customerMap;

This specifies that the Customer.SalesPerson navigation property, and it’s inverse SalesPerson.Customers, are stored in the salesPersonFK column in the dbo.Custs table. Because the fragment maps the salesPersonFK column to the c.SalesPerson.ID, where SalesPerson.ID is the Primary Key (or part of the PrimaryKey) of the related SalesPerson entity, and of course FK’s point to PK’s.

Specifying Join Table Mappings:

In the case of many to many relationships you must have a join table, so absent mapping information we produce a join table by convention.

If however you need more control you can do this:

var blogPostsMap = new AssociationMap<Blog, Post>(
b => b.Posts
).Map(
(b, p) => new {BlogId = b.ID, PostId = p.ID}
);

builder.Tables[“dbo.BlogPosts”] = blogPostsMap;

This says the many to many relationships between Blog and Post is stored in the ‘dbo.BlogPosts’ table which has two columns:

‘BlogId’ which is also an FK pointing to the column to which the ID property of Blog is mapped.
‘PostId’ which is also an FK pointing to the column to which the ID property of Post is mapped.

Entity Splitting:

Code Only can even support more advanced mapping strategies like entity splitting:

builder.Tables[“dbo.Customer”] = Map.OfType<Customer>(
   c => new {
      cid = c.ID,
      c.Name,
      active = c.IsActive
   }
);

builder.Tables[“dbo.CustomerDetails”] = Map.OfType<Customer>(
    c => new {
       cid = c.ID,
       c.Size,
       c.Industry
    }
);

This says that the Customer entity is split across the dbo.Customer and dbo.CustomerDetails tables.

Encapsulating all the Configuration:

You can also write a class that derives from EntityMap<T>, to hold all the mapping, facets etc. For example here a class that holds the configuration for Product

public class ProductMap: EntityMap<Product>
{
    public ProductMap{
         this.Map( p => new {
              pid = p.ID,
              pcode = p.Name,
              cid = p.Category.ID
         });
         this.ForProperty(p => p.ID).Identity();
         this.ForProperty(p => p.Name).MaxLength(100)
             .NonUnicode();
         this.ForProperty(p => p.Category).NonNullable();
         this.RegisterInverse(p => p.Category,
                              c => c.Products);
     }
}

This approach is highly recommended, because configuring the Product type is now trivial:

builder.Tables[“dbo.Products”] = new ProductMap();

Summary:

As you can see we are planning a lot of enhancements which will allow most core scenarios.

What do you think? Do you like the API? Are there things you’d like to see look a little different?

As always we are very keen to hear your feedback.

Alex James
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Code Only

efdesign — Wed, 10 Jun 2009 23:04:00 GMT

There are currently two ways to get Entity Framework apps up and running, we call these Database First and Model First respectively.

Database First has been with us since .NET 3.5 SP1, and involves reverse engineering an Entity Data Model from an existing database.

Model First is new to Visual Studio 2010/.NET 4.0 and allows you to create an Entity Data Model from scratch and then generate a database and mapping for it.

However many developers view their Code as their model.

Ideally these developers just want to write some Domain classes and without ever touching a designer or a piece of XML be able to use those classes with the Entity Framework.
Basically they want to write ‘Code Only’.

We are pleased to announce that we’ll be shipping a preview of “Code Only” on top of the .NET Framework 4.0 Beta 1 (more on that in the coming days).

How it works:

To use Code only you simply write some POCO classes, here is a simple example:

public class Category
{
   public int ID { get; set; }
   public string Name { get; set; }
   public List<Product> Products { get; set; }
}

public class Product
{
   public int ID { get; set; }
   public string Name { get; set; }
   public Category Category { get; set; }
}

Then write a class that derives from ObjectContext that describes the shape of your model and how you want to access your POCO classes, so something like this:

public class MyContext : ObjectContext
{
   public MyContext(EntityConnection conn)
    : base(conn){ }
   public ObjectSet<Category> Categories {
      get {
         return base.CreateObjectSet<Category>();
      }
   }
   public ObjectSet<Product> Products {
      get {
         return base.CreateObjectSet<Product>();
      }
   }
}

NOTE: this persistence / EF aware class could easily be in another assembly so you can keep it separate from your nice pristine domain classes.

At this point you have everything you need from a CLR perspective, but you won’t be able to use the MyContext class without some EF metadata, which is normally stored in the EDMX file.

But remember Code-Only means there isn’t an EDMX file.
So where do we get the metadata?
That is where the ContextBuilder class comes in:

SqlConnection connection = …; // some un-opened SqlConnection
MyContext context = ContextBuilder.Create<MyContext>(connection);

What is happening here is that the ContextBuilder is looking at the Properties on MyContext, inferring a default Conceptual Model, Storage Model and Mapping by convention, it the uses that metadata plus the SqlConnection you passed in to create an EntityConnection, and finally it constructs an instance of MyContext by passing the EntityConnection to the constructor we created previously.

Once you have an instance of your context, we ship several extension methods you can use to automatically create a database script, see if the database exists, drops the database, and/or create the database,. For example this code snippet uses two of those extension methods to create the database if it doesn’t already exist.

if (!context.DatabaseExists())
context.CreateDatabase();

The CreateDatabase call looks at the EF metadata, in particular the Storage Model, aka SSDL, and uses it to produce Database Definition Language (or DDL) which it then executes against against the connection.

Overriding Conventions

In the examples so far everything is done by convention. This is great if you are happy with our conventions. However you may want to override them, for example to register different keys, use different mappings or use a different inheritance strategy etc.

To override the convention instead of creating an ObjectContext directly you create an instance of a ContextBuilder that you can configure.

var contextBuilder = new ContextBuilder<MyContext>();

In the first Feature CTP, Code Only provides the ability to override how the key properties are inferred:

contextBuilder.RegisterKey((Product p) => p.ID);

This code tells the builder that the key of Product is its ID property.

When you’ve finished configuring how you want your model and mappings to work in your code, you create an instance of your context, this time by calling Create on the builder instance you have been configuring:

MyContext context = contextBuilder.Create(connection);

It is that simple.

Over time the features of the ContextBuilder will grow so you will be able to setup custom mappings, mark properties as transient, setup up Facets (e.g. MaxLength), change inheritance strategies, link properties that are the inverse of each other together (e.g. product.Category and category.Products) and more. In fact here is an example of the sort of thing we are aiming to support for the next release of the Feature CTP:

builder[“dbo.prds”] =
from c in builder.OfType<Product>()
select new {
    pcode = p.ID,
    p.Name,
    cat = p.Category.ID
}
);

This snippet:

Maps Product entities to the ‘prds’ table.
Maps Product.ID to a ‘pcode’ column.
Maps Product.Name to a ‘Name’ column.
Maps the FK under the Product.Category relationship to the ‘cat’ column.

Here is another example that does basic TPH inheritance:

builder[“dbo.Cars”] = (
from b in builder.OfTypeOnly<Car>()
select new {
     id = b.CID.AsKey(),
     b.Color,
     b.Manufacturer,
     disc = “C”
   }
).Merge(
from s in builder.OfType<SportsCar>()
select new {
     id = s.CID.AsKey(),
     s.Color,
     s.Manufacturer,
     disc = “S”,
     hp = s.HorsePower,
     tq = s.Torque
}
);

These two examples just scratch the surface of what we are planning on doing, hopefully though they give you a sense of where we are heading.

We are super excited about Code Only. But as of right now our plans to support overriding conventions aren’t finalized, so we’d love to get your feedback, this really is your chance to help us get it right.

Alex James,
Program Manager, Entity Framework Team, Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Self-Tracking Entities in the Entity Framework

efdesign — Tue, 24 Mar 2009 04:50:52 GMT

Background

One of the biggest pieces of feedback we received from the N-Tier Improvements for Entity Framework post as well as other sources was: “low level APIs are great, but where is the end-to-end architecture for N-tier and the Entity Framework?”. This post outlines some of the additional feedback we’ve received and describes the self-tracking entities architecture that will ship along side of VisualStudio 2010 and .NET Framework 4.0.

When an application goes multi-tier, there are important architectural decisions that the application developer makes around how to communicate between tiers. There are a lot of choices, and the choice depends on a variety of conditions. How rapidly is each tier expected to change? How much control is there over each tier? Are their specific protocol, security, or business policy concerns? The answers to these questions often drive the selection of whether to use data transfer objects (DTOs) or DataSets or something different altogether.

Self-Tracking Entities

Self-tracking entities know how to do their own change tracking regardless of which tier those changes are made on. As an architecture, self-tracking entities falls between DTOs and DataSets and includes some of the benefits of each.

Drawing from DataSet

DataSet on the client tier is very easy to use because there is no need to track changes separately or maintain any extra data structures that include change tracking information. DataSet takes care of serializing state information for each row of data. On the mid tier, applying the changes stored within a DataSet is straightforward. DataSets have also gained popularity because of the number of tools that work with DataSets, and because they are easily bound to many UI/presentation controls. Since it works in so many scenarios, for many applications there is never a need to transform data outside of a DataSet allowing a single paradigm to be used up and down the stack.

However, there are disadvantages to DataSets when used as the communication payload between tiers. The first is the cost of getting the data into a serializable format, and the second is that the DataSet serialization format is generally not very interoperable with other languages that are used to expose services. Another disadvantage of using a DataSet is that you can quickly lose the intent of your service call because so many kinds of things can be included in a DataSet. For example, if the service method is declared as:

DataSet GetCustomer(string id);

It is extra work to ensure that the DataSet that is returned contains only rows of data that pertain to “customers”. It is also more difficult to specify a service contract that says the DataSet is also supposed to return data for each customer’s orders and their order details.

Self-tracking entities share many advantages with DataSet. They also encapsulate change tracking information, which is serialized along with the data contained in the entity. On the mid-tier applying changes from a graph of self-tracking entities to a persistent context is equally straightforward. The Entity Framework will also provide tools for generating self-tracking entities and because these entities are just objects, they can be easily made to work with UI/presentation controls.

Drawing from DTOs

DTOs and SOA are used to give the developer more control over the service contract and payload for tier to tier communication. DTOs themselves do not have behavior, so they are typically very simple classes designed just to provide the needed information to perform a specific service operation. Not only does this provide the opportunity to optimize the wire format (some believe a DTO is all about the wire format), but it makes it possible and easy to capture the intent of each service method. The data contract used with DTOs is typically interoperable which makes it easy to use services that run on different platforms. DTOs also provide a way to separate messaging contracts from the presentation layer, the business logic, and the persistence layer which in many cases creates a maintainable architecture.

There are disadvantages to using DTOs and the primary one is complexity. DTOs are often hand-crafted to include only the specific information that is needed for an operation. When there is a common pattern for mapping DTOs to entity classes, there are some tools available that will do DTO generation and mapping but it is not always possible. With DTOs, it is up to the developer to decide how to do change tracking on each tier (especially the client) which increases the complexity of the presentation layer. The complexity of the service implementation also increases because the developer is responsible for translating the DTO into entities for doing business logic validation, as well as being able to report changes stored in the DTO to the persistence framework.

Not only do self-tracking entities share advantages with DataSets as mentioned above, they also share many advantages with DTOs. Self-tracking entities expose a simple and interoperable wire format and it is clear what kinds of data a service method requires or returns. You can also use a self-tracking entity as part of a message. However, self-tracking entities don’t give quite as much architectural separation as using pure DTOs, but you do gain a less complex solution that requires fewer data transformations. It is important to note that self-tracking entities can be made to be ignorant of any particular persistence framework making them essentially POCO objects. Particular persistence frameworks such as the Entity Framework will have the capabilities to create these entities and interpret the change tracking information when saving changes.

.NET Framework 3.5SP1 Challenge

With the .NET Framework 3.5SP1, this sort of solution was very hard to implement using the Entity Framework because change tracking was always done by a centralized ObjectContext which contains an ObjectStateManager. In particular, “reattaching” to report changes back to the ObjectStateManager was all but impossible without completely shredding your entity graph and applying changes one entity and one relationship at a time. The new API changes that are being added to the Entity Framework in .NET Framework 4.0 are enablers for an easier experience of reporting changes back to the ObjectStateManager, making self-tracking entities (as well as other architectures) easier to build.

Mid-tier experience

Using self-tracking entities on the mid-tier is about working with entity graphs and the Entity Framework. The service contract that is used in the following examples contains two simple methods for retrieving a Customer entity graph and applying updates to that entity graph:

interface ICustomerService
{
Customer GetCustomer(string customerID);
bool UpdateCustomer(Customer customer);
}

The implementation of the GetCustomer service method can be done using an Entity Framework ObjectContext and using LINQ or query builder methods to retrieve the entity you want. In the example below, a query is issued for a particular customer entity and all of the Orders and OrderDetails are included.

public Customer GetCustomer(string customerID)
{
    using (NorthwindEFContext context = new NorthwindEFContext())
    {
        var result = context.Customers.
              Include("Orders.OrderDetails").
              Single(c => c.CustomerID == customerID);
        return result;
    }
}

UpdateCustomer is example of how to save the changes that are made to a graph of self-tracking entities. Similar to DataSet, applying these changes to the persistence layer and saving them should be simple. In the below example, a new Entity Framework API, “ApplyChanges” is used which understands how to interpret the change tracking information that is stored by each entity and how to tell the ObjectContext’s ObjectStateManager about those changes.

public void UpdateCustomer(Customer customer)
{
    using (NorthwindEFContext context = new NorthwindEFContext())
    {
        context.Customers.ApplyChanges(customer);
        context.SaveChanges();
    }
}

Client Experience

The client experience when working with self-tracking entities is similar to how you would manipulate any object graph. You can make changes to scalar or complex properties, add or remove references, and add or remove from collections of related entities. The key part of the experience is that tracking changes is hidden from the client because it is done internally on each entity. There is no ObjectContext and there is no extra state that has to be maintained or passed from client to the tier that does persistence.

In this example, the test first queries for a customer, then deletes one of the orders. The resulting entity graph is sent back to the service tier using the UpdateCustomer method.

public void DeleteObjectsTest()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var customer = client.GetCustomer("ALFKI");
        customer.Orders.First().Delete();
        client.UpdateCustomer(customer);
    }
}

The next test shows how to add a new Order with two OrderDetails to an existing customer. By default, the constructor of a self-tracking entity puts the entity in the “Added” state. There are conveience methods on each self-tracking entity to change this state if needed (Delete() and SetUnchanged()).

public void AddObjectsTest()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var customer = client.GetCustomer("ALFKI");
        customer.Orders.Add(new Order() {
            OrderID = 100,
            OrderDetails = {
                new OrderDetail{
                     ProductID = 3,
                     Quantity = 7
                },
                new OrderDetail{
                     ProductID = 4,
                     Quantity = 8
                },
            }
        });
        client.UpdateCustomer(customer);
    }
}

The final example shows that a self-tracking entity graph can contain any number of changes and those changes can be any combination of adds, modifications, and deletes.

public void ComplexModificationsTest ()
{
    using (CustomerServiceClient client = new CustomerServiceClient())
    {
        var cust = client.GetCustomer("ALFKI");
        // remove first order - will do cascade in the database
        cust.Orders.Remove(cust.Orders.First());
        // modify one of the orders
        var order = cust.Orders.Last();
        order.RequiredDate = DateTime.Now;
        order.OrderDetails.Add(
            new OrderDetail{
                ProductID = 7,
                Quantity = 3
        });
       // add new order
        cust.Orders.Add(new Order()
        {
            OrderID = 100,
            OrderDate = DateTime.Now,
            ShipCity = "Redmond",
            ShipAddress = "One Microsoft Way",
            ShipRegion = "WA",
            ShipPostalCode = "98052",
            OrderDetails = {
                new OrderDetail{
                    ProductID = 3,
                    Quantity = 7
                },
                new OrderDetail{
                    ProductID = 4,
                    Quantity = 8
                },
            }
        });
        client.UpdateCustomer(cust);
    }
}

Generating Self-Tracking Entities
Building self-tracking entities should be as easy as using them. We plan to ship a T4 template that will do the code generation for creating self-tracking entities from your EDM. This template will show up in the list of available templates when you right click on your model and choose Add New Artifact Generation Item.

Design Notes

Inside a Self-Tracking Entity

There have not been final decisions about what exactly will be included inside of a self-tracking entity, but it is important to track:

The state of the entity, Added, Deleted, Modified, or Unchanged
The original value from reference relationship properties
Adds and removes from collection relationship properties

This information will be included with the data contract of the entity. As part of the code generation of each self-tracking entity, scalar and complex property changes will mark the entity as “dirty” meaning that its state will change from Unchanged to Modified.

Inside ApplyChanges

ApplyChanges is a new API on the ObjectContext and ObjectSet classes that attaches an entity graph and interprets the change tracking information stored in each entity. The design for discovering the change tracking information has not yet been finalized, but there are a couple of options available. One would be to code generate an interface as part of the T4 template for self-tracking entities as well as a version of ApplyChanges that knew about the interface. Another option would be fall back to runtime discovery of the particular properties using a convension and some of the new dynamic capabilities of the framework. The thing we want to avoid is causing the self-tracking entity implementation to have a dependency on any of the Entity Framework assemblies.

The algorithm that ApplyChanges uses is:

1. Attaching the entity graph to the ObjectContext

2. Changing the state of each entity using the ChangeObjectState API.

3. For any reference relationship property, if there is an original value changing the state of the original reference relationship to Deleted and change the state of the current reference relationship to Added. This is done using the ChangeRelationshipState API.

4. For any collection relationship property, use the ChangeRelationshipState API to mark any removed relationships as Deleted and to mark any added relationships as Added.

Summary

We’ve received a lot of feedback and suggestions on how to make developing multi-tier applications easier using the Entity Framework. One of the components of improving this experience is the introduction of an end-to-end architecture around self-tracking entities. We’d like to hear your feedback on this addition to the Entity Framework, and any other comments you have on the matter of multi-tier development using domain models and entities.

Jeff Derstadt,
Dev Lead, Entity Framework Team, Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Foreign Keys in the Entity Framework

efdesign — Mon, 16 Mar 2009 23:47:00 GMT

Background

A number of months ago we asked whether Foreign Keys (FKs) in Conceptual and Object models were important.

The feedback we got indicated that there are a real mix of opinions on the topic. Some people are all for them while others think that FKs pollute the conceptual model.

The fact that our customers were so divided meant we thought it was important to provide options here. Those who want FKs in their Entities should be able to do so, so they can gain all the benefits that having FKs in your Entities undoubtedly provide. On the other hand customers who are concerned that have FKs in their Entities in someway pollutes their model can continue to use the type of associations we had in .NET 3.5 SP1.

In the past we've called the .NET 3.5 SP1 style Associations "first class" associations. Which while accurate somehow implies that associations based on FKs in Entities are "second class". In fact when we talked about this new work recently with our MVPs one of the first questions was "are these second class?", which they most definitely are not.

The fact is these new associations are different but first class nevertheless.

So what's been added?

In .NET 4.0 we will add support for a new type of association called "FK Associations".

The way this works is we will allow you to include your FK columns in your entities as "FK properties", and once you have FK Properties in your entity you can create an "FK Association" that is dependent upon those properties.

Okay so we will have "FK Associations", what are we going to call the older style associations? Obviously saying something like ".NET 3.5 SP1 style Associations" every time isn't ideal. So we are going to call them "Independent Associations".

The term "Independent Association" resonates for us because they are independently mapped, whereas FK Associations need no mapping, simply mapping the Entity(Set) is sufficient.

How do I use an "FK Association"?

The real reason so many customers and partners are asking for "FK Associations" is that they significantly simplify some key coding patterns. So much so that we are convinced that for most people "FK Associations" will be an automatic choice going forward.

Some of the things that are hard with "Independent Associations" are trivially easy using FK Associations. Scenarios as diverse as DataBinding, Dynamic Data, Concurrency Control, ASP.NET MVC Binders, N-Tier etc are all positively impacted.

Let's have a look at some code snippets that will work with "FK Associations":

1) Create a new Product and FK Association to an existing Category by setting the FK Property directly:

using (var context = new Context())
{
    //Create a product and a relationship to a known category by ID
    Product p = new Product
    {
        ID = 1,
        Name = "Bovril",
        CategoryID = 13
    };
    //Add the product (and create the relationship by FK value)
    context.Products.AddObject(p);
    context.SaveChanges();
}

This sort of approach works both for insert and update, and is particular useful for things like databinding, where you often have the new Value of the FK in a grid or something but you don't have the corresponding object, and you don't want to wear the cost of a query to pull back the principal object (i.e. the Category).

2) Create a new Product and a new FK Association to an existing Category by setting the reference instead:

public void Create_new_Product_in_existing_Category_by_reference()
{
    using (var context = new Context())
    {
        //Create a new product and relate to an existing category
        Product p = new Product
        {
            ID = 1,
            Name = "Bovril",
            Category = context.Categories
                        .Single(c => c.Name == "Food")
        };
        // Note: no need to add the product, because relating
        // to an existing category does that automatically.
        // Also notice the use of the Single() query operator
        // this is new to EF in .NET 4.0 too.
        context.SaveChanges();
    }
}

This programming pattern is not new to the Entity Framework, you could do this in .NET 3.5 SP1. I called it out because it shows you can still write the sort of code you write with Independent Associations even when using FK Associations.

3) Update an existing Product without informing the Entity Framework about the original value of the CategoryID (not supported with Independent Associations):

public void Edit(Product editedProduct)
{
    using (var context = new Context())
    {
        // Create a stand-in for the original entity
        // by just using the ID. Of the editedProduct
        // Note: you don't have to provide an existing Category or CategoryID
        context.Products.Attach(
            new Product { ID = editedProduct.ID });

        // Now update with new values including CategoryID
        context.Products.ApplyCurrentValues(editedProduct);
        context.SaveChanges();
    }
}

In this example "editedProduct" is a product which has been edited somewhere, this is exactly the sort of the code you might write in the Edit method of a ASP.NET MVC controller, and is a great improvement over the code you have to write using Independent Associations.

There is much more you can do with FK Associations, but these 3 samples give you a flavor of the sort of coding patterns FK Associations allow.

Keeping FKs and References in Sync

One thing that hasn't been mentioned so far is that the Entity Framework tries to keep related References and FKs in sync as much as possible.

When this synchronization occurs depends upon when the Entity Framework is notified of changes, which depends upon the type of Entity Classes involved: be they POCO with Proxies, POCO without Proxies, IPOCO or EntityObjects etc.

Another post will cover this in more detail.

How do I create an "FK Association"?

There are a number of ways.

One mainline scenario is when using the tools to infer a model from a database. We have added an option to choose whether "FK Associations" or "Independent Associations" are generated by default. The same is true if you use EdmGen.exe (i.e. our command line tool).

The second mainline scenario for creating FK Associations is creating a model from scratch, aka Model First which is new to .NET 4.0.

Here are the steps involved:

Create two Entities (say Product and Category) that look something like this:
Add a property that will be the FK Property: i.e. CategoryID
Create an association between the two EntityTypes with the correct end multiplicities and NavigationProperty names:
Double Click the line between the two Entities, that represents the Association, and add a Referential Integrity Constraint:

This is the step that tells the Entity Framework that the CategoryID is an "FK Property" and that the "CategoryProduct" association is an "FK Association".
You end up with something that looks like this:

And you are done you've set up an FK Association. Notice that this association doesn't need to be mapped you simply need to make sure you map all the properties of both Entities.

Very easy.

Summary

In .NET 4.0 we will add support for FK Properties and FK Associations to the Entity Framework. The existing style Independent Associations will still be possible, but we expect that FK Associations will become most peoples automatic choice because they simplify so many common Entity Framework coding tasks.

As always we'd love to hear your thoughts on this work.

Alex James,
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Customizing Entity Classes in VS 2010

efdesign — Thu, 22 Jan 2009 22:15:00 GMT

When VS 2010 ships it will include some significant improvements to our code generation story for the Entity Framework. The basic idea is to make use of T4 templates for code generation and ship strong integration into the Entity Framework Designer to make the experience of customizing those templates as seamless as possible.

Below Sanjay from our Tools team outlines what will be possible once VS 2010 is released.

Customize the code generated by the Entity Designer with T4 templates

In Visual Studio 2008 SP1, the ADO.NET Entity Designer generates classes from the CSDL portion of the EDMX file using the EntityClassGenerator APIs. Numerous customers have asked us how to customize the code generation for a variety of scenarios including:

Make the generated ObjectContext internal
Make the generated ObjectContext and Entity classes implement a user-defined interface
Add user-defined CLR attributes to generated ObjectContext and generated Entity classes
Influence generated classes based on structural annotations in CSDL
Generate the ObjectContext and Entity classes into separate files
Generate “proxy classes” for the generated classes
Partially or fully change how classes are generated, maybe even generate additional (non code) artifacts in the project
Completely replace entity framework code generation with custom code
Generate POCO classes from the model to use as a starting point in my applications
Generate self-tracking entity classes
· ...

While the SingleFileGenerator techniques described here enable some of these scenarios these techniques are generally difficult to implement and install or are tricky to debug and customize.

In Visual Studio 2010, we plan to give customers the ability to use T4 templates to generate their classes, and are also planning deep integration with the Entity Designer and Visual Studio to provide a great end-to-end experience.

Walkthrough

The example below customizes code generation to make the generated classes implement a user-defined interface (e.g. IValidate).

Preconditions:

User has installed Visual Studio 2010
User is familiar with customizing T4 templates
User has a C# or a VB console project that targets FX4.0 with an EDMX file in the project

Walkthrough:

User: opens EDMX file in the Entity Designer. Note that the generated classes are in a child file of the EDMX file (i.e. in Northwind.Designer.cs)
User: right-clicks on an empty area of the designer surface and chooses “Add New Artifact Generation Item…” from the context menu.
Visual Studio: displays the standard “Add New Item” dialog and shows a filtered list of items for the user to select from. As expected, the “Add New Item” dialog only shows a list of item templates specific to the current project target framework version, project type and language.
User: selects “ADO.NET EntityObject Generator”, specifies a file name and clicks “Add"
Visual Studio: checks out the project from Source Control if necessary
Visual Studio: sets the “Custom Tool” property of the EDMX file to empty, which will cause the existing .designer.cs file to be deleted
Visual Studio: the VS item template adds a new .tt file to the project in the same project directory as the EDMX file
Visual Studio: configures the new .tt file to process the selected EDMX file. The EDMX file to process is a replaceable parameter in the .tt file, and VS updates it to point to the selected EDMX file. Thus, the .tt file now knows which EDMX file to generate code for.

[Screenshot of .tt file open in the Clarius T4 Editor Community edition for VS 2008]
User: double-clicks the .tt file in the project to open it in VS10
User: edits the .tt file in Visual Studio to make every Entity object implement the IValidate interface

[Screenshot of .tt file open in the Clarius T4 Editor Community edition for VS 2008]
User: saves and closes .tt file
Visual Studio: Transforms the .tt file to produce the generated classes as a child file of the .tt file
User: notes that the generated classes (under the .tt file) implement the IValidate interface as expected
User: switches to the EDMX file in the designer and makes some changes (e.g. add a new entity)
User: saves EDMX file
Visual Studio: Transforms all .tt files in the project that reference the EDMX file. As before, the generated classes are child files of the .tt files

Design

The design affects multiple parts of the Entity Framework and the Entity Designer and the key changes are described below:

Entity Framework Runtime

We started by creating C# and VB T4 templates to generate classes from CSDL and EDMX files
Since the T4 engine is installed with Visual Studio and not with the.NET FX runtime, we preprocess the templates as described here and include the compiled code into System.Data.Entity.Design.dll
We added a new class System.Data.Entity.Design.EntityCodeGenerator (in System.Data.Entity.Design.dll ) that generates code using the preprocessed T4 template
Finally, we enhanced a few runtime metadata APIs and added extension methods to load and iterate over models a lot easier. Our T4 templates make heavy use of these new methods.

Entity Designer

We took the C# and VB T4 templates used by the runtime and changed them to be more “VS friendly” as described below.

We made the EDMX file name referenced in the T4 templates a replaceable parameter
We created a wizard (with no GUI) that is launched by Visual Studio when the item is added to the project. The wizard does 2 things:
1. updates the EDMX file name referenced in the T4 templates and
2. sets the “Custom Tool” property of the EDMX file to empty
We rolled up the T4 templates (and wizard) into standard C# and VB Visual Studio item template which are installed with VS 2010; this makes them show up in the “Add New Item” dialog in Visual Studio
We added a new context menu to the designer surface to launch the “Add New Item” dialog and also added code to only show VS item templates whose names start with the prefix ADONETArtifactGenerator_
Our design ensures that user installed VS item templates in %MyDocuments% as well as the VS item templates we ship in box show up in the “Add…New…Item” dialog (per the template name prefix rule above)
We added code to the designer that finds all .tt files related to an EDMX file and automatically transform them when the EDMX file is saved.
Finally, we added a new property called “Process related T4 templates on Save” on the designer surface to let users control whether or not to transform .tt files related to the EDMX file on save.

Multi Targeting considerations

As many of you are aware, VS 2010 allows developers to target FX4.0 as well as FX3.5 (and older runtimes). The T4 templates that we ship in the Visual Studio box generate code that works with the Entity Framework in .NET FX 4.0. This means the VS item templates we ship in the box will not be available for projects that target FX3.5.

However, it is certainly possible (and supported) for users to create new T4 templates (and new VS item templates) that generate code from an EDMX file in projects that target FX3.5 and light up in the same experience. In fact, we might release some new templates ourselves later, on CodeGallery perhaps.

How can 3^rd parties plug in?

Since our design is based upon VS item templates that wrap T4 templates, we have opened the door to let 3^rd party code generators participate in the end-to-end experience. Our contract is straight forward and easy to implement:

Create a (C# or VB) T4 template to generate code as you desire. The T4 templates we ship in the box can be invaluable points of reference as you roll your own.
Make the EDMX reference in your .tt file a replaceable parameter by calling it $edmxInputFile$ like we do in our .tt file.
Wrap the .tt file in a VS item template. This is pretty easy and well documented on MSDN. You can also use the Export Template wizard included Visual Studio. Also specify additional VS item template settings such as target framework, project type, etc
Prefix the name of your .vstemplate file with ADONETArtifactGenerator_ if you want your VS item template to show up in the “Add New Item” dialog when it is launched from the context menu in the Entity Designer.
You can use the wizard we ship in Visual Studio to do parameter replacement in your VS item template by adding the following to your .vstemplate file:

<WizardExtension>
<Assembly>
Microsoft.Data.Entity.Design, Version=10.0.0.0, Culture=neutral,PublicKeyToken=b03f5f7f11d50a3a
</Assembly>
<FullClassName> Microsoft.Data.Entity.Design.VisualStudio.ModelWizard.AddArtifactGeneratorWizard
</FullClassName>
</WizardExtension>

Of course, you can also create your own wizard that does custom processing when the item is added to the project as described here.
Create a VSCONTENT file and zip up your custom item template & related files into a Visual Studio Content Installer (.vsi) file
Upload the .vsi file someplace your customers can download it from
Visual Studio already knows how to install .vsi files and everything you need is already available in Visual Studio.

A few key takeaways:

This design lets the EF team release new T4 templates outside the usual Visual Studio and .NET FX ship cycles and the experience to consume them is exactly the same.
There are several excellent resources on the web that describe advanced T4 template scenarios and our design enables them. In fact, we expect users to leverage these resources while creating new T4 templates or customizing the ones we ship. For example, code in a custom T4 template can use the EnvDTE APIs to access the project system and generate outputs to multiple files.
Over time, we expect to blog about some of these advanced scenarios and also work with MVPs, industry experts and Patterns & Practices to provide guidance to customers in a manner similar to the Guidance Automation Toolkit.
Custom T4 templates can generate anything you want, not just code. For example, we have an internal prototype of a T4 template that iterates over a model and generates an HTML file with a nicely formatted report of the entities and associations in the model
VS item templates can add multiple files to your project (more details here). This means your custom item templates can include all kinds of files (e.g. workflow files, multiple T4 files, pre-canned code files, etc). For example, you can easily imagine a 3^rd party VS item template that adds 2 .tt files in your project that process the same EDMX file but generate different kinds of outputs.

We hope this gives you an overview of our design and intent of this feature and we would love to hear your comments.

Sanjay Nagamangalam,
Lead PM, ADO.NET Entity Designer

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Update on Computed Properties

efdesign — Tue, 13 Jan 2009 07:27:00 GMT

A while back I wrote a post that introduced the concept of Computed Properties. Since that time we’ve had a number of conversations with both customers and internal partners and we’ve had some new ideas, that have changed our thinking somewhat.

First off we decided to implement Model Defined Functions, which are significantly more powerful and flexible than Computed Properties, and provide overlapping capabilities.
Computed Properties on the other hand, would never have been structurally part of the entity, which lead to some tough choices and probable customer confusion:

Do you always pay the cost of materializing the Computed Property even when it is not required?
What about if there are lots of Computed Properties on the Entity?
Today in order to materialize an Entity and it's properties, computed or otherwise, the Entity Framework needs access to setters. Unfortunately the existence of the Setter, sends an erroneous message that Computed Properties can be altered and persisted.
Upon further investigation we discovered that Computed Properties didn't actually address the requirements of Reporting Services, which was one of the original drivers for Computed Properties. Our original design called for computed properties to operate over an instance of an Entity, and it turned out that Reporting Services needed computed properties to operate over both an instance and a set of instances. Model Defined Functions on the other hand are flexible enough to handle both of these scenarios.

In the future it may be possible to gain all the benefits of computed properties without any of the above issues, by extending our integration of Model Defined Functions with LINQ so that the [EdmFunction] attribute can be applied to CLR properties too.

If we did this the first parameter of the specified Model Defined Functions could be mapped to be the CLR type containing the property itself. This would allow a Computed Property, backed by a Model Defined Function, to be used in a targeted way in LINQ queries, without any of the illusions that this might somehow be set-able or structurally part of the entity.

Given all these points we started to think Computed Properties just weren’t as important.

We think our best course of actions in the short term is 1) to focus on supporting Model Defined Functions and 2) ask for your feedback on the relative importance of Computed Properties given the points listed above.

So what do you think?

Do Model Defined Functions address most of the scenarios for Computed Properties?

Particularly if in the future we can treat CLR properties as stubs for Model Defined Functions?

As always we are very keen to hear what you think.

Alex James
Program Manager
Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

NB: For more background on the origins of [EdmFunction] check out this post.