ADO.NET team blog : Entity Framework

Automatic Generation of Stored Procedure Return Types

dpblogs — Thu, 12 Nov 2009 20:07:30 GMT

A new feature of the Entity Framework in .NET 4.0 is the ability to return collections of complex type instances from stored procedures. In the next public release of the designer, we have enriched this functionality by adding the ability to automatically create these complex types by querying stored procedure metadata from the database server. To demonstrate this feature, we will use the Northwind database, and we will focus on a stored procedure called “CustOrdersDetail”. Looking at the designer’s model browser window after we reverse engineer Northwind, we see:

Double-clicking “CustOrdersDetail” brings up the “Add Function Import” dialog:

We click “Get Column Information” in order to see the metadata returned by the server about this stored procedure’s return columns. Then, we click on the “Create New Complex Type” button, and the designer automatically creates a new complex type that matches the shape of the data returned from the stored procedure:

To make life more interesting (and realistic,) let’s rename some of the properties in the generated complex type: ProductName to Name, and UnitPrice to Price. To keep things running, we then need to fix the function import’s mappings to look like this:

Finally, to simulate some real-world churn, we change the stored procedure’s return shape: We’ll rename “UnitPrice” to “UnitPricing”, add a new column called “Foo”, and delete “Quantity”:

ALTER PROCEDURE CustOrdersDetail @OrderID int
AS
SELECT ProductName,
    UnitPricing=ROUND(Od.UnitPrice, 2),
    Discount=CONVERT(int, Discount * 100),
       Foo = '!', 
    ExtendedPrice=ROUND(CONVERT(money, Quantity * (1 - Discount) * Od.UnitPrice), 2)
FROM Products P, [Order Details] Od
WHERE Od.ProductID = P.ProductID and Od.OrderID = @OrderID

That done, we double click on the function import again, get column information, and this time, click on the “Update” button. The designer will now inspect the mappings, the existing complex type, and the new return shape, and produce these proposed changes to the complex type:

As you can see, “Foo” will be added, and “Quantity” will be deleted. In addition, the mappings are used to map “Name” to “ProductName” meaning no action will be taken for it. Finally, since “UnitPricing” is seen as new, “Price” will be deleted and “UnitPricing” will be put in its stead. Since we do not want this, we will rename the column in the function import’s mapping to “UnitPricing” and rerun the above process. This time we get:

Which is what we would want.

In conclusion, the designer now exposes the Entity Framework’s new ability to return complex type instances from stored procedure. On top of this, it adds some features to make this capability easier to use and to iterate over. In future releases of Visual Studio, we will improve the iterative features and look into supporting batch imports of stored procedures, so that you will not need to import them one by one. We look forward to your feedback.

Noam Ben-Ami
Program Manager, Entity Designer

Foreign Key Relationships in the Entity Framework

dpblogs — Fri, 06 Nov 2009 22:06:00 GMT

Last March Alex James posted to our design blog about our plans for adding foreign keys to the entity framework. Since then we’ve pushed forward with implementing FK associations and properties and have included them in Visual Studio 2010 Beta 2. This article offers a walkthrough of foreign key relationships in the EF and how they’re useful in VS2010 Beta2. For more information, the documentation and a number of blogs offer useful background about the logic behind adding EF foreign key relationships; here are a few links:

http://blogs.msdn.com/efdesign/archive/2009/03/16/foreign-keys-in-the-entity-framework.aspx

http://blogs.msdn.com/efdesign/archive/2008/10/27/foreign-keys-in-the-conceptual-and-object-models.aspx

Defining and Managing Relationship MSDN documentation

Walkthrough:

We’re going to look at the designer experience, the EDMX model differences and a few coding examples. We’ll use this extremely simple database model:

Note the fact that the Posts table has a foreign key to the Blogs table.

Designer Experience

Start by creating a new WPF application then add a new ADO.NET Entity Data Model item to it. Choose to generate the model from the database. Open the Tables portion of the treeview and check the Blogs and Posts tables. Note when choosing the tables with which to generate Entities “Include Foreign Key Columns in the Model” is checked by default. This means new models by default will use the new FK associations rather than Independent Associations. Independent Associations were previously the only type of association offered in the Entity Framework and still can be used. If you want just Independent Associations in your model, you’ll need to uncheck the Include FK Columns in the Model checkbox.

After clicking Finish your model should look as follows:

Double click the association and you’ll see the foreign key properties for the relationship:

Let’s leave this as is for now but note it’s possible to switch an existing Association between an FK or Independent Association type. In this case if you wanted to switch an Association to be Independent, you’d press the Delete button in the Referential Constraint dialog. After doing so you’d need to map the Independent Association appropriately and remove the BlogID property from the Post Entity. For now let’s move on without changing the association type.

For comparison sake, add a new entity to the design surface from the toolbox. Once done, right click on the entity and choose Add->Association:

Note that your association doesn’t have to be an FK association (association types can be mixed in the same model). Uncheck the ‘Add foreign key properties to the ‘Blog’ Entity’ check box then Click OK in the dialog. This will let us later look at the model differences between Independent and Foreign Key associations.

So far we’ve: created a WPF project, added an EDM Data Model to it, reverse engineered our Blogs and Posts tables to the model, then added a third entity and created an Independent Association with it.

XML model differences:

We’re going to briefly diverge and look at some of the differences in the XML generated for the different types of associations in our simple model. If you don’t want to do this, feel free to skip to the next section.

Right click on the EDMX file in your project and choose Open With->XML Editor. This will close the design surface and open the contents of the EDMX file in the XML Editor.

Scrolling down, the CSDL XML fragment for our Independent Association looks like this:

<Association Name="Entity1Blog">
<End Type="BlogsModel.Entity1" Role="Entity1" Multiplicity="1" />
<End Type="BlogsModel.Blog" Role="Blog" Multiplicity="*" />
</Association>

It would also have an MSL XML fragment that tells EF how to traverse the relationship if we’d mapped the new entity & association to storage objects. Since we didn’t complete the mappings the fragment is missing.

The CSDL XML fragment for our FK Associations looks like this:

<Association Name="FK_BlogPost">
          <End Role="Blogs" Type="BlogsModel.Blog" Multiplicity="1" />
          <End Role="Posts" Type="BlogsModel.Post" Multiplicity="*" />
          <ReferentialConstraint>
            <Principal Role="Blogs">
              <PropertyRef Name="BlogID" />
            </Principal>
            <Dependent Role="Posts">
              <PropertyRef Name="BlogID" />
            </Dependent>
          </ReferentialConstraint>
</Association>

Notice the ReferentialConstraint element and its contents. There is also no MSL XML fragment for the relationship.

One other thing of interest to note is the Post EntityType has this fragment in the CSDL:

Having the BlogID property on the Post Entity will allow us to directly read & manipulate its value plus use it for data binding (no more need for partial classes with this exposed as a property).

This divergence this tells us three things:

EF is handling the two different types of Associations completely differently.
If you want to dig into an EDMX and tell what types of Associations you have you can (but looking at the XML can hurt your eyes J).
Foreign Key Associations add the parent ID key properties as properties on the child object. This is useful in a number of ways.

Coding experience

Double click on the EDMX file in your project. After clicking Yes, this should reopen the EF Designer for your model. Delete Entity1 from the design surface. This should give you valid model with everything mapped (exactly what you had after running the Add->New Item wizard).

We’re going to run through a few scenarios of how to code against our model. To facilitate, add a button to your WPF form and double click it.

Scenario 1: Adding new Blog and a new Post together by setting Navigation properties.

Note this is what you used to do without FK Associations and is still the recommended way of adding two new dependent objects together. We’ll talk about why in a few minutes. Inside the button event handler add the following code:

using (BlogsEntities ctx = new BlogsEntities())
{
    Blog myBlog = new Blog { BlogID = 9, Name = "Tim's blog", Owner = "Tim" };
    Post myPost = new Post { PostID = 102, Title = "Post Title", PostContent = "TestContent",
= DateTime.Now, ModifiedDate = DateTime.Now };
    //Nav properties will work immediately.
    myBlog.Posts.Add(myPost);
    ctx.Blogs.AddObject(myBlog);
    ctx.SaveChanges();
}

This code opens a connection to the DB, creates a new blog object, creates a new post object, adds the post object to the blog object’s Posts collection, adds the new blog objects to the context, then saves the changes to the database. Note that as soon as we added the myPost object to the Posts collection of myBlog we could use the navigation properties to traverse between the objects.

Scenario 2: Adding new Blog and a new Post together but set the Post BlogID FK property instead of adding to the myBlog Posts collection.

Delete the code you just placed in the event handler and replace it with the following:

using (BlogsEntities ctx = new BlogsEntities())
            {
                Blog myBlog = new Blog{BlogID = 11, Name = "Tim's blog", Owner = "Tim"};
                Post myPost = new Post{PostID = 101,Title = "Post Title", CreatedDate = DateTime.Now,  
                    ModifiedDate = DateTime.Now, PostContent="Post Content", BlogID = 11};
                ctx.Posts.AddObject(myPost);
                ctx.Blogs.AddObject (myBlog);
                ctx.SaveChanges();
            }

Note that the navigation properties on the two new objects won’t map to each other until after SaveChanges is called. This is because the context doesn’t know about the parent object yet.

Scenario 3: Adding a new post to an already existing blog.

Replace the code in the event handler with the following:

using (BlogsEntities ctx = new BlogsEntities())
            {
                Post myPost = new Post {PostID = 102, Title = "Post Title", CreatedDate = 
DateTime.Now, ModifiedDate =DateTime.Now, PostContent = "Post Content", BlogID = 11};
                //Nav properties will work immediately b/c the Blog object already exists
                ctx.Posts.AddObject(myPost);
                ctx.SaveChanges();
            }

Note we never loaded the Blog object into memory. We knew BlogID 11 was a valid ID and set the post.BlogID property directly. Note also, the navigation property for myPost.Blog will work immediately after setting the FK property because the Blog object already exists in the context. Being able to set the property directly enables some previously difficult data binding scenarios as well as can make some coding experiences easier.

These three coding examples show different ways of creating new objects and setting up their relationships using navigation properties and FK associations.

Summary

In .NET 4.0 we’ve added support for FK Properties and FK Associations to the Entity Framework. It’s still possible to use Independent Associations and the two can be mixed in models. We’re excited to offer FK Associations because they simplify many common Entity Framework coding tasks. Hopefully this walkthrough has given you a feel for how you can use FK Associations and Properties.

As always we'd love to hear your thoughts.

Thanks,
Tim Laverty, Program Manager

Model First with the Entity Framework 4

dpblogs — Thu, 05 Nov 2009 17:52:00 GMT

In a previous post, we walked through the designer’s “out of the box” database schema generation experience. In this post, we show how some new designer features in VS2010 Beta 2 integrate with this capability, then we pop the hood and show how easy it is to replace or extend parts of the generation system.

We will begin with a simple model that contains two newly supported constructs: Employee.EmployerId is a foreign key that references Company.Id, and Person.Address is a complex type.

To show the details of the model, we also include the EDM’s XML:

   1: <Schema xmlns:annotation="http://schemas.microsoft.com/ado/2009/02/edm/annotation" Namespace="SimpleModel" Alias="Self" xmlns="http://schemas.microsoft.com/ado/2008/09/edm">   2:         <EntityContainer Name="SimpleModelContainer" annotation:LazyLoadingEnabled="true" >   3:           <EntitySet Name="Persons" EntityType="SimpleModel.Person" />   4:           <EntitySet Name="Companies" EntityType="SimpleModel.Company" />   5:           <AssociationSet Name="CompanyEmployee" Association="SimpleModel.CompanyEmployee">   6:             <End Role="Company" EntitySet="Companies" />   7:             <End Role="Employee" EntitySet="Persons" />   8:           </AssociationSet>   9:         </EntityContainer>  10:         <EntityType Name="Person">  11:           <Key>  12:             <PropertyRef Name="Id" />  13:           </Key>  14:           <Property Type="Int32" Name="Id" Nullable="false" annotation:StoreGeneratedPattern="Identity" />  15:           <Property Type="String" Name="FirstName" Nullable="false" />  16:           <Property Type="String" Name="LastName" Nullable="false" />  17:           <Property Name="Address" Type="SimpleModel.Address" Nullable="false" />  18:         </EntityType>  19:         <EntityType Name="Employee" BaseType="SimpleModel.Person">  20:           <Property Type="String" Name="Title" Nullable="false" />  21:           <NavigationProperty Name="Company" Relationship="SimpleModel.CompanyEmployee" FromRole="Employee" ToRole="Company" />  22:           <Property Type="Int32" Name="EmployerId" Nullable="false" />  23:         </EntityType>  24:         <EntityType Name="Company">  25:           <Key>  26:             <PropertyRef Name="Id" />  27:           </Key>  28:           <Property Type="Int32" Name="Id" Nullable="false" annotation:StoreGeneratedPattern="Identity" />  29:           <Property Type="String" Name="Name" Nullable="false" />  30:           <Property Type="String" Name="Address" Nullable="false" />  31:           <NavigationProperty Name="Employees" Relationship="SimpleModel.CompanyEmployee" FromRole="Company" ToRole="Employee" />  32:         </EntityType>  33:         <ComplexType Name="Address" >  34:           <Property Type="String" Name="Street" Nullable="false" Unicode="true" MaxLength="250" FixedLength="false" />  35:           <Property Type="String" Name="City" Nullable="false" Unicode="true" />  36:           <Property Type="String" Name="Zip" Nullable="false" FixedLength="true" MaxLength="5" Unicode="false" />  37:           <Property Type="String" Name="State" Nullable="true" FixedLength="false" MaxLength="100" Unicode="true" />  38:           <Property Type="String" Name="Country" Nullable="false" Unicode="false" />  39:         </ComplexType>  40:         <Association Name="CompanyEmployee">  41:           <End Type="SimpleModel.Company" Role="Company" Multiplicity="1" />  42:           <End Type="SimpleModel.Employee" Role="Employee" Multiplicity="*" />  43:           <ReferentialConstraint>  44:             <Principal Role="Company">  45:               <PropertyRef Name="Id" />  46:             </Principal>  47:             <Dependent Role="Employee">  48:               <PropertyRef Name="EmployerId" />  49:             </Dependent>  50:           </ReferentialConstraint>  51:         </Association>  52: </Schema>

Note the ReferentialConstraint element above, which defines the foreign key, and the ComplexType element that defines the Address type.

When we run the database generation script against this model, we get the following table definitions:

-- Creating table 'Persons'
CREATE TABLE [dbo].[Persons] (
    [Id] int IDENTITY(1,1) NOT NULL,
    [FirstName] nvarchar(max)  NOT NULL,
    [LastName] nvarchar(max)  NOT NULL,
    [Address_Street] nvarchar(250)  NOT NULL,
    [Address_City] nvarchar(max)  NOT NULL,
    [Address_Zip] char(5)  NOT NULL,
    [Address_State] nvarchar(100)  NULL,
    [Address_Country] varchar(max)  NOT NULL
);
GO
-- Creating table 'Companies'
CREATE TABLE [dbo].[Companies] (
    [Id] int IDENTITY(1,1) NOT NULL,
    [Name] nvarchar(max)  NOT NULL,
    [Address] nvarchar(max)  NOT NULL
);
GO
-- Creating table 'Persons_Employee'
CREATE TABLE [dbo].[Persons_Employee] (
    [Title] nvarchar(max)  NOT NULL,
    [EmployerId] int  NOT NULL,
    [Id] int  NOT NULL
);
GO

A few things to note:

The complex type was “rolled-up” into its owning type’s table, and the name of the complex property was appended to the name of the complex type properties, e.g. “Address_State”.
The foreign key property “EmployerId” was pulled through into the model, rather than a “hidden” foreign key being generated, as would have been the case for a relationship that did not specify a referential constraint. Another blog post will go into referential constraints in more detail.
The name of a table that stores instances of subtypes is composed of the name of the subtype’s set to which we append the name of the type, e.g. “Persons_Employee”.
We’ve set the Address.Zip property to be a fixed length non-unicode string, and so it is generated as a char(5).
If you do not specify unicode, the default is true. If you do not specify fixed length, the default is false. If you do not specify max length, the default is “max”.
Property Persons.Id’s StoreGeneratedPattern property is set to “Identity”, which makes it an IDENTITY(1,1) in the database.
Not shown in the above script are the key and constraint definitions, which are similar to those shown in the previous post.

Now, let’s take a look at how model first actually works, so we can see how to change what it does. Click on any empty area in the designer surface to see the top-level properties of the model:

Of interest to us are two of the three “Database Script Generation” properties:

Database Generation Workflow – Controls the overall process by which the conceptual model is translated into a database script. The default workflow is “TablePerTypeStrategy.xaml”.

DDL Generation Template – Is called by the default database generation workflow to transform the generated database model to DDL. More on this below.

First, let’s take a look at TablePerTypeStrategy.xaml – it is located in Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\Entity Framework Tools\DBGen. If we open this file in Visual Studio, we see that it is a Workflow Foundation (WF) workflow:

The first activity, “CsdlToSsdlAndMslActivity” generates the store model (SSDL) for the EDM, and then generates the mappings (MSL) that connect the two. It has the following properties:

The “MslOutputGeneratorType” specifies a class that generates MSL based on a table-per-type mapping strategy. The “OuputGeneratorType” generates SSDL. Rather than replacing this activity and all of the plumbing it provides, you can replace these two classes with your own if you want to change the mapping strategy or otherwise alter the system. In future posts, we will provide examples and more details on the specifics of these classes. You can also replace this entire activity with your own activity that fills the MslOutput and SsdlOutput parameters using any mechanism you wish.

The second activity, “SsdlToDdlActivity” generates DDL from the store model, and it does this by using a T4 template, which makes for very easy customization. In the same folder that TablePerTypeStrategy.xaml is installed, there is also SSDLToSQL10.tt which generates TSQL DDL. Templates for supporting other database platforms will become available in the future from Entity Framework provider writers. T4 templates are simple text files whose intent is to transform some sets of inputs, in this case, SSDL, to some output, DDL in this case. A full description of the template is outside the scope of this post, but we’ll take a brief look at one section of the template, the one that generates the table definitions. To help with readability, we have bolded those parts of the script that generate DDL:


-- --------------------------------------------------
-- Creating all tables
-- --------------------------------------------------

<#
    foreach (EntitySet entitySet in Store.GetAllEntitySets())
    {
        string schemaName = Id(entitySet.GetSchemaName());
        string tableName = Id(entitySet.GetTableName());
#>
-- Creating table '<#=tableName#>'
CREATE TABLE <# if (!IsSQLCE) {#>[<#=schemaName#>].<#}#>[<#=tableName#>] (
<#
        for (int p = 0; p < entitySet.ElementType.Properties.Count; p++)
        {
            EdmProperty prop = entitySet.ElementType.Properties[p];
#>
    [<#=Id(prop.Name)#>] <#=prop.ToStoreType()#> 
<#=WriteIdentity(prop, targetVersion)#> <#=WriteNullable(prop.Nullable)#>
<#=(p < entitySet.ElementType.Properties.Count - 1) ? "," : ""#>
<#
        }
#>
);
GO
<# 
    } 
#>

As you can see, the template iterates over the objects in the SSDL and the translation is fairly straightforward.

The experience when additional templates and workflows are installed is tremendously improved relative to Visual Studio 2010 Beta1. To demonstrate this, copy and paste TablePerTypeStrategy.xaml and rename it to CustomStrategy.xaml. Copy and paste SSDLToSQL10.tt and rename it SSDLToCustomDDL.tt. When we switch back to the designer and look at the designer property sheet, we will see these new files appear in the drop downs for the workflow and template properties:

And:

So now you can mix and match two different strategies with two different DDL generators, for example, you could mix and match a table-per-hierarchy strategy with either a SQL or a Firebird database schema template.

But there is a problem: Installing templates and workflows in the Visual Studio path is often neither advisable nor possible. So we’ve provided two additional install locations. The first is %localappdata%\Microsoft\[AppId]\10.0\Extensions\Microsoft\Entity Framework Tools\DBGen where [AppId] depends on your Visual Studio SKU, for example:

D:\documents and settings\sqlcl02\Local Settings\Application Data\Microsoft\VisualStudio\10.0\Extensions\Microsoft\Entity Framework Tools\DBGen.

Copy the same files to this folder and rename them to CustomUserStrategy.xaml and SSDLToCustomUserDDL.tt and they will immediately show up:

Finally, if you want a custom workflow or template specifically for your project, copy the template or workflow to your project and type in the path. For example:

Make sure that the (VS) or (User) postfixes are not in the path.

In summary, it is very easy now to mix and much strategies and target database DDL formats, as well as to install your own custom workflows and templates. In a future post, we will walk through examples of custom strategies and templates.

Noam Ben-Ami
Program Manager,
Entity Designer

ADO.Net Entity Framework Feature Community Technology Preview Released!

dpblogs — Wed, 04 Nov 2009 23:25:00 GMT

We’ve released an update of the Entity Framework Feature CTP! The updated CTP includes many requested improvements and added features we’ve gathered from the community since the release of our first CTP and supports installation with Visual Studio 2010 Beta 2 as well. Some of the improvements and added features added since the last CTP include:

Enhancements to Code Only, including-

Fine Grained Control over model

Specify Navigation Property Inverses
Specify Property Facets
Complex Types

Customizable Mappings

Change Table Name, Column Names
Specify Custom Inheritance Strategy
Entity Splitting
Join Table Mapping

The Self-Tracking Entities template, allowing code generation of EF entities that facilitate ease of use in WCF/N-Tier scenarios-

Foreign Key associations: Self-Tracking Entities can take advantage of the Foreign Keys in the Model feature added in .NET 4.0 Beta 2 and can contain both navigation properties and Foreign Key properties for the same association. Fix-up logic has been tidied up to be aware of FKs.
Support for Silverlight 3: Generated entity types can be compiled to target Silverlight 3 and can be used in combination with Silverlight-enabled WCF services.
Databinding support: Generated entity types now implement INotifyPropertyChanged and use ObservableCollections allowing them to work better with WPF and Silverlight databinding.
Richer concurrency control support: Self-Tracking Entities now support the same variations for optimistic concurrency control as the Entity Framework. Original values are preserved for all required properties according to their concurrency mode in the Entity Data Model.
Improved independent association support: The approach for managing entities with dangling references has been reengineered to avoid unnecessary database round-trips.
New and improved methods: AcceptChanges, StartTracking, StopTracking where added and the existing MarkAsX methods are now extension methods.
Generated code improvements and refactoring: The ApplyChanges implementation has been moved to the Context template so the template has no binary dependencies besides Entity Framework. Generated code for entity types have been refactored.

Over the next few weeks we’ll be posting walkthroughs that dive deeper into each feature. We also have a few features we plan to release in CTP format in the next few months including an updated POCO code gen template. Your comments and feedback are most welcome; please give us your comments here, our design blog, through Connect or through the ADO.NET Entity Framework pre-release forums. Please stay tuned for more news!

Thanks!

The Entity Framework Team

Upcoming ADO.Net Entity Framework Feature Community Technology Preview for Visual Studio 2010 Beta 2

dpblogs — Tue, 27 Oct 2009 00:42:03 GMT

Just a quick note to let everyone know we’re actively working on an updated Feature CTP for Visual Studio 2010 Beta 2. As soon as the bits are ready we’ll release and post here again. Currently the plan is to include some great new enhancements for Code Only and Self-Tracking Entities with the bits.

Please be patient & stay tuned!

Thanks!
The Entity Framework Team

New features in Entity Framework impacting providers

dpblogs — Thu, 22 Oct 2009 19:31:10 GMT

Providers for Entity Framework 3.5 will work unmodified against Entity Framework 4.0. Also, most of the features and improvements introduced in Entity Framework 4.0 will just work against an Entity Framework 3.5 provider. However, there are several features that require provider changes to be enabled. Below is a list of these features along with short description of the changes required to enable them.

New features in Entity Framework Runtime impacting providers

New EDM (Canonical) Functions

In Entity Framework 3.5 there is a set of EDM (canonical) functions, which are defined by the Entity Framework and should be supported by all store providers. In Entity Framework 4.0 we are expanding that set by adding a number of new string, aggregate, mathematical, and date/time functions. Providers would need to add support for these new functions, listed below.

Date/Time functions

CreateDateTime(year, month, day, hour, minute, second)

CreateDateTimeOffset(year, month, day, hour, minute, second, tzoffset)

CreateTime(hour, minute, second)

DayOfYear(expression)

TruncateTime(expression)

AddYears(expression, number)

AddMonths(expression, number)

AddDays(expression, number)

AddHours(expression, number)

AddMinutes(expression, number)

AddSeconds(expression, number)

AddMilliseconds(expression, number)

AddMicroseconds(expression, number)

AddNanoseconds(expression, number)

DiffYears(startExpression, endExpression)

DiffMonths(startExpression, endExpression)

DiffDays(startExpression, endExpression)

DiffHours(startExpression, endExpression)

DiffMinutes(startExpression, endExpression)

DiffSeconds(startExpression, endExpression)

DiffMilliseconds(startExpression, endExpression)

DiffMicroseconds(startExpression, endExpression)

DiffNanoseconds(startExpression, endExpression)

Math Functions

Power(value, exponent)

Truncate(value, digits)

Round(value, digits)

Aggregate functions

StDevP(expression)

Var (expression)

VarP (expression)

String functions

Contains(string, target)

StartsWith(string, target)

EndsWith(string, target)

Exposing provider functions in LINQ To Entities ()

Entity Framework 3.5 customers preferring to write their queries using LINQ often hit a limitation on the range of functions and query patterns supported in LINQ to Entities. To address that issue, we are providing an extensibility mechanism that can be used to map CLR methods to provider and to EDM functions.

The basis of the extensibility mechanism is a new method-level attribute that carry function mapping information. Here is the basic signature of the attribute’s constructor:

public EdmFunctionAttribute(string namespaceName, string functionName)

The namespaceName parameter indicates the namespace for the function in metadata (i.e. “EDM” or “SQLSERVER”, or other store provider namespace). The functionName parameter takes the name of the function itself.

We encourage providers to expose classes that represent their store functions, so that when their provider is used they are readily available via LINQ to Entities. Here are some recommendations with regards to which functions to expose and how:

Which functions to expose?

These that provide functionality not already provided via the base class library functions supported by LINQ to Entities or the EDM functions.

Overloads – Provide enough for pleasant user experience

Specify the scalar arguments and return types as nullable
Provide overloads for nullable and non-nullable collection element for collection arguments

Proxies Implementation

Trigger direct evaluation over the input ObjectQuery for methods taking collection arguments
Throw NotSupportedException in all other cases

Example:

[EdmFunction(“SampleSqlServer", "CHECKSUM_AGG")] 
        public static Int32? ChecksumAggregate( IEnumerable<Int32> arg)  { 
            ObjectQuery<Int32> objectQuerySource = arg as ObjectQuery<Int32>; 
            if (objectQuerySource != null)  { 
                return ((IQueryable)objectQuerySource).Provider
                      .Execute<Int32?>(Expression.Call((MethodInfo)MethodInfo
                        .GetCurrentMethod(),Expression.Constant(arg))); 
            } 
     throw new NotSupportedException("This function can only be invoked from LINQ 
                            to Entities."); 
        } 
         [EdmFunction(“SampleSqlServer", "CHECKSUM_AGG")]
         public static Int32? ChecksumAggregate( IEnumerable<Int32?> arg)  { 
        { …. }

Translating String.StartsWith, String.EndsWith and String.Contains to LIKE in LINQ to Entities

In Entity Framework 3.5, in LINQ to Entities we provided support for the String.StartsWith, String.EndsWith and String.Contains and we translated them to canonical functions in the following way:

Boolean String.Contains(String value) -> IndexOf(this, value) > 0

Boolean String.EndsWith(String value) -> Right(this, length(value)) = value

Boolean String.StartsWith(String value) -> IndexOf(this, value) = 1

However, due to the fact that these translations result in inability to utilize indexes on the tested string on some providers and that users found the resulting SQL more difficult to read, we are providing an option to translate these into DbLikeExpression, i.e:

Boolean String.Contains(String value) -> DbLikeExpression(this, ‘%’ + value’ + ‘%’, escapeCharacter)

Boolean String.EndsWith(String value) -> DbLikeExpression(this, ‘%’ + value’, escapeCharacter)

Boolean String.StartsWith(String value) -> DbLikeExpression(this, value’ + ‘%’’, escapeCharacter)

Here, value’ is the equivalent of value but with the wildcard characters and the escape character escaped, and escapeCharacter is the escape character. This translation only happens if value is a constant or a parameter.

Because different providers may have different wildcard characters, in order for this translation to happen provider writers need to override and provide implementations of the following methods that have been introduced in DbProviderManifest:

public virtual bool SupportsEscapingLikeArgument(out char escapeCharacter)

and

public virtual string EscapeLikeArgument(string argument)

The default implementation of SupportsEscapingLikeArgument returns false, thus if provider writers choose to not override these methods, they will still continue to get the old translations of these functions.

Please note that this feature is not any public release yet, but it will be available in .NET 4.0.

New Features in Entity Designer Impacting Providers

Model First

In Visual Studio 2010 the designer can generate mappings, storage model, and database DDL from a conceptual model. This feature is called “Model First.” In order to make this feature highly customizable, it is implemented using a Workflow Foundation (WF) workflow that users can replace. The workflow contains two activities: One which generates MSL and SSDL, and another which generates DDL.

DDL generation is done by a T4 template which processes the SSDL generated by the previous workflow activity. Providers who wish to allow users to generate DDL for databases other than SQL Server will need to install a T4 DDL generation template in:

%localappdata%\Microsoft\[AppId]\10.0\Extensions\Microsoft\Entity Framework Tools\DBGen

Where [AppId] signifies either VisualStudio, VBExpress, VCSExpress, or VNSExpress depending on what SKU the user is using at the moment. Note that when searching for templates, the Entity Framework Designer does not do recursive searches.
Templates from this path have “(User)” post-pended to their name.

For reference, built-in templates are installed in:

%programFiles%\Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\Entity Framework Tools\DBGen

Templates from this path have “(VS)” post-pended to their name.

Here is a screenshot. In this case, a template called “CustomDDL.tt” was added to the “(User)” path referenced above. (In the case of this specific machine, that path was D:\Documents and Settings\SQLCL02\Local Settings\Application Data\Microsoft\VisualStudio\10.0\Extensions\Microsoft\Entity Framework Tools\DBGen)

The same rules also apply to workflows. In this screenshot, we added a workflow called “CustomStrategy.xaml” to the “(User)” directory:

In both cases, there is no need to restart VS, as the templates and workflows show up immediately.

Please note that this feature is not any public release yet, but it will be available in Visual Studio 2010.

Thank you!

Kati Iceva
Software Development Engineer, Entity Framework

Noam Ben-Ami
Program Manager, Entity Designer

VS2010 and .NET Framework 4 Beta 2 Announced!

dpblogs — Tue, 20 Oct 2009 00:56:00 GMT

Visual Studio 2010 and the .NET Framework 4 Beta 2 are now available for download by MSDN subscribers and will available to the rest of the world on Wednesday. Beta 2 as well the VS2010 Launch date of March 22, 2010 were announced this morning on Soma’s blog.

Included in Beta 2 are some great new features and updates for the Entity Framework 4 and ADO.NET Data Services 4.

Entity Framework 4 Beta 2

In addition to all of the new EF 4 features that were included in Beta 1, Beta 2 also includes:

Foreign Keys: EF 4 now includes a new type of associations (Foreign Key Associations) that allow you to have Foreign Key properties on your entities. Foreign Key Associations simplify some key scenarios such as Data binding and can be included or excluded when using the model creation and update wizards. Independent Associations supported in .NET 3.5 remain as is, but Foreign Key Associations become the default type of association going forward. Referential integrity constraints can now be created from the designer.
Improvements to POCO Support: Fix-up of changes to navigation properties and FKs is now performed automatically on DetectChanges and SaveChanges. It is also now possible to declare collections as ISet<T> in POCO objects. When either ICollection<T> or ISet<T> is used, the default collection type materialized is HashSet<T>, which among other advantages does not allow duplicate entries.
Lazy Loading on by Default in new Models: When you create a new model in VS 2010, you get generated ObjectContext types that have lazy loading turned on by default (context.ContextOptions.LazyLoadingEnabled = true is placed in the constructor). This simplifies the default experience of working with objects. Of course, it is still possible to turn it off and do explicit and eager loading.
EntityDataSource support for Query Extenders and POCO: EntityDataSource now includes support for ASP.NET Query Extenders and POCO entities. Query Extenders are a new addition to ASP.NET that allow you to have more control over the data retrieval query of a Data Source, leveraging the LINQ capabilities of Entity Framework.
Support for Binary Keys: The EF now allows you to have binary property types (and varbinary storage columns) mapped as Entity Key / Foreign Key.
ObjectMaterialized event: It is now possible to write logic that is executed immediately after an object has been materialized. This event is raised after the scalar, complex and reference properties are populated (collections are filled afterwards).
Object Services API improvements to enable N-Tier and Self Tracking Entities: ObjectStateEntry.GetUpdatableOriginalValues method provides a access to an updatable data record that you can use to establish the original state of an entity at the property level.
Improvements to the generated SQL: Numerous simplifications and improvements of the SQL generated when querying using the Entity Framework have been done including: removal of joins, better translation of certain functions, elimination of constants used for internal purposes, which results in removing of levels of nesting and others.
Navigation Property Management: Users can now delete and add navigation properties, enabling the creation of one-way associations.
Improved Database Generation: SQL CE support has been added. In addition, customization of database generation is significantly simpler now, with the ability to specify the DDL generation template to be used, as well as automatic discovery of installed database generation workflows and templates.
New Extensibility APIs: New APIs enable users to add custom properties to any object in the model and have those properties show up in the property sheet. In addition, other extension points allow for customization of models on load, save, and after the model creation and update wizards run.
Generation of Complex Types from Stored Procedures: When creating a function import from a function, the UI can now detect the columns returned from a stored procedure and create a corresponding complex type. In addition, existing complex types can be updated when a stored procedure definition changes.
Greatly Improved Facet Management: The designer can now distinguish between empty and blank values, and various property facet defaults are handled more cleanly and consistently.
LINQ to Entities improvements: Support for recognizing additional patterns in queries, simplification of EntityFunctions and SqlFunctions static classes and OrderBy is now lifted over filters and other operations.

ADO.NET Data Services 4 Beta 2

Projections: This ADO.NET Data Services URI format has been extended to express projections (i.e. you can now work with a subset of the properties of an entity). Beta 2 includes both server and client library (including LINQ support) support for projections.
Data Binding: The data services client library for the .NET Framework 3.5 SP1 and Silverlight2 has been extended to support two-way data binding for WPF and Silverlight based applications.
Row Count: One scenario we heard a ton of feedback on after shipping V1 of ADO.NET Data Services in the .NET Framework 3.5SP1 is the ability for the a client of a data service to determine the total number of entities in a set without having to retrieve them all. To address this need, we have extended the data services addressing scheme to allow a client to obtain this type of information without having to download all the entities in a set.
Feed Customization (aka "Web Friendly Feeds"): A common ask we have received is to provide the ability to customize how entities are mapped into the various elements of an AtomPub feed. This feature does just that by providing a data service author declarative control over how the data service runtime maps the properties of an entity (e.g. a Customer, Order, etc) to the elements of a feed.
Server Driven Paging (SDP): This one is best described by example. If you had a data service that exposes photos, you likely want to limit the total number of photos a single request to the service can retrieve because the total collection of photos may be very large. This feature allows a service author to set per collection limits on the total number of entities returned for each request. In addition to limiting the number of photos returned per request, the server provides the client a "next link" which is simply a URI specifying how to continue retrieving the rest of the entities in the collection not returned by the first request. For those familiar with AtomPub, this feature adds support for AtomPub <link rel="next" ...> elements to the data service runtime.
Enhanced BLOB Support: This feature enhances the BLOB support provided in V1 to enable data services to stream arbitrarily large BLOBs, store binary content separate from its metadata, easily defer the loading of BLOB content when its metadata is requested, etc.
Request Pipeline: We have started to expose events throughout the data services server request processing pipeline. For this release we’ll expose request level events and in future we’ll look to expose more fine grained events based on your feedback. The goal of exposing our processing pipeline is to allow services further transparency into a data service such that a service author can do things such as setting HTTP response cache headers, wrapping interceptor processing and data service request processing in a single transaction, etc.
New "Data Service Provider" Interface for Custom Provider Writers: As the data services runtime has evolved, so has the number of ways people want to plug data into the data service framework. In V1, two methods (Entity Framework and arbitrary .NET classes) were supported to enable a data service to interact with various diverse data sources. To address another class of environments and data sources we have introduced a way to write a "custom" provider for those cases when the previous two provider models don't meet your needs.

Thank you,

Elisa Flasko
Program Manager,
Data & Modeling Group

Improvements to the Generated SQL in .NET 4.0 Beta1

dpblogs — Thu, 06 Aug 2009 07:31:00 GMT

One of the things that we are continuously working on improving is the quality and the readability of the SQL generated when querying using the Entity Framework. We have already made some improvements in .NET 4.0 Beta 1, and we are working on more for .NET 4.0 post Beta 1.

Most of the improvements we have made are in the Entity Framework query pipeline as opposed to specific SqlClient changes. Because the changes were in the query pipeline, this results in simpler output command trees and thus would affect the SQL generated by any backend, not only by our SQL Server provider.

Below we’ve highlighted the biggest improvements available in Beta 1 and then the ones that will be in the next public release of .NET Framework 4.0. Unless otherwise noted, the examples are based on the NorthwindEF Entity Data Model provided with the ADO.NET Entity Framework Query Samples (http://code.msdn.microsoft.com/EFQuerySamples). Also, in all the examples below, the parts of the SQL query highlighted in red have been removed and those in yellow have been added.

1. Avoid projecting all columns in nested queries

In some nested queries in the generated SQL, we used to explicitly project all columns that are brought in scope by the corresponding FROM clause, like the fragments marked red in the sample generated SQL below. Now, we instead only project the columns that are later referenced. This improvement is specific to our T-Sql Generation in SqlClient.

Example:

var query = (from p in context.Products
                 orderby p.ProductName
                 select p.ProductName)
                 .Skip(2).Take(2);

2. Avoiding Unnecessary IS NULL Checks in LINQ to Entities Queries

When translating joins in LINQ to Entities, we check whether the columns on which the join is specified are equal or both are null. In the cases when one (or both) of the columns is non-nullable, the “IS NULL” check is redundant and has been removed. This improvement is in the Entity Framework query pipeline.

Example:

var query = context.Products.Join(
context.Categories, 
p => p.ProductID,
c => c.CategoryID,
(p, c) => new { p.ProductID, c.CategoryID, p.ProductName, c.CategoryName });

3. Additional join elimination: Eliminate parent table in “Child Left Outer Join Parent”

In scenarios where a given table is explicitly or implicitly (e.g. via a navigation property) joined to a parent table (i.e. table to which it has a foreign key constraint) we were not eliminating the parent table even if the only columns referenced from that table were the columns comprising the primary key and could have been reused from the child table. (Note: The parent-child terminology assumes that there is foreign key relationship between these tables specified in the SSDL). This improvement is in the Entity Framework query pipeline.

Example:

Note: For this example the highlighted following fragments need to be added to the SSDL in the ADO.NET Entity Framework Query Samples noted above:

string entitySQL = "SELECT p.Category.CategoryId FROM Products as p";

For LINQ to Entities in .NET Framework 3.5 SP1 the equivalent query resulted in the same generated query for Entity SQL shown in the After sample.

4. Using IN instead of nested OR’s

Comparing a column to multiple values, either as a result of the explicit use of Entity SQL’s IN expression or as result of internally generated checks over a type discriminator value used to result in nested OR expressions in the generated SQL. Now, we are instead producing an IN expression. This improvement is specific to our T-Sql Generation in SqlClient.

For example:

string entitySQL = "SELECT p FROM Products as p where p.ProductId in {1, 2, 3, 4, 5}";

Starting with.NET 4.0 Beta 1 Entity Framework, a LINQ version of this query is also supported:

var query = context.Products.Where(p => new int[] { 1, 2, 3, 4, 5 }.Contains(p.ProductID));

The resulting generated TSQL also takes advantage of this improvement:

5. Translating more of LINQ’s GroupBy operator into GROUP BY

LINQ’s GroupBy operator is richer then SQL’s group by clause. For example, in addition to aggregates over a group it can return the entire group. When translating LINQ’s GroupBy, we try to recognize if it can be expressed by SQL’s group by (i.e. DbGroupByExpression without a group partition), otherwise we translate it into a more complex expression involving a join to the input table.

In Entity Framework 4.0 Beta1 we have expanded the cases that we are able to translate into SQL’s GroupBy. This improvement is in the Entity Framework query pipeline.

Example:

var query = context.Products.Where(p => p.ProductID < 4)
            .GroupBy(p => p.ProductID, p => p.ProductID, (key, group) => 
            new { Key = key, Max = group.Max() });

6. Avoiding “Cast (1 as bit)”, using “1” instead

In cases when we need an internally generated constant, like when translating EntitySQL Exists expression or IEnumerable.Count in LINQ to Entities, we previously used “true”, which translates to “cast(1 as bit)” on SQL Server. We now instead use the integer constant 1, which translates to “1”. This improvement is in the Entity Framework query pipeline.

var query = context.Categories.Select(c => new { c.CategoryID, count = c.Products.Count() });

7. Simplifying some queries that go against a subtype in a type hierarchy

In some cases when a query is only interested in entities of a particular subtype (specified via OfType or IsOf for example) we used to query using a view over the base type and thus possibly generate some unnecessary case statements to also check for the types that are not of the desired subtype. This improvement allows us to use the simplified query view that is only over the desired subtype and thereby generate a simpler provider query.

The following example that illustrates that is over a schema with a TPH mapping with the inheritance hierarchy as shown in the figure. This improvement is in the Entity Framework query pipeline.

var query = this.tph.Entities.OfType<Tph.Derived1>().GroupBy(d2 => d2).Select(p => p.Max(g => g.Id));

Improvements in .NET 4.0 post Beta1

Here is a quick overview of the improvements that we are working on for this release but are not included in Beta1:

Elimination of null sentinels – In the output query there is often the constant 1 projected both in nested and in top-most queries. By avoiding it and reusing other columns for the same purpose in many cases we are able to avoid levels of nesting.
Additional Join Elimination
Use of Inner Joins instead of Left Outer Joins when possible
Provide mechanism for efficient queries on non-Unicode columns – We now generate non-unicode constants and parameters when these are used in LINQ to Entities queries in comparisons with non-unicode columns. This allows indexes to be used by SQL Server.
Improve the translation of the functions String.StartsWith, String.Contains and String.EndsWith in LINQ to Entites to use LIKE.
Improve the translation of the canonical functions StartsWith, Contains and EndsWith to use LIKE (these canonical functions became available in .NET 4.0 Beta1)
Collapse multiple Case statements into one

Kati Iceva
Software Development Engineer, Entity Framework

Customizing T4 Templates

dpblogs — Wed, 22 Jul 2009 18:26:00 GMT

The whole point of us using T4 for code-generation is that it makes it easy to customize the generated entities.

A good example might be to support some sort of validation when setting string properties.

To illustrate lets re-use the code from the recent POCO template walkthrough post.

In that project there is a Person entity which has an EmailAddress property. If we want this property to only accept valid EmailAddresses, we need to modify the generated code to do some validation, probably using a Regular Expression.

One way to do this is to use Structural Annotations in conjunction with a customized T4 template. Structural Annotations, if you aren’t familiar with them, allow you to put custom annotations in the model and retrieve them using the Entity Frameworks metadata APIs. So we can use Structural Annotations to embed our Regular Expressions right in the model, and access them from inside the T4 template, which already uses the Entity Framework’s metadata APIs.

To do this, the first step is to register a custom namespace in the schema element of the CSDL portion of the EDMX file:

<edmx:ConceptualModels>
      <Schema xmlns="http://schemas.microsoft.com/ado/2008/09/edm" xmlns:store="http://schemas.microsoft.com/ado/2007/12/edm/EntityStoreSchemaGenerator" 
              xmlns:Regex="http://Regex"
              Namespace="Blogging" Alias="Self">
        <EntityContainer Name="BloggingContainer" >
          <EntitySet Name="Blogs" EntityType="Blogging.Blog" />
          <EntitySet Name="People" EntityType="Blogging.Person" />
          <EntitySet Name="Entrys" EntityType="Blogging.Entry" />

With that in place we can put the Regular Expression on our EmailAddress property by putting some custom XML, in the namespace we just registered, inside the corresponding Property.

Something like this:

<EntityType Name="Person">
          <Key>
            <PropertyRef Name="ID" /></Key>
          <Property Type="Int32" Name="ID" Nullable="false" store:StoreGeneratedPattern="Identity" />
          <Property Type="String" Name="Firstname" Nullable="false" MaxLength="50" />
          <Property Type="String" Name="Surname" Nullable="false" MaxLength="50" />
          <Property Type="String" Name="EmailAddress" Nullable="false" MaxLength="100">
            <Regex:Expression ErrorMessage="A valid emailaddress is required">^(([A-Za-z0-9]+_+)|([A-Za-z0-9]+\-+)|([A-Za-z0-9]+\.+)|([A-Za-z0-9]+\++))*[A-Za-z0-9]+@((\w+\-+)|(\w+\.))*\w{1,63}\.[a-zA-Z]{2,6}$</Regex:Expression>
          </Property>
          <NavigationProperty Name="Entries" Relationship="Blogging.PostPerson" FromRole="Person" ToRole="Post" />
          <NavigationProperty Name="Blogs" Relationship="Blogging.PersonBlog" FromRole="Person" ToRole="Blog" />
        </EntityType>

I’m using this regular expression: “^(([A-Za-z0-9]+_+)|([A-Za-z0-9]+\-+)|([A-Za-z0-9]+\.+)|([A-Za-z0-9]+\++))*[A-Za-z0-9]+@((\w+\-+)|(\w+\.))*\w{1,63}\.[a-zA-Z]{2,6}$” but the of course any Regular expression would work.

The ErrorMessage attribute is there so it can be used as the Message of an Exception in the event of a non-match.

Once you have this Regular Expression in your model, you can modify your T4 template (in this case the template is the POCO Types template that ships with the Microsoft Entity Framework Feature CTP 1), so that it looks for and handles Regular Expressions.

So rather than this:

public string EmailAddress { get; set; }

We want to produce something like this:

private string _EmailAddress; 
        private Regex _EmailAddressRegex = new Regex(@"^(([A-Za-z0-9]+_+)|([A-Za-z0-9]+\-+)|([A-Za-z0-9]+\.+)|([A-Za-z0-9]+\++))*[A-Za-z0-9]+@((\w+\-+)|(\w+\.))*\w{1,63}\.[a-zA-Z]{2,6}$");
        public string EmailAddress 
        { 
             get { return _EmailAddress; }
             set 
             { 
                if (value != null && !_EmailAddressRegex.IsMatch(value))
                   throw new Exception("A valid emailaddress is required");
                _EmailAddress = value; 
             }
        }

To make this change to the generated code there are three parts of the template that need to change:

A change to the part of the template that emits primitive properties:

Some utility functions (used above), that use the Entity Framework Metadata APIs to access the Regular Expression:

MetadataProperty GetRegexProperty(EdmProperty property)

{

return property.MetadataProperties.FirstOrDefault(mp => mp.Name == "http://Regex:Expression");

}

bool HasRegex(EdmProperty property)

{

return GetRegexProperty(property) != null;

}

System.Xml.Linq.XElement GetRegexElement(EdmProperty property)

{

var prop = GetRegexProperty(property);

if (prop == null) throw new Exception("No Regex found");

var node = prop.Value as System.Xml.Linq.XElement;

if (node == null) throw new Exception("No Regex found");

return node;

}

string GetRegexErrorMessage(EdmProperty property)

{

var node = GetRegexElement(property);

var messageAttr = node.Attribute("ErrorMessage");

if (messageAttr == null)

return "Regular Expression check for " + property.Name + " failed.";

else

return messageAttr.Value;

}

And something to make the generated classes use System.Text.RegularExpressions:

void WriteHeader(string namespaceName, CodeGenerationTools code, params string[] extraUsings)

{

CodeRegion region = new CodeRegion(this);

//------------------------------------------------------------------------------

// <auto-generated>

// This code was generated from a template.

// Changes to this file may cause incorrect behavior and will be lost if

// the code is regenerated.

// </auto-generated>

//------------------------------------------------------------------------------

using System;

using System.Collections.Generic;

using System.Text.RegularExpressions;

Now if you replace your POCO Types T4 template with the template attached below, the generated classes will enforce any regular expressions you add to your model. To find the changes in the T4 template just find ‘Regex’ and ‘RegularExpression’.

This is just scratching the surface though, you could really run with this idea, perhaps producing DataAnnotations directly from the EDM so that they can be used by frameworks like Dynamic Data?

Your options are almost unlimited.

Have fun!
Alex James
Entity Framework Team

Entity Framework Provider for Synergy DBMS

dpblogs — Wed, 01 Jul 2009 20:08:00 GMT

It is my pleasure to announce that Synergex has released their Synergy/DE Data Provider for .NET with support for the Entity Framework. The Synergy/DE Data Provider for .NET enables developers to access Synergy DBMS databases from .NET applications and Visual Studio 2008 and to take advantage of the Entity Framework and LINQ.

For more information, see the Synergex announcement page about the release here. Congratulations to everyone at Synergex who contributed to the release!

Enjoy!

Elisa Flasko
Program Manager, Data Programmability

Feature CTP Walkthrough: Code Only for Entity Framework

dpblogs — Tue, 23 Jun 2009 00:58:00 GMT

Overview

This post covers a new Entity Framework extension called Code Only. Code Only allows you to use the Entity Framework using POCO entities and without an EDMX file, i.e. with ‘Code Only’. This walkthrough shows you how to use the first release of Code Only, which is a very early and limited preview of our long term plans for Code Only. To get an idea of where Code Only is heading look at this post on the EF Design Blog.

Code Only ships as part of the Microsoft Entity Framework Feature CTP 1. The CTP works on top of the latest version of the Entity Framework released as part of .NET 4.0 Beta1, and you can download it here.

Requirements

This walkthrough requires Visual Studio 2010 Beta 1
The Microsoft Entity Framework Feature CTP1, that can be downloaded from here.
A local SQL Server 2008 Express instance has to be installed as SQLEXPRESS. Otherwise, changes to the connection string in the web.config file or regenerating the database from the included SQL scripts may be necessary.
Download and extract the initial solution attached to this post.

Feature CTP Walkthrough: POCO Templates for Entity Framework

dpblogs — Tue, 23 Jun 2009 00:38:00 GMT

Overview

This post covers a new Entity Framework extension called POCO Templates. POCO Templates is a T4 template and associated include files, that allow you to generate POCO classes or Plain Old CLR Objects for use with the Entity Framework. T4 templates are used because T4 makes it easy to customize your Entities.

POCO Templates ship as part of the Microsoft Entity Framework Feature CTP 1. The CTP works on top of the latest version of the Entity Framework released as part of .NET 4.0 Beta1, and you can download it here.

Requirements

This walkthrough requires Visual Studio 2010 Beta 1
The Microsoft Entity Framework Feature CTP1, that can be downloaded from here.
A local SQL Server 2008 Express instance has to be installed as SQLEXPRESS. Otherwise, changes to the connection string in the web.config file or regenerating the database from the included SQL scripts may be necessary.
Download and extract the initial solution attached to this post.

Feature CTP Walkthrough: Self Tracking Entities for Entity Framework

dpblogs — Mon, 22 Jun 2009 23:16:00 GMT

Overview

The goal of the following walkthrough is to demonstrate a basic scenario for the use of Self Tracking Entities with Entity Framework. Self Tracking Entities is a feature included in the Microsoft Entity Framework Community Technology Preview 1, which consists of a new API added to the ObjectContext and ObjectSet classes (implemented as extension methods for the Feature CTP) and a code-generation template based on Text Template Transformation Toolkit (T4) technology included in Visual Studio.

Self Tracking Entities are domain classes that do not have any dependency on Entity Framework but include mechanisms that allow them to track changes on scalar and complex property values, references and collections within themselves, independently of the Entity Framework.

The first scenario we are trying to address with Self Tracking Entities is one in which a WCF service exposes a series of operations that return entity graphs, then a client application can manipulate that graph and submit the modifications to a service operation that validates and performs the updates to a database store using Entity Framework.

Requirements

This walkthrough requires Visual Studio 2010 Beta 1
The Microsoft Entity Framework Feature CTP1, that can be downloaded from here.
A local SQL Server 2008 Express instance has to be installed as SQLEXPRESS. Otherwise, changes to the connection string in the web.config file or regenerating the database from the included SQL scripts may be necessary.
Download and extract the initial solution attached to this post.

POCO in the Entity Framework : Part 2 – Complex Types, Deferred Loading and Explicit Loading

dpblogs — Thu, 28 May 2009 19:03:00 GMT

In my post last week on the POCO Experience in Entity Framework, I covered the fundamentals of POCO support in Entity Framework 4.0. In this post, I’ll cover a few more aspects related to POCO.

Complex Types

Complex Types are supported in POCO just like they are with regular EntityObject based entities. All you have to do is declare your complex types as POCO classes and then use them for declaring the complex type based properties on your POCO entities.

As an example, here’s an InventoryDetail complex type to represent a part of my Product entity:

public class InventoryDetail
{
    public Int16 UnitsInStock { get; set; }
    public Int16 UnitsOnOrder { get; set; }
    public Int16 ReorderLevel { get; set; }
}

My Product class now has been modified to include a property of this type to group the inventory detail fields:

public class Product
{
    public int ProductID { get; set; }
    public string ProductName { get; set; }
    public int SupplierID { get; set; }
    public string QuantityPerUnit { get; set; }
    public decimal UnitPrice { get; set; }
    public InventoryDetail InventoryDetail { get; set; }
    public bool Discontinued { get; set; }
    public Category Category { get; set; }
}

You can then do everything you are used to doing with Complex Types. Here’s how I use it in a query:

var outOfStockProducts = from c in context.Products
                         where c.InventoryDetail.UnitsInStock == 0
                         select c;

As you can see, Complex Type support with POCO is really straightforward to use. There are a couple of things you need to keep in mind when using complex types support with POCO:

You must define your Complex Type as a class. Structs are not supported.
You cannot use inheritance with your complex type classes.

While we are on the topic of Complex Types, I thought I’d mention one other thing: Did you know that the Entity Framework designer in Visual Studio 2010 supports complex type declarations?

In Visual Studio 2008, you had to manually add the Complex Type declaration to the CSDL. That is all history with the Complex Type support in the designer in Visual Studio 2010.

And what’s cool is that because Visual Studio 2010 supports Multi-Targeting, you can use this capability when building applications that target .NET Framework 3.5, using Entity Framework 3.5 as well!

Deferred/Lazy Loading

In my sneak preview post on Deferred Loading two weeks ago, I mentioned that there is now Deferred Loading support in Entity Framework. It comes as no surprise then that the default code-generated entity types out of the box based on EntityObject will offer Deferred Loading. If you are wondering whether Deferred Loading is supported with POCO objects, then I think you will be happy to know that you can get Deferred Loading with POCO as well.

There are two things you need to do in order to get Deferred Loading support with POCO entities:

Declare the property that you would like to load lazily as virtual. These properties can be any collection type that implements ICollection<T> or they can be a reference representing a 1/0..1 relationship.

For instance, here’s a part of the updated Category entity class which I have modified to support Deferred Loading:

public class Category
{
    public int CategoryID { get; set; }
    public string CategoryName { get; set; }
    public string Description { get; set; }
    public byte[] Picture { get; set; }
    public virtual List<Product> Products { get; set; }
    ...

2. Enable deferred loading on the context:

context.ContextOptions.DeferredLoadingEnabled = true;

That’s it. You will now get automatic Deferred Loading for your POCO types without having to do anything else.

So how exactly does this work and what’s going on under the covers?

The reason why this works is because when I marked my collection property as virtual, this allowed the Entity Framework to provide a proxy instance for my POCO type at runtime, and it is this proxy that does automatic deferred loading. The proxy instance is based on a type that derives from my own POCO entity class - so all functionality you have provided is preserved. From a developer point of view, this allows you to write persistence ignorant code even when deferred loading might be a requirement.

If you were to inspect the actual instance in the debugger, you will see that the underlying type for the instance is different from the original type that I declared:

While the Entity Framework does its best to provide automatic deferred loading with minimal friction, this is something you need to be aware of when dealing with manual creation of instances that you want to then add or attach, or when you serialize/deserialize instances.

Manual instantiation of Proxy instances for POCO entities

In order to enable creation of proxy instances for adding/attaching, you can use the CreateObject factory method on ObjectContext for creating entity instances:

Category category = context.CreateObject<Category>();

Try to keep this in mind and use CreateObject when creating instances that you want to then use with the Entity Framework.

More Efficient Change Tracking with “Change Tracking Proxies”

The standard POCO entities we have talked about until now rely on snapshot based change tracking – i.e. the Entity Framework will maintain snapshots of before values and relationships of the entities so that they can be compared with current values later during Save. However, this comparison is relatively expensive when compared to the way change tracking works with EntityObject based entities.

There is another type of proxy that will allow you to get better performance out of change tracking with POCO entities.

If you are familiar with IPOCO, you know that IEntityWithChangeTracker was one of the interfaces you were required to implement to provide change notifications to the Entity Framework.

Change Tracking proxies subclass your POCO entity class to provide you with this capability during runtime without requiring you to implement the IPOCO interfaces yourself.

In many ways, you get the best of both worlds with this approach: You get persistence ignorance with POCO classes and you get the performance of EntityObject / IPOCO when it comes to change tracking.

To get change tracking proxies, the basic rule is that your class must be public, non-abstract or non-sealed. Your class must also implement public virtual getters/setters for all properties that are persisted. Finally, you must declare collection based relationship navigation properties as ICollection<T> only. They cannot be a concrete implementation or another interface that derives from ICollection<T> (a difference from the Deferred Loading proxy)

Here’s an example of my POCO class for Product that will give me more efficient proxy based change tracking at runtime:

public class Product
{
    public virtual int ProductID { get; set; }
    public virtual string ProductName { get; set; }
    public virtual int SupplierID { get; set; }
    public virtual string QuantityPerUnit { get; set; }
    public virtual decimal UnitPrice { get; set; }
    public virtual InventoryDetail InventoryDetail { get; set; }
    public virtual bool Discontinued { get; set; }
    public virtual Category Category { get; set; }
}

Once again, keep in mind that you must use CreateObject for creating proxy instances you want to then add or attach to the context. But pure POCO entities that don’t rely on proxies and proxy based entities can work together. You need to only use CreateObject when dealing with proxy based entities.

What if I want both Deferred Loading and better change tracking for the same POCO type?

The two are not mutually exclusive and you don’t have to choose between a Deferred Loading proxy and a Change Tracking proxy. If you want Deferred Loading and efficient change tracking, you just have to follow the rules for Change Tracking via proxies, and enable Deferred Loading. Change Tracking proxies will give you deferred loading if deferred loading is enabled.

Explicit Loading

All this deferred loading capability is great – but there are plenty of you out there that want to be in full control of how you load related entities. You might even choose to go the route of pure POCO without having to resort to any of the automatic proxy generation done for you.

This is a perfectly acceptable (and preferable in many cases) and you can use explicit loading of relationships and be in complete control of how you query for data from the database.

There are two options for doing Explicit Load with POCO:

One is to use ObjectContext.LoadProperty and specify the name of the navigation property you want to load:

context.LoadProperty(beveragesCategory, "Products");

This gets the job done – but it isn’t very type safe as you can see. If I don’t have the right name of the navigation property here, I will get a runtime exception.

Some of you might actually prefer this:

context.LoadProperty(beveragesCategory, c => c.Products);

I can use a lambda expression to specify the property that I want to load explicitly, and this is offers more type safety.

So that’s about everything I planned on covering in the second post. There’s more to come however – in the final part of this series, we’ll go over some of the things to be aware of when dealing with pure POCO (non proxy) instances and keeping things in sync between your object graph and the Object State Manager. We will also cover variations of SaveChanges that you might want to be aware of.

In the meantime, check out the sample code that I have updated to cover some of the things we have talked about in this post.

Faisal Mohamood
Program Manager, Entity Framework

POCO in the Entity Framework: Part 1 - The Experience

dpblogs — Fri, 22 May 2009 03:46:00 GMT

Last week I mentioned in the sneak preview on POCO that support for POCO entities is one of the new capabilities we have added to Entity Framework 4.0. This week, I’d like to go into the details of POCO support in Entity Framework 4.0.

There’s quite a bit to discuss here, including:

Overall POCO experience in Entity Framework 4.0
Change Tracking in POCO
Relationship Fix-up
Complex Types
Deferred (Lazy) Loading and Explicit Loading
Best Practices

In this post, I will focus primarily on the overall experience so that you can get started with POCO in Entity Framework 4.0 right away. I’d like to use a simple example that we can walk through so you can see what it feels like to use POCO in Entity Framework 4.0. I will use the Northwind database, and we’ll continue to build on this example in subsequent posts.

Step 1 – Create the Model, turn off default Code Generation

While POCO allows you to write your own entity classes in a persistence ignorant fashion, there is still the need for you to “plug in” persistence and EF metadata so that your POCO entities can be materialized from the database and persisted back to the database. In order to do this, you will still need to either create an Entity Data Model using the Entity Framework Designer or provide the CSDL, SSDL and MSL metadata files exactly as you have done with Entity Framework 3.5. So first I’ll generate an EDMX using the ADO.NET Entity Data Model Wizard.

Create a class library project for defining your POCO types. I named mine NorthwindModel. This project will be persistence ignorant and will not have a dependency on the Entity Framework.
Create a class library project that will contain your persistence aware code. I named mine NorthwindData. This project will have a dependency on Entity Framework (System.Data.Entity) in addition to a dependency on the NorthwindModel project.
Add New Item to the NorthwindData project and add an ADO.NET Entity Data Model called Northwind.edmx (doing this will automatically add the dependency to the Entity Framework).
Go through “Generate from Database” and build a model for the Northwind database.
For now, select Categories and Products as the only two tables you are interested in adding to your Entity Data model.

Now that I have my Entity Data model to work with, there is one final step before I start to write code : turn off code generation. After all you are interested in POCO – so remove the Custom Tool that is responsible for generating EntityObject based code for Northwind.edmx. This will turn off code generation for your model.

We are now ready to write our POCO entities.

Step 2 – Code up your POCO entities

I am going to write simple POCO entities for Category and Product. These will be added to the NorthwindModel project. Note that what I show here shouldn’t be taken as best practice and the intention here is to demonstrate the simplest case that works out of the box. We will extend and customize this to our needs as we go forward and build on top of this using Repository and Unit of Work patterns later on.

Here’s sample code for our Category entity:

    public class Category
    {
        public int CategoryID { get; set; }
        public string CategoryName { get; set; }
        public string Description { get; set; }
        public byte[] Picture { get; set; }
        public List<Product> Products { get; set; }
    }

Note that I have defined properties for scalar properties as well as navigation properties in my model. The Navigation Property in our model translates to a List<Product>.

Similarly, Product entity can be coded like this:

    public class Product
    {
        public int ProductID { get; set; }
        public string ProductName { get; set; }
        public int SupplierID { get; set; }
        public string QuantityPerUnit { get; set; }
        public decimal UnitPrice { get; set; }
        public Int16 UnitsInStock { get; set; }
        public Int16 UnitsOnOrder { get; set; }
        public Int16 ReorderLevel { get; set; }
        public bool Discontinued { get; set; }
        public Category Category { get; set; }
    }

In this case, since the relationship allows only one Category that a Product can belong to, we have a reference to a Category (unlike the List<T> collection we have for modeling the Navigation Property in Category).

Step 3 – Code up your Entity Framework Context

The last thing I have to do in order to pull all of this together is to provide a context implementation (much like the ObjectContext implementation you get when you use default code generation). The context is the glue that brings persistence awareness into your application, and it will allow you to compose queries, materialize entities as well as save changes back to the database. The context will be a part of the NorthwindData project.

For simplicity, I will include our context into the same class library that has our entity types – but when we discuss patterns such as Repository and Unit of Work, we will set it up such that you have a pure POCO class library without any persistence concerns.

Here’s a simple context implementation for our scenario:

public class NorthwindContext : ObjectContext
    {   
        public NorthwindContext() : base("name=NorthwindEntities", 
"NorthwindEntities")  
        {
            _categories = CreateObjectSet<Category>();
            _products = CreateObjectSet<Product>();
        }

        public ObjectSet<Category> Categories
        {
            get 
            { 
                return _categories; 
            }
        }
        private ObjectSet<Category> _categories;

        public ObjectSet<Product> Products
        {
            get
            {
                return _products;
            }
        }
        private ObjectSet<Product> _products;
    }

Note that ObjectSet<T> is a more specialized ObjectQuery<T> that we introduced in Entity Framework 4.0.

That’s it – now you have pure POCO entities with a simple context that will allow you to write queries like this:

    NorthwindContext db = new NorthwindContext();

    var beverages = from p in db.Products
                    where p.Category.CategoryName == "Beverages"
                    select p;

The entities that are materialized are pure POCO entities, and you can make and persist changes much like you would with regular EntityObject or IPOCO entities. You get all the services provided by Entity Framework - the only difference is that you are using pure POCO entities.

There are many possibilities here as far as how we can improve on this simplified example. Before we get into that however, I would like to get some basic questions out of the way.

Do I need an Entity Data Model before I can use POCO?

Yes – POCO support in Entity Framework 4.0 simply removes the need of having persistence specific concerns in your entity classes. There is still the need for you to have a CSDL/SSDL/MSL (collectively EDMX) metadata so that the Entity Framework is able to use your entities along with the metadata in order to enable data access. There is a separate effort that we are working on that will allow you to do true “code-first” development without the need to have a predefined EDMX model. A community preview of this feature will be released to the web in the coming months. We will roll this into the product the first chance we get. As always, your feedback will be helpful.

Do I have to always hand-craft these entities and the context?

No – there is a very powerful and flexible code generation mechanism in Entity Framework 4.0 that is based on T4 as Alex blogged about here. You can provide your own templates that allow you to build your entities in a way that you see fit. We are also working on providing standard out of the box templates that will generate POCO entities for you.

How is metadata mapped when using POCO entities?

In Entity Framework 3.5, both EntityObject and IPOCO based entities relied on the use of mapping attributes that were meant for decorating and mapping the entity types and properties back to the corresponding elements in the Conceptual model. Entity Framework 4.0 introduces convention based mapping for allowing mapping of Entity Types, Properties, Complex Types and Relationships back to the conceptual model without the need for explicit decoration. The simple rule here is that Entity Type names, Property names and Complex Types names used in your POCO classes must match those defined by the conceptual model. Namespace names are ignored and don’t have to match between the class definition and the conceptual model.

Do I need to have public getters and setters for all properties in my entity classes?

You can use any access modifier on your POCO type’s properties as long as none of the mapped properties are virtual and as long as you don’t require partial trust support. When running in partial trust, there are specific requirements on the visibility of access modifiers on your entity classes. We will be documenting the complete set of access modifiers that are supported when partial trust is involved.

What types of collections are supported for collection based navigation properties?

Any collection that is an ICollection<T> will be supported. If you don’t initialize the field with a concrete type that is an ICollection<T> type, then an List<T> type will be provided upon materialization.

Can I have uni-directional relationships? For instance – I would like to have a Category property in my Product class, but I don’t want to have a Products collection in my Category class.

Yes – this is supported. The only restriction here is that your entity types have to reflect what is defined by the model. If you are not interested in having the navigation property for one side of the relationship, then remove it from the model completely.

Is Deferred (Lazy) Loading supported with POCO?

Yes – Deferred (Lazy) loading is supported with POCO through the use of proxy types that are used to provide automatic lazy loading behavior on top of your POCO classes. This is something that we’ll cover when we get to deferred loading – until then know that eager loading via the use of “Include” is also supported, like so:

var beverageCategory = (from c in context.Categories
.Include("Products")
                        where c.CategoryName == "Beverages"
                        select c).Single();

If I were to use Deferred (Lazy) Loading, I don’t have to do this – we’ll discuss that when we discuss proxies.

Fixing up Relationships

There are two types of fix-ups to be aware of: 1)Fix-up during query, and 2) Fix-up during changes to your entities/relationships

Fix-up during Query

Fix-up during query happens when you load related entities using separate queries on the same ObjectContext. For instance, if I were to query for a category instance Beverages, and later query for a product instance Chai (that is in Category Beverages), I would want chai.Category to point to the instance of the Beverages category without additional work.

This is automatic and works today.

Fix-up during changes to your entities/relationships

This is the relationship fix-up between two entities when I add/remove an entity that is related to another entity or alter relationships. Think of this as the case when I create a new product called “Diet Chai” and want to associate it with the Beverages category.

In Beta1 of Entity Framework 4.0, you do not get automatic relationship fix-up in this case with POCO entities. When dealing with relationships that change between entities, you will have to make sure that your POCO types include the logic to manage fix-up on both ends of the relationship correctly.

For instance, in my Northwind example, I have defined the following method on Category to support adding / removing of Orders.

        public void AddProduct(Product p)
        {
            if (Products == null)
            {
                Products = new List<Product>();
            }

            if (!Products.Contains(p))
            {
                Products.Add(p);
            }            
            
            p.Category = this;
        }

In general, it is good to use a pattern like this to support adding/removing related items rather than using the relationship collection to add/remove related entities. You also have options of making the getter/setter for the actual EF backed collection private/internal to support more fine grained access – but as mentioned earlier, some of this depends on the requirements as to whether you require partial trust support or not.

So we have covered quite a bit of ground in this “quick look” at the overall POCO experience. There’s quite a lot more to talk about – and I will be continuing this discussion early next week. In the meantime, take a look at the complete solution (attached) if you are interested in the sample code covering all the concepts we covered here. Please keep in mind that you must have a local instance of Northwind database installed on the machine where you are running this sample.

In my next post we will look at Complex Types, Change Tracking, Proxies, Lazy Loading and Eager Loading. Until then, please look through the MSDN Documentation on POCO here for more details on POCO support in Entity Framework 4.0.

Let us know what you think!

Faisal Mohamood
Program Manager, Entity Framework