Entity Framework Design : EDM

Foreign Keys in the Entity Framework

efdesign — Mon, 16 Mar 2009 23:47:00 GMT

Background

A number of months ago we asked whether Foreign Keys (FKs) in Conceptual and Object models were important.

The feedback we got indicated that there are a real mix of opinions on the topic. Some people are all for them while others think that FKs pollute the conceptual model.

The fact that our customers were so divided meant we thought it was important to provide options here. Those who want FKs in their Entities should be able to do so, so they can gain all the benefits that having FKs in your Entities undoubtedly provide. On the other hand customers who are concerned that have FKs in their Entities in someway pollutes their model can continue to use the type of associations we had in .NET 3.5 SP1.

In the past we've called the .NET 3.5 SP1 style Associations "first class" associations. Which while accurate somehow implies that associations based on FKs in Entities are "second class". In fact when we talked about this new work recently with our MVPs one of the first questions was "are these second class?", which they most definitely are not.

The fact is these new associations are different but first class nevertheless.

So what's been added?

In .NET 4.0 we will add support for a new type of association called "FK Associations".

The way this works is we will allow you to include your FK columns in your entities as "FK properties", and once you have FK Properties in your entity you can create an "FK Association" that is dependent upon those properties.

Okay so we will have "FK Associations", what are we going to call the older style associations? Obviously saying something like ".NET 3.5 SP1 style Associations" every time isn't ideal. So we are going to call them "Independent Associations".

The term "Independent Association" resonates for us because they are independently mapped, whereas FK Associations need no mapping, simply mapping the Entity(Set) is sufficient.

How do I use an "FK Association"?

The real reason so many customers and partners are asking for "FK Associations" is that they significantly simplify some key coding patterns. So much so that we are convinced that for most people "FK Associations" will be an automatic choice going forward.

Some of the things that are hard with "Independent Associations" are trivially easy using FK Associations. Scenarios as diverse as DataBinding, Dynamic Data, Concurrency Control, ASP.NET MVC Binders, N-Tier etc are all positively impacted.

Let's have a look at some code snippets that will work with "FK Associations":

1) Create a new Product and FK Association to an existing Category by setting the FK Property directly:

using (var context = new Context())
{
    //Create a product and a relationship to a known category by ID
    Product p = new Product
    {
        ID = 1,
        Name = "Bovril",
        CategoryID = 13
    };
    //Add the product (and create the relationship by FK value)
    context.Products.AddObject(p);
    context.SaveChanges();
}

This sort of approach works both for insert and update, and is particular useful for things like databinding, where you often have the new Value of the FK in a grid or something but you don't have the corresponding object, and you don't want to wear the cost of a query to pull back the principal object (i.e. the Category).

2) Create a new Product and a new FK Association to an existing Category by setting the reference instead:

public void Create_new_Product_in_existing_Category_by_reference()
{
    using (var context = new Context())
    {
        //Create a new product and relate to an existing category
        Product p = new Product
        {
            ID = 1,
            Name = "Bovril",
            Category = context.Categories
                        .Single(c => c.Name == "Food")
        };
        // Note: no need to add the product, because relating
        // to an existing category does that automatically.
        // Also notice the use of the Single() query operator
        // this is new to EF in .NET 4.0 too.
        context.SaveChanges();
    }
}

This programming pattern is not new to the Entity Framework, you could do this in .NET 3.5 SP1. I called it out because it shows you can still write the sort of code you write with Independent Associations even when using FK Associations.

3) Update an existing Product without informing the Entity Framework about the original value of the CategoryID (not supported with Independent Associations):

public void Edit(Product editedProduct)
{
    using (var context = new Context())
    {
        // Create a stand-in for the original entity
        // by just using the ID. Of the editedProduct
        // Note: you don't have to provide an existing Category or CategoryID
        context.Products.Attach(
            new Product { ID = editedProduct.ID });

        // Now update with new values including CategoryID
        context.Products.ApplyCurrentValues(editedProduct);
        context.SaveChanges();
    }
}

In this example "editedProduct" is a product which has been edited somewhere, this is exactly the sort of the code you might write in the Edit method of a ASP.NET MVC controller, and is a great improvement over the code you have to write using Independent Associations.

There is much more you can do with FK Associations, but these 3 samples give you a flavor of the sort of coding patterns FK Associations allow.

Keeping FKs and References in Sync

One thing that hasn't been mentioned so far is that the Entity Framework tries to keep related References and FKs in sync as much as possible.

When this synchronization occurs depends upon when the Entity Framework is notified of changes, which depends upon the type of Entity Classes involved: be they POCO with Proxies, POCO without Proxies, IPOCO or EntityObjects etc.

Another post will cover this in more detail.

How do I create an "FK Association"?

There are a number of ways.

One mainline scenario is when using the tools to infer a model from a database. We have added an option to choose whether "FK Associations" or "Independent Associations" are generated by default. The same is true if you use EdmGen.exe (i.e. our command line tool).

The second mainline scenario for creating FK Associations is creating a model from scratch, aka Model First which is new to .NET 4.0.

Here are the steps involved:

Create two Entities (say Product and Category) that look something like this:
Add a property that will be the FK Property: i.e. CategoryID
Create an association between the two EntityTypes with the correct end multiplicities and NavigationProperty names:
Double Click the line between the two Entities, that represents the Association, and add a Referential Integrity Constraint:

This is the step that tells the Entity Framework that the CategoryID is an "FK Property" and that the "CategoryProduct" association is an "FK Association".
You end up with something that looks like this:

And you are done you've set up an FK Association. Notice that this association doesn't need to be mapped you simply need to make sure you map all the properties of both Entities.

Very easy.

Summary

In .NET 4.0 we will add support for FK Properties and FK Associations to the Entity Framework. The existing style Independent Associations will still be possible, but we expect that FK Associations will become most peoples automatic choice because they simplify so many common Entity Framework coding tasks.

As always we'd love to hear your thoughts on this work.

Alex James,
Program Manager, Entity Framework Team, Microsoft.

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Update on Computed Properties

efdesign — Tue, 13 Jan 2009 07:27:00 GMT

A while back I wrote a post that introduced the concept of Computed Properties. Since that time we’ve had a number of conversations with both customers and internal partners and we’ve had some new ideas, that have changed our thinking somewhat.

First off we decided to implement Model Defined Functions, which are significantly more powerful and flexible than Computed Properties, and provide overlapping capabilities.
Computed Properties on the other hand, would never have been structurally part of the entity, which lead to some tough choices and probable customer confusion:

Do you always pay the cost of materializing the Computed Property even when it is not required?
What about if there are lots of Computed Properties on the Entity?
Today in order to materialize an Entity and it's properties, computed or otherwise, the Entity Framework needs access to setters. Unfortunately the existence of the Setter, sends an erroneous message that Computed Properties can be altered and persisted.
Upon further investigation we discovered that Computed Properties didn't actually address the requirements of Reporting Services, which was one of the original drivers for Computed Properties. Our original design called for computed properties to operate over an instance of an Entity, and it turned out that Reporting Services needed computed properties to operate over both an instance and a set of instances. Model Defined Functions on the other hand are flexible enough to handle both of these scenarios.

In the future it may be possible to gain all the benefits of computed properties without any of the above issues, by extending our integration of Model Defined Functions with LINQ so that the [EdmFunction] attribute can be applied to CLR properties too.

If we did this the first parameter of the specified Model Defined Functions could be mapped to be the CLR type containing the property itself. This would allow a Computed Property, backed by a Model Defined Function, to be used in a targeted way in LINQ queries, without any of the illusions that this might somehow be set-able or structurally part of the entity.

Given all these points we started to think Computed Properties just weren’t as important.

We think our best course of actions in the short term is 1) to focus on supporting Model Defined Functions and 2) ask for your feedback on the relative importance of Computed Properties given the points listed above.

So what do you think?

Do Model Defined Functions address most of the scenarios for Computed Properties?

Particularly if in the future we can treat CLR properties as stubs for Model Defined Functions?

As always we are very keen to hear what you think.

Alex James
Program Manager
Microsoft

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

NB: For more background on the origins of [EdmFunction] check out this post.

Model Defined Functions

efdesign — Wed, 07 Jan 2009 23:32:00 GMT

Today the Entity Framework, and more specifically the Entity Data Model, have a limited notion of Functions.

We are currently restricted to Function Imports that allow stored procedures to be invoked, and Canonical / Store Functions for database independent and database specific functions respectively.

Now however we want to support functions defined, not just declared, in the EDM (aka. the CSDL).

An example would be:

<Function Name="GetAge" ReturnType="Edm.Int32">
      <Parameter Name="Person" Type="Model.Person" />
      <DefiningExpression>
            Edm.DiffYears(Edm.CurrentDateTime(), Person.Birthday)
      </DefiningExpression>
</Function>

Here are some things to notice:

The DefiningExpression is eSQL.
The function can have zero or more parameters.
The Function must have a return type.
Function Parameters are referenced directly by Name in the DefiningExpression: meaning there is no parameter denoting prefix like @. This means you must be careful to choose parameter names that don't collide with identifiers you need to use in the rest of eSQL expression.
Unlike functions in SSDL, functions in CSDL only support In bound Parameters (i.e Mode="In") because otherwise they become non-composable.
For this reason the Mode of a parameter cannot be set in CSDL (it is always Mode="In").
Functions are declared as Global Items and are declared within the <Schema> element. As such there identity is made up of the Schema's namespace and the function name.
The function parameters and return type can be any of the following:
- A scalar type or collection of scalar types.
- An entity type or collection of entity types.
- A complex type or collection of complex types.
- A row type or collection of row types (See below).
- A ref type or collection of ref types.
Functions with a DefiningExpression do not require mapping, since the eSQL expression is composed out of eSQL fragments that are already mapped.
Functions without a DefiningExpression are simply declarations. Today the Entity Framework doesn't complain when loading a CSDL with such a function, but you can't invoke it. In the future these functions might be used to support mapping Table Value Functions in the store to functions in the Conceptual Model.
Since it is trivial in eSQL to create arbitrary un-named types, imagine a projection that projects 3 of the properties from an Entity, we now need a mechanism for defining these "RowTypes" inline, so that they can be used when defining Function Parameters and ReturnTypes. For example:

<Parameter Name="Coordinate">
   <RowType>
      <Property Name="X" Type="int" Nullable="false"/>
      <Property Name="Y" Type="int" Nullable="false"/>
      <Property Name="Z" Type="int" Nullable="false"/>
   </RowType>
</Parameter>
Since eSQL is primarily set based, we also need a way of defining parameters and return types that are collections of RowTypes:

<Parameter Name="Coordinates">
   <CollectionType>
      <RowType>
         <Property Name="X" Type="int" Nullable="false"/>
         <Property Name="Y" Type="int" Nullable="false"/>
         <Property Name="Z" Type="int" Nullable="false"/>
      </RowType>
   </CollectionType>
</Parameter>

Using the Function via eSQL:

It is trivial to use the function via eSQL. For example:

SELECT Namespace.GetAge(p)
FROM Container.People AS P
WHERE P.Firstname = ‘Jim’

Here we get Jim's age, assuming of course there is only one Jim!

It is also possible to compose functions together, you must simply ensure return types and target parameter types are the same (in the case of named types) or structurally equivalent (in the case of row types or collections of row types).

Things get a little trickier when you are dealing with functions that return sets, for example imagine a function that returns someone's friends, used in conjunction with the GetAge function:

SELECT VALUE (F)
FROM Container.People AS P
CROSS APPLY Namespace.GetFriends(P) AS F
WHERE Namespace.GetAge(P) > 21

Here we get all the friends of people older than 21.

As you can see to do this sort of thing you need a crash course in eSQL.

Using the Function via LINQ:

It is also possible to use these functions in LINQ, but this does require a extra step to create an appropriate stub function in the CLR.

This solution is based on the techniques described here, and involves creating a Stub function in the CLR language of your choice, and annotating it something like this:

[EdmFunction("Namespace", "GetAge")]
public static int GetAge(Person p)
{
throw new NotSupportedException(…);
}

The Entity Framework uses the signature of the function and the EdmFunction attribute to map calls to this function when encountered to the appropriate Model Defined Function.

Once you have this stub it is then trivial to use it in a LINQ query like this:

var peopleOver21 =
     from p in ctx.People
     where GetAge(p) < 21
     select p;

Indeed if you are familiar with LINQ you will probably find composing functions together a lot easier too:

var friendOfPeopleOver21 =
     from p in ctx.People
     from f in GetFriends(p)
     where GetAge(p) < 21
     select f;

Notice that the CLR functions don't need to be directly callable, in the example above the CLR stub throws an exception if called directly.

However the existence of the stub allows you to create LINQ expressions that compile correctly, and then at runtime, when used in a LINQ to Entities query, the function call is simply translated by the entity framework into a query that runs in the database.

Summary

As you can see Model Defined Function's are very powerful, and this post has barely scratched the surface of possibilities they open up.

The Entity Framework team would love to hear your comments.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Structural Annotations - One Pager

efdesign — Tue, 12 Aug 2008 22:50:00 GMT

In V1 of the Entity Framework it is possible to annotate a schema using attributes declared in another XSD.

However XML attributes are a very limited form of annotation. It would be better if we could annotate using full elements.

This is what we are calling Structural Annotations.

This feature will allow both customers and partners like Reporting Services to modify the model so that it includes information important to them which can’t be captured in vanilla EDM format.

1 EDM extensions

While it should be possible to annotate any level in the XML hierarchy, there are however some restrictions.

The general rule is you can annotation any CSDL / SSDL element that has a corresponding MetadataItem which in practice means everything except <Using>, <Schema>, <Key> and <PropertyRef> elements.

These annotations should be named, so that they can be accessed using the same API as today.

i.e. something like this:

1.1 Key points to notice:

In the above example both the CLR and RS namespaces must have been declared somewhere. Probably at the root of the CSDL somewhere, i.e. something like this:
<Schema xmlns:RS="http://schemas.microsoft.com/RS/2006" xmlns:CLR=http://schemas.microsoft.com/net/3.5…
Alternatively the namespace could be in lined in the annotation:
<Security xmlns="http://schemas.microsoft.com/RS/2006"> …
</Security>
In both cases the unique identity of the annotation is in the form {namespace}:{elementname}. So for the RS Security annotation example, irrespective of how the RS namespace is introduced the identity would be:
http://schemas.microsoft.com/RS/2006:Security
Structural annotations should always follow all other sub-elements i.e. when structurally annotating an <EntityType> element the annotation element should follow all <Key> <Property> and <NavigationProperty> elements.
Attribute based annotations (as supported in V1 and used for things like CodeGen) are scoped to the same annotation identity namespace. Hence care should be taken to verify that annotations using either approach don’t have colliding identities.
It should be possible to have more than one named “Structural Annotation” per CSDL (or SSDL) element. Indeed element names can collide so long as the combination of Namespace + Element Name is unique for a particular element (or more specifically each MetadataItem). This means for instance you can’t have two <RS:Security> elements under any one MetadataItem.

1.2 Positive and Negative Cases:

This:

…would be invalid, because of the identity collision, likewise this would also be invalid:

where-as this is fine:

… since all the above annotations have unique identities.

1.3 What can an <Annotation> contain?

A structural annotation is simply an XML element. As such it can be considered a root of an XML document that can contain any valid XML structures. These structures are simply ignored by the Entity Framework and Metadata APIs.

1.4 What elements can be annotated?

If an element in the CSDL or SSDL has a corresponding MetadataItem in the Metadata API that element should support structural annotations. Since <Using>, <Schema>, <Key> and <PropertyRef> elements have no corresponding MetadataItem(s) they don’t support structural annotations.

Notice while TypeUsage(s) have no CSDL representation today,they are MetadataItem(s) so they could in theory be annotated in the future. For example if a public mutable metadata API is produced.

2 Metadata API Changes:

In this example we are using the Entity Frameworks Metadata API to get the EdmType for an EntityType called Content. From that we then get the MetadataProperty called http://schemas.microsoft.com/RS/2006:Security:

EdmType contentType = metadataWorkspace.GetEdmType(“MyNamespace.Content”);
if (contentType.MetadataProperties.Contains(
"http://schemas.microsoft.com/RS/2006:Security")
{
    MetadataProperty annotationProperty = contentType.MetadataProperties[ "http://schemas.microsoft.com/RS/2006:Security"];
    object annotationValue = annotationProperty.Value;
    //Do something...
}

Notice that the way you access structural annotations is the same as the way you access attribute annotations, i.e. by name/identity, in the MetadataProperties collection.

This of course means that their names can’t collide, name collisions should be picked up when metadata is loaded from persistent formats (like CSDL) or when annotations are created by hand in the any future mutable metadata API.

2.1 MetadataProperty TypeUsage

When creating a MetadataProperty to hold the XElement for the structural annotation, we need to assign a TypeUsage too. For V1 style attribute annotations, the TypeUsage we chose was based on Edm.String. Ideally we should use Edm.XML etc, but since this is not currently available, we will use Edm.String in the meantime. We have filled a bug to change this if/when [j1] XML becomes an EDM type.

3 What is the MetadataProperty.Value?

The Value of the MetadataProperty will be a XElement for structural annotations originating from CSDL.

So if you know that the annotation is a structural annotation you can access it like this:

MetadataProperty annotationProperty =
contentType.MetadataProperties["CLR:Attributes"];

XElement myAnnotation = annotationProperty.Value as XElement;

3.1 XElement is rooted at the <Annotation> element

The XElement returned will be the XElement containing everything including the structural annotation itself.

3.2 XElement is mutable

There is nothing to stop the consumer of the Metadata API from modifying the XElement since it is simply a reference, to an instance of a type the Entity Framework doesn’t control.

We could address this by cloning on access to make things a little safer. But this negatively affects the mainline scenario, namely where a customer uses annotations purely for read, and is therefore not required.

The important thing to understand is that should a user modify the XElement all bets are off. We should provide guidance both via API hints:

/// <summary>
/// ....
/// <remarks>
/// Should this return a reference type like an XElement, you should consider
/// this immutable, no guarantees are made about behavior if this reference
/// type is modified in anyway.
/// </remarks>
/// </summary>
public object Value { get {

And via appropriate documentation.

3.3 No universal annotation format

Rather than try to invent a new universal data format for annotations, we decided to be pragmatic.

So we are limiting ourselves to XML for structural annotations loaded from CSDL (an XML format).

However since the MetadataProperty.Value is of type object, in a world where we have an alternative EDM representation (like in memory using mutable metadata for instance) we could easily assign something other than an XElement to the MetadataProperty.Value, like for example a xaml Serializable CLR type instance.

4 Out of Scope

The current plan is to simply support annotation in the CSDL & SSDL. Meaning annotating MSL is currently out of scope.

If we uncover meaningful important scenarios for MSL annotations, we will undertake that work separately.

Incidentally supporting structural annotations in MSL touches a completely separate code path, and as a result will require roughly the same amount of work. Additionally in order to make undertaking this work meaningful we would first need to expose a public mapping metadata API.

....

As always we are keen to hear any feedback you have on this item.

Do you think you will find a use for Structured Annotations?

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Using Stored Procedures to load structured data.

efdesign — Fri, 18 Jul 2008 20:59:00 GMT

V1 of the Entity Framework allows you to use stored procedures in two main ways:

Mapping Create, Update and Delete entity operations to appropriate stored procedures.
Doing a FunctionImport that allows you to return an enumeration of Entities*

Now the thing is, in order to return an enumeration of Entities, you have to map the Entity too.

Why?

Well in V1 FunctionImport attaches the returned entities, so you can make and save changes.

But of course sometimes you don't want that functionality, you just want a structured way of moving information around, so it is a shame to be required to do the mapping too.

This leaves us two possibilities for V2.

Allowing FunctionImports to return unattached/untracked Entities.
Allowing FunctionImports to return ComplexTypes.

The ComplexTypes option is the topic of the following one-pager by Asad a Program Manager on the Entity Framework team:

Scenario:

Customer Goal:

The customer wants to do something like this in their .NET code:

public CustomerInfo GetCustomerInfo(int Id)
{
    using (NorthwindEntities ctx = new NorthwindEntities())
    {
        CustomerInfo info = ctx.GetCustomerInfoById(Id).FirstOrDefault();
        return info;
    }
}

Without needing to create mappings etc, as this lowers the level of friction inherent in using stored procedures for queries with the Entity Framework.

NB: today we only support Collection(Type) as the return type of the FunctionImport, which means it results in an Enumeration, hence the call to FirstOrDefault(). A separate work item is required to support returning just one ComplexType (or EntityType) rather than collections.

Detailed Scenario walk through:

User defines a Stored Procedure in store:

Create Procedure GetCustomerInfoById(
        @Id int
    )
As
    SELECT first_name As Firstname, last_name As Lastname, city As City
    FROM CustomerTable
    WHERE id = @Id

Complex type definition in EDM:

User wants to map the result set from this stored procedure to a Complex type “CustomerData” defined in Entity Data Model as:

The mapping involves following three steps:

Define import function definition in SSDL:

Expose function definition in CSDL schema file

Define mapping (convention based) MSL file:

Design Details:

- Mapping between Complex type and result type is by convention.

- Explicit property to column name mapping is not supported.

- The result from the stored procedure has to exactly match the shape and the property names of the complex types.

- In EDM the result is captured in a Complex type therefore no EntitySet definition or mapping is required.

Assumptions/suppositions:

- A future work item (covered by a separate feature entry) would enable richer mapping capability by allowing column renames. However for now Convention based mapping is the only supported feature.

Looking forward to hearing your feedback.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

*In V1 it is also possible to do a FunctionImport that returns a scalar value, but this FunctionImport is not available in Object Services, if you want to use it you have to drop down to eSQL in the EntityServices layer.

Computed Properties One Pager

efdesign — Fri, 27 Jun 2008 21:21:00 GMT

UPDATE: this blog post is out of date. Please review this instead.

If you've been paying attention to what the Entity Framework team has been saying. You will remember hearing us talk about using the Entity Data Model (EDM) as a common way of describing models that can be understood by many different applications.

As of V1, the Entity Framework supports ad-hoc line of business applications. In the future though we also want to add extra services, such as reporting. To make this work we need to make a number of key extensions both to the EDM and the Entity Framework itself.

The first of these is Computed Properties.

The Reporting Model requirement, is the ability to create reports in terms of properties that are not mapped to fields in the database, but are instead computed.

Supporting this idea is then vital if we imagine a world where Reporting Services works over the EDM.

Computed Properties however are useful for more than just Reporting scenarios. What follows is the initial one pager (okay more like 5 pages) on the topic:

Scenario:

We want to support Reporting Services (and other customers too) that want to be able to have some properties of the entity be computed in the store.

Examples range from a Person’s Fullname which is simply the Person’s Firstname and Surname concatenated, to more complicated scenarios that use Navigations, and / or completely random eSQL.

Scenario Detailed Walkthrough:

In the EntityType definition a user creates a Computed Property, either with or without the help of tooling:

In this example the “Fullname” property is computed using the eSQL included in the DefiningExpression under the Property.

Interesting Points:

We re-use the <Property> element for calculated properties, the thing that distinguishes computed properties is that the property has a <DefiningExpression> sub element.
Calculated Properties are always read only.
We also want to introduce a ReadOnly attribute to the Property element that can be used to mark non-calculated properties as read only too (See the CreatedDate property above).
In the DefiningExpression’s eSQL we add support for a contextual variable, tentatively called “it”, which can be used by programmers to reference properties and navigation properties on the Entity.
While any eSQL can be written in the DefiningExpression, the expect shape of the resulting query must be projectable into the Type specified at the Property level. For example in the above, the Fullname property is of type string, so the DefiningExpression is valid only because its return shape is expected to be a single row with a single string column.
Calculated Properties can’t be referenced in the Key of the Entity.
Can Calculated Properties reference other Calculated Properties in the same Entity? Ideally yes.
Can Calculated Properties reference Calculated Properties in other Entities? Ideally yes.
Invalid eSQL or invalid return type of the expression should ideally result in a Parsing Error at runtime.

Computed Property Return Types:

The example given so far, is a special case, in that the eSQL in the DefiningExpression only references properties of the current entity. We can therefore deduce that there will be just one row and from the shape of the projection we can deduce there is one string column, hence we can safely assume this eSQL can be materialized into a single string column or property in the reader or object respectively.

However a DefiningExpression can easily produce multiple rows, either by using a NavigationProperty hanging off the current Entity, or by virtue of being a standard query with an undetermined set of results.

This means we need to also look at handing computed properties that return a Collection.

Additionally the eSQL can return more than one column, so the collection may not necessarily be a Collection of scalars.

Ideally then we should support a collection of ComplexTypes, Entities and perhaps even RowTypes.

Collections of Scalars:

An example would be this:

<EntityType Name="Person">
    <Key>
      <PropertyRef Name="ID" />
    </Key>
    <Property Name="ID" Type="Int32" />
    <Property Name="Firstname" Type="String" MaxLength="50" />
    <Property Name="Surname" Type="String" MaxLength="50" />
    <Property Name="Fullname" Type="String">
        <DefiningExpression>
        it.Firstname + ‘ ’ + it.Surname
        </DefiningExpression>
    </Property>
    <NavigationProperty Name="Friends" Relationship="FriendShip" FromRole="FriendOf" ToRole="Friend" />
    <Property Name="Acquaintances" Type="Collection(String)">
        <DefiningExpression>
        SELECT VALUE f.Fullname
        FROM it.Friends AS f
        </DefiningExpression>
    </Property>
</EntityType>

In this example Acquaintances navigates through a navigation property “Friends”, which returns a Set, so the result is a set of the projected Fullnames.

Another way of writing the calculated property that hard-codes the target set in the model would be:

<Property Name="Acquaintances" Type="Collection(String)">
<DefiningExpression>
SELECT VALUE f.Fullname
FROM [Person] AS P, P.Friends AS F
WHERE P.ID = it.ID
</DefiningExpression>
</Property>

This approach is not to be recommended because it mixes the Entity structure with the extents or entitysets, which of course won’t work well with MEST.

Incidentally there is an unrelated proposal for RS to extend eSQL to make it possible to express the above in this simplified syntax

<Property Name="Acquaintances" Type="Collection(String)">
<DefiningExpression>
SELECT VALUE P.Friends..Fullname
FROM [Person] P
WHERE P.ID = it.ID
</DefiningExpression>
</Property>

Note if in the future ‘..’ notation is supported in eSQL, then it should be a no-op on our end to support this too:

<Property Name="Acquaintances" Type="Collection(String)">
<DefiningExpression>
it.Friends..Fullname
</DefiningExpression>
</Property>

You can also imagine a query that doesn’t use “it” at all, and is completely hardcoded:

<Property Name="TopGuns" Type="Collection(String)">
<DefiningExpression>
SELECT VALUE f.Fullname
FROM [Person] AS P
WHERE P.Firstname IN {‘Colin’, ‘Kati’}
</DefiningExpression>
</Property>

Object Services:

Computed properties that return a single scalar value are treated the same as normal properties, the only difference is that they are read-only, which probably only means that the ObjectStateManager needs to ignore changes to these properties rather than making them impossible to change altogether by removing the Setter.

For Collections of Scalar types the Property should be a IEnumerable<T>, i.e. in the Acquaintances example above there should be a IEnumerable<String> property called Acquaintances on the entity. Again changes should be ignored.

Entities, ComplexTypes and RowTypes:

The eSQL in the DefiningExpression can easily project multiple columns so we need a way of setting the type of Property accordingly. There are only 3 real options: Entities, ComplexTypes and RowTypes.

In this example we reverse a navigation property as a computed property, and return a collection of Person entities. I.e. the People who have the current person as a friend:

    <Property Name="FriendOf" Type="Collection(Person)">
        <DefiningExpression>
        SELECT VALUE p
        FROM [Person] AS p, p.Friends AS f
        WHERE f.ID = it.ID
        </DefiningExpression>
    </Property>

The query itself it unimportant, what is important, is that we are returning a set of Entities. Likewise we could do something similar to return a list of ComplexTypes:

    <Property Name="FriendsAddresses" Type="Collection(Address)">
        <DefiningExpression>
        SELECT VALUE f.Address
        FROM it.Friends f
        </DefiningExpression>
    </Property>

If we use a navigation property that returns a single Entity then you could do something like this too:

    <Property Name="MaternalGrandmother" Type="Person">
        <DefiningExpression>
        it.Mother.Mother
        </DefiningExpression>
    </Property>

Or a single ComplexType:

    <Property Name="MothersAddress" Type="Address">
        <DefiningExpression>
        it.Mother.Address
        </DefiningExpression>
    </Property>

Ideally we should support, both single and Collections of both Entities and ComplexTypes.

Random projections (i.e. RowTypes) are much more difficult.

If they can be coerced into a wellknown shape by property name matching then it might be possible to project into ComplexTypes. But if not, then there is no O-Space equivalent of a RowType that is appropriate.

Random Examples:

Including an FK in the structure of an Entity:

Note: While this is a handy, we should not confuse this with a solution to requirement for collocation of FK data, because as mentioned previously calculated properties are always read-only and we need a read-write solution to the collocation problem, to allow setting the FK in the Entity directly.

Why DefiningExpression?

DefiningExpression is something we will probably use for C-Side Functions (i.e. functions defined in Entity Data Model in the CSDL). So it makes sense to re-use this term, in order to keep the CSDL vocabulary and mental load smaller.

Also today we already have DefiningQuery in the storage model and QueryView in the mapping, so re-using either of those terms could easily lead to confusion.

Also and perhaps most interestingly, if we interpret the eSQL as a expression, then:

1 => int

Whereas if we interpret as a query:

1 => Collection(int)

This former has useful semantics because we can call functions in the eSQL fragment and declare that the property has a scalar return type rather than a Collection(scalar) return type.

...eof...

I for one would love to hear your thoughts.

Alex James
Program Manager,
Entity Framework Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Transparency in the design process

dpblogs — Tue, 24 Jun 2008 02:12:00 GMT

Today marks the first day of engineering for the ADO.NET Entity Framework V2.0. V2 of the product is a combination of a continued investment in our greater data platform vision as well as a focus on new and expanded ORM scenarios. Some of the things that we are trying to do in this release are as follows:

Persistence Ignorance : We are looking at ways to introduce a full POCO solution for state management and interaction with the ObjectContext.
N-Tier Support : Today we support Data Contract serialization of entities or exposing entities via Astoria, in V2 we would like to expand to a DataSet like experience where one can remote graphs and changes to the graphs across the wire using standard WCF services.
Code-First : We want to enable a convention based, code only experience with EF wherein one may start with classes and opt-in to database generation and deployment. We expect that we would provide attributes and external mapping capabilities for people who wanted something beyond the convention based mapping.
TDD Scenarios: With the introduction of POCO classes some of the TDD scenarios get a lot easier, and we are looking at incorporating some other asks to better fill out the scenario, such as making our ObjectQuery<T> and other generated members of our context and classes virtual.
FK's : Today we support bi-directional relationships, and we are looking at introducing model concepts to facilitate the definition of FK like experiences in the model or in one's POCO classes.
Lazy Loading: Today we support explicit lazy loading (calling .Load), and we are looking at various options around LoadOptions as well as outright implicit lazy loading.
Query Tree Re-Writing: This allows framework develolpers to contextualy, vertically and horizontally filter query results.
...

We have been looking at various options to ensure that the design of Entity Framework V2 truly reflects the requirements of the day to day challenges that our developer community faces when building real-world applications and services. We would like to start by being as transparent as possible in the design process. The approach we take will be similar to what you have seen on Astoria Team blog (ADO.NET Data Services), but I wanted to briefly describe how we’ll go about it so you know exactly what you are looking at when reading one of our design-related posts.

How did we get here? Version 1 of the ADO.NET Entity Framework is set to ship as a part of Visual Studio 2008 SP1 and the .NET Framework 3.5 SP1. This release has been the result of a number of years of effort. During V1 the team was focused on implementing an initial set of scenarios as well as the groundwork necessary to continue to build out the Entity Framework in future versions.

Where are we now? Today marks the first day of engineering for ADO.NET Entity Framework V2.0. What this means in practice is that we have a team with developers, testers, program managers, etc., that we have regular design meetings, and that we have a timeline (or more accurately, that we’re required to have a timeline but of course we’re still working on it :).

Transparency in the design? Over the years Microsoft has been opening up the engineering processes incrementally. Long ago there were only betas, and that was the only chance to see and give feedback about a product before it shipped. Then we started to do Community Tech Previews (CTPs). CTPs enabled us to provide bits more often and gather feedback frequently. The goal with increasing the transparency of design is to take this one step further: we would like to enable folks that are interested in Entity Framework version 2 to follow the design topics as we discuss them, and have the opportunity to provide feedback right during the time where we are actively discussing a certain aspect and before we have made a final decision.

What exactly would we make visible? In short, our design process. To be more concrete, I’m not talking about some fancy set of specifications. What I mean is that as we go through the detailed design of the various aspects of Entity Framework V2, we would post to this blog a) the meeting notes from our design meetings (the team has a design meeting twice a week, plus a lot of impromptu hallway chats), and b) deeper write-ups of specific design challenges where we’d like folks to understand how we’re approaching a problem and provide a channel for deep, detailed feedback.

How transparent is transparent? We want to be completely clear about the scope of the information we are sharing. One of the things we need to learn both from the Microsoft side and from the community side is whether the model works within a practical set of restrictions. We will post as much of our discussions as it is practically possible. However, we have to make sure we don’t compromise the interests of Microsoft as a company. There are certain things that can range from ideas to specific implementation details that we could consider trade secrets, high-value Microsoft intellectual property or something along those lines. It *will* happen that in some cases we will not discuss a topic publicly, either for a certain term (e.g. until a proper IP protection mechanism is in place) or until we ship or ever. This is nothing new, but I haven’t seen folks from large companies discuss this explicitly before, so we wanted to make sure it is clear here.

About your feedback: We would love to hear your thoughts, be it comments, suggestions, ideas or anything else. However, in the end we are designing a commercial Microsoft product. So we’ll happily take your feedback but you need to understand that by providing us feedback in any form you are agreeing that we may use it to develop our product, that others may use it in connection with the product and that you will not be compensated for any of these things. We may incorporate ideas or make changes based on the comments you make, or we may make changes to the product that are indirectly influenced by discussions that we have with you and other folks in the community. Again, this is nothing new, but instead of having some fancy statement written in legal lingo we wanted to be upfront about this here in this first post on the topic. Of course our legal folks looked at this, and they were cool enough to understand that the informal nature of the process is what makes it work, and they let us get away with this statement in which I think we clearly delineate what will happen with whatever feedback you send our way.

We will start posting design notes and challenges soon and tweak the process as we go.

Tim Mallalieu
Program Manager, Entity Framework

Entity Framework Design : EDM

Foreign Keys in the Entity Framework

Background

So what's been added?

How do I use an "FK Association"?

Keeping FKs and References in Sync

How do I create an "FK Association"?

Summary

Update on Computed Properties

Model Defined Functions

Using the Function via eSQL:

Using the Function via LINQ:

Summary

Structural Annotations - One Pager

1 EDM extensions

1.1 Key points to notice:

1.2 Positive and Negative Cases:

1.3 What can an <Annotation> contain?

1.4 What elements can be annotated?

2 Metadata API Changes:

2.1 MetadataProperty TypeUsage

3 What is the MetadataProperty.Value?

3.1 XElement is rooted at the <Annotation> element

3.2 XElement is mutable

3.3 No universal annotation format

4 Out of Scope

Using Stored Procedures to load structured data.

Scenario:

Customer Goal:

Detailed Scenario walk through:

User defines a Stored Procedure in store:

Create Procedure GetCustomerInfoById( @Id int ) As SELECT first_name As Firstname, last_name As Lastname, city As City FROM CustomerTable WHERE id = @Id

Complex type definition in EDM:

The mapping involves following three steps:

Define import function definition in SSDL:

Expose function definition in CSDL schema file

Define mapping (convention based) MSL file:

Design Details:

Assumptions/suppositions:

Computed Properties One Pager

Scenario:

Scenario Detailed Walkthrough:

Interesting Points:

Computed Property Return Types:

Collections of Scalars:

Object Services:

Entities, ComplexTypes and RowTypes:

Random Examples:

Why DefiningExpression?

Transparency in the design process

Create Procedure GetCustomerInfoById(
@Id int
)
As
SELECT first_name As Firstname, last_name As Lastname, city As City
FROM CustomerTable
WHERE id = @Id