Tim Mallalieu's Blog.

Just a PM's random musings on data, models, services...

Vote of No Confidence

Vote of No Confidence

Rate This
  • Comments 37

So,

It's been a long, long time since I have posted anything on my blog. Reality is I tried to maintain a blog where I thought I could come up with wonderfully profound things to share with the world but clearly that was not the case. Having said that a few events have happened that prompted me to start blogging again.

1: We are about to ship the first version of EF
2: We have an interesting thread going on about a "
vote of no confidence" about our product
3: We are starting work on our next version of the product

Seems like this would be a good time to throw myself out there. First, to respond to the general thread in the community and secondly to be more available for conversations around the current and future versions of the product.

Anyway... I will attempt to be more available and present. People can always just ping me timmall@microsoft.com directly about any of this.

In response to the community threads I will paste a response here that I made, yesterday, on an internal thread on the topic....

The unfortunate reality is that these are scenarios that we care deeply about but do not fully support in V1.0. I can go into some more detail here. One point to note is that the choices on these features were heavily considered, but we had to deal with the tension between trying to add more features vs. trying to stay true to our initial goal which was to lay the core foundation for a multiple-release strategy for building out a broader data platform offering. Today, coincidentally, marked the start of our work on the next version of the product, and we are determined to address this particular developer community in earnest while still furthering the investment in the overall data platform. Here is my take on the points below and things we are thinking about moving forward:

INORDINATE FOCUS THE DATA ASPECT OF ENTITIES LEADS TO DEGRADED ENTITY ARCHITECTURES

In v1.0 we chose to support default scenarios where one would have a distinguished based class (our EntityObject) and we would provide a mapping solution from these classes to our EDM model and from the EDM model to the database. The reason for this was because the investment we were making was in terms of the EDM – a new conceptual data model that allowed people to describe the shape of their data in terms of their domain instead of the layout in the database. Our goal is to allow people, in the fullness of time to be able to have a set of common services in terms of the Entity Data Model. Today they have query (ESQL) and REST based Services (Astoria describes its metadata using the EDM), they also get a first generation Object Persistence facility with the Object Layer component in the Entity Framework. The Object Layer, of course, also comes with our first class LINQ implementation.

There are a couple of interesting things here…

1: We expose the start of a pure value based API using ADO.NET (Entity Client) so that you can issue ESQL queries against the database in terms of your domain model, in a store agnostic manner (i.e. using ESQL instead of TSQL of PLSQL…) . There are many partners in the building for whom this is the only way they would want to leverage the EF and the EDM. Partners that fall in this camp tend to be interested in queries and presentation, data transformation or stores where the schema is frequently changing (Universal Table implementations) so that relying on CLR classes becomes intractable.

2: The focus on the EDM for the ORM developer can appear to be a red herring, as we heard from customers. The EDM describes the shape of the domain objects but does not allow for encapsulation of behavior and provides limited constraints and no specific grammar for action-semantics. In order to define a domain model where one wanted to leverage the EDM and EF today one would have to do one of the following:

  • Define an EDM model, code-gen classes and implement logic in partial classes.
  • Define an EDM model and use this as a pure DAO with a parallel set of domain objects – you could abstract the EF via many patterns… the common one we here is the repository pattern (gotchas will be pointed out in the discussion of points below).
  • Define an EDM model for query and update and then use LINQ to project into Domain Objects…   You get a case for simpler query expression and use LINQ for a projection facility to retrieve and transform from the EDM objects to the true Domain Objects.

I think there are a couple of things we are trying to do in the next version that can help here, and we actively want the feedback of folks in the community to help make sure we do the right thing:

  • Code-First scenarios…  In the code first scenarios we allow people to define their own classes (POCO) which they can work with as they see fit. We then infer a model from the classes. For situations where one desires to have a model that cannot be inferred directly from the classes, developers will be able to use new CLR attributes or an external mapping specification.
  • Code-Gen Pipeline… For some framework scenarios it is desirable to start with an EDM model (possibly with custom annotations) and then generate either the DB, the classes or both. We are looking at a general pipeline that developers can plug into that will give a lot of flexibility in generating classes and databases as well as in the way a model can be derived from the database, CLR classes or some other form (maybe a model repository built by a third party, a UML tool, or some other artificat like sharepoint).
  • Adding action-semantics to the EDM...  We have looked at things we can do here. There are things that would be quite nice but likely they would come at a cost of addressing particular feedback we have heard or our other investments. We will share thoughts and designs on these and see what people think regardless.

EXCESS CODE NEEDED TO DEAL WITH LACK OF LAZY LOADING

The EF, today, does support lazy loading. It does not support IMPLICIT lazy loading… this distinction is subtle but important. In EF we, by default, do not load ends of references or collections we wait until asked to load these. Today, however, one must explicitly ask by calling a .Load method. We took a fairly conservative approach in v1.0, because we wanted developers to be aware of when they were asking the framework to make a roundtrip to the database… our take on “boundaries are explicit”. When one builds out a repository pattern or starts to abstract the EF in some other form of data access abstraction this can become an issue because the need to call an explicit method to perform a lazy load operation causes the EF abstractions to bleed up and now you are struggling with an intersection of concerns as opposed to a clean separation of concerns.

We have heard this feedback and are looking at supporting optional implicit lazy loading as well as other strategies around eager loading (such as general LoadOptions ala LINQ to SQL) in a future release of the EF  Until then, it is possible to build implicit lazy loading on top of the explicit mechanisms which the EF support in v1, and one of our team members has published to his blog and code gallery a sample which demonstrates how this can be done.

SHARED, CANONICAL MODEL CONTRADICTS SOFTWARE BEST PRACTICES

There seems to be confusion around this topic. We are not recommending that folks return to the days where we were evangelizing the use of XSD for “canonical schemas”. I don’t believe that people think that this is tractable. What we do believe, however, is that it is desirable to have a single meta-model (EDM if you will) with which you can describe many domain models and that by having a single grammar we can provide a set of common services on any given domain model. For example, consider an application that is to be written against a database with 600 tables. Do I believe that this app should have a single model with 600 Entity Types in it? No… Furthermore, do I believe that any given domain entity (say Customer) has only one shape in that app and that this shape must be the canonical shape for the entire Enterprise?… Heck no.

I would expect, however, that with a common way to describe these models I could do some interesting things in the fullness of times…

  • I could use reporting services over any instance of one of these models and define reports in terms of the domain entities instead of the underlying tables.
  • I could perform ETL tasks between two stores in terms of entities.
  • I could write sync services between a local store and a remote store where the sync contract and programming model on both sides are in terms of EDM.
  • If I invested in learning how to build a model to use with some tool like reporting services, I would not have to learn a new tool and model description language when it comes time to build a model for one of the other tools.

This is the world that we are working towards… We are not there yet, but a number of these scenarios are what we are aggressively pursuing now.

LACK OF PERSISTENCE IGNORANCE CAUSES BUSINESS LOGIC TO BE HARDER TO READ, WRITE, AND MODIFY, CAUSING DEVELOPMENT AND MAINTENANCE COSTS TO INCREASE AT AN EXAGGERATED RATE

Agreed. There are different developer segments for whom PI is either important or not. We do not have a good story about PI today. We are working on one. Based on customer feedback from over a year ago, we started work on PI, but we said from the first that we would be unable to complete that work in v1 given the other demands of the release.  We made some initial steps for v1, and then began planning future steps for the next release.  At the last MVP summit we presented an early look at one of the techniques we were investigating which involved performing IL rewrite tricks to support these scenarios. We got strong feedback from many of the signatories on the “vote of no confidence” letter that this was not the right approach, and as a result we are not doing the IL rewrite in the next release. Instead we are doing a full POCO implementation that we hope to get feedback from the community on.

Tomorrow we will post some of the initial thinking on the new “EF Design Blog” including a little video of some of the initial dev work we are doing here. It is worthwhile pointing out that this work is a direct response to the feedback we have gotten from this community.

EXCESSIVE MERGE CONFLICTS WITH SOURCE CONTROL IN TEAM ENVIRONMENT

At the end of the day, this is feedback that we have heard, tried to address and are actively addressing moving forward. We are taking steps that we think should help in general:

1: We intend to have a more transparent design process.
2: We have a new advisory council to be more proactive in getting thoughts on our work and our directions.  This advisory council has some key community folks participating:

    Eric Evans - http://www.domainlanguage.com/about/ericevans.html 
    Stephen Forte -
http://www.stephenforte.net/ 
    Martin Fowler -
http://martinfowler.com/ 
    Pavel Hruby -
http://www.phruby.com/ 
    Jimmy Nilsson -
http://jimmynilsson.com/

3: We intend to drop more frequent interim builds. These builds will be unsupported interim builds where people can see what we have rolled into the product earlier and more often. We are still working the logistics of this out.
4: For our internal development we will be doing significantly shorter iterations.  We had quite long milestones in Orcas which made it more difficult to respond to customer feedback, and we hope that the transparent design process, the advisory council and the frequent drops will give us more feedback, and that with shorter iterations we will be more capable of responding.

Here is the thing though, even though we attempt to address the feedback, we know we won’t get everything.  From a product ownership perspective it pains me greatly to read the “no confidence” letter.  Especially as we have spent so much of the last couple of months working internally on how we can get better.  It is ironic that we find out about this letter the same week that we were already planning to roll out our goals for our engineering process for this next release, but such is life. Many of the signatories have given us great feedback, and I only hope that they will continue to do so as we proceed.

 

 

Leave a Comment
  • Please add 6 and 5 and type the answer here:
  • Post
  • This "vote of no confidence" is a tad harsh and I do feel for you guys.  It must be terribly disheartening for the EF team.  Please don't let this discourage you though!  The goals in principle are definitely a step in the right direction and it is reassuring to hear that you're taking the community's criticisms onboard with plans to address the issues in v2.  In a way it's good that you're getting so much feedback – this is obviously an important area that people are very passionate about.

    That said... I do have to say it is *very* frustrating to get all excited about a product, only to find that there are limitation that make real world scenarios painful to implement.

    As Brian (above) mentions, WPF is a perfect example.  I can't believe that WPF only recently got support in SP1 for hooking up events in the designer!  It's *basic* things like this that really should have been in from day one that frustrate developers the most.

    Same goes for the EF.  You already know what the major pain points are in v1.  So why release something that you *know* isn't done?  Is it sensible to tie yourselves to the SP1 release schedule?  Why not start on v2 right now and release that as an out of band release?  To be honest, I'm a little surprised that so much new technology is being introduced in a service pack.

    Just a suggestion, but why not release the whole of the EF on CodePlex.  Do frequent releases and let the product mature, and when it's ready, then bake it into the .NET Framework.  A really good example of this approach was the ASP.NET AJAX (Atlas) project.  That was a resounding success and the project quickly matured and made it into the framework.

  • I think the fundamental issue here is that the "coding in a box, then seeking feedback on preview or final releases" is a flawed development model *when*, and only when, you're dealing with generic frameworks that emphasize developer productivity and/or manageability, like NHibernate or Subversion, rather than highlighting and supporting a proprietary Microsoft product like Office SDKs.

    There's no question you guys are working hard, and kudos for your work. But this is very specifically an area where only the brightest and most experienced software architects currently living in the universe CAN lead, and if the intent is not to lead, why bother?

    Great list of members in the advisory committee, but now the questions become how much are you willing to rely in their advice, and to what extent are you willing to submit to their leadership?

  • I came across the "ADO .NET Entity Framework Vote of No Confidence" at http://efvote.wufoo.com/forms/ado-net-entity-framework-vote-of-no-confidence/

  • I often hear that MS don't listen, I think your response shows it does.

  • "but we had to deal with the tension between trying to add more features vs. trying to stay true to our initial goal which was to lay the core foundation for a multiple-release strategy for building out a broader data platform offering".

    The question of course arises whether the cart should be put before the horse. If the consumers have told you what they want, and the product does not meet these needs, why flog the dead horse? Shouldn't customer needs drive or at least guide product development?

  • Brian Grant is a Half Balked Trout Stuffer

    Learn how to spell pal, then you can fling the poo.

  • Let's start this post with a couple of warnings / disclaimers: Be advised that this post mixes metaphors

  • .Net -i arendajate kommuuni poolt on postitatud väga asjalik "Vote of no confidence" artikkel ADO.NET

  • i went to read the list of people and when i press for page 2, nothing happened (loading data...)   is it nHibernate performance (lol) ?

  • When I first heard about the vonc, I was one of those that thought that it was a little too harsh and unfair. After all, the EF is a very promising piece of technology and it's just making its first step in its 1000 mile journey.

    But then I got it. The team should actually be proud that the vonc exists at all. You see, when you deeply care about something, this is when you honestly try to fix what you think needs to fixed, otherwise you couldn't care less and you'd just be ignoring it. I think the whole industry has high hopes from the EF and that's why the vonc. And who better to make the loudest resonating claim than the best the .net community has, the mvps. Most of the industry trusts them to know what ticks right and what needs a little bit of tweaking. They are eye openers for people like me who are less experienced. Without them I would have thought that the EF is perfect. And if they make a such loud call, all of us should listen.

    And because of this heavyweight figures, the EF team should be proud of their baby, not hurt. The best of the best do really care about the EF that they gotta be sure that it shapes up nicely and the opportunity won't be missed. So, the bottom line is, this is all constructive criticism coming from good intentions. Keep up the great work.

  • None of the other things matter if I can't reliably version my source code. The merge problem is the biggest one, and while you seem to be rather flip about it, it really makes the difference between whether I can use the EF in the real world or not. That is the only reason I signed the petition. I can deal with the rest of the problems - they are differences of opinion, and will work out over time, but without reliable version control, I may as well use Access.

  • I had to be in Los Angeles a couple weeks ago now for some cross training and to meet my awesome development

  • I had to be in Los Angeles a couple weeks ago now for some cross training and to meet my awesome development

  • I'm sure many of you have heard about a very common open source acronym: LAMP . If you haven't

  • I am confused about the need for yet another language like ESQL when there is LINQ. This is all very confusing ? I was considering adding LINQ to SQL but now you saying that LINQ can not handle this because of flaw in LINQ or performance or something. Dump ESQL altogether.

Page 2 of 3 (37 items) 123