LINQ: Building an IQueryable Provider - Part II

Roller — Tue, 31 Jul 2007 21:41:25 GMT

Now, that I’ve laid the groundwork defining a reusable version of IQueryable and IQueryProvider, namely

Dealing with Linq’s Immutable Expression Trees

Jomo Fisher -- C#, LINQ and Whatnot — Tue, 31 Jul 2007 23:08:58 GMT

Jomo Fisher --I recently got a question via my blog that dovetailed nicely with something I’ve been working

re: LINQ: Building an IQueryable Provider - Part II

Roger Jennings — Tue, 31 Jul 2007 23:17:13 GMT

Pingback from http://oakleafblog.blogspot.com/2007/07/linq-changes-from-orcas-beta-1-to-vs.html (updated)

re: LINQ: Building an IQueryable Provider - Part II

kfarmer@microsoft.com — Wed, 01 Aug 2007 01:05:38 GMT

Shout, shout, let it all out.

These are the things we can't code without.

Come on.. We're talkin' to you.. Come on!

(Even DLinq folks have home projects they'd like the visitor exposed for...)

re: LINQ: Building an IQueryable Provider - Part II

Frans Bouma — Wed, 01 Aug 2007 11:57:25 GMT

Thinking it all through and re-reading also your second post, I still think it's not good to have the queryprovider in the queryable as it gives unnecessary clumbsyness to the code.

Let me give an example :)

in some BL class B, I create the query:

EntityStore s = new EntityStore();

var q = from c in s.Customer select c;

Pretty simple. According to your articles, q now has a reference to the queryprovider that will be used to produce (or will call into additional providers):

a) the query (which is db specific)

b) the objects to return when q is enumerated.

the problem steps in when q is passed back to the caller of the method in B to be executed somewhere else (it's deferred execution after all :)).

The code which will consume q by enumerating over it then has to assume the DB specific engine is already in q. Otherwise how is q able to produce Oracle specific SQL? So, the code which consumes q can't simply do:

foreach(IEntity e in q)

{

// do something

}

as q needs extra info, namely the db specific engine, be it sqlserver, oracle etc.

A typical place to put the engine in is in that method in B, where EntityStore is instantiated. However, that would make B being tied to the different databases supported by the application, which is not likely a thing one wants to do. After all, the entity store contains entity definitions which are database agnostic, and are persistable on any store supported by the o/r mapper core.

So how I understand it, the db specific engine to use at _execution time_ has to be injected into the provider inside q AFTER q has been created and also possibly at a different location than where q was created.

Typically, one without the necessity for deferred execution would likely say:

to execute the query, do:

IEnumerable<IEntity> list = q.Execute(dbSpecificQueryProvider);

How would MS suggest solving this problem? Sure I can create a queryprovider which produces objects in our own query system, no problem, but how are these executed on the db specific provider if there's no spot to inject a db specific provider later on, as q is enumerated + executed at the same time!

re: LINQ: Building an IQueryable Provider - Part II

mattwar — Wed, 01 Aug 2007 18:28:01 GMT

Frans, I think you are still confusing LINQ provider and ADO provider. A LINQ provider is an implementation of the standard query operator pattern. A LINQ IQueryable provider is a little more specific since you don't writer your own implementation of the pattern, you implement the IQueryable interface and build some kind of query translator.

You should look at a LINQ IQueryable query the same way that you would look at a LINQ to Objects query. For example, imagine you apply the Enumerable.Where operator to an array of customers. The query is deferred, yet it is also tied both to the 'data' in the array and to the means of executing the query operation.

If you build an IQueryable provider that allows execution against a database your queries will be built based on 'data' exposed by your provider and will be tied to the execution that your provider provides. That execution does not have to target only a single ADO provider or use ADO at all. You don't have to make the determination of what specific translation or execution API you need to use until at runtime when your provider is called upon to execute the query.

Again, a LINQ IQueryable provider is not the same thing as an ADO provider. Your LINQ provider is your ORM product in general, not the specific use of it. In your example above the user of the query will never need to assign the query to execute against a particular provider. That information was locked in when the query was originated. If you follow the LINQ to SQL example, or even the example in this post you will see that queries originate and are tied back to the 'context' that captures all the how-to-execute state.

re: LINQ: Building an IQueryable Provider - Part II

Frans Bouma — Wed, 01 Aug 2007 19:56:03 GMT

Sure, where the query originates, there's where the db specific information has to be injected into the query provider somehow so when the query is executed, the information is there.

But that's precisely what I don't want, as it would tie the place where the query originates to the selection requirement of the db to use. Perhaps that's not up to that place to decide (yet).

Linq to sql carries the context along with the queryable produced, Linq to nhibernate does the same thing. This is IMHO not ideal as it could force you to make decisions at a level you don't want to make these decisions yet: a query is a formulation of what should happen, a series of commands, not the engine as well.

I'll figure something out, but IMHO the design is flawed, and the root cause is the fact that it's a combination of concerns: a query and an enumerable resultset. I know MS won't change this, but I do think that this requirement to choose the db specific engine when the query is created and also the fact that this info is carried along the path that the query follows (which is IMHO not good, as it has no relevance along that path, only at execution time), is a big downside of the whole Linq system.

re: LINQ: Building an IQueryable Provider - Part II

Michael Friis — Wed, 01 Aug 2007 21:18:49 GMT

These to posts have been great, thanks. I've spent the longest time poring over various descriptions and trying to understand other implementations - now it all comes together :-)

Great with the Visitor - I managed to extract and compile the internal one using Reflector, using yours is probably a more kosher.

Michael

LINQ: Building an IQueryable Provider - Part III

The Wayward WebLog — Thu, 02 Aug 2007 00:13:58 GMT

Part III? Wasn’t I done in the last post? Didn’t I have the provider actually working, translating, executing and returning a sequence of objects? Sure, that’s true, but only just so. The provider I built was really fragile. It only understood one major

How to build custom LINQ provider?

Galin Iliev [Galcho] Blog! — Thu, 02 Aug 2007 12:08:55 GMT

LINQ: Building an IQueryable Provider - Part IV

The Wayward WebLog — Thu, 02 Aug 2007 23:27:16 GMT

I just could not leave well enough alone. I had the crude LINQ provider working with just a translation of the Where method into SQL. I could execute the query and convert the results into my objects. But that’s not good enough for me, and I know it’s

LINQ: Building an IQueryable Provider - Part V

The Wayward WebLog — Sat, 04 Aug 2007 01:55:54 GMT

Over the past four parts of this series I have constructed a working LINQ IQueryable provider that targets ADO and SQL and has so far been able to translate both Queryable.Where and Queryable.Select standard query operators. Yet, as big of an accomplishment

Créer provider Linq expliqué de A à Z

Wed, 08 Aug 2007 10:53:30 GMT

Matt Warren présente sur un blog une implémentation d'un provider Linq vers SQL en plusieurs étapes.

LINQ: Building an IQueryable Provider - Part VI

The Wayward WebLog — Fri, 10 Aug 2007 01:59:08 GMT

This is the sixth in a series of posts on how to build a LINQ IQueryable provider. If you have not read

Risorse su Linq to SQL

jankyBlog — Mon, 27 Aug 2007 10:58:34 GMT

Risorse su Linq to SQL

LINQ: Building an IQueryable provider - Part VII

The Wayward WebLog — Wed, 05 Sep 2007 03:00:16 GMT

This is the seventh in a series of posts on how to build a LINQ IQueryable provider. If you have not

Community Convergence XXXI

Charlie Calvert's Community Blog — Wed, 26 Sep 2007 03:38:49 GMT

Welcome to the thirty-first edition of Community Convergence. This issue features links to seven very

LINQ: Building an IQueryable Provider - Part VIII

The Wayward WebLog — Wed, 10 Oct 2007 00:49:26 GMT

This is the eighth in a series of posts on how to build a LINQ IQueryable provider. If you have not read

TerraServer Sample: A LINQ Provider

Charlie Calvert's Community Blog — Mon, 07 Jan 2008 04:34:54 GMT

Over the holidays Alex Turner, Mary Deyo and I added a new sample to the downloadable version of the

TerraServer Sample: A LINQ Provider

Noticias externas — Mon, 07 Jan 2008 05:33:47 GMT

Over the holidays Alex Turner, Mary Deyo and I added a new sample to the downloadable version of the

LINQ: Building an IQueryable Provider - Part IX

The Wayward WebLog — Thu, 17 Jan 2008 06:06:17 GMT

This is the nineth in a series of posts on how to build a LINQ IQueryable provider. If you have not read

LINQ Tips: Implementing IQueryable Provider

Shahed Khan (MVP C#) — Thu, 07 Feb 2008 08:49:24 GMT

Check out the following from Matt Warrens blog posts, if you are interested on how to implement IQueryable...

LINQ Tips: Implementing IQueryable Provider

Shahed Khan (MVP C#) — Thu, 07 Feb 2008 08:49:49 GMT

Check out the following from Matt Warrens blog posts, if you are interested on how to implement IQueryable

Linq to NHibernate - The Day After

Kyle Baley - The Coding Hillbilly — Wed, 09 Apr 2008 15:34:50 GMT

What a difference a day makes. Spent the better part of yesterday trying to figure out the Linq to NHibernate

LINQ to Entities: Combining Predicates

meek — Sat, 03 May 2008 03:31:13 GMT

Someone asked a great question on the ADO.NET Entity Framework forums yesterday: how do I compose predicates

Using LINQ Expressions to Generate Dynamic Methods II

meek — Wed, 14 May 2008 00:37:04 GMT

A beta of Visual Studio 2008 SP1 was released on Monday and the ADO.NET Entity Framework (EF) is now

LINQ: Building an IQueryable Provider - Part X

The Wayward WebLog — Wed, 09 Jul 2008 02:27:44 GMT

This is the tenth in a series of posts on how to build a LINQ IQueryable provider. If you have not read the previous posts you'll want to find a nice shady tree, relax and mediate on why your world is so confused and full of meaningless tasks that it

LINQ: Building an IQueryable Provider - Part XI

The Wayward WebLog — Mon, 14 Jul 2008 22:18:31 GMT

This is the eleventh in a series of posts on how to build a LINQ IQueryable provider. If you have not read the previous posts you’ll want to do so before proceeding, or at least before proceeding to copy the code into your own project and telling your

[PL] Własny dostawca w LINQ (dla opornych) - cz. 3

Only Human | Devoted to technology v.2.0 — Tue, 15 Jul 2008 16:42:02 GMT

Artykuł został rozbity na wiele części, poniżej spis treści: I.  Budowa zapytania II. Podstawy analizy

Book Links

Programming Linq Book Log — Tue, 19 Aug 2008 20:24:39 GMT

Chapter Page Url 4 111 http://msdn2.microsoft.com/en-us/library/bb386907.aspx 4 111 http://msdn2.microsoft.com/en-us/library/bb399400.aspx

Desenhando Edms e criando queries dinâmicas

Arquitetura em Pauta — Sun, 16 Nov 2008 16:49:12 GMT

Visitamos outro dia os arquitetos de uma empresa de ERP que estão desenvolvendo com o Entity Framework

Building a LINQ IQueryable Provider - Part XII

The Wayward WebLog — Tue, 18 Nov 2008 04:00:05 GMT

This is the twelfth in a series of posts on how to build a LINQ IQueryable provider. If you have not

LINQ links

The Wayward WebLog — Tue, 18 Nov 2008 20:24:23 GMT

Part I - Reusable IQueryable base classes Part II - Where and reusable Expression tree visitor Part II

LINQ: Building an IQueryable provider series

Floating Heart — Thu, 19 Mar 2009 17:49:58 GMT

http://blogs.msdn.com/mattwar/pages/linq-links.aspx Here's a list of all the posts in the building