All About Interop : LINQ

Many many LINQ Providers

DotNetInterop — Fri, 12 Sep 2008 18:28:00 GMT

When LINQ shipped, it included a provider model, to encourage people to make other systems and data stores accessible from the coolness-that-is-LINQ.

Back in February, Charlie Calvert published a long list of LINQ providers. In July, Robert Shelton later reblogged it. Charlie also has a list of LINQ Links. Nice to see. By my count there are 33 different LINQ providers listed. I'll bet there are more by now.

You can also search for more on LINQ providers.

LINQ and interop with other (non-Microsoft) databases

DotNetInterop — Tue, 22 May 2007 20:53:00 GMT

This topic came up on an internal mailing list at Microsoft.

The question is, will LINQ be usable with databases other than Microsoft SQL Server?

Before we answer that, first of all let us understand, what is LINQ?

What is LINQ?

LINQ is Microsoft’s Language Integrated Query. It is a set of extensions to the .NET Framework that provide common capabilities within the programming language for querying against in-memory data as well as external data sources. I had posted previously on LINQ, here.

This is pretty cool when you understand the implications. Rather than writing a for loop to scroll through a collection of objects, imagine if you could apply a query to the collection itself, to select some of the members of the collection, based on criteria you define. Imagine getting the query verbs you know and love from a RDBMS, but applying those things to random collections of objects, whether or not those objects were persisted in or retrieved from a database. This is what LINQ can provide.

When does LINQ become available?

LINQ will ship with the .NET Framework v3.5, which was previously code-named the "Orcas" release. (My esteemed colleague Paul Andrew blogged earlier about the content in the .NET Framework v3.5.) The .NET Framework v3.5 ("Orcas") will ship before the end of 2007; an update of Visual Studio will also ship at the same time. As with prior versions of the .NET Framework, the .NET Framework redistributable and SDK will be separately downloadable items. In other words, you can just use the SDK and the command line tools to build apps, but the visual toolset will add significant value.

As for the LINQ stuff in Orcas, originally the plan called for LINQ over Objects (query on arbitrary object collections), LINQ-to-XML (query over data derived from XML), LINQ-to-Datasets (you get the idea), as well as LINQ-to-SQL. Later we added LINQ-to-Entities into the plan for Orcas; this would be possible with the inclusion of the ADO.NET Entity Framework in Orcas as well. (but as Microsoft's Mike Pizzo notes, the ADO.NET Entity Framework has since been removed from Orcas!). You can read about the ADO.NET Entity Framework here. So to summarize, LINQ is in Orcas, the .NET Framework v3.5, which will release before the end of 2007. Also, the LINQ stuff in Orcas will not include Linq-to-Entities.

You can read Mike Pizzo's post yourself; here is the summary: We considered the ADO.NET Entity Framework important and big enough that we've decided to get it right, with good tools support, which meant holding it for release until after Orcas. This of course is why the the LINQ support for the Entity Framework won't ship in Orcas either! The ADO.NET Entity Framework (including LINQ to Entities support) will be marketed with SQL Server "Katmai" (read the press release, or check the datasheet), and will ship at the same time (first half 2008), but the two will not be technically linqed ;). The ADO.NET Entity Framework will ship as an update to the .NET Framework, which is generally available to developers. Using the update will not require SQL Server "Katmai" to be installed.

Where does the interop part come in?

Why are you reading all about LINQ on an interop blog? Here's the interop part: The LINQ-to-Entities that ships in Orcas+1 as part of the ADO.NET Entity Framework will include a provider model. What this means is that there will be multiple providers for multiple stores. Lance Olson tells me:

We’ve been working with a significant number of 3^rd party provider writers for more than a year now to ensure that support for all major databases is available for LINQ to Entities at RTM (RTM of Orcas+1 !! -dino) . LINQ to Entities is built on an extension of the ADO.NET 2.0 provider model so provider writers just need to update their 2.0 providers to support a new interface in order to get LINQ support. SQLLite has an open source provider on the web today working against Beta 1 for example. A number of these partners are planning to attend Tech Ed and will be announcing their support and demonstrating working providers there. We’re obviously very excited to be able to show such broad support for LINQ.

So, look for good interop support in LINQ-to-Entities, which is coming in the first half of 2008.

Next obvious question - what about a public provider model for LINQ-to-SQL, which would enable LINQ-to-any-other-RDBMS? We have not announced plans in that direction. We think the LINQ-to-Entities

other relevant posts:
http://blogs.msdn.com/data/archive/2007/04/28/microsoft-s-data-access-strategy.aspx

-Dino

LINQ versus DataSet ?

DotNetInterop — Thu, 15 Feb 2007 21:41:00 GMT

Today, someone asked on an internal mailing list, what does LINQ offer, than I cannot do with DataSet.Select()?

Luca Bolognese, my compadre from the old country, provided a clear and insightful response. Just for some background - LINQ is a codename for a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. Language-integrated query means, the query "verb" is built-in to the language itself. It is not a query library; it is integrated into the language. So you can do queries across in-memory graph of objects, as easily as you might send a query to a database to retrieve results. Here is Luca's response:

[DataSet.Select() and LINQ] are very different.

The dataset is an in-memory relational representation of data, Select works against this in-memory representation.
An object model is, well, an in-memory graph of objects. LINQ query works over this object view.
Select takes a string as input, so it is not strongly typed; LINQ queries are strongly typed.
LINQ queries work seamlessly across domains, you can submit the same query against in-memory objects, databases or, also, a dataset.

If you know at compile time the structure of your query and you are working with datasets, I suggest you investigate “LINQ to DataSet”.

By the way, here's the blog post he mentioned, LINQ-to-DataSet.

LINQ: the January CTP is out.

DotNetInterop — Sat, 13 Jan 2007 01:08:00 GMT

The January CTP for the next version of Visual Studio, code-named "Orcas" is now available. By CTP, Microsoft implies a Community Technology Preview - it is essentially a lightweight beta ship vehicle for a future product. By shipping this preview, Microsoft is trying to solicit feedback on the feature set ("I really like having that new dial to adjust my frimble."), on the realization of those features ("I like having that lever, but when I flip it this other thingy comes loose."), and on the prioritization of features ("The new dial makes my life easier, but I don't find much use for that new lever you've added.") The main highlights of this CTP are the new ADO.NET stuff (including the Entity Data Model) and the latest LINQ bits.

You might ask, just what is this LINQ thing? LINQ, the name, derives from "Language Integrated Query." LINQ is a codename for a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. It extends C# and Visual Basic with native language syntax for queries and provides class libraries to take advantage of these capabilities.

I took that description right from the main LINQ page on MSDN. Let me decompose that a little, because it's sort of dense. It makes sense if you already know what LINQ is, but if you come into it without any prior knowledge, I'm not so sure it clarifies anything.

First, LINQ is a set of extensions to a couple .NET languages, as well as to the base framework (or base class library). It's important to realize that LINQ is not just a new class library. LINQ also implies language syntax changes. You might be a skeptic and say, languages schmanguages - they're all Turing-complete, right? But no one who looks at Perl, T-SQL, and C# could avoid the conclusion that different languages have different balance points - they are optimized for different things. What LINQ is doing is extending the existing language syntax that we know and love in C# and VB today, to add some new stuff.

New stuff to do what? Ahh, to do query, set and transform operations. A query, like "Give me all the customers who ordered $10000 worth of product last month." Today, querying data is done differently, depending on where you get your data. If your data is in a relational database, then you use the DBMS query engine to do the queries. You write some SQL code as a string and send it to the database and it gives you rows of data back. (if you are an EJB programmer, then you write an EJBQL string, and a findByXxx method and a bunch of other stuff. ) On the other hand if you have your data in an XML document, then you can perform XPath or XQuery on it. Finally if you have your data in a plain old in-memory collection of objects, you may have to manually iterate through the collection to extract the data items that match your constraints. foreach anyone?

What LINQ attempts to do is standardize the way these queries are performed, regardless of the datasource, and secondly, to add query as a language feature, so that developers can get all the benefits (intellisense, and so on) of that.

For years different teams and projects in the software industry have grappled with the problem of object-relational mapping. The whole point was to save time in development. We all realize that much (most?) of the world's transactional data is stored in relational stores, and most new app development is done in object-oriented languages. So the challenge we've been grappling with has been, how to map the relational data to an in-memory object state, in effect making it simpler and easier for an OO programmer to fiddle with data. This effort has seen some small successes, but also some large failures. Also as more and more data moves into XML, mapping between relational stores and in-memory object representations seems to be missing half the point. We need something different!

Microsoft is taking a different tack on the question of how best to improve developer productivity around data handling, given the two constraints above - that data is relational (but more and more XML) and code is OO. With LINQ, Microsoft is saying, we think the opportunity for improvement is in the ability to query data more simply. It's not so much in the mapping - there are solutions here that get people "most of the way there". The challenge now is, once you mapped your DBMS schema to your Object schema, once you have filled a collection of objects from your database, then what? Can you filter that collection ("Give me all the customers that ordered over $10000 in the last month. Ok, now give me the ones whose orders included this particular product. Wait, now let me see the ones that ordered this other product.")

Or think a little more generally and consider an object that aggregates data from multiple stores - maybe some of them relational and some not. And you'd like to do "join" queries across all of that data. Wouldn't it be nice if that were possible without requiring the developer to speak 3 different dialects of query language - T-SQL, XPath and something else?

So that's what LINQ represents - Microsoft's integration of this query capability into the programming language itself. It is worth checking out; it is one of the things Microsoft believes will distinguish the next version of .NET from other alternatives, like Java. A big innovation. It seems that at least some people outside of Microsoft tend to agree. And with the January CTP of Visual Studio "Orcas", now you can check it out.

What about Interop, you say? Well that brings me to my point. Currently the design of LINQ allows for extensions so that one could integrate this in-language query capability with any backing store. Microsoft will do the work to integrate SQL Server, as well as XML content with the capability. We're hoping that given enough interest, we'll see a clear justification for doing integration with other relational stores. If you have strong opinions on this, please let me know.

Cheers,
-Dino