LinqLite
Working on .NET RIA Services has given me a new appreciation for LINQ. In fact, I even got up the nerve to try writing my own LINQ provider. Ouch, not easy. Authoring a LINQ provider is a bit like writing a compiler, requiring you to parse a myriad of complex LINQ expressions, each of which can contain a never-ending array of sub-expressions and clauses.
It’s well worth the investment to create a fully functional LINQ provider when your back-end data source is a relational database or a rich data abstraction layer like Entity Framework or nHibernate. When you’re working with a limited data provider which can’t support most complex query operations anyway, like a simple WCF service for example, the overhead in authoring the LINQ provider might be much more than it’s worth.
Enter LinqLite. LinqLite is a small library I put together which takes care of parsing the nasty LINQ expressions for you. In return, it hands back a much more simplistic set of clauses describing the original query. These clauses maintain the original query expression order, and give you the data you need to know in an easy-to-absorb format. You still need to write code to handle the query clauses for your specific data provider, but doing so is dramatically simpler and more contained than handling the full set of LINQ expressions.
The catch is that LinqLite does not support the full set of LINQ syntax. It does not support joins, for example. However, it does support the core set of LINQ expressions that you’re likely to find yourself using 90% of the time for simple queries over a simple data provider. And again, using LinqLite is a much less expensive way to expose an IQueryable over your back-end data provider.
In this blog post, I’ll describe the subset of syntax supported by LinqLite, and walk through an example implementation.
Supported Syntax
By default, LinqLite supports a limited set of LINQ syntax:
1. LinqLite supports the following LINQ expression types: Where, OrderBy, ThenBy, GroupBy, Skip, Take, Count
2. A LinqLite query may only operate over a single entity type. For instance the following query can be handled by LinqLite since it only includes a single entity type, Customer:
var result = this.customers.Where(c => c.Gender == "M");
3. The OrderBy, ThenBy, and GroupBy expressions are limited to accepting the name of a property on the entity and cannot take an IComparable parameter. For example:
var result = this.customers.OrderBy(c => c.Name).ThenBy(c => c.Gender);
4. The Where expression is limited to the following operator types: ==, !=, >, >=, <, <=, StartsWith, EndsWith, Contains
In addition, a Where expression must compare an entity property with another property on the same entity, an entity property with a constant value, or a constant value with an entity property. For example, the following queries can all be handled by LinqLite:
var result = this.customers.Where(c => c.Issues > 2);
var result = this.customers.Where(c => c.FirstName != c.LastName);
var result = this.customers.Where(c => “M” == c.Gender);
var result = this.customers.Where(c => c.Name.StartsWith(“Fr”));
If the query expression passed to the LinqLite query provider does not fall within the above constraints, LinqLite will throw an InvalidQueryException.
If you need support for additional LINQ syntax, it may be simpler for you to modify the LinqLite library to add this support, rather than writing your own LINQ provider from scratch.
Example Implementation
The following sample shows an implementation of LinqLite for use within a .NET RIA Services project (naturally). To keep the example very simple, the back-end data source is a simple hardcoded array of data. The example code shows how to parse each of the query clauses returned from LinqLite and then return the correct data from the original data set, based upon the parameters provided in the original query.
The full sample code can be downloaded here.
This sample code requires:
- VS2008 SP1 (Which includes SQL Express 2008)
- Silverlight 3 RTM
- .NET RIA Services July '09 Preview
As with most RIA Services projects, our sample project contains an ASP.Net Web Server project and a Silverlight client project, as well as the LinqLite library project. RIA Services allows the Silverlight client to pass LINQ queries back to the server over a WCF channel, which then grabs and returns the requested data. Since our back-end data store on the server is simply a hard-coded array of data, we need a way to break apart the query received from the client, and then filter and sort the data accordingly. Of course, we could use LINQ to Objects to query the data array directly, but for demo purposes we’re ignoring that and executing the query manually.
To start with, we need to create our strongly typed entities on the server. In our case we’re using two entities, SuperEmployee and SuperEmployeeOrigin, which are simple POCO classes.
Next, we create the back-end data set, which is just an array. This is stored in the static SuperEmployeeData class.
private static SuperEmployee[] superEmployees = new SuperEmployee[]
{
new SuperEmployee(0, "Foo Basdasdar", "Male", "Human", 123, "DC", "123"),
new SuperEmployee(1, "Cool Name", "Male", "Human", 982, "DC", "first appears in Batman #16"),
new SuperEmployee(2, "Alfred E. Neuman", "Male", "Human", 518, "Ec", "first appears in MAD #21"),
new SuperEmployee(3, "Angel", "Male", "Mutant", 1061, "Marvel", "first appears in X-Men #1"),
...
The SuperEmployeeData class returns two status properties, a SuperEmployees property which returns the entire array of SuperEmployees, and a SuperEmployeeOrigins property which returns an array of all the unique SuperEmployee origins.
Now that our entities and data are in place, it’s time to author our RIA DomainService class. In our simple example, the DomainService class will return a set of read-only query methods, each of which returns an IQueryable<SuperEmployee> or IQueryable<SuperEmployeeOrigin>. RIA Services will expose these query methods to the client, allowing the client to write LINQ queries against our data. The outline of our DomainService looks like:
[EnableClientAccess]
public class SuperEmployeeDomainService : DomainService
{
/// <summary>
/// Gets a query to match all super employees.
/// </summary>
public IQueryable<SuperEmployee> GetSuperEmployees()
/// <summary>
/// Gets query to match a single super employee with the given ID.
/// </summary>
public IQueryable<SuperEmployee> GetSuperEmployee(int employeeID);
/// <summary>
/// Gets a query to return all origins.
/// </summary>
public IQueryable<SuperEmployeeOrigin> GetOrigins();
}
Each of these methods needs to return an instance of a LINQ Query Provider which implements IQueryable<T>. The query provider will be called automatically when the query is composed and executed. The query provider will then need to parse the query, gather and return the requested data.
To go any farther we need to create our custom LINQ Query Provider, based on LinqLite. In our case we will actually create two Query Providers, one which knows how to query against SuperEmployees, and another to query for SuperEmployeeOrigins.
Because we’re creating two query provider classes, it makes sense to put as much of the code as possible into a base class. So, we’ll create a class in our server project called QueryableEntityBase, which inherits from the LinqLiteQueryable<T> class. LinqLiteQueryable<T> implements the IOrderedQueryable<T> interface, and does all the hard work of parsing the original query expression and translating it into a simple set of query clauses for your code to consume.
Then, we’ll create a couple of specific implementations of QueryableEntityBase for the two types of entities we want to be able to query against, QueryableSuperEmployee and QueryableSuperEmplolyeeOrigin.
The next step on our journey is to override the ExecuteQuery method within our QueryableEntityBase class. This method gets called after LinqLite has parsed the query, and it’s where you will inspect the query clauses returned from LinqLite and do whatever work needs to be done to return the requested data. Our implementation looks like:
protected override object ExecuteQuery(QueryExpression simplifiedQuery)
{
// Start off with the entire data set.
T[] resultSet = this.GetInitialResultSet();
// Iterate through the clauses.
foreach (QueryClause clause in simplifiedQuery.Children)
{
if (clause.ClauseType == QueryClauseType.Skip)
{
resultSet = this.HandleSkip((SkipClause)clause, resultSet);
}
else if (clause.ClauseType == QueryClauseType.Take)
{
resultSet = this.HandleTake((TakeClause)clause, resultSet);
}
else if (clause.ClauseType == QueryClauseType.OrderBy ||
clause.ClauseType == QueryClauseType.ThenBy)
{
resultSet = this.HandleOrderBy((OrderByClause)clause, resultSet);
}
else if (clause.ClauseType == QueryClauseType.Where)
{
resultSet = this.HandleWhere((WhereClause)clause, resultSet);
}
else if (clause.ClauseType == QueryClauseType.Count)
{
return resultSet.Length;
}
else if (clause.ClauseType == QueryClauseType.GroupBy)
{
throw new ArgumentException("Not supported.");
}
}
return resultSet;
}
Notice that this method first calls GetInitialResultSet, which is an abstract method which needs to return the full dataset. The QueryableSuperEmployee class implements this by simply returning the full collection of SuperEmployees:
protected override SuperEmployee[] GetInitialResultSet()
{
return SuperEmployeeData.SuperEmployees;
}
After obtaining the original data set, the ExecuteQuery method begins looping through each of the simple QueryClauses returned from LinqLite, which can be of type WhereClause, OrderByClause, ThenByClause, GroupByClause, SkipClause, TakeClause, or CountClause. In each case, it calls a special handler method which will return a new subset of the data matching the parameters in the query clause.
For example, the SkipClause class looks like:
public class SkipClause : QueryClause
{
public int Size
{
get;
}
}
This simple class just returns the requested number of items to skip.
And the skip clause handler looks like:
protected virtual T[] HandleSkip(SkipClause skip, T[] dataSet)
{
if (skip.Size <= 0)
{
return dataSet;
}
else if (skip.Size >= dataSet.Length)
{
return new T[] { };
}
else
{
T[] resultSet = new T[dataSet.Length - skip.Size];
for (int i = 0; i < resultSet.Length; ++i)
{
resultSet[i] = dataSet[i + skip.Size];
}
return resultSet;
}
}
Since our back-end data source is a simple array, this method simply skips past the requested count and returns the rest of the items in the array, if any.
Perhaps it’s a bit more interesting to look at how we handle an OrderBy expression. For instance, consider the case where the original query looks like:
var x = this.superEmployees.OrderBy(s => s.Gender);
For this query, LinqLite will hand you an OrderByClause object with the PropertyName property set to “Gender”, and the Direction property set to OrderByDirection.Ascending. Since the value of the PropertyName property is a simple string, we need to map this back to the actual property on the SuperEmployee entity which we’re supposed to order by.
Our sample code does this by calling to an abstract method called GetOrderByComparisonDelegate, which needs to return a Comparison<T> that, when called, will perform the comparison against the appropriate entity property. The rest of the code then performs a simple bubble sort to do the actual sorting.
protected virtual T[] HandleOrderBy(OrderByClause orderBy, T[] dataSet)
{
Comparison<T> comparer = GetOrderByComparisonDelegate(orderBy);
if (comparer != null)
{
// Perform a simple, inefficient bubble sort for demo purposes.
for (int i = 0; i < dataSet.Length; ++i)
{
for (int j = i + 1; j < dataSet.Length; ++j)
{
int comparison = comparer(dataSet[i], dataSet[j]);
… (ommitted)
}
}
}
return dataSet;
}
The QueryableSuperEmployee class implements GetOrderByComparisonDelegate as:
protected override Comparison<SuperEmployee> GetOrderByComparisonDelegate(OrderByClause orderBy)
{
Comparison<SuperEmployee> comparer = null;
switch (orderBy.PropertyName)
{
case "EmployeeID":
comparer = delegate(SuperEmployee x, SuperEmployee y) {
return x.EmployeeID.CompareTo(y.EmployeeID); };
break;
case "Gender":
comparer = delegate(SuperEmployee x, SuperEmployee y) {
return string.Compare(x.Gender, y.Gender, StringComparison.OrdinalIgnoreCase); };
break;
case "Issues":
comparer = delegate(SuperEmployee x, SuperEmployee y) {
return Nullable.Compare<int>(x.Issues, y.Issues); };
break;
case "Name":
comparer = delegate(SuperEmployee x, SuperEmployee y) {
return string.Compare(x.Name, y.Name, StringComparison.OrdinalIgnoreCase); };
break;
case "Origin":
comparer = delegate(SuperEmployee x, SuperEmployee y) {
return string.Compare(x.Origin, y.Origin, StringComparison.OrdinalIgnoreCase); };
break;
case "Publisher":
comparer = delegate(SuperEmployee x, SuperEmployee y) {
return string.Compare(x.Publishers, y.Publishers, StringComparison.OrdinalIgnoreCase); };
break;
case "Sites":
comparer = delegate(SuperEmployee x, SuperEmployee y) {
return string.Compare(x.Sites, y.Sites, StringComparison.OrdinalIgnoreCase); };
break;
default:
break;
}
return comparer;
}
The Where clause is handled in much the same way, by providing specific implementations for SuperEmployee and SuperEmployeeOrigin that know how to filter against specific properties on those entities, with two caveats. First, a LinqLite Where-clause is allowed to compare two properties on the entity, or it can compare an entity property against a constant. The QueryableEntityBase class handles this by first checking what type of value we find on the right and left-hand sides of the Where operator, and then calling either FilterOnProperties or FilterOnConstant, as required:
private T[] HandleWhere(WhereClause where, T[] dataSet)
{
WhereFilterPropertyMember p1 = where.LeftHandSide as WhereFilterPropertyMember;
if (p1 != null)
{
WhereFilterPropertyMember p2 = where.RightHandSide as WhereFilterPropertyMember;
if (p2 != null)
{
// Both sides are entity properties.
return this.FilterOnProperties(p1, p2, where.Operator, dataSet);
}
else
{
WhereFilterConstantMember c2 = where.RightHandSide as WhereFilterConstantMember;
if (c2 != null)
{
// Left side is a property, right side is a constant.
return this.FilterOnConstant(p1, c2, where.Operator, dataSet);
}
}
}
else
{
WhereFilterConstantMember c1 = where.LeftHandSide as WhereFilterConstantMember;
if (c1 != null)
{
WhereFilterPropertyMember p2 = where.RightHandSide as WhereFilterPropertyMember;
if (p2 != null)
{
// Left side is a constant, right side is a property.
return this.FilterOnConstant(c1, p2, where.Operator, dataSet);
}
}
}
// A comparison with two constants or something unsupported.
throw new InvalidQueryException("Invalid where query parameters");
}
The FilterOnProperties and FilterOnConstant methods set up a delegate which knows how to extract the value from the appropriate entity property at the appropriate time, much like we did with the OrderBy handler described above.
The second caveat is that a LinqLite Where-clause can support multiple types of comparison operators, such as equals, not-equals, greater than, StartsWith, EndsWith, etc.
The QueryableEntityBase class takes care of handling these operators based on the type of the entity property being filtered against. For instance, for a string property, the comparison looks like:
private static bool StringCompare(string s1, string s2, WhereFilterOperator @operator)
{
// Handle nulls.
if (s1 == null && s2 == null)
{
return @operator == WhereFilterOperator.Equals ||
@operator == WhereFilterOperator.LessThanOrEqualTo ||
@operator == WhereFilterOperator.GreaterThanOrEqualTo;
}
if (s1 == null || s2 == null)
{
return @operator == WhereFilterOperator.NotEquals;
}
switch (@operator)
{
case WhereFilterOperator.Equals:
return s1 == s2;
case WhereFilterOperator.NotEquals:
return s1 != s2;
case WhereFilterOperator.GreaterThan:
return string.Compare(s1, s2) > 0;
case WhereFilterOperator.GreaterThanOrEqualTo:
return string.Compare(s1, s2) >= 0;
case WhereFilterOperator.LessThan:
return string.Compare(s1, s2) < 0;
case WhereFilterOperator.LessThanOrEqualTo:
return string.Compare(s1, s2) <= 0;
case WhereFilterOperator.StartsWith:
return s1.StartsWith(s2);
case WhereFilterOperator.EndsWith:
return s1.EndsWith(s2);
case WhereFilterOperator.Contains:
return s1.Contains(s2);
default:
return false;
}
}
Though it’s not shown here, the WhereFilterPropertyMember class also exposes a CaseSensitivity property, allowing you to handle case-sensitive queries such as:
var x = this.superEmployees.Where(s => s.Name.ToLower().StartsWith(“s”));
Putting all of this together, we’ve now got a set of classes which allow us to author LINQ queries against the SuperEmployee data type or the SuperEmployeeOrigin data type, provided that the expressions used in the query are constrained to the set of clauses and operations supported by LinqLite. Whenever a query is executed, LinqLite will parse the original LINQ expression, throw an exception if it finds something it cannot support, and otherwise hand back the query as a set of QueryClauses. These lightweight classes are relatively easy to parse and apply to your data-set.
Of course, LinqLite supports all of the query expression types that RIA Services queries are allowed to use. We can now complete our RIA Servcies DomainService, which exposes the queryable objects from the server to the client:
[EnableClientAccess]
public class SuperEmployeeDomainService : DomainService
{
// Reference to the linq provider for our super employee service.
private QueryableSuperEmployee superEmployeeContext = new QueryableSuperEmployee();
// Reference to the linq provider for our super employee origins service.
private QueryableSuperEmployeeOrigin superEmployeeOriginContext = new QueryableSuperEmployeeOrigin();
public IQueryable<SuperEmployee> GetSuperEmployees()
{
return this.superEmployeeContext;
}
public IQueryable<SuperEmployee> GetSuperEmployee(int employeeID)
{
return this.superEmployeeContext.Where(emp => emp.EmployeeID == employeeID);
}
public IQueryable<SuperEmployeeOrigin> GetOrigins()
{
return this.superEmployeeOriginContext;
}
}
}
This means that I can now run my sample app, and my Silverlight UI can query against my server data to its heart’s content. The UI below contains a TextBox which allows me to enter an Origin value. When I type in “Human”, the UI sends a new query back to the server which asks for all SuperEmployees where the Origin property starts with “Human”, and displays only the matching rows in the grid:
References
The SuperEmployee placement service example used in this sample is a modification of the code written by Brad Abrams in his extremely prolific series on RIA Services.
The LinqLite query provider is a heavily modified version of the LINQ To TerraServer example on MSDN.
Disclaimers
Much of the code in the MyApp and MyApp.Web projects was written by Brad Abrams originally, so don’t flame me for the millions of StyleCop violations!
