Looking for feedback: query caching in data services

Airline Travel » Looking for feedback: query caching in data services

Airline Travel » Looking for feedback: query caching in data services — Tue, 01 Apr 2008 02:09:21 GMT

PingBack from http://www.travel-hilarity.com/airline_travel/?p=2357

Query caching in ADO.NET Data Services

Marcelo's WebLog — Tue, 01 Apr 2008 02:30:52 GMT

Pablo has a new post on the Project Astoria Team blog about a few ideas we're considering on how to cache

re: Looking for feedback: query caching in data services

Eric J. Smith — Tue, 01 Apr 2008 07:35:21 GMT

What happens if you want to restrict the data by what the user specifically has access to and not by roles? Would that still be possible?

re: Looking for feedback: query caching in data services

pabloc — Tue, 01 Apr 2008 07:40:03 GMT

Eric: yes, that would be possible in both of the approaches discussed here; in fact, that's why we're looking at this in so much depth; we want to make sure this is a good low-level mechanism to implement various row-level access policies on top. For the particular case of per-user access, you could have a table in the database that holds the access-control entries indicating which users get to see which records. At the interceptor level you'd write a predicate that injects lookups into those tables as conditions (which is what the "CustomerAccess" property hints at in the samples of the blog post).

-pablo

re: Looking for feedback: query caching in data services

Aleksandar — Sun, 06 Apr 2008 09:58:12 GMT

I would favor performance in expense of usability of interceptor code and here is why. We are using ADO.NET Data Services for our core banking application and our mechanics for restricting access to entity instances is based on combination of entity ownership, OU hierarchy, team membership and entity sharing. In runtime expressions are supplied with user attributes extracted from security context and the pattern is the same for all entities. I expect that anyone writing security interceptors will avoid case by case expressions for different entities and seek to apply uniform pattern. Since devising this pattern is one time job, usability shouldn't be a problem.

Aleksandar

re: Looking for feedback: query caching in data services

pabloc — Sun, 06 Apr 2008 11:03:09 GMT

Aleksandar: thanks for the feedback. Do you have examples of interceptors you can share (in this blog or offline)? I'd be curious to see what are the patterns for the interceptors you're using, both from the query perspective and from the point of view of dependencies from the environment.

Do you use information from the runtime context (e.g. request header, login, etc.) to establish the criteria for filtering?

Thanks,

-pablo

re: Looking for feedback: query caching in data services

Michael Brundage — Wed, 23 Apr 2008 09:04:14 GMT

Hi Pablo,

Well, now OnStartRequest and InitializeService need to stay in sync somehow. Either you statically verify this, or developers are going to experience considerable debugging pain.

Perhaps this could be automated...

What's really going on here is that you've allowed implicit variables (environment methods like GetUser() and the properties on the objects those methods return). I could write a query interceptor that looks at your user agent or any other http headers that were passed with the request (?).

Astoria controls the environment APIs that the query interceptors access -- methods like GetUser(). You could return a proxy object from every such method, and on those proxies intercept the property accessors (UserDescriptor.IsAdmin, UserDescriptor.UserID, etc.). When the query runs the first time, Astoria could record every implicit parameter upon which the query depended, and include those values as part of the cache key. When a subsequent query passes the same parameters, you can return the result (result cache). When it passes different parameters, you may be able to re-evaluate only the interceptors, or you might need to re-compile the whole query.

In your example above, suppose IsAdmin is true the first time the query is executed. Astoria memoizes the query tree with all its dependent parameters includeing IsAdmin. Subsequently another query comes in with IsAdmin false. Astoria re-evaluates the interceptor, which now reads UserID which was not previously read, invalidating the query cache.

You could add an extra layer or two of cleverness to this. Clever #1: diff the two query plans and cache the part they had in common. Clever #2: push the if test into the predicate. See, when IsAdmin was true, there's no where clause in the query plan, and when IsAdmin is false, there is. So you can push the IsAdmin test into the where clause also -- this query interceptor can be entirely pushed into the database.

Two other approaches that are more wildly divergent from your current model:

1. 80% rule. Recognize that paging and user auth are the things that tend to vary most, and special-case those.

2. Treat queries as resources -- stored queries, if you will. The user stores the query resource (with all parameters made explicit, including those that interceptors depend on) and can then GET the query results by hitting the endpoint and passing parameters. The query cache becomes just an ordinary endpoint cache. Personally I like this model best, because it's simple and implements the system using the system.