Man vs Code

Peter Qian and row count

2009-03-18T04:51:20Z

Peter Qian, the newest member of the ADO.NET Data Services, now has his own blog.

In his initial post, Peter talks about the row count feature we included with v1.5 of ADO.NET Data Services. He was one of the main designers and the implementer of this feature, so he knows what he is talking about. What he doesn't know about quite yet is what happens when the community finds he owns the #1 customer ask for ADO.NET Data Services and he now has a blog.

updated Linq to Sql IUpdatable implementation

2009-03-17T23:52:49Z

Based on feedback from Robert Fonseca-Ensor, I have made a bunch of bug and performance fixes to the IUpdatable implementation for Linq to Sql and posted those to code gallery. Rob is also now my first official contributor to the project.

As part of ADO.NET Data Services v1.5 we have provided a new way to plug in data sources that provides the following features sorely missing from v1:

1) Does not require the data provider to provider CLR types representing the entities exposed by the data source. This meta data can be provided dynamically at runtime (or even per query) via a new metadata interface and does not require any static typed entities.

2) Provides a much cleaner way for third party provider writers to write and ship stand alone ADO.NET Data Service providers. (No more partial classes like my Linq to Sql IUpdatable implementation).

In the coming days I will posting a walk through for converting the Linq to Sql IUpdateable implementation to a full blown Linq to Sql provider for ADO.NET Data Services to demonstrate the new interface.

Astoria v1.5 CTP now available

2009-03-17T22:28:00Z

In the fine tradition of Saint Patrick's Day we have released the first CTP of ADO.NET Data Services v1.5 today. Download here. Team blog announcement and "Getting Started" walk through video here.

This is definitely a green release - with features like server driven paging and streaming blob support which will save you millions of server cycles. We are planning on having several post on the team blogs explaining these and the rest of the v1.5 features soon. Also probably some more of our design videos if I can get a spare moment...

VSlive Presentations and demos

2009-03-05T06:08:00Z

Last week I presented two talks about ADO.NET Data Services at VSLive in San Francisco. As promised to the talk attendees, here are the slides and demos.

OOB == Out of Band

2009-03-02T20:08:16Z

Yesterday we announced that we will be releasing an update to ADO.NET Data Services that we are calling ADO.NET Data Services v1.5. For more details, see Mike Flasko's write up here on the team blog.

We should have a CTP out very soon. I am personally very excited about having a ship vehicle that will allow us to ship quickly and hence get some new features out quickly (i.e. Row Count, Binding, Server Driven Paging, etc) to solve some key problems that we have heard about from the community.

Now we just have to find some time to film a few more design videos...

Using timeout setting with Astoria client

2009-01-13T21:06:21Z

Believe it or not, but we did ship ADO.NET Data Services with a few bugs. One issue in particular is somewhat nasty. However the code developers might be tempted to write to work around the bug could cause far worse problems down the road when we fix the initial bug. Hence I wanted to write a short post to describe the issue and give a quick sample of how to write code that will work now and in the future when we fix the issue.

The problem exists in the ADO.NET Data Services client (System.Data.Services.Client.dll) we shipped with .NET 3.5 sp1 and Silverlight 2.0 SDK. It also is a issue for developers using the Windows Azure Table Storage API since that is built on top of the ADO.NET Data Services client.

In particular, we do not throw the correct exception when a HTTP request has problems while being sent to the server. I.e. a request timeouts, there is a bad server name, etc. In other words, any problem that would normally raise a System.Net.WebException. Unfortunately, in V1 of ADO.NET Data Services a NullReferenceException is raised instead of a WebException and even worse, the exception message is lost:

 DataServiceContext context = new DataServiceContext(new Uri("http://BadServer/Northwind.svc"));
 context.Timeout = 1; // Set small timeout amount to intentionally timeout request
 DataServiceQuery<Customer> query = context.CreateQuery<Customer>("Customers");

 try
 {
     foreach (Customer c in query) { }
 }
 catch (NullReferenceException nre)
 {
     // in V1, NullReferenceException is thrown instead of WebException.
     // Also, no useful description
     Console.WriteLine("Something bad happened!");
 }

We are planning on fixing this bug in a future version of ADO.NET Data Services client (both in .NET and Silverlight) so if your code relies exclusively on catching the NullReference exception, it will be broken when we ship the fix. Hence, to make your code robust – have your code explicitly catch both exception types:

 DataServiceContext context = new DataServiceContext(new Uri("http://BadServer/Northwind.svc"));
 context.Timeout = 1;  // Set small timeout amount to intentionally timeout request
 DataServiceQuery<Customer> query = context.CreateQuery<Customer>("Customers");

 try
 {
      foreach (Customer c in query) { }
 }
 catch (WebException e)
 {
     // When fix is shippped, will ADO.NET Data Services will correctly throw webException
     Console.WriteLine(e.Message);
 }
 catch (NullReferenceException nre)
 {
     // in V1, NullReferenceException is thrown instead of WebException.
     // Also, no useful description
     Console.WriteLine("Something bad happened!");
 }

We will update the team blog when we have a better idea of when the fix will be available. All I can say now is we are looking into trying to get it out sooner than later, but we hope to have more information on this in the near future.

MSDN Webcast: geekSpeak: Silverlight and ADO.NET Data Services with John Papa

2008-12-16T14:52:17Z

On Wednesday John Papa will be giving a webcast about anything and everything you ever wanted to know when consuming Astoria Data Services from a Silverlight 2.0 app. Should be a good one.

I also noticed that his new book will be out January 2nd. I have already preordered my copy. I do wonder where O'Reilly is finding animals for the covers these days.

What is IUpdatable? Why should I care?

2008-12-15T22:59:05Z

Over the past few weeks I have posted a couple of samples of implementing IUpdateable for non-EF DALs so those can be used as data sources for ADO.NET Data Services. A handful of people have asked me why this is interesting, so I now realize I should have given more background up front.

Out of the box, the only DAL that gets read/ write behavior for free in ADO.NET Data Services is Entity Framework. This is because internally in the ADO.NET Data Services bits, there is an implementation of an interface we call IUpdatable:

namespace System.Data.Services
{
    public interface IUpdatable
    {
        void AddReferenceToCollection(object targetResource, string propertyName, object resourceToBeAdded);
        void ClearChanges();
        object CreateResource(string containerName, string fullTypeName);
        void DeleteResource(object targetResource);
        object GetResource(IQueryable query, string fullTypeName);
        object GetValue(object targetResource, string propertyName);
        void RemoveReferenceFromCollection(object targetResource, string propertyName, object resourceToBeRemoved);
        object ResetResource(object resource);
        object ResolveResource(object resource);
        void SaveChanges();
        void SetReference(object targetResource, string propertyName, object propertyValue);
        void SetValue(object targetResource, string propertyName, object propertyValue);
    }
}

The reason behind this interface is that when we were designing V1 of ADO.NET Data Services we wanted to the user to have the ability to plug in LINQ data sources(.e. Linq to Sql, Entity Framework, nHibernate, etc). For read only access, this is really straight forward and general uniform across LINQ DALs. All the data source has to expose is a type with public IQueryable<T> entry points for each of their EntitySets. For write operations, it was a completely different story since there is no common API for updating LINQ based DALs. So we invented IUpdateable.

So any data source for Astoria that wants to support PUT, POST, and DELETE they must have an implementation of IUpdateable for their specific DAL. Hence why I started projects on Code Gallery to provide reference implementations of implementing IUpdatable for SubSonic and Linq to Sql.

There actually are a few more smaller interfaces to implement to become a full fledge ADO.NET Data Service provider. At some point in the near future, I will try to get a post up describing that stuff plus some preview of the refactoring we are doing in V2 of ADO.NET Data Services to make a nicer provider plugin model.

Attention! Phani on Deck!

2008-12-12T18:23:43Z

One of the most common feature requests we hear for Astoria is the ability to mark properties on an entity as hidden. Currently in V1 we support this on the server, but not with the .NET or Silverlight clients.

Phani, resident Astoria hacker (and a really good tester), has written a very good post on how you can roll your own override of client serialization behavior to support this functionality with the v1 clients.

IUpdateable for Linq to Sql - Fixing a few issues

2008-12-12T00:32:25Z

Finally got IUpdateable.ResetResource() for Linq To Sql working correctly. This means that REPLACE operations are now working correctly. I have uploaded those changes to code gallery.

Going forward all changes to the code with be available exclusively on code gallery. Also, I am trying to track known open issues there with the issue tracking capability.

Finally, if anyone wants to be a contributor to this project or the SubSonic one, please let me know. Particularly for the latter, there is some considerable work to do.

Developing aN Astoria data provider for SubSonic

2008-12-05T18:37:43Z

I have developed a data provider for SubSonic to enable it as a ADO.NET Data Services data source. This ended up being a bit harder then I initially thought, and is definitely still a work in progress. If you are interested, I have started a code gallery project for it.

I decided to base the first implementation of my provider on SubSonic 3.0 preview 2. This is the first version of Sub Sonic that supports IQueryable<T> so getting read-only ADO.NET Data Service support over SubSonic is quite trivial. In fact, Jay Kimble has written a very nice post about how to do it here.

To use my SubSonic Astoria data provider, first follow Jay’s instructions to set up your SubSonic DAL. This includs a very nice addition he has made to the SubSonic T4 templates to automatically add the Astoria attribute for Key properties.

Next, down load the file SubSonicIUpdateable.cs class from the code gallery project. Add it into your project and make it a partial class with the same type as the DB type generated by SubSonic code gen:

public partial class DB : IUpdatable

Interestingly, SubSonic (at least version 3.0) doesn’t really have a traditional O/R context type per se. In a way, this is practically a requirement for being a Astoria data source. Hence, my implementation utilizes a local context for each set of requests. Since this is only required to support scenarios required by IUpdateable, this was rather trivial to implement.

A few known bugs/ issues with the initial code:

Binary fields support is currently not working. Hence inserting/ updating binary data is not happening. I think this is a SubSonic bug, but I need to spend some more cycles looking into it.
The IUpdateable implementation currently does not support batch mode. So even if the Astoria client is specifying batching, the DAL will send in the changes one at a time.
Only using SubSonic metadata to find primary key properties for a given type for concurrency checks. Probably should switch this to using Astoria metadata for this.
Identity values not working. I can’t currently figure out how to get this to work with SubSonic 3.
I currently don’t support relationships between entity types because that is not currently supported with the preview of SubSonic.
I will try to keep this up to date. Also, along with the code I have included the test project I use to test the implementation.

One other interesting design issue that I discovered while implementing this is how Astoria Data Providers must be integrated at the source level. For example, what I really would like to do is implement something like SubSonicAstoriaProvidr<T> (where T is the DAL instance) and ship a assembly for that. With Astoria V1, that is next to impossible so one has to utilize something like partial classes to make this work. More to come on this later, but this has become one of the most important things we want to improve on in vNext.

Slides for my ASP.NET Connections Sessions

2008-11-11T09:16:00Z

As promised the slides & demos for my two ASP.NET Connections talks from 11/11/08:

MDA01: Deep Dive: ADO.NET Data Services Framework—Application Patterns

MDA04: LINQ to XML, SQL, Entities, DataSets and Co.: Data Access Technologies Explained - AKA Linq to Anything

Update: 11/12/08 - download now includes all demos including Linq to Twitter, PetBook, and my LINQ samples from our LINQ to Anything (MDA04) talk. Will update again with Shyam's demos from LINQ to Anything sometime in the next few days.

IUpdateable for Linq To Sql

2008-11-06T04:52:00Z

IUpdatable for Linq to Sql

I have had an implementation of IUpdateable for Linq to Sql about half done for quite some time now. Yesterday I decided to bite the bullet and finish it up.

Below I have included that code. I’ve done some light testing, but I am sure there are some bugs – and perhaps some perf fixes that can be made. My plan is to get this out on CodePlex as soon as I get a moment.

Update - 11/06/08, I actually ended up putting it on CodeGallery here.

Update 2 - 12/02/08, I found I had implemented ResetResource incorrectly to set the property values to the original values instead of the default values. Fixed this and uploaded the file to CodeGallery.

Update 3 - 12/11/08 - Found another set of fixes for ResetResource. Fixed this and uploaded the file to CodeGallery

Note: Going forward all updates to the code will be exclusively on Code Gallery and will not posted to this blog.

A couple of notes about this code:

1) I haven’t implemented ClearChanges() yet. As far as I can tell there is really no straight forward to do this with Linq to Sql. It isn’t that big of deal because ClearChanges is only required by Astoria when processing batches with multiple ChangeSets. Something that is not supported by the Astoria .NET client.

2) The easiest way to use this is to define a partial class for your DataContext type and provide the implementation there. i.e.

public partial class nwDataContext : IUpdatable

That way you don’t need to change your generated context class.

Next step - implementing IExpandProvider

    public partial class myDataContext : IUpdatable
    {
        /// <summary>
        /// Creates the resource of the given type and belonging to the given container
        /// </summary>
        /// <param name="containerName">container name to which the resource needs to be added</param>
        /// <param name="fullTypeName">full type name i.e. Namespace qualified type name of the resource</param>
        /// <returns>object representing a resource of given type and belonging to the given container</returns>
        public object CreateResource(string containerName, string fullTypeName)
        {
            Type t = Type.GetType(fullTypeName);
            Debug.Assert(t != null);  // assume can find type
            ITable table = (ITable)this.GetType().GetProperty(containerName).GetValue(this, null);
            object resource = Activator.CreateInstance(t);
            table.InsertOnSubmit(resource);
            return resource;
        }

        /// <summary>
        /// Gets the resource of the given type that the query points to
        /// </summary>
        /// <param name="query">query pointing to a particular resource</param>
        /// <param name="fullTypeName">full type name i.e. Namespace qualified type name of the resource</param>
        /// <returns>object representing a resource of given type and as referenced by the query</returns>
        public object GetResource(IQueryable query, string fullTypeName)
        {
            object resource = null;

            foreach (object o in query)
            {
                if (resource != null)
                {
                    throw new Exception("Expected a single response");
                }
                resource = o;
            }

            // fullTypeName can be null for deletes
            if (fullTypeName != null && resource.GetType() != Type.GetType(fullTypeName))
                throw new Exception("Unexpected type for resource");
            return resource;
        }

        /// <summary>
        /// Resets the value of the given resource to its default value
        /// </summary>
        /// <param name="resource">resource whose value needs to be reset</param>
        /// <returns>same resource with its value reset</returns>
        public object ResetResource(object resource)
        {
            Type t = resource.GetType();
            Debug.Assert(t != null);
            object newResource = Activator.CreateInstance(t);
            MetaTable table = this.Mapping.GetTable(t);
            foreach (var member in table.RowType.IdentityMembers)
            {
                object keyValue = member.MemberAccessor.GetBoxedValue(resource);
                member.MemberAccessor.SetBoxedValue(ref newResource, keyValue);
            }
            return newResource;
        }

        /// <summary>
        /// Sets the value of the given property on the target object
        /// </summary>
        /// <param name="targetResource">target object which defines the property</param>
        /// <param name="propertyName">name of the property whose value needs to be updated</param>
        /// <param name="propertyValue">value of the property</param>
        public void SetValue(object targetResource, string propertyName, object propertyValue)
        {
            PropertyInfo pi = targetResource.GetType().GetProperty(propertyName);
            if (pi == null)
                throw new Exception("Can't find property");
            pi.SetValue(targetResource, propertyValue, null);
        }

        /// <summary>
        /// Gets the value of the given property on the target object
        /// </summary>
        /// <param name="targetResource">target object which defines the property</param>
        /// <param name="propertyName">name of the property whose value needs to be updated</param>
        /// <returns>the value of the property for the given target resource</returns>
        public object GetValue(object targetResource, string propertyName)
        {
            PropertyInfo pi = targetResource.GetType().GetProperty(propertyName);
            if (pi == null)
                throw new Exception("Can't find property");
            return pi.GetValue(targetResource, null);
        }

        /// <summary>
        /// Sets the value of the given reference property on the target object
        /// </summary>
        /// <param name="targetResource">target object which defines the property</param>
        /// <param name="propertyName">name of the property whose value needs to be updated</param>
        /// <param name="propertyValue">value of the property</param>
        public void SetReference(object targetResource, string propertyName, object propertyValue)
        {
            this.SetValue(targetResource, propertyName, propertyValue);
        }

        /// <summary>
        /// Adds the given value to the collection
        /// </summary>
        /// <param name="targetResource">target object which defines the property</param>
        /// <param name="propertyName">name of the property whose value needs to be updated</param>
        /// <param name="resourceToBeAdded">value of the property which needs to be added</param>
        public void AddReferenceToCollection(object targetResource, string propertyName, object resourceToBeAdded)
        {
            PropertyInfo pi = targetResource.GetType().GetProperty(propertyName);
            if (pi == null)
                throw new Exception("Can't find property");
            IList collection = (IList) pi.GetValue(targetResource, null);
            collection.Add(resourceToBeAdded);
        }

        /// <summary>
        /// Removes the given value from the collection
        /// </summary>
        /// <param name="targetResource">target object which defines the property</param>
        /// <param name="propertyName">name of the property whose value needs to be updated</param>
        /// <param name="resourceToBeRemoved">value of the property which needs to be removed</param>
        public void RemoveReferenceFromCollection(object targetResource, string propertyName, object resourceToBeRemoved)
        {
            PropertyInfo pi = targetResource.GetType().GetProperty(propertyName);
            if (pi == null)
                throw new Exception("Can't find property");
            IList collection = (IList)pi.GetValue(targetResource, null);
            collection.Remove(resourceToBeRemoved);
        }

        /// <summary>
        /// Delete the given resource
        /// </summary>
        /// <param name="targetResource">resource that needs to be deleted</param>
        public void DeleteResource(object targetResource)
        {
            ITable table = this.GetTable(targetResource.GetType());
            table.DeleteOnSubmit(targetResource);
        }

        /// <summary>
        /// Saves all the pending changes made till now
        /// </summary>
        public void SaveChanges()
        {
            this.SubmitChanges();
        }

        /// <summary>
        /// Returns the actual instance of the resource represented by the given resource object
        /// </summary>
        /// <param name="resource">object representing the resource whose instance needs to be fetched</param>
        /// <returns>The actual instance of the resource represented by the given resource object</returns>
        public object ResolveResource(object resource)
        {
            return resource;
        }

        /// <summary>
        /// Revert all the pending changes.
        /// </summary>
        public void ClearChanges()
        {
        }
    }

Linq to anything

2008-10-31T04:12:00Z

Along with my manager Shyam Pather, I am giving the following talk at the upcoming Dev Connections conference:

MDA04: LINQ to XML, SQL, Entities, DataSets and Co.: Data Access Technologies Explained
With the release of the Microsoft .NET Framework 3.5, Microsoft introduced several new data access technologies based on its LINQ technology, such as LINQ to SQL and LINQ to DataSets. Shortly after, we expect Microsoft to release the ADO.NET Entity Framework which enables LINQ to Entities in addition to Entity SQL. Finally, we have all the existing ADO.NET data access patterns. This raises one important question: Which data access technology is best suited for which situation? In this session, we look at different scenarios and requirements that can inform your technology decision.

As part of this talk, we are trying to give as many Linq demos as possible. So far we have:

Linq to Objects

Linq to Sql

Linq to Entities

Linq to DataSet

Linq to Xml

Linq to ADO.NET Data Services

Linq to Windows Azure Storage

Linq to Sql Data Services

Plus a few surprises.

I am looking at something real esoteric to write a Linq provider. So if anyone has any ideas, please leave a comment or email me.

Astoria team is hiring!

2008-03-24T21:56:42Z

The Astoria team is hiring!

We have positions open for Developers and Program Managers. Currently, we are finishing up V1 and planning for V2, so it is a great time to join the team. For V2, we are thinking about expanding the project into new areas (e.g. offline support, sync, dynamic language support) so there will be plenty of new and exciting work.

Please forward your resume to aconrad@microsoft.com if you’re interested. The project covers a diverse set of technical areas (databases, Linq, O/R, REST, etc) so we value passion for the technology more than deep expertise in any particular area. The Astoria team uses an agile development model, so the project moves fast and we encourage individuals to take on a variety of roles and responsibilities. If you’re a developer who wants to be responsible for more than just coding (i.e. API design, specification writing, working with customers and partners) this team is the place for you!

And finally, team members must be willing to participate in the grueling Team Astoria off-site events.

Update 4/23/08 - As of the start of this week, we now have a block of additional new positions on the Astoria Team we need to fill. These include development, program management, and quality assurance positions. We are trying to fill these very quickly, so if you are interested please contact us soon.