Jeff's thoughts on Software Architecture, Large Scale Services and the Technical world at large : SQL Server Data Services

SDS Channel 9 Video is up

jcurrier — Tue, 28 Oct 2008 06:27:00 GMT

A little while ago Jason and I recorded a Channel 9 video to discuss some of the newer features in SDS (SQL Data Services). Now that all the Azure components have been announced this is now available up at the Channel 9 site. You can view it here.

Channel 9 interview for PDC

jcurrier — Thu, 23 Oct 2008 20:39:48 GMT

Jason and I just finished recording a session on some of the new features of the services that will be announced at PDC next week (along with some others such as Blobs). Keep in eye out for it. I think you'll enjoy it!

--Jeff--

What happened to SSDS PUT/POST responses?

jcurrier — Fri, 25 Jul 2008 20:06:03 GMT

Since the rollout I've seen at least one or two messages regarding the altered responses we're now sending with the REST head of the service. There are several reasons why we've changed the service to now not return the entity body back to you again. These are as follows:

Version - Prior to this rollout the only mechanism that we had to communicate version information to you was via the entity body of the response (specifically the version element in the body). However, now that we have true ETag support we now can simply return this value back to caller via the ETag header per the Http spec.
Cost - Although at this point we don't charge for the service we eventually will. Included within this billing structure are charges for ingress & egress (bandwidth usage in essence). Therefore, by not returning the body to the caller (which the caller originally provided us with anyway) this is a net cost savings for you the caller.

Hope this helps to explain a bit more of the thinking behind the change. As always we're open to feedback.

--Jeff--

Check out the SSDS coverage in the latest issue of MSDN magazine.

jcurrier — Thu, 03 Jul 2008 22:28:32 GMT

Hey all,

If you have a subscription to MSDN magazine, heck even if you don't, you read a bit more about using SSDS in the latest issue. If you don't have a subscription then you can still read the article on-line here.

Enjoy!

Optimistic Concurrency with SOAP in SSDS

jcurrier — Sun, 29 Jun 2008 23:47:00 GMT

In my last post I described how ETag's can be used in the upcoming sprint 3 version of SSDS. In this post I'd like to talk about how this same functionality is exposed through our SOAP service head.

In many ways designing this particular feature for the REST head was much easier than for the SOAP service head. This is because the Http specification describes, in a fair bit of detail, what the semantics are around things like ETag's. The SOAP specification provides no such guidance and as a result we've rolled our own implementation here to support this feature in a way that makes sense for our service.

VersionMatch

As I mentioned earlier, since there is no ETag concept natively in SOAP we've introduced a new object called, "VersionMatch". We've augmented the Scope struct in the system to now include this so that version is now also considered a qualifier for operations that take place via the SOAP service head.

The VersionMatch object, as shown in the class diagram below, captures two things. The first, is the version of the entity in question. This could be the version of the entity we've just created, the version that I, as a caller, want to retrieve, or finally the one I want to update or delete.

The second concept is something we refer to as the, "MatchType" or VersionMatchType. This is used to determine how the service should compare the version when executing the requested operation. This could mean that we only perform the operation if the version matches the version in the store (the VersionMatchType.Match case), or only perform the operation if it doesn't match (the VersionMatchType.NotMatch case) or finally to ignore the version completely when performing the operation. NOTE: By default we use the Ignore semantic unless otherwise instructed.

Class Diagram of the modified Scope object along with the new VersionMatch object and VersionMatchType enum.

Now, that we've covered a bit of the changes to how we address things in the system let's see how we can apply these new features to some common scenarios.

Conditional Get

Conditional retrieval is an interesting case if a mid-tier or client application is maintaining a cache. This shows up many times in the web browser case where large files are cached until they've been either refreshed or updated on the service side.

We can now perform a similar operation now using the SOAP service head. I've created a sample below which illustrates how a caller can now conditionally retrieve an entity only if the version has changed.

            using (SitkaSoapServiceClient client = new SitkaSoapServiceClient("SitkaSoapEndpoint"))
            {
                try
                {
                    // Create a reference to some entity that we know exists and one that
                    // we have a reference to in our cache (the version we have is 2000).
                    Scope entityScope = new Scope();
                    entityScope.AuthorityId = "some_authority";
                    entityScope.ContainerId = "some_container";
                    entityScope.EntityId = "123";
                    entityScope.VersionMatch = new VersionMatch();
                    entityScope.VersionMatch.Version = 2000;
                    entityScope.VersionMatch.MatchType = VersionMatchType.NotMatch;

                    // Then, retrieve the entity *only* if the version doesn't match the
                    // one that I already know about.
                    Entity theEntity = client.Get(entityScope);

                }
                catch (FaultException<Error> ex)
                {
                    if (ex.Detail.StatusCode == ErrorCodes.EntityNotModified)
                    {
                        // there is no later version.
                    }
                }
            }

Conditional Update and Delete

The next two cases are really very similar to one another. In both of these cases we really only want to update (or delete) the entity if we know that there has been no other modifications to the entity in question. This way we can be certain that we've not overwritten someone else's changes (in the update case) or perhaps deleted an entity which now (since the update) may not have needed to be deleted.

The code listing below illustrates both of these cases.

Conditional Update in SOAP

            using(SitkaSoapServiceClient client = new SitkaSoapServiceClient("SitkaSoapEndpoint"))
            {
                try
                {
                    // Create a reference to some entity that we know exists.
                    Scope entityScope = new Scope();
                    entityScope.AuthorityId = "some_authority";
                    entityScope.ContainerId = "some_container";
                    entityScope.EntityId = "123";

                    // Then, retrieve the entity (any version will do).
                    Entity theEntity = client.Get(entityScope);

                    // Update some properties on the Entity.
                    theEntity.Properties["FavoriteStorageService"] = "SSDS";

                    // Next, update the version match properties on the scope we'll be using to update
                    // the entity with.
                    entityScope.VersionMatch = new VersionMatch();
                    entityScope.VersionMatch.Version = theEntity.Version;
                    entityScope.VersionMatch.MatchType = VersionMatchType.Match;

                    // Finally, update the entity if the version check clears.
                    client.Update(entityScope, theEntity);

                }catch(FaultException<Error> ex)
                {
                    if(ex.Detail.StatusCode == ErrorCodes.PreconditionFailed)
                    {
                        // Our version check failed so we don't have the latest and greatest.
                    }
                }
            }

Conditional Delete in SOAP

            using(SitkaSoapServiceClient client = new SitkaSoapServiceClient("SitkaSoapEndpoint"))
            {
                try
                {
                    // Create a reference to some entity that we know exists.
                    Scope entityScope = new Scope();
                    entityScope.AuthorityId = "some_authority";
                    entityScope.ContainerId = "some_container";
                    entityScope.EntityId = "123";

                    // Next, set the version match properties on the scope we'll be using to remove
                    // the entity with.
                    entityScope.VersionMatch = new VersionMatch();
                    entityScope.VersionMatch.Version = 2000; // the version of the entity we know about.
                    entityScope.VersionMatch.MatchType = VersionMatchType.Match;

                    // Finally, remove the entity if the version check clears.
                    client.Delete(entityScope);

                }catch(FaultException<Error> ex)
                {
                    if(ex.Detail.StatusCode == ErrorCodes.PreconditionFailed)
                    {
                        // Our version check failed so we don't have the latest and greatest.
                    }
                }
            }

That covers the optimistic concurrency examples for the Sprint 3 bits of SSDS. I'm certain that there are other scenarios here that I'm not covering and I'd really like to hear other scenarios if you have them to share.

Next time, I'll cover perhaps the biggest feature of the sprint (and one of our most requested features of the service that we have now completed for sprint 3....). Look for more shortly :-)

ETag's, Optimistic Concurrency and SSDS

jcurrier — Sun, 22 Jun 2008 10:17:03 GMT

I've been kind of quiet the last month or so but that's because I've been totally heads down implementing some major features for this sprint in SSDS. Our next release to production, which should be in the next few weeks, is coming up fast so I thought over the next few posts to the blog I would cover some of the highlights of the release.

We've covered a lot of ground in this sprint. We've added some features, which I believe, that people will be pretty excited about. I've decided to start the ball rolling with one of the features that was requested early on of the service... ETag support.

ETag's in SSDS

It may be useful to quickly review the role that ETag's play in a RESTful service. The ETag header value is typically used by a client application to determine if the content of a give resource has changed over some period of time. In SSDS we use the value of the, "Version" property as the ETag header value for a given entity. As documented previously, the version value will be changed when a update takes place to a entity.

At this point, you may be asking yourself when do I get one of these things back to me? Beginning with the next rollout ETag's will be returned to the caller on all POST, PUT, and single entity GET operations. Now, with query operations we can't return back a ETag value simply because the response doesn't actually refer to any one particular entity in the service. Instead, if you would like to determine what would be the ETag value for any one entity in the EntitySet simply extract the Version property value from the entity of interest.

Now that we know when ETag's come back to us we likely should review how we would get the ETag value from the response. ETag's are a common Http header and are well supported as shown in the sample below which retrieves an ETag from a HttpWebResponse.

HttpWebRequest request = CreateRequest();
using(HttpWebResponse response = request.GetResponse())
{
    string etagValue = response.Headers[HttpResponseHeader.ETag];
    // do something interesting with the ETag.
}

Conditional Operations

Now that we have ETag's we can use these to execute operations in a conditional manner. In order to support these sort of operations we've added support for two other headers to the service the, "If-Match" and, "If-None-Match" headers. We support these headers over all operations except for query though the following scenarios will likely fallout to be the most common:

Conditional GET - This is actually the most common case, browsers typically operate in this manner. In short, they first retrieve a entity which retrieves a entity with a ETag. Then, on any subsequent calls to the same resouce a, "If-None-Match" header is provided on the request with the ETag from the initial GET operation. If, and only if, the resource has changed then the complete entity definition is returned to the caller.

If the entity hasn't been updated then a 304 (NotModified) is returned and only the headers are returned to the caller.

Conditional PUT - In this case, we only want to update the contents of an entity if we know that we have the latest and greatest. In this case we would provide a, "If-Match" header value with the ETag value of the entity that we wish to update.

In this case, if a later version of the entity is found in the service then we will return a 412 (PreConditionFailed) to the caller and the caller will have to retrieve the entity again and decide if they still need to update the entity again.

Conditional DELETE - This is case pretty much the exact same as the Conditional PUT case. The only difference here being that the verb we've chosen to use is the DELETE verb rather then PUT. I wanted to explicitly call out though just to illustrate our support for it.

HEAD support

There are some cases where the caller wishes to discover if the content they have is the latest greatest. Typically, you would like this operation to be light weight in nature (which would exclude the returning of the entire entity) and only give you back the ETag or other relevant headers. The Http specification provides a verb for this called, "HEAD" and with this sprint we've added support for it to the service.

You use the HEAD verb just like you would the normal GET operation except that we will only return back to you the headers that are of interest. This verb, will perhaps not so obvious in usefulness here, will become more obvious in my one of my next posts.

What about SOAP?

While I haven't covered SOAP in this post we have added support for these operations to the SOAP service. I'll be covering the specifics of how this works in my next post. Keep an eye out for this shortly!

Speaking at TechEd

jcurrier — Thu, 01 May 2008 18:58:00 GMT

I'm heading down to Orlando in a couple of weeks to give a talk about best practices when it comes to SSDS. I'm working on my talk outline and was curious if there were any topics in particular that the community wanted to hear about in the talk?

Thoughts?

Good overview of using threads for multi container query in SSDS

jcurrier — Tue, 15 Apr 2008 06:30:18 GMT

Eugenio has a good blog entry today describing using multiple threads to execute multi-container query. Well worth the read.

Interested in partaking in a design review for SQL Server Data Services?

jcurrier — Sun, 13 Apr 2008 09:54:00 GMT

Are you interested in taking part in a design review of SQL Server Data Services? Are you located in Silicon Valley? If you answered yes to both of these questions then you're going to want check out Ryan's blog here to find out details for attending. Space is limited so act fast!

--Jeff--

cURL'ing up with SQL Server Data Services

jcurrier — Sun, 13 Apr 2008 09:47:59 GMT

We've begun working with some external customers (and a considerable number of internal ones) on the project and one of the first requests that came up was regarding the use of cURL.

cURL, as many of you already know, is a pretty cool command line utility which can be used to issue HTTP requests. However we haven't, until this point, documented exactly how you can use the service from cURL. This post will address that.

Enumerating your Authority

In this example I'll illustrate how you can query an authority for all of the containers that are located within it. Before we get to this however I'll explain a bit more about how cURL functions.

cURL uses various command line parameters to construct the HTTP request on your behalf. As you might guess the -X parameter is used to specify the verb (HTTP method) that you wish to use on the request. The -u parameter is used to specify the credentials to use and finally the last parameter is the URL you want to use.

Command line syntax to query the authority:

curl -X GET -u "<userid>:<password>" http://<authority-id>.data.beta.mssds.com/v1/?q=''

Output

<s:EntitySet xmlns:s="http://schemas.microsoft.com/sitka/2008/03/" xmlns:xsi="http://www.w3.org/2001/XMLSc
hema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema">
<s:Container>
    <s:Id>testcontainer</s:Id>
    <s:Version>1</s:Version>
</s:Container>
<s:Container>
    <s:Id>testcontainer2</s:Id>
    <s:Version>1</s:Version>
</s:Container>
</s:EntitySet>

Creating Containers and Entities

As I've mentioned previously here we use the POST verb to create entities within SSDS. Now, because specifying the entire payload for a POST could be cumbersome cURL allows you to specify the filename where the payload can be found. This done use the, "--data" parameter and the, "@" symbol.

We'll begin with creating a container with cURL. We'll start by creating a simple file called, "CurlContainer" containing the following XML payload:

<s:Container xmlns:s="http://schemas.microsoft.com/sitka/2008/03/">
<s:Id>CurlContainer</s:Id>
</s:Container>

Now that this is complete we can construct the command line syntax that we'll need to send the request to the service.

Command line syntax to create the container:

curl -X POST --data @CurlContainer -H "Content-Type: application/xml" -u "<userid>:<password>" http://<authority-id>.data.beta.mssds.com/v1/

A couple of things that you might notice here that are different from our prior syntax. The first is that the verb has changed (that's pretty obvious). The second item is that we've used the, "--data" parameter to specify the payload. We didn't need to use that before since we were only querying the service.

Finally, and perhaps most importantly, is that we specify the content type of data in the request. This is done using the, "-H" parameter along with the header data. If this isn't specified then the request will be denied with a 400 so do remember to use it.

Now, when we query the authority as we did earlier I get the following output returned to me.

<s:EntitySet xmlns:s="http://schemas.microsoft.com/sitka/2008/03/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema">
<s:Container>
    <s:Id>CurlContainer</s:Id>
    <s:Version>1</s:Version>
</s:Container>
<s:Container>
    <s:Id>testcontainer</s:Id>
    <s:Version>1</s:Version>
</s:Container>
<s:Container>
    <s:Id>testcontainer2</s:Id>
    <s:Version>1</s:Version>
</s:Container>
</s:EntitySet>

Switching to entity creation we find that not a lot has changed. The only meaningful difference is the URL that we provide and the payload content. Following the steps just described I can create a new entity, "CurlEntity" within the CurlContainer I just created. Listed below is the payload I've created as well as the syntax used to create the entity.

Payload

<CurlEntity xmlns:s="http://schemas.microsoft.com/sitka/2008/03/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema">
<s:Id>CurlEntity</s:Id>
<Name xsi:type="x:string">Jeff Currier</Name>
</CurlEntity>

Command line syntax to create the entity:

curl -X POST -H "Content-Type: application/xml" --data @CurlEntity.xml -u "<userid>:<password>" http://<authority-id>.data.beta.mssds.com/v1/CurlContainer

Finally, let's just query the container to see my new data.

Command line syntax to query the container:

curl -G -u "<userid>:<password>" http://<authority-id>.data.beta.mssds.com/v1/CurlContainer?q=''

Output

<s:EntitySet xmlns:s="http://schemas.microsoft.com/sitka/2008/03/" xmlns:xsi="http://www.w3.org/2001/XMLSc
hema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema">
<CurlEntity>
    <s:Id>CurlEntity</s:Id>
    <s:Version>1</s:Version>
    <Name xsi:type="x:string">Jeff Currier</Name>
</CurlEntity>
</s:EntitySet>

Updating Entities

Now that we can create and query things we need to have a way of updating them otherwise things just aren't that interesting. We do this, as always in a RESTful service, using the PUT verb. In this case I'm going to change the, "Name" property on my newly created entity to another name.

Like we did previously we'll create a file which will contain the payload definition for the request. I've created this file, "CurlEntity2.xml" and will provide this on the command line. Listed below is both the payload as well as the command line syntax for updating.

Update Payload:

Command line syntax for updating an entity:

curl -X PUT --data @CurlEntity2.xml -H "Content-Type: application/xml" -u "<userid>:<password>" http://<authority-id>.data.beta.mssds.com/v1/CurlContainer/CurlEntity

Output from listing the contents of the container

<s:EntitySet xmlns:s="http://schemas.microsoft.com/sitka/2008/03/" xmlns:xsi="http://www.w3.org/2001/XMLSc
hema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema">
<CurlEntity>
    <s:Id>CurlEntity</s:Id>
    <s:Version>2</s:Version>
    <Name xsi:type="x:string">Jason Hunter</Name>
</CurlEntity>
</s:EntitySet>

Deleteing Entities

All things must come to an end even when it comes to entities. We do this in RESTful services using the Delete verb. Notice here that there is no need to specify a payload or a content-type header. None of this is required as the service will determine what to delete based upon the URL.

Command line syntax:

curl -X DELETE -u "<userid>:<password>" http://<authority-id>.data.beta.mssds.com/v1/CurlContainer/CurlEntity

Output from listing the contents of our container:

<s:EntitySet xmlns:s="http://schemas.microsoft.com/sitka/2008/03/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema"/>

That wraps up our cURL example for this evening. If there are other languages or tools that you would be interested in seeing examples in please do let us know!

--Jeff--

Check out Nigel & Istvan over at Channel 9

jcurrier — Tue, 08 Apr 2008 07:08:34 GMT

You can watch a good interview with Istvan and Nigel over on Channel 9. The talk covers some of the interesting architecture, security and other topics. Well worth the watch!

Interacting with SQL Server Data Services using SOAP

jcurrier — Thu, 03 Apr 2008 06:43:34 GMT

In one of my prior posts I promised some SOAP examples to better illustrate how you can use SOAP to interact with SSDS. Therefore, to honor that I've decided to write some up some examples of using SOAP with SSDS.

Setting Scope

In the SOAP client we really don't have the luxury of being able to easily determine the scope of a particular operation from the different segments of a URI. Therefore, to make establishing the scope of a given operation easy we introduced the notion of a scope object to the SOAP interface.

This object captures all of the possible scopes that are valid for the service. I've listed below both the Scope object definition as well as a description of what each of the settings mean in the context of the service.

Service Scope - This the scope when a scope object instance is passed to the service without any of the properties set. This would typically be the case when the intent is to create a new authority object in the service.

Authority Scope - When just the authority id is specified on the scope object. This is generally the case when the caller is attempting to create a container, retrieve the set (or a restricted set) of containers from a given authority or finally when attempting to simply retrieve a particular authority instance.

Container Scope - As you might imagine this scope is established when both the authority and container id properties are set on the scope object. This scope is typically used when querying within a container for a set of entities (or all entities), creating new entities, or deleting a specific container.

Entity Scope - Used to narrow the focus of the operation to be performed down to the entity level. This is typically used when the caller is attempting to update, retrieve or delete a specific entity within the service.

The generated C# code for the Scope object.

    public partial struct Scope : System.Runtime.Serialization.IExtensibleDataObject, System.ComponentModel.INotifyPropertyChanged {
        
        [System.NonSerializedAttribute()]
        private System.Runtime.Serialization.ExtensionDataObject extensionDataField;
        
        private string AuthorityIdField;
        
        [System.Runtime.Serialization.OptionalFieldAttribute()]
        private string ContainerIdField;
        
        [System.Runtime.Serialization.OptionalFieldAttribute()]
        private string EntityIdField;
        
        public System.Runtime.Serialization.ExtensionDataObject ExtensionData {
            get {
                return this.extensionDataField;
            }
            set {
                this.extensionDataField = value;
            }
        }
        
        [System.Runtime.Serialization.DataMemberAttribute(IsRequired=true)]
        public string AuthorityId {
            get {
                return this.AuthorityIdField;
            }
            set {
                if ((object.ReferenceEquals(this.AuthorityIdField, value) != true)) {
                    this.AuthorityIdField = value;
                    this.RaisePropertyChanged("AuthorityId");
                }
            }
        }
        
        [System.Runtime.Serialization.DataMemberAttribute()]
        public string ContainerId {
            get {
                return this.ContainerIdField;
            }
            set {
                if ((object.ReferenceEquals(this.ContainerIdField, value) != true)) {
                    this.ContainerIdField = value;
                    this.RaisePropertyChanged("ContainerId");
                }
            }
        }
        
        [System.Runtime.Serialization.DataMemberAttribute()]
        public string EntityId {
            get {
                return this.EntityIdField;
            }
            set {
                if ((object.ReferenceEquals(this.EntityIdField, value) != true)) {
                    this.EntityIdField = value;
                    this.RaisePropertyChanged("EntityId");
                }
            }
        }
        
        public event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;
        
        void RaisePropertyChanged(string propertyName) {
            System.ComponentModel.PropertyChangedEventHandler propertyChanged = this.PropertyChanged;
            if ((propertyChanged != null)) {
                propertyChanged(this, new System.ComponentModel.PropertyChangedEventArgs(propertyName));
            }
        }
    }

So, now that we have some context about what scope is and what it's used to indicate within the service let's show some examples of using the service.

Creating Authorities

In this example I'd like to illustrate how to create an authority in the system. As you may recall from some of other posts on our data model the Authority is the top level domain, so to speak, for a set of containers.

All the containers created within a specific authority will be contained in a single data center. During the authority creation process a domain name will be created automatically on your behalf. This last point tends to be more important in the REST examples (from a programmatic point of view) but it will also have a greater significance in our later service releases.

The code...

            // Begin by creating an instance of our SOAP proxy class.
            using (SitkaSoapServiceClient client = new SitkaSoapServiceClient("SitkaSoapEndpoint"))
            {
                // Next, set the credentials that will be used to associate the request with a known
                // user in the system.  This user will also then become the, "owner" of this authority.
                client.ClientCredentials.UserName.UserName = "my_user";
                client.ClientCredentials.UserName.Password = "my_password";

                // Create a new scope object at the service level.
                Scope serviceScope = new Scope();

                // Next, create an Authority object and set the Id of that authority.
                Authority myNewAuthority = new Authority();
                myNewAuthority.Id = "ssdsrocks";

                try
                {
                    // Proceed to create the authority object.
                    Scope authorityScope = client.Create(serviceScope, myNewAuthority);

                    // Finally using the returned Scope object which is initialized to my 
                    // new authority object retrieve the authority.
                    Authority theAuthority = (Authority) client.Get(authorityScope);

                    Console.WriteLine("Here's my ID {0}", theAuthority.Id);

                }catch(FaultException<Error> ex)
                {
                    Console.WriteLine(ex.Message);
                }
            }

The Results

Creating Containers

In the last example I gave an example of creating new authorities in SQL Server Data Services. Now, let's go ahead and create some containers. Recall that containers are the mechanism in our service that we use for partitioning data. All of the data for a particular container will be located on a single machine within the cluster (with multiple replicas existing on other machines in the cluster).

Now, with no further delay creating containers.

The code...

            // Begin by creating an instance of our SOAP proxy class.
            using (SitkaSoapServiceClient client = new SitkaSoapServiceClient("SitkaSoapEndpoint"))
            {
                // Next, set the credentials that will be used to associate the request with a known
                // user in the system. 
                client.ClientCredentials.UserName.UserName = "my_user_id";
                client.ClientCredentials.UserName.Password = "my_password";

                // Create a new scope object at our new authority level.
                Scope authorityScope = new Scope();
                authorityScope.AuthorityId = "ssdsrocks";

                // Next, create the new Container object and set the Id 
                string containerId = "MyMusic";
                Container newContainer = new Container();
                newContainer.Id = containerId;

                try
                {
                    // Proceed to create the container object.
                    Scope containerScope = client.Create(authorityScope, newContainer);

                    // Now, rather than simply retrieve the new container let's query the authority
                    // for it.
                    string query =
                        String.Format(@"from e in entities where e.Id ==""{0}"" select e", containerId);

                    List<Entity> containers = client.Query(authorityScope, query);

                    // Finally, let's print out the results of all the containers we've found.
                    foreach(Container c in containers)
                    {
                        Console.WriteLine("Container id:{0}", c.Id);
                    }
 

                }catch(FaultException<Error> ex)
                {
                    Console.WriteLine(ex.Message);
                }
            }

and the Results...

Creating Entities

Entities represent the really interesting stuff in the service. These are the objects where the real data will be stored and later queried for. In this example I will create some new entities (with some properties) and then query for a subset of the entities.

The code.

            // Begin by creating an instance of our SOAP proxy class.
            using (SitkaSoapServiceClient client = new SitkaSoapServiceClient("SitkaSoapEndpoint"))
            {
                // Next, set the credentials that will be used to associate the request with a known
                // user in the system. 
                client.ClientCredentials.UserName.UserName = "my_user_id"; 
                client.ClientCredentials.UserName.Password = "my_password";

                // Create a new scope object at our new container level.
                Scope containerScope = new Scope();
                containerScope.AuthorityId = "ssdsrocks";
                containerScope.ContainerId = "MyMusic";

                try
                {
                    // Create some entities with various properties.
                    Entity[] entities = new Entity[3];
                    entities[0] = new Entity();
                    entities[0].Id = "1";
                    entities[0].Kind = "Album";
                    entities[0].Properties = new Dictionary<string, object>();
                    entities[0].Properties["Artist"] = "The Shins";
                    entities[0].Properties["Title"] = "Chutes Too Narrow";

                    entities[1] = new Entity();
                    entities[1].Id = "2";
                    entities[1].Kind = "Album";
                    entities[1].Properties = new Dictionary<string, object>();
                    entities[1].Properties["Artist"] = "Brett Dennen";
                    entities[1].Properties["Title"] = "So Much More";

                    entities[2] = new Entity();
                    entities[2].Id = "3";
                    entities[2].Kind = "Album";
                    entities[2].Properties = new Dictionary<string, object>();
                    entities[2].Properties["Artist"] = "The Shins";
                    entities[2].Properties["Title"] = "Wincing the Night Away";

                    // NOTE: This is a unique property to this entity instance.
                    entities[2].Properties["Released"] = 2007m; // treat as decimal. 

                    // Go ahead and create the entities in the service.
                    foreach(Entity currentEntity in entities)
                    {
                        client.Create(containerScope, currentEntity);
                    }

                    // Okay, so the entities are there now.  Let's first retrieve all the entities.
                    List<Entity> allEntities = client.Query(containerScope, "from e in entities select e");
                    
                    Console.WriteLine("All Entities in container are:");
                    foreach(Entity e in allEntities)
                    {
                        Console.WriteLine("Entity id = {0}", e.Id);
                    }

                    // Now, let's restrict the set of entities returned to the one (and only one) that
                    // had a release date of 2007.
                    String query = String.Format(@"from e in entities where e[""Released""] == 2007 select e");
                    List<Entity> entityResults = client.Query(containerScope, query);

                    Console.WriteLine("All Entities with a Released property set to 2007:");
                    foreach (Entity e in entityResults)
                    {
                        Console.WriteLine("Entity id = {0}", e.Id);
                    }

                }catch(FaultException<Error> ex)
                {
                    Console.WriteLine(ex.Detail.Message);
                }
            }

The Results...

So, that wraps up the SOAP examples for tonight. Hope you enjoy them and if you have questions (or other examples that you would like to see) please let me a comment back so that I can try to get it out there for you.

--Jeff--

What questions do you have about SQL Server Data Services?

jcurrier — Fri, 21 Mar 2008 06:15:00 GMT

So,

I've written several posts now around coding (in C# and in Java) about using SSDS. So, I'm a bit curious if anyone out there has specific questions that either I (or others on team) might be able to answer. This could be code related, design related, etc. So, what questions do you have?

--Jeff--

SQL Server Data Services Meet Java

jcurrier — Wed, 19 Mar 2008 08:20:00 GMT

In my last post I covered a simple example of querying SSDS using C#, HttpWebRequest and XLinq. Today, I thought I'd try to use another language (one we at Microsoft didn't have a hand in creating) Java.

As I did in my last post I'm going to start with a little bit of infrastructure. It's worth noting at this point that I'm using the HttpClient libraries that are freely available from the Jakarta Commons project of Apache. This makes working with REST services pretty painless in the Java development environment.

Making Requests

The first thing that we'll need to construct is a way for us to issue requests to the SSDS service. As I did previously, I've created a simple helper method to make this easier for me to do my work. This method simply takes in the HTTP Method, the URI, and finally a string to use as the request body. It will return to the caller the body of the response that we received back from the server.

    private static String IssueRequest(String method,
                                       String url,
    		                       String body) throws Exception
    {
    	String response = "";
    	
    	// Define the HTTP client that we'll use.
        HttpClient client = new HttpClient();
	HttpMethod requestMethod = null;
		
	// Choose the method we'll be using to issue the request.
	if(method.equals("GET"))
	{
            requestMethod = new GetMethod(url);
	}
	else if (method.equals("POST"))
	{
            requestMethod = new PostMethod(url);
            ((PostMethod)requestMethod).setRequestBody(body);
            ((PostMethod)requestMethod).setRequestHeader("Content-Type", XmlContentType);
	}
	else if(method.equals("PUT"))
	{
            requestMethod = new PutMethod(url);

            ((PutMethod)requestMethod).setRequestBody(body);
            ((PutMethod)requestMethod).setRequestHeader("Content-Type", XmlContentType);
	}
	else if(method.equals("DELETE"))
	{
            requestMethod = new DeleteMethod(url);
	}
	else
	{
            throw new Exception("Unsupported method type provided");
	}
		   
	URL requestUrl = new URL(url);
	try
	{
	    // Next, set the credentials we'll be using on the request.
            Credentials defaultcreds = new UsernamePasswordCredentials(UserId, UserPw);
            client.getState().setCredentials(
	       		new AuthScope(requestUrl.getHost(), 80, AuthScope.ANY_REALM), defaultcreds);
			
	    // Then, state that we do want to do authentication on these requests.
            requestMethod.setDoAuthentication(true);

            // Provide custom retry handler if a retry is necessary.. Quite Nice actually
            requestMethod.getParams().setParameter(
		HttpMethodParams.RETRY_HANDLER,
		   new DefaultHttpMethodRetryHandler(3, false));

            // Execute the method.
            int statusCode = client.executeMethod(requestMethod);
            if (statusCode != HttpStatus.SC_OK)
            {
                System.err.println("Method failed: "
                    + requestMethod.getStatusLine());
            }

            // Read the response body.
            byte[] responseBody = requestMethod.getResponseBody();

            // NOTE: Simplifying assumption.  Assume all returned content at this
            //       is string data.
            response = new String(responseBody);
         } 
         catch (HttpException e)
         {
             return e.getMessage();
         } 
         catch (IOException e)
         {
             return e.getMessage();
         } 
         finally
         {
             // Release the connection.
             requestMethod.releaseConnection();
         }
		
         return response;
    }

It's all in the presentation

Okay, so now we can make requests to the service and we can read in our responses fine let's go ahead try to make something a bit more interesting.

In the last example you might recall the data model that we used had a Northwind origin if you like (read more here). I've decided to keep with this basic storage model and construct a JFC (I still think of it as Swing) application which will allow me to issue ad-hoc queries against my service. I've inserted a screen shot of this below, forgive the UI my JFC skills aren't quite as sharp as they used to be ;-)

What I've got here is a data bound combobox which allows me to determine the container I'd like to issue my query against. Next to that is a textbox whose text value will be used for my query text. Then, as you might imagine, after I execute the query the results (in wire format) will be displayed in the results TextArea.

Let's start with how we retrieve the container Id's. Listed below is the code I'm using to retrieve the container id's.

    private static String[] retrieveCustomerIds()
    {
    	// Construct the URI with our query in that will be used to retrieve
    	// all of the containers.  In this case, since the query is empty we will
    	// retrieve all the containers within the, "mix08-demo" authority.
    	String customerQuery = "http://mix08-demo.data.sitka.microsoft.com/v1/?q=";
    	String[] customerIds = null;
    	
    	try
    	{

            // Next, issue the GET request to retrieve the data.  No body is necessary
            // in the request (nor is it allowed).
            String response = IssueRequest("GET", customerQuery, "");

            DocumentBuilderFactory factory =  DocumentBuilderFactory.newInstance();
            factory.setNamespaceAware(true);
            DocumentBuilder docBuilder = factory.newDocumentBuilder();
            InputSource inStream = new InputSource();
            inStream.setCharacterStream(new StringReader(response));
        	
            // Read the response we got back from the server into a document.
            Document responseDoc = docBuilder.parse(inStream);

            // Create an XPath expression to retrieve just the customer id's from the set of the containers
            // that were returned.
            XPathFactory xpathFactory = XPathFactory.newInstance();
            XPath xpath = xpathFactory.newXPath();

            MyNamespaceContext ctx = new MyNamespaceContext();
            xpath.setNamespaceContext(ctx);

            XPathExpression expr 
             = xpath.compile("/s:EntitySet/s:Container/s:Id/text()");

            // execute the XPath expression and copy the results into a string array for data binding.
            Object result = expr.evaluate(responseDoc, XPathConstants.NODESET);
            NodeList nodes = (NodeList) result;
            int containerCount = nodes.getLength();
            customerIds = new String[containerCount];
        	
            // finally, add each of the container id's to the list of strings that
            // we will return to the caller.
            for (int i = 0; i < containerCount; i++) 
            {
            	customerIds[i] = nodes.item(i).getNodeValue();
            }
    	}catch(Exception ex)
    	{
    	    String errorMsg = ex.getMessage();
    	    // Do something more interesting here later.
    	    System.out.println(errorMsg);
    	}

    	return customerIds;
    }

Pretty easy stuff, most of the work is actually in processing the response (this is where the JAXB strongly typed sample I referred to last time would be quite handy). Anyway, keeping things relatively simple we simply read the response into a XML Document, compose a XPath expression over it extracting out just the Id's of the containers and then place each of these into a String array which is then returned back to the caller to later be used for data binding the combo box. Not that bad.

It's all in the Query

Now, that we can select the container that we wish to query over let's go ahead implement the code which actually does the query. The code listing below illustrates how to do this.

    private static void processQueryRequest()
    {
    	String queryUrl = String.format("%s%s?q='%s'", 
    			"http://mix08-demo.data.sitka.microsoft.com/v1/", 
    			containerId, java.net.URLEncoder.encode(txtQuery.getText()));
    	
    	try
    	{
    	    String response = IssueRequest("GET", queryUrl, "");
    		
    	    txtResults.setText(response);
    		
    	}catch(Exception ex)
    	{

            // Do something more interesting here...
    	    System.out.println(ex.getMessage());
    	}
    }

The first thing we do is to construct the URI that will be used on our request. There's a couple of components to this. The first is the location of the service (prefixed by our authority id), the second bit is the id of the container that we wish to query and the final piece is the encoded query text that we wish to use for the query.

Next, since we have the URI properly formatted we then go ahead and issue the GET request to the server and receive back a response. The text of the response body is then inserted directly into the results area. I've included some sample screenshots here illustrating requests that were valid and some that I made which were intentionally invalid just to, hopefully, illustrate that everything is really working here.

Query of the entire container

Query for just the customer within a container

Where's the Where

Hopefully, this post illustrates a bit of what you can do with SQL Server Data Services with Java. As I mentioned I'll try to post a Strong Typing example here using JAXB shortly. I'll do the same for C# as well if Jason doesn't beat me to it :-)

--Jeff--

Jason's got some SSDS examples as well

jcurrier — Tue, 18 Mar 2008 22:39:00 GMT

Jason Hunter, dev lead on the SQL Server Data Services team, has posted up some coding examples as well for SSDS. You can check these out here. In this example Jason does a Master-Details WinForm app. Pretty cool stuff...