Life in the clouds...

What kind of examples would be helpful?

Here is a question. Myself and other members of the SSDS team are always asking the “What do you want to see” question. I am going to ask a similar question…

• What kind of examples would be helpful?
• Would you rather see code snippets or a fully working application?
• Would you rather see a console app? WinForm? ASP.Net? WPF? Silverlight?
• What type of app?
• What are the challenging scenarios that you are facing that you would like a reference application to help you solve?

Feel free to email me at david.robinson AT microsoft.com

Been quiet lately

Hey All,

I have been really quiet lately. We had a ton of great stuff hit when we rolled out the Sprint 3. I had all these plans for samples and blog posts, but the last two weeks I have been completely out of pocket with a sick child.

The good news is that I have been back for a few days and I am almost done digging out of email. So I am ready to start rocking and rolling :)

Hope everyone has been having a tremendous summer…

Dave

Back from vacation...well, almost

Hey Everyone,

Just a quick post. I have been on vacation for the past 10 days in New York visiting family and relaxing with my wife and kids. I am getting ready to head to the airport to fly home and I just started digging out of email. Luckily for me I have a 6 hour flight back to Seattle. We are flying Jet Blue so hopefully my kids (ages 8, 6 and 3 (twins)) will enjoy the complementary TV and let me get some solid work done. It’s kind of weird travelling with kids. When I travel by myself, I am the guy that prays before I step foot on the flight that I am not sitting next to any younger kids or babies. Don't get me wrong, I absolutely love kids, but I look forward to the distraction free time on a plane to get work done. My kids are used to flying so they are usually well behaved...Usually...anyone who has kids knows that at the drop of a hat, a three year old can turn from a sweet little thing to a demon. I know they have the term terrible twos, but my experience has been that three has always been the age to watch out for...Needless to say, I am keeping my fingers crossed.

Also everyone should have received my email blast last week from Microsoft Connect on the new Sprint 3 features coming. The text from that can be seen here. Those features are getting rolled out this upcoming Thursday and should be completed by Friday. Our super - duper - magnificent operations team has been phenomenal at these rollouts and although the product team works really hard to crank out these new features, it is the unsung operations heroes that deserve some serious kudos.

Hope everyone didn't miss me too much :)

Dave

Best Practice - Take Advantage of Concurrency

In the first best practice, we looked at how we can gain maximum performance out of SSDS by partitioning our data across multiple containers. In this post, we are going to look at how we can take advantage of the many SSDS front end servers to maximize our applications performance.

SSDS is an Internet based data storage and query processing utility service. Lets face it, SSDS can only be as fast as your Internet connection. With that being said, when we architect our applications, we need to keep that in mind and be sure that we take whatever steps necessary to ensure that our application performs well. There are a handful of access and caching patterns we can use to ensure that any latency doesn't impact our applications performance. In this post we are going to concentrate on concurrency.

If we once again look at how SSDS is architected, we will see that we have many, many front end servers that expose the service via our SOAP and REST interfaces. This is the area in the diagram below in blue. In addition, those front end servers can accept many concurrent connections.

Those front end servers look at the operation you are performing, and route the request to the back end server that your container is located on. The back end servers are in the orange area in the diagram above.

To gain maximum performance from SSDS, don't just perform your operations in a singleton fashion. Issue those operations concurrently!!! If you architect your application so that it can make multiple simultaneous asynchronous calls, you can be assured that you are best utilizing SSDS and that your application will perform well - at least from a data access perspective ;)

One thing to watch out for....SSDS only supports a simple concurrency model today. Cases where you need to perform multiple entity updates as part of an single operation, this batch of updates needs to be managed in your application (SSDS doesn't support batched updates today, but its a feature we have in our plans). These operations can be multithreaded, but you need to ensure that you save both the original version of your entity and the updated version of the entity within your application. This way, should one of the operations fail, you can have your application perform any remediation necessary.

So to recap...

• Multithread your SSDS calls when appropriate
• Use Asynchronous access patterns
• Helps combat latency
• Many front-end servers able to handle many requests
• Things to watch out for
• SSDS Supports Simple Concurrency Today
• Multiple entity updates need to be managed by your application
• No support for batching today, but its coming

If anything is not clear, or if you have specific questions around your scenarios, let me know. I'd be happy to help.

-Dave

SSDS MSDN Article Published

Just a quick July 4th post. My MSDN article is up. You can read it for yourself here.

Happy July 4th to those fellow U.S. citizens out there.

-Dave

Still waiting for your golden ticket?

No I am not talking about the chocolate bar movie, I am talking about your invite to the SSDS Private Beta. All the cool kids are in it, and we are ratcheting up the speed at which we add people. We are adding hundreds and hundreds each day. If you want to start testing the most powerful, cloud based, data and query processing service now is the time to sign up. In a couple short weeks we will be rolling out our Sprint 3 bits and there is a great deal of functionality coming.

To sign up go to Register for Free Beta

Best Practice - Data Partitioning

As you all should know by now, SSDS uses what we call the ACE model, compared to a traditional relational model. ACE standing for Authority, Container, and Entity. Keep in mind that you could directly map your relational tables directly to SSDS Entities, we will cover that in another posting. This is specifically about how to partition your data and what we consider the "Best Practice"

The first thing you need to do when thinking about partitioning your data is ask yourself some questions...

• What do my queries look like?
• How can I maximize my throughput by spreading my data across containers?
• How much data do I have?
• What do my use cases look like?

These are all valuable questions you need to ask yourself when thinking about how you are going to store your data in SSDS. Lets take a moment and review the SSDS Architecture and how data is placed on the nodes...

If you recall, SSDS is comprised of a series of front end servers which expose our web services, and a series of backend servers which store the data. The key here is that when you create a container, that container is placed on a backend node that is selected using a proprietary algorithm. So if you store all of your data in a single container, you can guarantee that all your data will be on a single machine (the data is replicated for DR purposes). Since SSDS has many, many, many backend servers, Why not take advantage of them? Think about it this way, if you issue multiple queries to SSDS, do you want a single machine processing the queries, or do you want to have many machines process the queries? That is where partitioning your data comes in...

At TechEd I talked about a Movie Showtimes application and how best to partition that data.

The relational data model looked like this (many props to the MSN Movies team for giving me their data)

In looking at my queries, access patterns and use cases I chose to use the Zip Code to partition my data. Why you ask? Well, first of all the application is to present Movie Showtimes. Most if not all users either start with a movie and then choose a location to see the corresponding showtimes, or they pick a location to see what's playing. By placing each zip codes data into its own authority, I have a very quick and easy query pattern. I also have spread my data across all the machines in the SSDS backend. So if I have 100 users looking for showtimes in 100 different zip codes, I know that chances are that I will get the benefit of having many machines process those request in parallel. Since I might want to see what movies were playing in a theater near me, I can use one of the many readily available web services out there to give me a list of zip codes that are in close proximity to me and then issues the query to those containers as well.

The point to all this is since SSDS has a ton of machines in the backend, you should take advantage of them. Now, one thing I want to make perfectly clear. SSDS is a multi-tenant system. My containers will be placed on machines that have other user's containers on them. I don't want anyone to get the impression that each container is placed on a dedicated machine. We do have the necessary mechanisms in place to ensure that a query to a single container won't consume all the available resources of a single backend machine.

Another point that should be made is with regards to Authorities. Authorities are the unit of geo-location. What that means is when you provision an Authority, you will get to choose the data center that the Authority is hosted in. With that being said, create your Authority in a data center that is in close proximity to your users. While this functionality is not turned on during the beta, it will be by the time we go live.

For the showtimes app, I put all west coast zip codes in a west coast authority and all east coast zip codes in an east coast authority like so...

So to recap...

• Take advantage of the Geo-Location aspect of Authorities
• Choose an authority closest to your users
• Take maximum advantage of Containers
• Containers are placed on individual nodes
• Partitioning your data across containers maximizes your throughput on Query and CRUD operations.

If anything is not clear, or if you have specific questions around your specific scenarios, let me know. I'd be happy to help.

-Dave

TechEd 2008

I have been back for a few days from 2 weeks in beautiful and extremely hot Orlando, FL at TechEd 2008. Everything was amazing. I gave a SSDS Best Practices talk which was well received and we also filmed a TechEd Online video which gives you a very candid look into what SSDS is and isn't and the business scenarios it supports. It will be posted in the coming week here. I will also post something when it comes online.

We got to chat with many customers and got a great deal of positive feedback and comments. Thank you everyone.

Over the next day or so, I plan on starting a new series of blog postings on SSDS Best Practices. I will start with the content from my TechEd presentation and I will continue to add to it as we add more and more features to SSDS.

That's it for now,

Dave

Peace & Quiet?

Roger Jennings wrote an interesting post to his blog here. He questions the fact that the SSDS forums have been quiet...

He does have a valid point. Not having a view into the day to day customer interactions the team has could lead people to believe that things are quiet, but that is certainly not the case :)

Who are we working with? I can't name names other than the ones showcased on our DevCenter and Public sites. What I can tell you is that we are working with companies of all shapes and sizes, from some of the worlds largest banks to small start-ups, customers covering almost every industry imaginable.

Do we expose a limited set of functionality today? Compared to SQL Server, absolutely - we have been very vocal about where we started this journey and where we will end up. Now compare our functionality to the other players in our space and I think you will see that SSDS is feature rich, comparatively speaking.

Another thing that I think I need to stress is that our team uses the SCRUM development methodology and we run 8 week sprints which makes us extremely agile. New features every 8 weeks...WOW

So, I have said this before, if there is a specific scenario that SSDS does not fit for, TELL ME ABOUT IT. I want to help. Post it here, or email me at david DOT robinson AT microsoft DOT com

SSDS Tech-Ed 2008 North America Sessions

Hi everyone,

I am off to Tech-Ed 2008 in Orlando, FL. I have the pleasure of presenting at both the Dev week and the IT week.

Below is a list of all the SSDS sessions at Tech-Ed. Please stop by my sessions and say hi :)

-Dave

SSDS MSDN DevCenter is live

Hi everyone,

Just a heads up. Our new DevCenter on MSDN is up. You can check it out at http://msdn.microsoft.com/en-us/sqlserver/dataservices/default.aspx All the things you usually expect from a DevCenter are there, plus we will be adding some downloads in the future.

If I was you, I would bookmark it :)

-Dave

SSDS Documentation is live on MSDN

Hi All,

Sorry for the lack of blogging as of late. We have been hard at work making SSDS the ultimate data utility service :)

As many of you have requested, the SSDS documentation is live on MSDN at

http://msdn.microsoft.com/en-us/library/cc512417.aspx

With TechEd just around the corner, I have been working on lots of new stuff to share, so expect some really cool posts in the upcoming days and weeks.

<Teaser>

As we talk about value props, the Microsoft Data Platform, and what a data utility service from Microsoft offers the community, the one point I don't think we stress enough is the fact that SSDS has SQL Server at its core. While SSDS is not SQL Server on the web, it does belong to a really powerful family of products and tools from the SQL Server Team.

How awesome would it be to take some data out of some data source, use SSIS to pump that data into SSDS and then use SSRS to report off of it?

</Teaser>

That's it for now. As always, feel free to ping me if you have any questions.

-Dave

Red Bull Video is Up

As I mentioned in a previous post, While we were at Red Bull, the folks from TechZulu.com started interviewing me while I was writing the drink database. The video is up and can be seen here

-Dave

SSDS, The Code Trip & Red Bull

I just posted an entry on The Code Trip site about an app I was challenged to write before I was to present during our stop last week at Red Bull’s HQ.

Check it out here

Yes, before you ask, it IS that easy to provision an Authority and start coding against SSDS…

-Dave

SSDS for game developers

I was at Cal State Fullerton as part of The Code Trip yesterday giving a talk to game developers. It was XNA Day there and in preparing for my presentation I was thinking about some of the challenges that game developers face. I am not talking about collision detection or bounding boxes..I am talking about data…

So, I focused the talk on SSDS and how game developers can use it to persist game data such as high scores, game saves and what have you. With that data up in the cloud you can start to build communities and social networking around your games. You can even use it simply to store user feedback on a new version...

The best part - no infrastructure required...The last thing a game developer needs to worry about is setting up and maintaining that infrastructure...

Not to mention if you write a game like Halo 3 that all of a sudden millions of user want to save their game data...SSDS has all the capacity you need...

You worry about what you do best - games..Let us worry about your data storage..

-Dave