Life in the clouds... : SSDS

Eugenio's experience migrating an app to SDS

davidrob — Mon, 15 Jun 2009 19:13:00 GMT

Quick heads up. Eugenio has a blog posting on his experience migrating an app to SDS. The part about the posting that I like the most is his developing locally on SQL Express and just changing his connection string. Check it out here.

-Dave

Getting ready to re-surface

davidrob — Sat, 07 Mar 2009 20:57:00 GMT

Hey All,

Its been 3 or 4 months since I blogged and I am getting super excited to start showing the community the forthcoming enhancements to SDS. Mix is only a handful of days away and that will be the first public, non super-duper NDA, demo of the new SDS features. I figured it would be good to take Live Writer off the shelf, wipe the dust off and get ready to start blogging...

I have to keep this short. My wife is out shopping and I am home with the kids and World War III is about to break out between the twins over a sippy cup, so I better intervene.

More info coming soon,

-Dave

PDC2008 - Who's the idiot with the video camera?

davidrob — Sun, 26 Oct 2008 17:49:00 GMT

Ok...So today is PDC2008 Pre-Con day. Just a quick heads up...

You will see some guys walking in and around the LA Convention Center with nice decked out video equiptment. Those are the Channel 9 guys.

You might also see some idiot walking around with a backpack and an average run of the mill HandyCam. That would be me :) What am I doing? Gathering data for my presentation and demo of course. What is the demo you ask? Well, I can't go into it right now. I wouldn't want to spoil the surprise. I can tell you its uber nerdy. I am married with 4 kids, no need to act cool anymore - EMBRACE YOUR INNER GEEK!!!

So, you want to see what it is? You have 2 options. Come to my talk on Tuesday @ 12:45, or wait until after my talk and I'll start posting details here.

So if you see me walking around, grab me and say Hi.

-Dave

2 Days to PDC2008!!!!!!!!!!!!!

davidrob — Sat, 25 Oct 2008 07:01:00 GMT

Tommorow afternoon, I get on a plane and head down to LA for PDC2008. I have to be completely honest. We have some really super, pimped out, mack daddy, completely awesome stuff coming. Both in the teams presentations and in new features for Sprint 5. I just sent out the new features list to some internal softies for "Word Smithing" which you all should see in a few days, and the list is long and chock full o' goodies.

Thats all for now. If I keep typing I am bound to leak something and will wind up ruining the suprise :)

-Dave

PS - I promise much more posting after Monday.

Venga a Barcelona y óigame que hablo en TechEd EMEA

davidrob — Sat, 11 Oct 2008 07:10:00 GMT

Hola!!!

Just a quick note. Sorry for the lack of posts as of late. We have all been heads down preparing for PDC.

I wanted to give a heads up to all the SSDS users on the other side of the pond. Myself and Istvan Cseri will be presenting at TechEd EMEA in Barcelona. For more information click here.

-Dave

Who’s coming to PDC?

davidrob — Wed, 24 Sep 2008 05:55:26 GMT

So…

I have gotten pinged by a bunch of people asking if we were planning any evening events for SSDS users. So, If there is sufficient interest in such an outing, please let me know by shooting me an email at david.robinson@microsoft.com

I am getting super excited about PDC. It is going to be so cool. So much stuff to show and talk about…plus I have to tell you, I love hanging out with our users. You folks are so cool.

See you all soon,

Dave

SSDS and SSIS…like peanut butter and jelly

davidrob — Wed, 24 Sep 2008 02:02:00 GMT

Hey All,

One of the great things about SSDS is that we belong to a rich family of products. One of those products is SQL Server Integration Services – everybody’s favorite ETL tool. Well, thanks to some hard work by Matt Masson and his team, we now have an SSDS Adaptor for SSIS.

You can now use SSIS to take data from virtually anywhere and pump it into or out of SSDS. This is one of the things I showed during my TechEd talk, but now you can try it for yourself.

Matt has posted to source and binaries over on CodePlex.

Enjoy!!!!

SSDS Tech-Ed Online Video Posted

davidrob — Fri, 29 Aug 2008 21:55:55 GMT

Hey all

During Tech-Ed North America 2008, Soumitra, Niraj and I did an interview for Tech-Ed online. Lots of good info.

The video can be found here http://www.microsoft.com/techedonline/

-Dave

What kind of examples would be helpful?

davidrob — Wed, 06 Aug 2008 01:20:00 GMT

Here is a question. Myself and other members of the SSDS team are always asking the “What do you want to see” question. I am going to ask a similar question…

What kind of examples would be helpful?

Would you rather see code snippets or a fully working application?
Would you rather see a console app? WinForm? ASP.Net? WPF? Silverlight?
What type of app?
What are the challenging scenarios that you are facing that you would like a reference application to help you solve?

Feel free to email me at david.robinson AT microsoft.com

Back from vacation...well, almost

davidrob — Sun, 20 Jul 2008 22:23:00 GMT

Hey Everyone,

Just a quick post. I have been on vacation for the past 10 days in New York visiting family and relaxing with my wife and kids. I am getting ready to head to the airport to fly home and I just started digging out of email. Luckily for me I have a 6 hour flight back to Seattle. We are flying Jet Blue so hopefully my kids (ages 8, 6 and 3 (twins)) will enjoy the complementary TV and let me get some solid work done. It’s kind of weird travelling with kids. When I travel by myself, I am the guy that prays before I step foot on the flight that I am not sitting next to any younger kids or babies. Don't get me wrong, I absolutely love kids, but I look forward to the distraction free time on a plane to get work done. My kids are used to flying so they are usually well behaved...Usually...anyone who has kids knows that at the drop of a hat, a three year old can turn from a sweet little thing to a demon. I know they have the term terrible twos, but my experience has been that three has always been the age to watch out for...Needless to say, I am keeping my fingers crossed.

Also everyone should have received my email blast last week from Microsoft Connect on the new Sprint 3 features coming. The text from that can be seen here. Those features are getting rolled out this upcoming Thursday and should be completed by Friday. Our super - duper - magnificent operations team has been phenomenal at these rollouts and although the product team works really hard to crank out these new features, it is the unsung operations heroes that deserve some serious kudos.

Hope everyone didn't miss me too much :)

Dave

Best Practice - Take Advantage of Concurrency

davidrob — Mon, 07 Jul 2008 06:43:00 GMT

In the first best practice, we looked at how we can gain maximum performance out of SSDS by partitioning our data across multiple containers. In this post, we are going to look at how we can take advantage of the many SSDS front end servers to maximize our applications performance.

SSDS is an Internet based data storage and query processing utility service. Lets face it, SSDS can only be as fast as your Internet connection. With that being said, when we architect our applications, we need to keep that in mind and be sure that we take whatever steps necessary to ensure that our application performs well. There are a handful of access and caching patterns we can use to ensure that any latency doesn't impact our applications performance. In this post we are going to concentrate on concurrency.

If we once again look at how SSDS is architected, we will see that we have many, many front end servers that expose the service via our SOAP and REST interfaces. This is the area in the diagram below in blue. In addition, those front end servers can accept many concurrent connections.

Those front end servers look at the operation you are performing, and route the request to the back end server that your container is located on. The back end servers are in the orange area in the diagram above.

To gain maximum performance from SSDS, don't just perform your operations in a singleton fashion. Issue those operations concurrently!!! If you architect your application so that it can make multiple simultaneous asynchronous calls, you can be assured that you are best utilizing SSDS and that your application will perform well - at least from a data access perspective ;)

One thing to watch out for....SSDS only supports a simple concurrency model today. Cases where you need to perform multiple entity updates as part of an single operation, this batch of updates needs to be managed in your application (SSDS doesn't support batched updates today, but its a feature we have in our plans). These operations can be multithreaded, but you need to ensure that you save both the original version of your entity and the updated version of the entity within your application. This way, should one of the operations fail, you can have your application perform any remediation necessary.

So to recap...

Multithread your SSDS calls when appropriate

Use Asynchronous access patterns
Helps combat latency
Many front-end servers able to handle many requests

Things to watch out for

SSDS Supports Simple Concurrency Today
Multiple entity updates need to be managed by your application
No support for batching today, but its coming

If anything is not clear, or if you have specific questions around your scenarios, let me know. I'd be happy to help.

-Dave

Technorati Tags: SSDS,SQL Server Data Services,Best Practice

SSDS MSDN Article Published

davidrob — Fri, 04 Jul 2008 22:11:00 GMT

Just a quick July 4th post. My MSDN article is up. You can read it for yourself here.

Happy July 4th to those fellow U.S. citizens out there.

-Dave

Technorati Tags: SSDS,MSDN,SQL Server Data Services

Still waiting for your golden ticket?

davidrob — Thu, 26 Jun 2008 19:49:39 GMT

No I am not talking about the chocolate bar movie, I am talking about your invite to the SSDS Private Beta. All the cool kids are in it, and we are ratcheting up the speed at which we add people. We are adding hundreds and hundreds each day. If you want to start testing the most powerful, cloud based, data and query processing service now is the time to sign up. In a couple short weeks we will be rolling out our Sprint 3 bits and there is a great deal of functionality coming.

To sign up go to Register for Free Beta

Best Practice - Data Partitioning

davidrob — Wed, 18 Jun 2008 21:57:51 GMT

As you all should know by now, SSDS uses what we call the ACE model, compared to a traditional relational model. ACE standing for Authority, Container, and Entity. Keep in mind that you could directly map your relational tables directly to SSDS Entities, we will cover that in another posting. This is specifically about how to partition your data and what we consider the "Best Practice"

The first thing you need to do when thinking about partitioning your data is ask yourself some questions...

What do my queries look like?
How can I maximize my throughput by spreading my data across containers?
How much data do I have?
What do my use cases look like?

These are all valuable questions you need to ask yourself when thinking about how you are going to store your data in SSDS. Lets take a moment and review the SSDS Architecture and how data is placed on the nodes...

If you recall, SSDS is comprised of a series of front end servers which expose our web services, and a series of backend servers which store the data. The key here is that when you create a container, that container is placed on a backend node that is selected using a proprietary algorithm. So if you store all of your data in a single container, you can guarantee that all your data will be on a single machine (the data is replicated for DR purposes). Since SSDS has many, many, many backend servers, Why not take advantage of them? Think about it this way, if you issue multiple queries to SSDS, do you want a single machine processing the queries, or do you want to have many machines process the queries? That is where partitioning your data comes in...

At TechEd I talked about a Movie Showtimes application and how best to partition that data.

The relational data model looked like this (many props to the MSN Movies team for giving me their data)

In looking at my queries, access patterns and use cases I chose to use the Zip Code to partition my data. Why you ask? Well, first of all the application is to present Movie Showtimes. Most if not all users either start with a movie and then choose a location to see the corresponding showtimes, or they pick a location to see what's playing. By placing each zip codes data into its own authority, I have a very quick and easy query pattern. I also have spread my data across all the machines in the SSDS backend. So if I have 100 users looking for showtimes in 100 different zip codes, I know that chances are that I will get the benefit of having many machines process those request in parallel. Since I might want to see what movies were playing in a theater near me, I can use one of the many readily available web services out there to give me a list of zip codes that are in close proximity to me and then issues the query to those containers as well.

The point to all this is since SSDS has a ton of machines in the backend, you should take advantage of them. Now, one thing I want to make perfectly clear. SSDS is a multi-tenant system. My containers will be placed on machines that have other user's containers on them. I don't want anyone to get the impression that each container is placed on a dedicated machine. We do have the necessary mechanisms in place to ensure that a query to a single container won't consume all the available resources of a single backend machine.

Another point that should be made is with regards to Authorities. Authorities are the unit of geo-location. What that means is when you provision an Authority, you will get to choose the data center that the Authority is hosted in. With that being said, create your Authority in a data center that is in close proximity to your users. While this functionality is not turned on during the beta, it will be by the time we go live.

For the showtimes app, I put all west coast zip codes in a west coast authority and all east coast zip codes in an east coast authority like so...

So to recap...

Take advantage of the Geo-Location aspect of Authorities

Choose an authority closest to your users

Take maximum advantage of Containers

Containers are placed on individual nodes
Partitioning your data across containers maximizes your throughput on Query and CRUD operations.

If anything is not clear, or if you have specific questions around your specific scenarios, let me know. I'd be happy to help.

-Dave

TechEd 2008

davidrob — Tue, 17 Jun 2008 22:38:59 GMT

I have been back for a few days from 2 weeks in beautiful and extremely hot Orlando, FL at TechEd 2008. Everything was amazing. I gave a SSDS Best Practices talk which was well received and we also filmed a TechEd Online video which gives you a very candid look into what SSDS is and isn't and the business scenarios it supports. It will be posted in the coming week here. I will also post something when it comes online.

We got to chat with many customers and got a great deal of positive feedback and comments. Thank you everyone.

Over the next day or so, I plan on starting a new series of blog postings on SSDS Best Practices. I will start with the content from my TechEd presentation and I will continue to add to it as we add more and more features to SSDS.

That's it for now,

Dave