Best Practice - Data Partitioning

re: Best Practice - Data Partitioning

jamiet — Thu, 19 Jun 2008 01:13:03 GMT

Hi David,

"when you provision an Authority, you will get to choose the data center that the Authority is hosted in"

Hmm...what if my application uses a single authority at the back-end but is used globally? Do you have recommendations to share for that scenario?

Thanks

Jamie

re: Best Practice - Data Partitioning

davidrob — Thu, 19 Jun 2008 01:20:51 GMT

Hey Jaime,

That is a great question. I would say at this point choose an Authority that is closest the majority of your users. We are looking at some scenarios to support replication of data across Authorities, but it is too early to comment on it.

HTH,

Dave

re: Best Practice - Data Partitioning

jamiet — Thu, 19 Jun 2008 09:03:13 GMT

Hi David,

Thanks for the reply.

As ever when you make far-reaching decisions such as data replication there are going to be trade-offs.

If you were to do this what would the trade-off be? Would you introduce eventual consistency?

-Jamie

re: Best Practice - Data Partitioning

davidrob — Thu, 19 Jun 2008 20:03:19 GMT

Jaime,

We are still ironing out the details on this. One point I would like to make is around eventual consistency.

SSDS is from the SQL team at Microsoft. We are an enterprise database team and to be honest things like eventual consistency don't fly here. How can you trust a data platform when if you do an operation and then try to retreive the updated record you may or may not get the most recent copy?

Try telling this to a customer...

"I'm sorry Mr. Jones, but even though you removed the 1000 widgets from your order, due to our data platforms EVENTUAL CONSISTENCY model when you clicked complete order, the delete didn't happen as of yet. Feel free to return the 1000 widgets and unfortunately we need to charge you a 20% restocking fee"

That won't fly will it?

re: Best Practice - Data Partitioning

jamiet — Fri, 20 Jun 2008 11:18:20 GMT

That's music to my ears David. I've never been comfortable with the idea of eventual consistency.

As a user I shouldn't really care about HOW you're going to achieve this (i.e. global replication whilst still upholding ACID) but nonetheless I'm intrigued. If you want to share anything around it then I'll be listening!

-Jamie (not Jaime :)

P.S. I've subscribed to email alerts for this comment thread but haven't received any emails - I never have for any MSDN blogs. Good job I subscribed to the RSS feed.

Best Practice - Take Advantage of Concurrency

Life in the clouds... — Mon, 07 Jul 2008 06:43:13 GMT

In the first best practice , we looked at how we can gain maximum performance out of SSDS by partitioning

re: Best Practice - Data Partitioning

nickfr — Fri, 01 Aug 2008 15:21:34 GMT

David,

an interesting article. I have a scenario where I have in relational terms a table with millions of rows of data. It sounds like I should partition the data into multiple containers - I guess almost like a hash table. I was wondering on the practical limits on this. Can I have a million containers? each with say up to 1000 entities? I guess I don't really understand the performance characteristics of SSDS yet and your articles are the first that address this. Could you or the team expand on this area?

thanks

-NiCK