Trip Report – HPTS (High Performance Transaction Systems) Workshop – Oct 25^th-28^th, Asilomar, Pacific Grove, CA.   (www.hpts.ws).    This is Part 1 of the trip report…  I am chiseling my way through my notes and expect this to take a couple more blog entries before I can describe everything I plan to cover.  Believe it or not, this 11 page “Part 1” is only taking us through Monday morning – there’s Monday afternoon, all Tuesday, and Wednesday morning yet to cover!

Overview:

HPTS was founded by Jim Gray, Dieter Gawlick, Andreas Reuter, and some other industry luminaries in 1985.  It is a workshop designed to bring together practitioners and academics working in super large, scalable, high throughput systems.  There is a lot of overlap between this community and the folks doing DBMS systems.  In the early days, the biggest challenges included getting 1000 transactions per second through a mainframe.  At the time, that felt like getting a man to land on the moon and come back safely.  Now, this includes huge web based sites like Amazon, eBay, Google, and Bing.  Our area of interest also includes integrating massively complex heterogeneous Enterprise and Internet (B2B) systems.  Of course, we are fascinated with hardware trends, networking, and all sorts of geeky stuff.

I first attended HPTS in 1985.  It was a magic opportunity to be with legends in the industry.  Gene Amdahl drove up in his Rolls Royce and parked next to my beat up old van.  Mike Blasgen(System R), Mike Stonebraker, Bruce Lindsay, Mohan, Andy Heller, Dieter Gawlick, Hector Garcia-Molina, Irv Traiger, Phil Bernstein, Jim Gray, and many more folks who had become well known to me through their papers were there in the flesh and treating me (a 29 year old punk) with kindness and respect.  It was a magical 3 day gathering.

Let me explain the Asilomar Conference Center (www.visitasilomar.com).  Asilomar sits right by the Pacific Ocean between Pebble Beach and Monterey in the Pine Trees and includes dunes leading right to the beach.   It was built as a YWCA around 1910 or so and contains a number of beautiful historic buildings in the arts and crafts style built by architect Julia Morgan.  In the 1950s, it was sold to the state of California which has operated it as a conference center (and built more buildings in the 1960s).  The rooms are rustic (no TVs or phones – crappy mattresses), the food is served in a large dining hall when the bell rings on the roof and it has a magic feel.   It is normal to walk out of your room, hear the surf, and see deer walking around.  I absolutely love the evenings by the real wood fireplaces.

In 1985, Gray decided we could all go cheap and lined us up to sleep four in a room.  I shared a room with Dieter Gawlick (inventor of IMS/FastPath among other things), Hector Garcia-Molina, and Theo Haerderer (a wonderful professor from Germany).   After the first night, Hector informed us we ALL snored too much and moved to stay in a hotel down the street.  For the next 8 or 10 gatherings, we were two to a room but now most of us pay a bit more to have a private room.

I’ve attended HPTS every two years (for three days) since 1985.  I was program and general chair in 1989 and general chair in 2007.  Since Jim Gray’s disappearance, I’ve assumed the role of local arrangements chair and “godfather” – kinda’ watching out for the whole workshop.  This year, James Hamilton (now of Amazon) was general chair and Margo Seltzer (of Harvard) was the program chair.  We had a wonderful (but exhausting) three days and I will tell you about them soon.

One of the traditions is that the folks leading HPTS wander around during the first two days trying to figure out who gets suckered into doing the work as the next workshop happens two years later.  I am thrilled to report that Mohan (from IBM) will be the general chair in 2011 and Mike Carey (now of UC Irvine) will be the program chair.  I will remain as local arrangements chair and “godfather”.

So, every HPTS starts with Sunday dinner and a casual beer/wine afterwards.  Monday is all day presentations and after dinner we have poster sessions (volunteer 10 minute talks – we had 15 this year).  Tuesday is all day presentations and this time we had a 90+ minute panel after dinner.  Finally, we have sessions Wednesday morning and adjourn after lunch.  It is a long, exhausting, and exhilarating 3 to 4 days.

Of course, the most important part of the workshop is the breaks and the opportunity to talk with people working in the same area as you are working.  In addition to the grizzled old luminaries, this year we went out of our way to get more graduate students and invited 16 of the brightest and best from Berkeley, Stanford, MIT, Harvard, Brown, and more.   I spent each of the three evenings staying up late talking by the fire with these wonderful young folks (and some other diehard old farts).   As I type this the following weekend, I am still tired from the sleep shortage!

Session 1) The New Memory Hierarchy –
                            Session Chair: David Cheriton (Stanford)

Steve Kleiman – Chief Scientist at NetApp:   “Flash on Compute Servers”

So, Steve Kleiman is the Chief Scientist at NetApp so you would expect him to be focused on network attached storage and we were not disappointed.

Steve started with a discussion of hardware trends (not all of which I was able to capture in my notes).  He observed that with multi-core and VMs, more bandwidth (BW) and IOs per Second (IOPS) are needed for each node.  Also, as a trend, Steve pointed out that flash memory has more than 5X the IOPS and less than 100 microseconds access time.

The original case for network storage lies around the benefits of aggregating disk which include more random IOPS and bandwidth as well as the management simplicity of concentrating data into a centralized place.  It has been the case that a single application per host has somewhat limited ability to consume bandwidth.

Considering flash in a networked storage environment, it does not have the same benefits of sharing.  Since there are enough IOPS in a flash drive to placate most apps, the equation changes.  Is network storage fast when the network latency is noticeable compared to flash?  We can do 5 SSDs to get 40,000 IOPS but will the latency be a problem?

We will see flash on compute servers.   Servers can already accommodate flash and it is cheap enough with 250GB < 10% of the server cost.

A new tier in the storage hierarchy:  Steve argues we will see storage split into two tiers: the IOPS tier and the capacity tier.  The IOPS-Tier will offer the best $/IOPS with low latency and will have sufficient capacity for the active data in most typical enterprise systems.  It will be the new “primary storage”.   The Capacity-Tier will offer the best $/GB – Deduplication, compression, and good serial performance.

Steve then outlined three broad categories of Host-Based Flash usage.  DAS (which I think means “Direct Access Storage but my notes failed me), Primary Storage Service, and Cache.

·         Host-Base Flash: DAS:  Single point of failure and so you would use normal backup or application based replication like Exchange or Oracle).  You can have boot files, VM, temp files. 

·         Host-Based Flash: Primary Storage Service:  File or Block.  Requires mirroring or RAID to peers.  Allows capacity sharing.  Steve believes it is likely that Microsoft and VMWare are likely to compete here – the integration of this is complex.  Making reliable primary storage is hard and this may impact the roles of folks in IT.

·         Host-Based Flash: Cache –File or block.  This will be a writethrough cache.  If you do a write-back you will occasionally lose stuff.  Using a cache can allow for a centralized data management (in a remote network storage server).  This can help with keeping the current IT storage management roles by providing an automatic “tiering”.

So, data management spans the IOPS tier and the Capacity tier.  You need to have automated data movement and global access.  It is impractical to have manual placement at huge scale.  It is going to be important for us to have a technology independent language to specify the desired properties for the data.  Say what properties you want and not how to get them.   New SLO (Service Level Objectives) for Max Latency, Bandwidth, Availability, etc.

In summary, the new host-based flash is the new IOPS-Tier.  It will replace high-performance primary storage in the cloud.  This IOPS-Tier s is not going to be economical as a secondary or as archival since the $/GB is higher than spinning media.   The Capacity Tier will be implemented as network storage.  It is optimized for the best $/GB.  Data management between these two tiers will be important.

There was a lot of discussion around the SLO (Service Level Objectives).   Steve indicated that their vision was not super-flexible set of knobs but more a small set of options for defining the data characteristics.  Margo offered that one aspect of what was important was the “value” of different parts of data which can, in turn, tell the system how hard to work to make the data remain available.

Someone asked if there was any performance analysis for 3-tier storage (memory, IOPS-Tier, Capacity-Tier) over more traditional 2-tier storage (Memory and Disk).  Steve answered that there had not been an explicit performance analysis.

Stonebraker observed that enterprises seem to like NAS/SAN but that the Cloud guys are using direct attached storage.  Are the forward looking guys walking from NAS/SAN?  Steve replied there are people very interested in NAS/SAN because it gives them a way to control the data and make servers stateless.  Adding in the IOPS-Tier is a simple extension.

This was a fascinating and engaging presentation!

Memory Technologies for Data Intensive Computing –Andy Bechtolshein (Sun Computers)

What an honor to get Andy to come and present to us.  Unfortunately, I didn’t get a chance to chat with him personally as he was only at the workshop for a short while.

Andy is a well respected and talented hardware engineer who was a founder of both Sun Microsystems and of Arista Network.  His slides are not going to be posted and I typed as aggressively as I could during his lightning fast presentation.  I would recommend you look both at my friend James Hamilton’s blog post of Andy’s talk at: http://perspectives.mvdirona.com/2009/10/26/AndyBechtolsheimAtHPTS2009.aspx and then read my notes attached below.  I have not attempted to coalesce the information between James’ notes and mine and there may be overlap.  James has done a nice job cleaning his up and I will focus on some of the other sessions in my editing.

Here’s my rough notes:

è Andy speaks VERY fast and I lost a lot of what was covered – The slides were very dense – Also, I apologize for some inaccuracies… These are RAW notes that I made during the fast paced presentation and some of them don’t completely make sense to me. 

 

Semiconductor Technology Roadmap

Still increasing

Number of devices on the chip still doubling

Flash is increasing faster than logic and DRAM – Flash faster than Moore’s law

Between 2010 and 2022, Flash should increase by 100X – After that (10nm) optical problems may become severe – don’t know after 2022

<more content lost…>

Emerging devices

·         Carbon-nano tubes

·         Graphene Nanoribbons

·         … <more that I lost…>

Silicon Roadmap Predictions

·         128X increase in transistors by 2022

·         1000 cores per chip by 2022

CPU Module Roadmap

·         IO bandwidth will not allow the same bandwidth per core.  IO going a little faster but number of pins hasn’t increased in 10 years.  

·         1000 cores would need several terabytes of bandwidth to memory.  Of course, this would need power and cooling

·         <more lost…>

CPU Challenge

·         Power per core -- 

·         Clock rates not increasing – need lower power level per core

·         <more lost…>

Power efficiency – Increasing frequency – a Square effect on the power

·         CPU power strategy –

·         Clock rates look flat

·         High clock rates are less power efficient

·         To reduce power, simplify architecture

·         <more lost…>

Multi-Chip 3D packaging

·         Thru-Si via Stacking

·         Need to combine CPU and Memory on one module

High Density 3D Multi-Chip Module

·          

Benefits of MCM Packaging

·         Enables much higher memory bandwidth

·         More channels, wider interfaces, faster I/o

·         Greatly reduces power

Challenges with MCM packaging

·         Total Memory – 64 device <…??...>

·         With 64 GNB/device – 4TB

·         Assuming 1K cores – 4BG/core

MCM enables  Fabric I/O

·         2010 –1 32 Gb/direction

·         2020 1.73Tbps/direction

·         Mesh or higher radix fabric topologies

·          

Expected Link Data Rates

·         10GBps shipping today

·         2012 – 25 GBps

·         2016 – 50 GBPS

·         2020 – 100 GBps

ExaScale Storage

·         Forget Hard Disks – Disk are not getting faster

Flash Experience so far

·         Not easy to get to 1000000 IOPS

·         Writes are a problem

·         SAS/SATA interface not optimal

·         Direct PCI-Express interface looks more promising.  – Move FLASH closer to the CPU  -- Lower latency and more throughput

Flash in the Memory Hierarchy

·         Flash is not random access  more like a block device  -- random access about 1000 slower than DRAM

·         Flash can be used as a stable storage device  -- writes are committed – Supercap magic behind the scenes.

·         Tremendous throughput and size – terabytes of capacity

Flash Latency

·         Latency expected to drop by a factor of 10

·         Flash transfer rates will double each year for the next several years. – 250MBS/sec 2012

Flash prices

·         2004 -- $100 GB

·         2009 $1 – GB

·         2012 -- $.25 / GB 

·         Prices are dropping dramatically –

Flash Summary

·         FLASH will be very expensive, very dense,

·         Costs falling by 50% per year

·         Access times falling by 50% per year

Tech Summary

·         Moore’s Law for at least 10 more years

·         Frequency Gains  not happening

·         Flash will be interesting

Software Challenges

·         Limits of App Parallelism

·         Need to exploit Parallelism

 

Session 2) Using the New Memory Hierarch
                                          Session Chair: David Cheriton (Stanford)

Scaling-Out without Partitioning – Phil Bernstein and Colin Reid (Microsoft)

This session was, in many ways, one of the most interesting at HPTS.  It was not new to me since I had been privileged to spend a number of hours talking to Colin (and a bit to Phil) about Hyder back at Microsoft.  Phil did a great job explaining a radical new approach to scale-out and serializable database!

Hyder is a software stack for transactional record management.  It can offer full database functionality and is designed to take advantage of flash in a novel way.

Most approaches to scale-out use partitioning and spread the data across multiple machines leaving the application responsible for consistency.

In Hyder, the database is the log, no partitioning is required, and the database is multi-versioned.  Hyder runs in the App process with a simple high-performance programming model and no need for client server.  This avoids the expense of RPC.

Hyder leverages some new hardware assumptions.  I/Os are now cheap and abundant.  Raw flash (not SSDs – raw flash) offers at least 10^4 more IOPS/GB than HDD.  This allows for dramatic changes in usage patterns.  We have cheap and high performance data center networks.  Large and cheap 64-bit addressable memories are available.   Also, with many-core servers, computation can be squandered and Hyder leverages that abundant computation to keep a consistent view of the data as it changes.

The Hyder system has individual nodes and a shared flash storage which holds a log.  Appending a record to the log involves a send to the log controller and a response with the location in the log into which the record was appended.  In this fashion, many servers can be pushing records into the log and they are allocated a location by the log controller.  It turns out that this simple centralized function of assigning a log location on append will adjudicate any conflicts (as we shall see later).

The Hyder stack comprises a persistent programming language like LING or SQL, an optimistic transaction protocol, and a multi-versioned binary search tree to represent the database state.  The Hyder database is stored in a log but it IS a binary tree.  So you can think of the database as a binary tree that is kept in the log and you find data by climbing the tree through the log.

The Binary Tree is multi-versioned.  You do a copy-on-write creating new nodes and replace nodes up to the root.  The transaction commits when the copy-on-write makes it up to the root of the tree.

For transaction execution, each server has a cache of the last committed state.  That cache is going to be close to the latest and greatest state since each server is constantly replaying the log to keep the local state accurate [recall the assumption that there are lots of cores per server and it’s OK to spend cycles from the extra cores].  So, each transaction running in a single server reads a snapshot and generates an intention log record.   The transaction gets a pointer to the snapshot and generates an intention log record.  The server generates updates locally appending them to the log (recall that an append is sent to the log controller which returns the log-id with its placement in the log).  Updates are copy-on-write climbing up the binary tree to the root.

Log updates get broadcast to all the servers – everyone sees the log.  Changes to the log are only done by appending to the log.  New records and their addresses are broadcast to all servers.  In this fashion, each server can reconstruct the tail of the log.

Now we get to transaction commit.  All machines are marching ahead rolling the log forward.  When an intention record is generated (indicating a proposed new root for the log), each machine in the cluster checks for a conflict.  If any of the records updated conflicts with previously committed work, the transaction is aborted.  Conflict in this case is traditional read/write conflict using all the classic set of rules for various degrees of consistency.  If there are no conflicts, the transaction is committed.  If there are conflicts, the transaction is aborted. 

Because there is an append in the log controller which assigns order, each intention record will race to the log and the controller will pick a winner.  Because log ordering is deterministic, each of the server nodes can reliably come to the same conclusion about which transaction won the race and, hence commits rather than aborts in the face of conflict.  Conflict detection and transaction commit is done redundantly and predictably at each node.

So, each node generates an intention log for a transaction it wants to do.  Every server checks for conflicts and then either aborts or commits.  The log order provides determinism.

Performance of Hyder:  The system scales without partitioning.  The system-wide throughput of update transactions is bounded by the update pipeline.  It is estimated this can perform 15K update transactions per second over a 1GB Ethernet and 150K update transactions per second over a 10GB Ethernet.  Conflict detection and merge can do about 200K txs per second.

The abort rate on write-hot data is bounded by the transaction’s conflict zone.  This is determined by the transactions end-to-end latency.  This conflict zone is about 200 microseconds in the prototype.  It is estimated that this would handle about 1500 TPS (assuming all the transaction have conflicts).

Major Technologies:  This is using flash as append only (and it is not using SSD flash but raw flash with a special controller).  The custom controller over raw flash has mechanisms for sync and fault tolerance.  Storage is striped with a self-adaptive algorithm for storage allocation.  It includes a fault tolerant algorithm for a totally ordered log.  Also, it has a fast algorithm for conflict detection and merging of records.

Status:  Most of Hyder has been prototyped but there is a long way to go.  Phil Bernstein and Colin Reid are working on a paper and HPTS is the first time Hyder has been discussed publicly.

Again, this was a wonderful talk with some fascinating and different ideas.  Great work!

A Performance Puzzle:  B-Tree Insertions are Slow on SSDs (Bradley Kszmaul – Tokutek – Research Scientist at MIT)

So, this talk was about some surprises in SSD performance in supporting MySQL.  Much of the discussion was about the datasheets for some of the drives not matching the measured reality.

Bradley had measured the value of running MySQL against SSDs and expected the SSDs (with higher IOPS) to get better Btree performance.  Here’s my rough and raw notes:

Intel X25E Specs

·         Read BW up to 250 MB/s

·         Write BW up to 170 MB/s

·         Random 4KB read rate – 35 KIO/s

·         Random 4KB

·         <lost some…>

Measure Berkeley DB

·         Trending to 4500 write per second

Performance Model  --

·         Startup cost plus bandwidth cost

·         Looking at read performance as a function of block size

·         As block size gets bigger, bandwidth goes up

Discussion à Margo points out that many Filesystems see large blocks and do read ahead.  That can perturb the results…

When you read and write large blocks, Bradley sees Filesystems doing the right thing and keeping contiguous blocks adjacent.

The datasheet for the drive is not matching the reality for larger block sizes

Mixing reads and writes seems to impact the performance quite a bit…

What Block Size to Use?

·         For point-queries, BTrees are insensitive to block size. 

·         For range queries, block size if important

·         Tension: Large block sizes make range queries faster; Small block sizes make exact match faster

·         Set block size so half of the time is accounted by seek, half by bandwidth

·         Reads: 50KB for SSD but 500KB-1000KB for rotating disk

·         Writes 10KB SSD , 500KB-1000KB for HDD

·         Mixed read/write 21KB for SSD

Cache-Oblivious Approach

·         Use data structures that are fast for any block size

·         Also can speed insertions without slowing searches

·         Tokutek’s MySQL storage engine uses these ideas.

Discussion: James Hamilton layering of software and release by release changes makes it hard to get predictable behavior.   When all these releases are separately evolved, the performance behavior is very unpredictable.   Bradley: fragmentation starts messing with this, too.  James: all the layers interact and we need to measure by the workload and adapt.  The complexity makes performance very hard to predict.

Implications of Storage Class Memories (SCMs) on Software Architectures (C. Mohan – IBM)

Storage Class Memory (SCM)

 

Mohan gave a very thought provoking talk on the implications of Storage Class Memories.  Where do we need them?  What good will they do?

If we have non-volatile memory that is as easy to access as DRAM, will we have corruption problems?  Will we see a running system trash the non-volatile memory like it trashes DRAM?  One of the advantages of our current system design is that we have to pass a “sanity test” of an IO driver or messaging stack to get nastiness outside of a running system.  That dramatically reduces the chance of this happening.

Anyway, Mohan’s talk was really thought provoking and enjoyable!  It was good to see my old friend back at HPTS after taking a stint in India for a few years.

Raw notes:

·         Blurs the distinction between DRAM and Disk

Industry SCM Activities

·         IBM

·         Intel/ST spun out of Numonyx (FLASH and PCM)

·         Samsung, Numonyx sample PCM chips –

·         <lost som>

Speed/Volitility/persistenct

·         DRAM plus SWCM – Fast and persistent

·         Persistent storage will not lose data

Competing technologies for SCM

·         Phase Change RAM (most promising)

SCM as a part of the storage stack

·         The distinction between RAM and Disk – the line in between is blurring

·         Memory-like and also Storage-Like

SCM Design Triangle

·         Speed – endurance – Cost/bit

·         Speed with Write endurance is more memory-like

Phase-Change RAM

·         When density increases, the failure rate goes up as bits flip more often

·         SCM is on Moore’s law

PCM is fast

·         100-1000 ns

Architecture

·         SCM may be used as Memory

·         SCM may, alternatively be used as a storage (through IO controllers)

·         Discussion à Pat:  Bugs in non-volatile memory can sometimes be preserved

PCM Use Cases

·         PCM as Disk

·         PCM as Staging device

·         PCM as memory

·         PCM as extended memory

Exploring DBMS as an exploiter of PCM

·         Should log records be written directly to PCM?

·         Should log be more like Disk (first to buffer and then forced to disk)?

·         PCM is not as reliable as disk à do you need to offload to a disk

·         If you use Group Commit, it is unclear that PCM helps a DBMS

PCM replaces DRAM – Buffer Pool in PC

·         The PCM Buffer Pool access will be slower than DRAM

·         Writes will suffer more than reads!

·         Should we instead have a DRAM BP backed by a PCM BP?

·         Similar to DB2 z in parallel sysplex environment with BPs in coupling facility (CF)

·         But DB2 situation has well defined rules on when pages a move from DRAM CP to CF BP

·         Variation was used in SafeRAM work at MCC in 1989

Assume the whole DB fits in PCM

·         Apply old main memory DB concepts design directly

·         Shouldn’t we leverage persistence specially?

·         Every bit change persisting may not be good!

·         Today’s failure semantics has flexibility on tracking changes to DB pages.

·         Memory overwrites will cause more damage

·         If every write assumed to be persistent as soon as write completes, then L1 and L2 caching can’t be leveraged.

Assume whole DB fits in memory

·         Need to externalize periodically since PCM won’t have good endurance

·         If DB spans both DRAM and PCM, hard to figure out what goes where!

What about Logging?

·         If PCM is persistent and whole DB is in PCM, do we need logging?

 

The end of Monday morning!  More soon… 

 

-          Pat

 

As I mentioned in a previous blog post, I am very impressed with a recent paper from my friends at the Berkeley RAD Lab called: Above the Clouds: a Berkeley View of Cloud Computing.   it seemed to me that it would be important for these concepts to be better understood by a broad audience within Microsoft so I took the time to create a presentation covering the contents of the paper.  You can see the presentation I created covering the paper’s contents here. 

If you are interested in cloud computing, you want to understand these ideas.

Also, I highly recommend the Berkeley blog on cloud computing available at:  http://abovetheclouds.cs.berkeley.edu.

Thanks to the folks at Berkeley for their hard work and permission to show this to you.

- Pat

I know this is a LONG time ago but these slides were never made public before and there's some interesting (and still relevant) stuff in this collection.  At the time, it was confidential but that is no longer the case.

In August of 2001, the SQL Service Broker team did a design preview and I wrote five presentations totaling 280 slides.  I presented these over a 12 hour period (actually, that time also includes a sixth talk with another 60 slides on "Fiefdoms and Emissaries" but I'll push that out later).

CAUTION:  These slides are from August of 2001 and SQL Service Broker shipped in late 2005.  Many things are changed!  Do NOT count on these slides for accuracy of the product but they can be a big help with inspiration.

The following slide decks are now out in my new public skydrive folder.

SQL Service Broker Overview

An overview and introduction to SQL Service Broker.  This slide deck has the following major sections:

• Introduction:  Motivates the challenges in loosely-coupled messaging, the advantages of deeply integrating messaging with the database, support for scaling and fault tolerance, and some scenarios for SQL Service Broker.
• High-Level Architecture: This covers web services, brokers, message fragmentation, service environments (which provide a namespace for services), message-types, service-contracts, services, and service programs.
• Programming Model:  This section covers an overview of the programming model, service instances and state management, dialogs, and again ties these programmatic concepts to the service instance.  Caution: the syntax has changed between this and the shipped product!
• Relationship to Other Stuff: Here we visit management issues, interoperation, and look at products that create services.
• Conclusion:  We revisit the scenarios and summarize the points made.

The SSB overview talk has 87 slides and can be found here.

SQL Service Broker Messaging

This talk examines the message protocols used in SQL Service Broker.  Reminder:  This is not an accurate depiction of what is in the product!

This slide deck has the following major sections:

• Introduction:  This covers the concepts of reliable messaging, brief discussion of web services, Internet and Intranet issues, transport support, binary and lossless transformations, and a sketch of the protocol stack.
• Adjacent Broker Protocol: The adjacent broker protocol talks to a nearby SQL Server to propagate messages and get them closer to their destination.  This section outlines the challenges at this level with delivery (sometimes tunneled through firewalls), lifetime management, and format negotiation.
• Endpoint Broker Protocol: The endpoint broker protocol functions across the SQL Server engines that actually constitute the two endpoints for the communication (and it layers atop the adjacent broker protocol).  This section covers dialog endpoint state, fragmentation and sequence numbers, piggy-backed acknowledgements, and unsequenced ack messages.
• Message Formats: This covers pieces of the messages, encryption challenges, and ASCII versus binary sections of the SOAP message.
• Monolog Contracts: Monologs were not implemented but they are VERY interesting if you care about Pub-Sub.  This short section on Monologs covers some of the issues related to the messaging formats.  It discusses dialog contracts to implement monologs.
• Web Services and SOAP Bubbles:  Some brief framing of the relationship of the SSB protocols to Web Services and SOAP.
• Conclusion: a summary of the content covered.

The SSB messaging talk has 31 slides and can be found here.

SQL Service Broker Performance

This talk examines performance, scaling, and high-availability issues with SQL Service Broker.   Reminder:  This is not an accurate depiction of what is in the product!

This slide deck has the following major sections:

• Introduction 
• Work Within a Service: This covers messages in and out of a service, service instance state management, data access from services, and scalable and highly-available ASP.NET apps.
• Fault Tolerant Services: In this section, we cover designing services for fault tolerance, deploying services for fault tolerance, and more issues with fault tolerance.
• Scaling and High-Performance: Here, we examine application issues for scale-up, execution of scale-up services, application issues for scale-out, and execution of scale-out services.
• Conclusion

The SSB performance talk has 46 slides and can be found here.

SQL Service Broker Monologs

One of my favorite talks, this describes the unshipped feature of SSB Monologs.  Monologs implement a very flexible form of pub-sub.  While they are an unimplemented feature in SSB, the serious implementer could follow the guidance in this talk and avoid a lot of difficulties.   Reminder:  This is not an accurate depiction of what is in the product (it has not shipped)!

This slide deck has the following major sections:

• Monolog Overview:  This covers "What's a Monolog?", programmatic interfaces, and broker-to-broker contracts.  Included are the notions of "History versus Currency", Monolog-Groups, and Subscriptions.
• Programming with Monologs:  This section includes programming a publisher, requesting a subscription, and monolog delivery in a subscriber.
• Some Special Monolog Features: A really cool concept called a Monolog-Checkpoint is covered as well as the lifetime of monologs.
• Plumbing Monologs Over Dialogs:  We examine fan-out trees for scale, plumbing the fanout using dialogs, publisher agents, and subscriber agents.
• Conclusion:  Here we examine content syndication and content aggregation and then summarize the talk.

The SSB monologs talk has 53 slides and can be found here.

SQL Service Broker Security

Again, one of my favorite talks, this describes some of the special challenges with security that arise in a messaging and services system like SQL Service Broker.  Classic session-oriented security only works when messages are received in order.  That is just one of the challenges.   Reminder:  This is not an accurate depiction of what is in the product!

This slide deck has the following major sections:

• Security Behavior: This section covers identity, dialog authorization, monolog authorization, and auditing.
• Encryption Concepts:  Here we examine symmetric key encryption, public key encryption, and some practical rules for using encryption.
• Encrypting Dialogs:  In this section, we introduce a concept called "handshakes" which helps optimize the performance of certificate management in a dialog-based system.  We look at handshake certificates, messages and dialogs, and cryptographic guarantees.
• Encrypting Monologs: Monologs are N-party communications.  We examine in this section why 2-party security works best and then how we can use a fan-out tree to implement a secure scalable publisher.
• Conclusion

The SSB security talk has 61 slides and can be found here.

I hope these talks are fun as there's still lots of interesting concepts in them 8 years later.

- Pat

Sunday, January 4th through Wednesday, January 7th was the Conference on Innovative Database Research (CIDR)  at Asilomar. 

First, let me talk for a minute about Asilomar. It is located in the forest right off the beach in Pacific Grove, CA (right next to Monterey).  From 1913 to 1956, it was a YWCA and has a number of magnificent and beautiful historic buildings.  In 1956, it was sold to the State of California which has run it as a conference center since then.  Asilomar has a deep and special place in my heart.  I've spent 3 days every two years there since 1985 at the HPTS (High Performance Transaction Systems) workshop.  This gathering has been attended by many wonderful friends in the transaction processing and database community.  Every job I've gotten since the 1980s has been through contacts forged at HPTS.  I chaired the HPTS workshop in 1989 and (believe it or not), my first grandson was born DURING the workshop (after the keynote, before the first session, and immediately after I ordered more coffee in a crisis for the first break).  I still smile at the location of the (now removed) pay phone where I continually called to find out how the birth was progressing.  Nick (my grandson) is now at Cal State San Jose.    It was at Asilomar that I last saw my dear friend, Jim Gray.  Now, I go two times each two years for CIDR (January of the odd years) and HPTS (October of the odd years).  The smells, the feel, the fireplaces, the wonderful Chapel Building (where we meet to present to each other), and the amazing participants offer me a rich combination of memories and current stimulus.

So, this year's CIDR was a good one.  The agenda was packed.  There was a healthy combination of seasoned database folks and young grad student.  We listened to a lot of presentations and had a lot of stimulating conversations.  I saw a bunch of old friends and made a bunch of new ones.  Someone (who... me??) managed to bring some nice sipping whiskey each night and get a gang off to chat quietly by the fire and ensure we ended up sleep deprived but with a better understanding of our new and old friends.  It was a very, very nice event.

On Tuesday evening, I was on a panel with my old friend from Tandem days, Shel Finkelstein.  We were joined by a friend of mine I met last year, Michael Armbrust, who is a graduate student at UC Berkeley studying in the Berkeley RAD Lab.  The panel was on the ways in which cloud-based systems are, in many cases, evolving towards availability over consistency.  I presented a summary of "Building on Quicksand".   Shel did a marvelous job presenting "Principles for Inconsistency".  Finally, Michael presented about his project at Berkeley called "SCADS: Scale-Independent Storage for Social Computing Applications" and then we had a joint Q&A.  You can find the papers for these talks at http://www-db.cs.wisc.edu/cidr/cidr2009/program.html and then navigating the program to Tuesday evening.

The panel was a blast...  I've known Shel for almost 20 years and knew he would do a great job.  He did.  This was Michael's first conference but I was THRILLED with his talk.  He was clear, organized, and thought provoking.  An absolute home run for someone's first conference! 

FInally, I want to thank Mike Franklin, Joe Hellerstein, and Gerhard Weikum (and many others) for all their hard work to make CIDR 2009 possible.

- Pat

Mark Dunn of .NET Rocks asked me to do an interview at TechEd EMEA last week.  We chatted about one of my talks called "If You Have Too Much Data, then Good Enough Is Good Enough".  The link to the interview is here.

Thanks, Mark!  It was fun!

- Pat

Well, those folks at TechEd EMEA continue to be nice to me.  Looks like I get to do four talks for the architecture track AND the closing general session.  It is quite an honor to be told I can make up whatever I feel like and they will let me present it.

Here are the titles and abstracts for the four talks for the architecture track:

Building on Quicksand

Quicksand is a mixture of sand and water that looks like it is solid but will slowly consume anyone that tries to walk over it.  This talk will argue that for many years our systems have built their reliability by layering on top of unreliable components and, effectively, built on top of quicksand.  Furthermore, the granularity of the unreliable components has been steadily increasing through the years and is gradually showing new and challenging semantics to the application developer.

Fault tolerant algorithms are built by creating pieces that are idempotent or repeatable without harm.   Ensuring these idempotent operations can be performed by components that are unreliable, a reliable design must capture the state ACROSS the failure units and keep going if something fails.  This has been done with circuitry, mirrored disks, multiprocessors, and more.  In the past, failures were usually masked from the running application.  Now, the size of the unreliable component is inexorably getting larger as the economics of our systems changes.  This is making it impossible for the underlying hardware and software platforms to hide the failures.  Increasingly, applications must design to cope with the failures of these unreliable components.

This talk will sketch the theory of fault tolerance and show a number of examples of how systems have been designed through the years and how their evolution has gradually impacted application design.  Hopefully, this discussion will help application designers understand the new responsibilities they face in building reliable distributed systems.  It is possible to stay afloat on top of quicksand with sufficient care!

When You Have Too Much Data, “Good Enough” is Good Enough

Classic database management systems have very tight control over their data and over the correctness of the answers they provide.   A number of pressures are being placed on classic database systems:

· We are awash in vast quantities of data
· There are many sources of the data, it frequently has inconsistent schema, and its interpretation is up to… well… interpretation
· Data are streaming into the system(s) at the same time as we are attempting to analyze it
· The machines holding the data may not all be concurrently available

This talk will examine some of these forces on classic database management systems and propose that there are a number of new ways to think about data.  We will explore a number of different dimensions in which the “classic” database world is being relaxed to meet the needs for our customers in a changing world.

RIAs and Emissaries

The emerging field of RIA (Rich Internet Applications) is a hybrid between the classic rich-client application and a browser.  Designed to provide the richness of the client environment without installation, RIA applications must be small and lightweight.  To provide safety, they must have strictly limited access to resources on the client machine.   Sandboxing of the client access limits the RIA app to per-app/per-user/per-machine state.   The state of the art is evolving in how sharing can be done within RIA environments across applications and across users while providing safe computing.

The emissary design pattern is one where there is non-trusted code running outside of a service which is specifically designed to support interactions with the service.  Leveraging reference data published by the target service, the emissary prepares requests with a sophisticated enough understanding of the service’s expectations that the request is likely to be accepted and processed.  All of this is done while respecting the trust boundaries imposed by the target service.   This pattern is clarified in the talk by showing parallels to the usage of paper forms in the business processes that proceeded the advent of computers.

We will explore the use and evolution of the Rich Internet Application and discuss some of the fun challenges the industry faces as RIAs evolve.  Then, we will describe the emissary design pattern and show how RIAs are very well suited to working with their parent web environment using this pattern.

This talk is simply the wild ideas of the presenter and does not reflect any product commitments.  It might, however, help you understand and build your applications differently.

Metropolis: Buildings and Applications

The Metropolis series of talks refers to a collection of presentations which examine cities and their evolution since being connected by the railroads and shows how Information Technology is undergoing the same transformations as IT shops are connected by the Internet.  This second talk in the series examines the parallels between buildings in the city and applications in the IT shop.

Buildings come in many shapes, forms, and uses.  Some are built with major investments and the builders anticipate a long and vibrant lifetime for the building.  Others are built for quick turnover and short lives.  As buildings age, different aspects of the building have different lifetimes and amortizations.  It is common to see the roof, siding, plumbing, or wiring redone a number of times before the building is discarded.  We are also seeing an increasing trend towards buildings being designed for a class or category of occupant without knowing the specific occupant.  You see this in retail malls, office buildings, and more.  As time moves on, a larger percentage of buildings are designed for a generalized use and a smaller percentage customized for the buyer.

Applications are following the same trend (although lagging the construction industry by a number of decades).   We see major investments in some pieces of software and then surprisingly quick-and-dirty implementations of others.  We see renovation of applications across releases with some aspects of the application remaining intact while other aspects may change many times over the lifetime of the code.  Finally, we are gradually seeing the creation of patterns or classes of applications and supporting infrastructure to ease their development.

This talk will cover a number of parallels between buildings and applications.  We intend to show that the same forces that have shaped the building industry are shaping the development of applications.  It is possible to look to the building industry to tell us where applications will be going.

Some (but not many) of you may have seen this talk a few years ago but it is still guaranteed to make you think differently about the composition and evolution of applications.

And, finally, the closing general session at which the poor attendees have no alternative entertainment but listening to me!  The organizing folks offered me this spot and asked me what I wanted to present.  Since I was curious about data centers and felt REALLY uneducated, I had to select that as a topic.  This led me to the following title and abstract for the big presentation.

Green Computing through Sharing
Reducing both Cost AND Carbon

Data centers consumed 1.5% of the total electricity in the US in 2006 and are on track to double as a percentage every five years. It is about 2% of the US total in 2008. Western Europe’s use is increasing at a slightly faster rate (from a slightly lower base percentage). The consumption of electicity within data centers is of significant financial and environmental importance.

Where the heck is all this power going? Why is the electrical load increasing so much? What can be done about it?

This talk will examine both traditional and emerging data center designs. We will start by examining how a data center is laid out, constructed, and managed. We will show two emerging trends: the change to designing data centers for the optimization of power and the emergence of new economies of scale in data centers which is contributing to the drive towards cloud computing. Microsoft is actively moving to compete in the space of cloud computing as we are seeing at the PDC (Professional Developers Conference) a few weeks before TechEd EMEA Developer.

Next, we will examine the sources of waste in the system today and examine why so many of our resources are underutilized. Because we are reluctant to share computing resources, they are left idle much of the time. Why is this currently the dominant choice? What can be done in the design of applications, systems, and data centers to make them more green (both carbon and cash)? What can developers do to make a difference?

Anyway, I am super excited to be going to Barcelona and humbled by the trust and opportunity I've been given!  Hope to see you there!  If so, please come up and say hi!  In the meantime, I'm going to work on my slides...

- Pat

Blogging on the bus again!!  This was written on August 18th and 19th but it's taken a few weeks to get around to editing the photos...  No computer stuff in this blog, just personal fun!

So, on Wednesday, August 13th, Lisa and I flew to Chicago to attend a wedding of a friend from our building in Seattle.  We met John at the building in which we live.  He recently fell in love with a wonderful lady named Helen and when we were invited, we had to say yes!   Neither of us had been to Chicago (I am NOT counting two work trips where I didn't get out of the conference rooms).   We were very excited to go and simply had the BEST time.

The wedding was at the downtown Ritz-Carlton right next door to the John Hancock Tower.   As we were leaving home and walking out the front of the building to get to the Seattle airport, we saw John and Helen loading up to go!   After some hugs, we determined that they we on an Alaska airlines flight landing 30 minutes before our United flight and we would see them at the hotel later in the evening since they were having dinner with Helen's parents who live in Chicago.   We landed, got to the hotel, unpacked, and had dinner in the hotel.  We stayed up late Wednesday night drinking at chatting with the almost newlyweds since we were the first to arrive into town.

Helen is a beautiful Chinese lady originally from Beijing and an only child.  Her father came over to become a university professor (I think in material sciences) and started a successful company... the real American dream!   John is from Philadelphia and is the 11th of 12 children from an Irish family.   He has worked his butt off for years in the financial industry and has held amazing positions at very influential companies having climbed up solely with his wits, hard work, and honor.  It is a joy to have them both as friends.  They waited to later in life to marry since neither of them found their true love until recently.  Just watching them together (and knowing they are our neighbors) is so cool!

Thursday, August 14th -- Walking, checking out the Chicago Spire, an amazing dinner, and partying with new friends!

We were going to be attending the wedding on Saturday but had Thursday and Friday as a mini-vacation to acquaint ourselves with downtown Chicago.   The weather was perfect!  The town was alive and pumping.  We were in the middle of the Magnificent Mile of shopping on Michigan Avenue and we just had a blast.   After sleeping in on Thursday, we had to try Chicago Pizza!   We headed to Gino's East and were one of the first there for lunch (since this was our breakfast after sleeping in).  I wanted to try deep dish pizza.   Lisa wanted a thin crust... So, not wanting either of us to be disappointed, we ordered both!   It takes a LONG time for deep dish pizza...  When they both came, we were astonished that it is DEEP dish pizza!   It seemed like a lasagna more than like a pizza.  I actually liked Lisa's pizza better!   The locals told us later that it is always better the next day after sitting in the refrigerator all night.   I'm not convinced!

Next, since Lisa and I are fascinated by cool buildings, we went to the presentation center for the Chicago Spire (also see).   This condo building will be 2000 feet tall hand hold about 1100 homes in it!  It is a really novel architecture right where the Chicago River meets Lake Michigan.   It will punch up on the Chicago Skyline about 500 feet taller than the Sears' Tower and be the tallest building in North America when it opens in 2012.  The construction is just about at ground level now.   Each of the 1100 (or so) homes has a different floor plan since the tower gets gradually smaller as it rises and the window (and structural) placement gradually rotates (about 2.5 degrees per floor).  It was a BLAST to see the project even though we are currently quite content with our condo home in Seattle that we are planning to move into next year (and the commute to Redmond from downtown Chicago would be harder than from downtown Redmond).  Still, it was fascinating checking it out and we will certainly remember on subsequent visits when it is a world famous landmark!

After leaving the Chicago Spire offices, we walked to Navy Pier which is wonderful public pier extending into Lake Michigan with amusement rides and boat tours and much more.  There were lots of people just having fun in the sun on a beautiful day.  Soon, the roar of the jet engines told us that the Blue Angels were in town!  Now, we had just spent an afternoon watching the Blue Angels less than two weeks ago!  Also, Lisa had bumped into the pilots at the Washington Athletic Club where we exercise.  The WAC is where they stay when in Seattle and Lisa had a wonderful conversation with these fine servicemen.  So, when we saw them flying over us again, it was just a LOT of fun to watch them practice on Thursday.  

 

By 3:30PM, we needed to start our walk back to the hotel as we had 6:00PM dinner reservations at Charlie Trotter's Restaurant and needed to get all dressed up!

So, Lisa had heard about Charlie Trotter's and we were told it was one of the finest restaurants in the world.  Now, you never know how that should be interpreted but we have to say, it was phenomenal!  There was a seven course menu with wine pairing.  A coat and tie is required and it was my first opportunity to try out my new suit (since the old one from 18 months ago was too loose). 

We had the most amazing food and had a wonderful time!   They have two seatings (6PM and 9PM) and by 8:30, we were heading back to the hotel.  Unfortunately, Lisa was getting a headache so she crashed for the night.  I shot a text message to the groom but the recently arrived gang was still at dinner so I walked and explored the beautiful city in the Magnificent Mile area on an absolutely lovely evening.  The place was abuzz with action and it felt SO good to walk and burn some calories.

Soon, I was meeting some of John and Helen's friends and family and we were out partying!  Helen's friend Eva is currently living in Shanghai.  Most of John's friends are from Washington DC but his family (six brothers and five sisters) are from Philadelphia.  I met Mike and Karen (from Austin), Reza (from DC), TJ and Emily (from DC), and John's brother, Tony!   John, Tony, and I closed the place down and had a blast!

Friday, August 15th -- Sushi Lunch, The Bus Tour, the Boat Tour, and Meeting more friends!

So, I slept in until 10AM (Lisa had popped to life at around 8AM but let me sleep).  We started by tracking down sushi for my lovely (sushi-loving) wife.  We had a lovely lunch and the set out to figure out what to do.  We decided to hop onto one of those double-decker busses where you can sit on top, see a LOT, and get a sunburn.  We did all these!

We rode around on the tour bus for two hours in the sun and saw SO many fascinating buildings, parks, the Chicago River, and more.  

           

We decided to bail out near the Chicago River and walk down to Grant Park.   This would only lose us the last mile of the tour loop and that was the Magnificent Mile of shopping and we had already walked that.   As we crossed the river, we saw a tour boat on the river and decided that would be more fun and hiked over to Navy Pier (where we already knew they originated).  So, off we were on an "Architecture Tour" from a boat up the Chicago River.   This time, it was an hour in the sun and again, snapping photos like crazy!   Here's some of the best!

We had promised to get back to the hotel by 5:30 to rendezvous with some of the folks flying in from other places.  That evening was the rehearsal dinner and, while we weren't in the wedding party, we were hanging out with some of them, their spouses, etc, while the rehearsal itself happened.  It was a BLAST.  Soon, we were running off with our new friend, Reza, to grab a steak a couple of blocks away while the wedding party did the dinner together.  We came back to the hotel later and had drinks with everyone until we bagged it at around 1AM.   Again, an absolutely LOVELY day of adventures, partying, and nice people.

Saturday, August 16th -- A lovely lunch and an AMAZING Wedding!

So, again we slept in and rested.  We got ourselves going at around 10:30 and knew we couldn't be too adventurous because we needed to be in formal clothing and ready for the wedding by 3:30.  We decided that the Chicago River was just TOO cool and so we hiked the mile down Michigan to the River and started exploring for a restaurant along the river to sit for lunch.  It took a little while but we found a wonderful place called Flatwater and had a GREAT lunch.  Walking back, we explored the hotel lobby for the new Trump Tower along the river.  Having an OPEN hotel while major construction is going on 90 stories above is really cool!  But there they are open while the building is being built!   We walked back and checked out the Fourth Presbyterian Church of Chicago which was only a block away from our hotel.  It is a beautiful 140 year old church!

   

Here's the building with our hotel, the Ritz-Carlton.

And, right next door is the John Hancock Center.

Soon, it was back to a shower and getting the fancy clothes on.

We showed up at around 3:30 for the 4:00PM wedding and it was gorgeous!  Our new friends were arriving and soon we saw the groom (our neighbor, John) and his six brothers.  There was a string quintet playing while everyone was seated and a trumpet player just waiting for the "Here Comes the Bride" procession... that's ALL the trumpet player did!

 

The wedding itself was short and beautiful!   Soon, we were outside the room for appetizers and cocktails while the wedding party had jillions of photos taken and the room was reset for dinner.  It was wonderful!

We sat at dinner with some of John's family (a sister, nephew, niece, and nephew's girlfriend).   Let me clarify... the nephew was 30, built like a mountain, and back from his second tour in Iraq.  They were fun!  There were also some of John's good friends from some of his jobs in the financial industry.  During the (lovely) meal, there were four musicians playing traditional Chinese music while we ate.  It was amazingly cool.

Then, out came the THIRD group of musicians, a 9 piece orchestra to do the dance music.  Helen has spent years doing ballet and John and Helen did the most impressive "first dance" I have ever seen.  It was elegant, graceful, and charming.  The singer from the musicians said she had seen a LOT of first dances and that was the classiest.  Then dancing began, the wedding cake was served, and more and more dancing!   Lisa had to (temporarily) abandon her beautiful (but painful) high-heels in order to dance!

 

Our new friend, Reza, caught the garter belt!

We managed to dance on and off until about 1AM.  My new suit DEFINITELY needed dry cleaning and we had a wonderful time.   It was just one of the classiest and most fun weddings I've ever been to.

So, back to the room by 1AM... Packing... Too little sleep... Catching a ride to the airport by 9AM... Change planes in Denver and glad to be home!   Happy (and tired) to be at the gym by 6AM Monday morning... Between the cruise and Chicago, I need more time at the gym!

A trip we won't forget!  Chicago is a great city!

-- Pat

Here is a first draft of a new presentation.   I gave it a couple of months ago just after TechEd and thought I would share it as I try to write up some of my thoughts on RIAs.  I plan to rework this a bit more and present it again at TechEd Europe.   The talk is titled: "The Emissary Design Pattern and RIAs (Rich Internet Applications)"

Abstract:

The Emissary design pattern was first described in 1999 in the old "Fiefdoms and Emissaries" talk.  The concept of a "fiefdom" is very similar to what we today call a service in a Service Oriented Architecture.  The fiefdom is a separate trust sphere and transactional boundary.   An emissary is a prescriptive pattern for interacting with a service (or fiefdom) which leverages reference data and a deep understanding of the service to prepare requests for service and maximize the chance those requests will comply with the requirements of the service.   An emissary may be richly interactive and anticipate the validation requirements of the service.

The emerging world of RIAs (Rich Internet Applications) is a fascinating blend of a classic smart client and a browser-based web application.   In a RIA app, client code runs in the browser but still must comply with the browser enforced sand-boxing and not cause harm to the host client machine.  Navigation, naming, linking, and much more are being defined in a fashion drawing from both the web style and the client style.  Many of the design issues with RIAs are under discussion today as this support for these applications is emerging.

This talk examines both the emissary design pattern and the nascent space of Rich Internet Applications.  It motivates how one can look to the workflow patterns contained in our parents' use of paper forms for workflow to understand the possibilities of implementing user-centric workflow as shared replicated data.  The talk concludes with some preliminary concepts of a shared and declarative definition of the "paper form" model and its constraints and how these may someday be used in the automatic generation of emissary-based RIA clients.

More RIA stuff soon!

- Pat