Prakash Sundaresan (孙博凯)

SQL China R&D - Update

2008-12-01T06:59:00Z

Well, what can I say – it’s been a long, interesting and busy year. Certainly, a lot of things have changed since the last time I made a post:

· What started off as a financial crisis in one part of the mortgage market in the US has now become a world-wide economic crisis, impacting virtually everyone in every sphere of economic activity.

· We have had major political developments across the world, not the least of which is the election of a new President-to-be in the US.

· Closer to home in our own field, at PDC 2008, Microsoft unveiled the Azure Services Platform – a major push into the world of Software + Services that is unrivaled in the industry for its breadth and depth. While still in its early days, this marks the beginning of a transformation that promises to impact virtually every product the company sells and indeed the entire software industry in a fundamental way.

In the SQL Server team, we have also had a very busy and productive year. We have continued to make strong progress on our vision of the Complete Data Platform. We released SQL Server 2008 which represents a big step forward in our mission to provide a platform for all data, across the entire life-cycle from birth to archive. We launched the product in events all across the world, including here in China at TechEd 2008 in November in Beijing, where Ted Kummert, VP for SQL Server, launched the product during his keynote address. The response from across the industry – customers, partners, press and analysts from across the world – has been extremely positive and we look forward to strong market momentum in the years ahead. Having been intimately involved in the planning of SQL Server 2008, it’s personally extremely gratifying to see this strong validation of our core vision and value proposition, as well as of the quality of the implementation.

Here in SQL China R&D, we have also had a busy year. Much of it has been heads-down work in doing our part in shipping SQL Server 2008 and other Data Platform components. The team is now in the thick of planning the work for the next releases of SQL Server. In addition to our R&D work, we have also been working hard to connect with our local China customers and partners, through a variety of channels: via our SQL Server Chinese Language Forum; via events such as TechEd where the team presented several talks and hosted the first-ever Ask the Experts session on SQL Server in China; by engaging with MVPs, customers, partners etc. You can read more about the work of the team in the words of team members themselves at our SQL CRD Team Blog (in Chinese).

Building a world-class development team for a complex systems-level product such as SQL Server is a challenging process that takes time and patience. However, I am extremely pleased with the progress we are making towards this goal as we execute on a number of fronts in a systematic and methodical manner. I look forward to sharing more of our experiences and learning with you as we make progress in this journey.

Until next time – 再见!

孙博凯

Product Development Disciplines at Microsoft

2008-01-27T11:01:00Z

Over the last several months in my role here in China, I have given talks at several leading universities and met with many of the leading faculty and students working on technologies related to the Data Platform. I’ve also spoken at several industry conferences, meeting with customers, partners, analysts and other industry folks. There are many topics that come up at these meetings – changing technology trends, distributed development, the tremendous growth of Asia etc. But one topic that seems to come up more than almost any other is the question of how we organize and conduct our product development in Microsoft. I suppose this is only natural – Microsoft is one of the most successful software companies in the world, and the software industry here in this region is poised for tremendous growth, so it makes sense that people in the industry are eager to learn from the our experience over the last quarter century.

This is actually a very big topic and within Microsoft we have an Engineering Excellence group that actually runs courses that can span several days and provide an overview of Microsoft’s software development methodology, our engineering system, organizational structures, best practices, tools and technologies we use internally ensure quality, reliability, security etc and a variety of related topic. By no means would we claim that we have all this figured out perfectly and have a perfect system, but there is indeed a lot of accumulated knowledge and experience that we can share. And we do actually share this information, in appropriate form, with others in our industry, worldwide and also in this region.

As this is indeed a large topic, I don’t want to get too deep into this here, but I do want to address one aspect of our engineering system – the core disciplines that we organize our R&D teams around and the particular roles that each of these disciplines plays. I want to discuss this because I believe Microsoft does this a little bit differently from the rest of the industry even in the US, and especially here in China there is not a good understanding of these core disciplines and what role each of them plays.

Traditionally, the Microsoft engineering system has consisted of 3 “core” disciplines: “Development”, “Test”, and “Program Management”, also known as Dev/Test/PM for short. I’m going to touch on each of these briefly here, but I like to introduce them in a different order:

PM: When we think of engineering disciplines, most people start with “Dev”. For me however, things really start with the Program Management discipline. At Microsoft, “PM” means many different things, but for me the core essence of the PM role is two things:

1. The first part of the PM’s job is to understand the customer’s requirements and translate that into a functional specification of what we should build. This is where it all begins. If we don’t understand the customer, it is not very likely that we’ll end up building the right thing.

2. The second part of the PM’s job is to work with Dev and Test to translate the initial specification into a living, breathing product.

I find that many people, especially here in China, think “Project Management” when they hear PM. Indeed, Project Management is part of a PM’s job (under #2 above), but it is only a part of the PM’s job. The real skill that a PM brings is the expertise to listen to customers, understand the world from their point of view, and then to design a solution for their problem. This does not just mean giving customers what they ask for literally, but to truly understand them and design a solution that solves their problems even if the customers could never imagine the solution – as the famous saying goes, if we had only listened to customers, we would have looked for a faster horse, not come up with the automobile.

Dev: Of all the engineering disciplines, this one is probably the one people think about the most commonly. Dev is short-hand for “Development”, the folks who responsibility it is to actually design and build the software that we ship. The essential job of Dev is to take the functional specification produced by PM and translate that into an actual implementation. In the world of mission-critical system-level software, this implementation better be extremely reliable, secure, manageable, scalable and high-performance. And the designs and implementations Dev produces better stand the test of time and last for several versions and years to come.

Test: The test discipline in Microsoft is much misunderstood, certainly externally, but sometimes internally as well. When I first came to Microsoft many years ago, I was (pleasantly) surprised to find that Microsoft had almost as many, if not more, testers as developers. Coming from a company that had a much less developed testing discipline (and where as a result, quality assurance was considerably weak), it took a little while to get used to what the essence of the Test discipline really is. The reality is that, in Microsoft, how fast we can ship software depends on not how quickly we can design and implement it but rather on how quickly we can test it. This is because every piece of software we ship, especially on the systems-software side, has to pass an extremely high quality bar. The Test discipline is really an complex area, and one where have learnt a lot over the years in terms of different types of testing that we employ – unit tests, functional test, integration tests, stress and long-haul tests, performance tests, security tests, localization tests, etc. etc. The set of tools and techniques we employ in test is truly some of the most impressive and complex – automated test harnesses, automated test generators, automated test failure analyzers, automated security “fuzzers”, fail-point and state-machine based testing.

The three “core” engineering disciplines described above are like the 3 legs of a chair – you need all three of them, and in a balance, to have a proper engineering organization. No one leg can dominate the other – otherwise, you get an organization that may not be in touch with customers needs or one that does not pay enough attention to quality. Indeed, the three disciplines are a little bit like the branches of government – they form a system of checks and balances that ensures we understand what customers want, we design and build that with high quality, and we ensure that we deliver a product that meets customer expectations in every regard.

It is also important to emphasize that we aim to attract the best talent to all three core disciplines – the bar is equally high for all the disciplines, it just happens to be that the passion and skill-set for each is a little different. PMs usually have a passion for working with customers, conceptualizing what the product should do, and then working with their Dev and Test peers to coordinate all the work to make sure we deliver exactly that. Developers have a passion for building top-quality software – software that is innovative, simple, reliable, secure, scalable, high-performance and stands the test of time. And Testers are passionate about finding all kinds of ways to break software and making sure making sure we find all the issues and bugs before we ship it to customers.

When we interview candidates, a very important part of what we do is find out which discipline the person’s talent and passion really lie in and directs them accordingly. Of course, over the course of one’s career, one’s passion and talent may change, and the person may change disciplines as a result – I myself started in the Dev discipline before switching to PM. This is only natural and we actually encourage that as a way to build better teams.

Other disciplines

It is also important to point out that although the three disciplines mentioned above are what have traditionally been considered the “core” disciplines at Microsoft, there are several other disciplines that are also becoming increasingly important. For example, User Experience (UX) professionals are essential to ensuring that products are intuitive and natural for users to use. A great user experience can make the difference a product that customers love versus one they merely tolerate. UX is certainly very important for products aimed at end consumers, but it is also important for all our audiences – Developers, IT Professionals, Information Workers.

As we move into the Software+Services era, a variety of disciplines related to architecting, building and running extremely large-scale infrastructure becomes increasingly important. Again, while this has been true for some time for our consumer facing web properties such as MSN and Live, it is now becoming increasingly important for all our product groups as more and more of them take steps to evolve their products along the Software+Services model.

Many candidates I talk to often want to discuss what role at Microsoft would be the best fit for them and how they can grow their careers. The best advice I can think of is to work on a technology and a role that they are really passionate about. As I mentioned above, we value all the disciplines equally and a well-balanced organization needs great people in all the different roles. While different disciplines appeal to people with different passions and skill-sets, all the disciplines offer opportunities for innovation and great work. And all of them offer opportunities for advancement and leadership. Indeed if you look across the senior levels of Microsoft, there are leaders who emerged from various disciplines – what they shared was a passion for what the work they were doing.

I hope this discussion of the different engineering disciplines at Microsoft and the approach we take to them shall be useful for the many people who seem to be interested in this topic. If you have any questions or comments, feel free to post a reply to his entry.

Until next time – 再见!

Happy New Year!

2008-01-06T16:52:00Z

It’s been quite a while since I made my last posting – things have been a bit hectic as we move ahead on various fronts on our R&D efforts here in China. A New Year is always a good time to take stock of achievements past as well as gear up for the challenges that lie in the year ahead. And of course to make some resolutions! So let me do all of those briefly here.

As I look back on my last few months in China, it is amazing to think of all the things we have accomplished. It seems like only yesterday that I moved here, family and all (it was actually Aug 1^st). In the few short months since, we have made a lot of progress in establishing the presence of the SQL Server China R&D team. We have clearly articulated who we are (our identity), what we seek to accomplish (our mission), why we are here in China, and our path forward – how we intend to achieve that mission. We have connected with a huge number of people – customers, partners, students, faculty, people from our industry – all our different constituencies who take an interest in our being here. We have made a lot of progress in building our R&D team, hiring some of the very best talent in the Asia-Pacific region interested in working in the Data Platform space. We have continued to execute successfully on our product delivery commitments on a number of different technology fronts. At the same time, we have been growing our knowledge and skill-set, individually and as a team, ramping up to take on even greater challenges in the future. And finally, we’ve had some great fun doing all this!

The path ahead is challenging, but exciting at the same time. This coming year shall see the launch of the next version of SQL Server, SQL Server 2008. The SQL Server team, and the China team as part of that larger team, has some hard work ahead of us as we push to get this product out the door. It is important to keep in mind that what awaits us is the reward of shipping a terrific product, with features customers cannot wait to get their hands on, with the high quality that they have a right to expect. Initial customer reaction to the CTPs we have been shipping the last few months has been very positive indeed and we cannot wait to see the impact this release shall have on the market. And of course, once we ship, we get to turn around and do it all over again – indeed, initial planning efforts for the subsequent (as yet un-named) release of SQL Server are already under-way. And what is especially exciting for us in the China team is that as we build up our capability, we get to play an even greater part in that next release, and in other future releases. Some parts of our China team are currently working on our deliverables into the next version of Windows, and they are already getting a taste of what it means to own a product or component end-end and deliver a new version to hundreds of millions of customers world-wide. It is not an easy task, especially in the case of a platform product like ours – but then that is what makes it exciting and challenging.

So, looking forward, I see an exciting and important year ahead of us as we ship SQL Server 2008, continue to build up our China team, ramp up our skills, and get ready to play a much bigger role in building the next release of our product. As I look at the quality of the team we are building, the breadth of technology we are working on, and the strength of our vision and our long term commitment to this team, I have no doubt that we are moving ever closer to our goal of building the premier data platform R&D lab in Asia-Pacific.

Every new year should come with some resolutions and this year is no exception J. While I, like most people, have some resolutions on the personal side that shall remain private, one resolution seems appropriate to share here – to be more regular with this blog!

Here’s to a Happy and Healthy New Year – 新年快乐!

SQL Server Culture

2007-10-10T05:13:00Z

In my last post on the SQL Server China R&D team, I brought up culture as one of the most important leadership challenges in building up a team. How do we ensure that a team grows up with the “right” culture? And what exactly is this “right” culture anyway? In this post, I want to delve into these questions in a bit more detail.

Let’s start with the definition of the word first – Merriam-Webster defines culture as

Main Entry: ¹cul·ture Pronunciation: 'k&l-ch&r
Function: noun
Etymology: Middle English, cultivated land, cultivation, from Anglo-French, from Latin cultura, from cultus, past participle

the integrated pattern of human knowledge, belief, and behavior that depends upon the capacity for learning and transmitting knowledge to succeeding generations b : the customary beliefs, social forms, and material traits of a racial, religious, or social group; c : the set of shared attitudes, values, goals, and practices that characterizes an institution or organization d : the set of values, conventions, or social practices associated with a particular field, activity, or societal characteristic

So, culture is about shared attitudes, values, goals and practices. Every organization has a culture, whether one that has been fostered intentionally, has evolved in a grass-roots manner or some combination of the two. Sometimes, organizations are very explicit about verbalizing the kinds of values and cultural attributes they would like to foster. But it should be no surprise that most people cannot understand, or even remember, what the “mandated” culture of an organization is supposed to be. Each of us grew up with some cultural background, but probably not many of us would say that we acquired our cultural leanings by reading a set of values in a document. Most of us imbibe culture through our daily lives, through everyday experiences, actions, reactions, positive and negative rewards and reinforcements, from a variety of quarters – our elders, superiors and peers. Over time, these values and patterns of behaviors become ingrained in our very being and we then become part of the “established culture”, doing our part in passing that culture on to others in the organization.

Research shows that culture, once ingrained, can be very difficult to change or “unlearn”, for individuals and even more so for groups or organizations. At the same time, over the long term, culture has been shown to be a much more reliable and important factor that differentiates successful organizations from average or below-average ones, much more so than business strategy, a particular technological or process advantage, or even particular leadership howsoever charismatic or visionary it might be. That is why it is so important, IMHO, to start off a team with the “right” culture, howsoever each organization might define right.

For a team such as the SQL Server R&D team in China, clearly a lot of the “right” culture for us follows from the culture of our “parent” team in Redmond. Of course, we will interpret and adapt and bring our own local flavor to this culture here in China, but many of the essential characteristics that define who we are and how we behave we will inherit from the larger SQL Server R&D team. So what are some of the elements of this “SQL Server culture”? It is hard to give a definition or boil down to a few bullet points to things that have been accumulated over a period of many years, but let me try to list here a few things that I believe are core to the SQL team:

Systems Culture: We are a systems team. We build mission-critical platform software which is used by tens of thousands of organizations to build and run business-critical applications that are in turn used by hundreds of millions of people world-wide every day, 24 hours a day, 365 days a year. We build software that has a lifetime of years, indeed decades. Any mistake or weaknesses, whether in design or implementation or in our processes or methodology, can and will show through. The bar is high! To be successful in this kind of an environment, we need a mind-set of being professional grade engineers (see below). Amateurs need not apply!

IC Culture: Building mission-critical systems software is a craft that takes years to learn. We often tell engineers when they’re hired, especially those coming straight out of universities, that they won’t even be fully productive until 3 to 4 years into their job! While this might sound hyperbolic, it is actually true. By the time an engineer learns what customers want, how to translate that into a product or feature, what is good design versus bad design (versus great design), what tradeoffs to make, how to write robust, secure, reliable, scalable high-performance code that is maintainable and supportable in the field, and to do all this in a productive way working in a team environment, it is going to be several years. This is the process of transforming a smart fresh graduate into a professional grade engineer and there are no short-cuts.

So what is IC culture? IC is Microsoft-speak for Individual Contributor – as opposed to lead or manager. When a business depends on deep technical skill and knowledge like ours does, we have to value the individual contributor. We cannot build a product like SQL Server if every smart engineer wants to stop writing code 4 years into their career and start only managing people. I often cite this statistic – Microsoft has dozens, perhaps hundreds, of VPs world-wide, but only 14 Technical Fellows (and we are fortunate to have 2 of those in SQL Server). The take-way is not that a VP is not an important post, but that at Microsoft, and definitely in SQL Server, we very highly value people who want to remain technical through their career. In a way, Bill Gates himself is the ultimate IC – sure he has a staff, but his main role as Chief Software Architect (CSA) of the company is to help chart the technical direction for the company. There can be no greater testament to the value placed on ICs and deep technical knowledge in our environment.

In fact, I believe this point is so fundamental, I’d like to step back here for a moment from SQL Server or Microsoft or any particular company. In my travels around the region, many people express a belief/hope that the Asia-Pacific region as a whole and in particular countries such as China and India shall increasingly take on the mantle of technological leadership in the world economy. Obviously there is a wealth of talent in this region that provides the necessary conditions for this to happen. However, there is also no doubt in my mind that if this region is really to get to that next step where long term product and industry innovations are being driven from labs and companies here, we have to create a culture that values deep technical achievement. In today’s environment, when I talk to college students about their career aspirations, 8 out of 10 want to be managers within a few years of graduating. We simply cannot expect to build the next generation of technological leadership on such a foundation. I see valuing deep technical knowledge and the IC culture as an imperative for the entire eco-system in this Asia-Pacific region, not just for a particular company or group.

Innovation culture: The database industry is a mature one: 30+ years, 20+ billion USD in annual revenues world-wide. It’d be easy to think that innovation is not a driving force in this industry at this stage. You’d be dead wrong! In my earlier post on the transformation from Database to Complete Data Platform, I described the unprecedented breadth and depth of challenges facing our field. To tackle these challenges successfully, innovation needs to be a core value in our genes, or else we’ll be extinct soon enough. Innovation can be big or small, technological or process oriented, but innovation needs to be a daily value that every employee lives and breathes. SQL Server has had a history of innovating in ways that fundamentally changed even this mature industry – pioneering ease of use and auto-management, integrated BI capabilities in the core platform, dramatically simplified developer experience. Today, we continue that tradition with many break-through technologies in the next release of SQL Server such as the Entity Data Model. We will absolutely need to continue to this focus on innovation going forward.

Customer-focus: In today’s super-charged environment, it is easy to focus on the competition and forget about the customer. But it is vital to understand that the way to beat the competition is by serving your customer, not the other way around. Customer-focus is not the job of one particular department or role. Sure there will be customer-facing departments like our Customer Support Services (CSS) and various other field organizations. But customer-focus is vital to job of every single individual throughout the organization. Without knowing what the customer ultimately wants, you cannot do your job properly – regardless of whether you are a developer, tester, program manager, architect or even an admin! There is no easier way to become obsolete than to lose touch with what the customer wants. On our team, it is mandatory for every employee to spend some portion of their time with customers, whether it be through participation in newsgroups and forums, attending customer conferences, or a variety of other ways.

“Do the right thing”: I could go on about other values we cherish within SQL Server and Microsoft, such as respect for diversity, being open and honest yet respectful, taking on big challenges etc. While these are all great values and ones we do indeed cherish, if you talk about everything you talk about nothing. So I would like to conclude by focusing on a summary value that I call “do the right thing”. I admit it may sound a bit sappy, but it is true that this is something we strive hard to live by in SQL Server. Whether it is with regard to a customer issue, a product issue or an internal issue, this is a motto that guides us in our day-day work. I’ll give you a couple of examples. I’m sure every SQL Server customer remembers the Slammer worm to this day. This one event completely changed our approach to software development within SQL Server. When the worm hit, we had actually just gone through a security push and released SQL Server SP3 (which actually included a fix for the exploit used by this worm). However, once this problem hit, it was no longer a question of whether we had issued a fix or not. We had to help our customers get their systems back up and running as quickly as possible in a safe way. The SQL Server team mounted a Herculean effort in the short term to produce the tools and guidance needed for customers to recover their systems. But it is the long term changes we made – in our development processes, in our disaster-preparedness, in our entire approach to security – that stand out in my mind. The results of those efforts have proven themselves over the last several years as SQL Server has proved itself to be one of the most secure products on the market. Does this mean we will never have a vulnerability or an exploit again? Clearly, the answer is no. But what we can say is that we’re working hard every day to make that as unlikely an event as possible, and we’re also constantly prepared to be ready to respond if such an event does occur again.

Another example I’ll cite is the case of the Database Mirroring (DBM) feature in SQL Server 2005. As one of the most important and popular features in the release, clearly there was a pressure on us to ship this feature. However, as we got close to the end-game of shipping SQL Server 2005, it became clear to us that we would not have the time to run this feature in production (both internally within Microsoft and at selected customer sites) for the duration we would need to have a crucial high-availability feature like DBM be certified for production use. So we had a difficult set of choices – delay the release, ship DBM anyway, or pull the feature from the release. None of them were easy choices – if they were, we would not be talking about them. But in the end, we decided the right thing to do was to ship SQL Server 2005 but not certify DBM for production use, complete the production-test runs in due course, and then certify DBM for production use with SP1. Sure, there was pain associated with this decision in the short term, but looking back customers now tell us they appreciated the extra care and effort that we put in to make sure the feature was completely ready before we let them use it in production.

Hopefully these couple of examples give you an idea of what we mean by “do the right thing.” There are so many more examples that come to mind but this post is long enough already. I hope you also got a sense for what we mean when we say “SQL Server culture.” It is going to be an interesting challenge to make sure our SQL Server China R&D team grows up with the important elements of this culture ingrained in us as we grow the team over the next few years. But that is a core part of the commitment we made when we decided to create this team.

Until next time – cheers and zai-jian!

Prakash – 孙博凯

SQL Server China R&D – Why, What etc.

2007-09-17T17:22:00Z

These couple of weeks I’ve been on the road again, speaking at TechEds as well at universities across the Asia-Pacific region. Last week I was at TechEd South-East Asia and also spoke at a faculty summit in Kuala Lumpur as well as at the National University of Singapore (NUS). This week, I’m in Taipei for TechEd Taiwan and related activities. At most events, people have been somewhat surprised (most of them pleasantly) to know that a core mission-critical product such as SQL Server is setting up an R&D Center in China and are curious to know the background of why we decided to build the team and what kind of work we’ll be doing there. I thought it would be a good idea to explain some of this here.

First, some history J. It was in the early 90’s that Microsoft got serious about the database business and started assembling a top-notch team in its SQL Server division. There were names like Hal Berenson, Peter Spiro and Dave Campbell (my current boss) who joined from DEC, Bill Baker from IRI, James Hamilton from IBM, Anil Nori from Oracle, Pedro Celis from Tandem and several other leading minds from virtually every corner of the database industry. And of course on the research side, there were names like Jim Gray, Dave Lomet, and Phil Bernstein among others. It was this set of people that took ownership of SQL Server and transformed it from being a bit-player in the database market to the leadership position that it now occupies today.

Throughout this time, until about 1995, all SQL Server development was done entirely in Redmond, Washington. We did this because, essentially, we could. Microsoft could hire the talent it needed – whether from academia or industry, mostly from the US but also from other locations – and bring them to Redmond. And if you can do this, there are certainly some advantages to having everyone co-located in one place: if you have a problem with a piece of code, you walk down the hall to talk to the person who wrote it; if you need to call a meeting, you grab the people you need and walk over to the coffee shop to hash out the matter. So it was a great model, while it lasted.

In the last few years, however, a couple of important things began to change. First was the geographical distribution of where talent resided, now and especially in the future. It has been true for some years now that countries outside the US, especially in Asia, have produced a lot more engineers than the US itself, and that gap has been widening every year. Increasingly however, rising standards of living in Asia coupled with increased difficulty in obtaining H1-B visas and Green Cards for foreign nationals in the US have resulted in many more of these engineers staying in their home countries instead of moving to the US as had previously been the norm. It was clear that in the future we were increasingly looking at a world where world-class talent was more dispersed around the world, especially in countries such as China and India.

The second factor that changed was the increasingly geographically diverse nature of our partner and customer-base. As SQL Server became more and more successful, we started seeing increasing numbers of partners and customers, including some very large ones, in regions outside the US. It is hard for a development team located in a single place, no matter how smart, to be able to understand the needs and desires of this global partner customer-base. Thus, it was no longer clear that it was the best policy to locate the entire team in one place, even if we could do so.

It is in this context that the SQL Server leadership team looked at the situation in 2005 and made the decision to expand our R&D footprint beyond Redmond by creating two additional strategic R&D sites, one in India and another in China. While this might seem like an easy decision to make, the hard part is actually in the execution. How do you take a complex product such as SQL Server and build R&D teams for it in these new locations? Yes, it is true there is a lot of talent in India and China, but it is also true that a lot of it is what I call “raw talent” – very bright, but lacking deep experience in developing commercial-grade products, especially systems-level platform products. It takes a lot of effort and energy and experiences to transform an individual from being a “smart coder” to being a “professional engineer” with all the implications that term has for me – someone who is at the top of their game with respect to translating ideas into mission-critical industrial-strength code in the most efficient and effective manner possible. It also takes a lot of time, effort, energy, experiences and most importantly, commitment to transform a set of very talented individuals into a world-class engineering team. It is not an easy challenge; but then again, if it were easy it would not be much fun, would it?

But that is precisely what we have set out to do – build a world-class engineering team that works on some of the broad and exciting challenges in the Data Platform world that I described in my previous post. So we investing in several areas in the China R&D team – in our core Data Access technologies, in XML, in BI, in Tools, and in some very new and exciting areas that we are not ready to share externally just yet. I’ll discuss more details about some of these areas in future posts, but clearly we’ll be creating products and technologies for the world market out of our China team.

But we are going beyond that. Part of the reason for being in China, as I mentioned before, is to be able to connect more deeply with our increasing base of customers and partners in this region. So the China team will also focus on creating products and technologies that address the specific needs of the local market, in China specifically, but more generally for the entire Asia-Pacific region. In the long run, this is actually very exciting both for the team and for our business in this region.

Make no mistake, this is going to be a long and sometimes difficult journey. There are many challenges that the team has to tackle and overcome - technical challenges, process challenges, communication challenges and so on. But perhaps the most important and interesting leadership challenge is that of culture – how do we make sure that we build a team that has the “right” culture? And what exactly is that culture? Is it the same as the culture the team in Redmond has? Or does it have to be customized and adapted in some ways to the location and context of this team in China? These are very interesting questions and ones I believe are going to fundamental to the success or otherwise of our team and others like it. I’ll talk about some of these issues in a future post. For now, let me end by saying that we are very clear in this goal – we will build the Premier Data Platform R&D team in the Asia-Pacific region, and we are really excited to be on this journey.

Until next time – cheers and zai-jian!

Prakash – 孙博凯

From Database to Complete Data Platform

2007-09-03T07:59:00Z

This past week, I was in Hong Kong in connection with Microsoft TechEd Hong Kong. Ron Jacobs and I gave the closing keynote where we highlighted the new wave of innovation in Microsoft’s Application Platform. I covered the Data Platform part of course, and I had the pleasure of giving the audience a quick glimpse into the many innovations coming to market in SQL Server 2008 (which is scheduled to be released next calendar year). We’ll talk more about SQL Server 2008 in a future posting, but if you are interested in learning more and playing with the bits right now, here’s the place to start.

In addition to speaking at TechEd, we also met with members of the local HK media and briefed them on SQL Server’s future direction as a product. And we met with some of our valued customers in the region – learning about the latest projects customers are undertaking on the Microsoft platform and the types of issues they need help with is always an educational experience.

But perhaps the most interesting aspect of the trip for me this time were the talks I gave at the University of Hong Kong (HKU) and also at the Hong Kong (HKUST) of Science and Technology. The sessions were entitled From Database to Complete Data Platform and the goal was to connect with students and faculty doing research in areas that we now broadly refer to as the “Data Platform” and give them an industry perspective of what we believe is a historic transformation of the field from its traditional roots in “Databases”.

The field of modern databases has been around for over 40 years. From the early work on Hierarchical and Network models, through Codd’s ground-breaking work on the Relational model, through the many innovations in the area of transactions, isolation levels, access methods, declarative query languages and query processing, cursors, APIs, and so on, the field that has provided the basis for building the robust enterprise-scale mission-critical applications that drive much of today’s “Information Economy”. As such, when an average student in a university thinks about the field of “Databases” they may picture a mature, somewhat musty, field in which all the exciting problems were solved many years ago and all that remains is incremental advancements that wring the last bit of life out of a dying field.

How wrong they would be.

Over the last few years, a powerful confluence of trends – technology trends, consumer and business trends, application trends – has resulted in the broadening and redefinition of Database field like never before, with more exciting problems to solve than at any previous time in the long history of the field. Let’s examine these trends briefly:

Technology Trends: Everyone is familiar with Moore’s law – processing capacity doubling every 18 months, first manifested as increasing MHz and now as multiple cores. The trends in hard disk storage capacity (and price) have actually been even more amazing. Just to give an example, hard disk prices per GB have dropped from about USD 40,000/GB in 1980 to about USD 0.5/GB today!!! Memory and flash memory prices are on an even steeper trajectory down, with capacity going up exponentially to match. At the same time, there is a tremendous proliferation of devices – cell-phones, PDAs, gaming devices, GPS devices and so on – all of which generate, store, process and send/receive/synchronize data at tremendous rates. And of course, the ubiquitous Internet has not only made possible new types of applications but also changed expectations around the characteristics of existing applications – more on this in a minute.

Consumer and Business Trends: Enabled by the technology trends mentioned above, there have been huge changes in how consumers and businesses relate to data and information. First, there is the sheer explosion of data – the amount of new data being generated, much of it born electronically, has been growing exponentially (ever notice how your hard-disk, no matter how large, is always significantly full?). Email, documents, digital photos, music, video, streaming data from sensor, satellite imagery, all are part of this great data explosion. But it is not just about storing all this data – consumers and businesses want to derive value from all this data – being able to search, share, synchronize, analyze, visualize and manipulate the data so it becomes useful information – the notion of “Your Data, Any Time, Any Place”. And all this while ensuring the data is secure, privacy is protected, and all regulatory requirements –both external and internal – are met.

Application Trends: First, there was “batch processing” – basically an automation of human processes that had previously existed for centuries. Then came “OLTP” – Online Transaction Processing that in many cases changed the way business was done – what used to take many days could now be done instantaneously. The OLTP architecture and underlying platform technology went through several iterations, but the core concept remained the same. As companies started to accumulate more and more data from these OLTP systems, they discovered an opportunity to gain a vital competitive advantage by analyzing this data and understanding their customers better – thus was born the Business Intelligence, including Data Warehousing, Online Analytical Processing (OLAP), Reporting, Data Mining and so on. But today we live in a Web 2.0 world – applications surface through a variety of end-points (rich client, browser, device, laptop/desktop, …), they seamlessly bring together data from a variety of data sources, and they provide a variety of rich services including query, search, analysis (increasingly real-time analysis), reporting, visualization, and so on. And they do all this while operating at unprecedented levels of scale, reliability and security.

A Complete Data Platform

The changing trends described above are driving a fundamental transformation of our field – from just Databases to what we now call a Complete Data Platform. This Data Platform builds upon and extends the concept of a Database in three different dimensions:

All Data: It is no longer sufficient for the Data Platform to be able to store and manipulate words and numbers as databases have done for a long time now. A Complete Data Platform must be able to work with all kinds of data – including text, XML, Objects, documents, files, streaming data from sensor networks, and any kind of user-defined data. And it must be able to provide the appropriate services for each type of data – storage, indexing, query etc.

All Tiers: Gone are the days when a Database ran only on a “server”. Today, a Complete Data Platform must provide data services across the entire spectrum of hardware tiers – from phones and mobile devices, to laptops, desktops, servers, server farms and finally cloud-scale mega-service infrastructure. And it must do so with seamless interoperability for data and applications across these tiers.

All Services: The range of services on data is no longer restricted to store, query, backup, restore and a few other verbs. A Complete Data Platform must provide a broad range of services including those and others – search, cache, synchronize, analyze, mine, integrate, report, visualize, secure, audit, archive, … In short, it must service the entire data life-cycle, from birth to archival.

We can drill into a lot more details (and we likely will in future posts) but that in a nutshell is the scope of what we call a Complete Data Platform – one that handles all data, on all tiers, and provides all relevant services on that data. And does so while maintaining consistency in some vital dimensions – for example in the data model, security model, management model, data access APIs, development tools, etc. And of course it goes without saying it does all this with high performance, rapid time to solution, and low TCO. Quite simple – isn’t it? J

Our Opportunity

I hope the discussion above gives you a sense about the unprecedented opportunity for innovation in the Data Platform space. At no other time in the history of the field of modern Databases has there been such a wide array of technical challenges, such a broad canvas on which to paint. If you are a student in university – as were some of the bright minds I met in Hong Kong this last week – this is a time of unprecedented opportunity for you. The spectrum of problems to pick from is so vast and varied. And not just for “Database” majors - there are interesting problems in this space for those with an interest in virtually any aspect of computer science – computer architecture, networking, programming languages, data mining, XML, search, visualization, web-scale computing, semantic web – the list is quite long. The database field has always been one where people easily spend a long time – it is not unusual to find people who have built multi-decade careers, indeed spent their entire professional life, in this space. The opportunity to do so, if you choose to, is even more compelling today. After all, we live in the information age – this is our time.

Until next time - cheers,

Prakash

Introduction

2007-08-27T13:16:00Z

Hi,

My name is Prakash Sundaresan (and I have a Chinese name as well - 孙博凯) and I lead the SQL Server R&D team in China. I just moved to China (for the 2nd time) a few weeks ago. As I begin to get to work here on building the team, ramping up our R&D efforts, working with customers and partners in the region etc., I find that there is a great deal of interest, from a variety of quarters, in what SQL Server R&D will be doing in China – hence this blog. Here, I plan to discuss not only the work we are doing in the SQL Server China R&D team, but also broader topics – the ongoing transformation of our field from Database to a modern comprehensive “Data Platform”; interesting applications customers and partners in this region are building using our technology; issues that may be of interest to students and others looking to build careers in this technology area; and anything else people want to discuss about working in China, global development etc.

Before I dive into any of these topics, let me first introduce myself a little bit. As you may guess from the name (how I got my Chinese name is a different story for another time J), I was born and raised in India. After getting my Bachelor’s degree, I went to the US, as many others do, for graduate studies. I spent a couple of years (and got a Masters’ degree) at the University of Wisconsin, Madison. At this time, Madison had a really strong program in Databases with pioneering faculty such as Dave Dewitt, Mike Carey and several others. More importantly, if you were a poor graduate student looking for a research assistantship, this was the crowd with all the $$$! Thus began my first professional fascination – with the database and information technology field – one that has lasted to this day.

After graduating, I spent a year at Digital Equipment Corporation (fondly known as DEC – for those who still rememberJ) working in the Advanced Development lab in San Francisco with the esteemed Dr. Jim Gray, Tom Barclay and others. This was my first exposure to industry, and I learnt so much in that year, it was an absolutely amazing experience. I will save more detailed thoughts for another time, but suffice it say that to this day, I owe a large part of my professional development to that first year and to Jim’s tutelage, as I’m sure numerous others do. All of us who have known him continue to hope for Jim’s safe return.

Part-way through that year, DEC sold their Rdb product line to Oracle, so there was no longer going to be a database group at DEC going forward. So, after my year at DEC was up, I went to work at Informix, which at that time was starting a team in Portland Oregon to work on XPS, their massively parallel database. I had a great time up there, helping to build a state-of-the-art parallel database and bringing it to market. We had some spectacular benchmark numbers and some great customer wins, achievements those of us involved cherish to this day. During this period however, Informix went through some challenging times, with a troubled acquisition of Illustra, an ill-advised public battle of billboards and lawsuits with Oracle, and some accounting missteps (to put it mildly). I learnt some valuable lessons in my time at Informix, not only in the art of software engineering but in terms of what is really important in life. Eventually, I decided to leave Informix to go work for Microsoft, which at the time was trying to become a serious player in databases with a little-known product called SQL Server for Windows.

It’s been an interesting ride ever since. I have been with Microsoft for close to 10 years now, all with the SQL Server R&D team – I suppose you call me a certified “database-head” by now! Over the years, I have played various roles in this team. In my earliest days, I was part of the core SQL Server Engine development team, working in the Query Processor and helping to ship SQL Server 7.0. It was the “good old days” as they say – working on pure code, getting SQL Server to scale on the multiple-processor machines that were becoming commodity at the time. Subsequently, I played a variety of roles, from being a Performance Lead for the RDBMS, to managing the Query Execution team, to leading the team working on core Engine improvements for the WinFS project (I’m sure this topic will come up again – it always seems to J).

Even as I took on these roles of increasing responsibility, there was a growing discomfort inside me. I was being asked to provide leadership and make decisions on the future direction of the product, but I did not feel like I had a good understanding of the needs of the customers for whom ultimately we were building the product. Why did a customer (or partner) choose SQL Server over the competition, or vice-versa? What was working well for them, and what wasn’t? What were the biggest pain points for them with SQL Server and the Microsoft platform in general – for the developer, for the DBAs / IT folks, and for the CIO? I had some general ideas of course, but did not feel like I had a really clear picture. So I decided to go fix this. How? What better way than to go work with customers directly for a while. I was very fortunate that there is a team within the SQL Server product group, called SQL Customer Advisory Team (or SQL CAT for short), that worked with our highest-end customers, helping them with architecture and best-practices so they would be successful with the product, and thus helping us build design wins and references. This team had originated in the US, but was now expanding internationally into Europe and Asia. They were looking for someone to cover Asia and the timing worked out so I joined the team, relocating to Shanghai for my first stint in China. I spent close to 18 months on this team and during this time, I had the pleasure of working with over 20 customers all across the Asia-Pacific region – from building the first-ever core-banking system on SQL Server (in Japan) to the first-ever end-end telecom OSS system (in Korea), to a variety of data-warehouses and BI systems, large-scale OLTP systems, SAP systems, and everything in between. It was an incredible learning experience – learning about the product and how customers used it of course, but also about our customers themselves, our field, our partners, and more generally about how business is done in Asia.

I suppose this is where my second professional fascination started to develop – with business in Asia. Of course, being originally from Asia, I had somewhat of an innate “connection” here. But I had never actually worked in Asia and when I was growing up, India was not exactly what you would call a “dynamic economy”. However, as anyone who has spent any time in Asia in the recent past can attest to, there is now a different energy in the air here – you can feel it right from the moment you land as your fellow passengers get ready to disembark the plane, if you know what I mean! It is stating the obvious to say that this is a time of tremendous transformation – for China, India, and the entire region. If one could list the top 5 global phenomena of our age, there may be some less pleasant items such as global warming and the geo-political tensions that bedevil us today, but amongst the positive ones I’m sure would be this historic rise of Asia. And of course, the tremendous impact of information technology, the Internet and other technological innovations on all of our lives and work. So in terms of future career direction, I had a strong suspicion that my life in the future was likely to revolve around the intersection of these two great phenomena of our time, Information Technology and Asia.

After spending 18 months in the CAT team, I returned back to Redmond. Around this time, the SQL Server team was transitioning from shipping SQL Server 2005 to turning the focus on the next release. While SQL Server 2005 was a great release (and has since proven to be the most successful release of SQL Server ever in the market), it took a long time coming. We had clearly struggled to match our engineering processes to the tremendous growth in the size of the team and the business. It was also a time of some considerable change in our industry – a lot had transpired since we had started the SQL Server 2005 journey 5 years prior, and it was time to take stock and point the ship in the right direction for the next decade of growth. Coming off my stint in the field, I was given a unique opportunity to serve as Director in our Strategy team, with responsibility to oversee the planning for the next release as well as to help craft the long term direction for the SQL Server product and business.

It was a great job, one that stretched me in many different ways. We drove the planning for what will now ship as SQL Server 2008, focusing up-front on release themes, complete end-end customer scenarios within each theme, and distinct “improvements” (aka features) within each scenario. And we drove a complete re-engineering of the way we build our software, with much more emphasis on making sure these improvements are really complete and high-quality before they show up in main-line builds of the product. The jury is still out on these changes to some extent, and any change is very hard, at least initially; but indications so far are that they well were worth the effort and pain for what they are buying us in terms of quality and predictability.

We also drove a ground up review of our product and technical strategy. I don’t want to discuss publicly much of what resulted, but suffice it to say we ended up with a much clearer understanding of our strategy and direction going forward. Paul Flessner, Senior VP for SQL Server, summarized some of this in his open letter in April 2006. Internally, we crystallized this notion of a Complete Data Platform which defines the scope of what we think is needed to serve customer needs going forward. It is a pretty fundamental and broad redefinition of our field as we know it. We have not talked much about this externally at this stage, but you will see us begin to do that much more going forward.

While it was extremely gratifying to be able to work on issues of such breadth and significance to our product and business, in the back of my mind a clock was ticking, and getting louder as time went by. Asia was calling! It was time to go. Luckily for me, just in the last year, we had decided to start R&D teams in both India and China, our first real R&D forays outside of Redmond. We were looking for someone to lead our newly formed China R&D team, and I jumped on the chance.

So here I am, in China again. Based in Shanghai, but with teams in both Shanghai and Beijing.

If you have read this far, you must have been quite bored when you started and even more so by now J. I’ll stop here. Next time we’ll discuss what a Complete Data Platform really means, or what exactly we’re doing in our China R&D team, or something of the sort.

Until then – Cheers!