Mark Cliggett's WebLog

Embedded Windows aka XP Embedded

markcli — Thu, 02 Mar 2006 07:46:00 GMT

It’s been a year since I’ve posted, so I should provide a little context. In my previous job, I ran a team in the Developer Division that was focused on increasing the flow and quality of communication between our customers (developers) and our product teams (the people who build VB, Visual Studio, the .NET Framework, etc.). We drove things like Product Feedback Center, the new Forums, and some internal initiatives designed to encourage people to do more to make customers successful. A fun job, and I learned a lot, but about 3 months ago I took a new job in the Mobile and Embedded Division, running the team responsible for building the componentized/embedded version of Windows – XP Embedded being the most recent release. I leave the community work in the awesome hands of Josh, Scott, Joe, Shayan, Keen, Sara, Eric, and others, and wish them the best. It was incredible privilege to work with them.

With that, onto Embedded Windows…

I was inspired to write this (in other words get my blog-posting rear end in gear) by a thread today in the XP Embedded newsgroup today that included these criticisms:

The tools are truly dreadful and so below standard it's unbelievable

when it comes this type of essential prototyping during optimising an

imnage.

Not to disparage what MS has achieved in componentising a complex OS

but I would be ashamed being a mega billion $$$ company and shipping

this s/w...

and

The sad part is that Microsoft would read this and not be concerned.

They would assume its just a few people and not a big deal.

This is the perfect opportunity to talk about where we’re headed with Embedded Windows (I use that term to distinguish it from Windows CE, and to avoid tying it to a particular version of Windows - our charter is long-term, not specific to XP Embedded).

Let me start by acknowledging the validity of the criticism. There are many many things I’ve seen/heard about in the few months that we need to improve on. We’ve underinvested in this product/technology. As a result, things that were (maybe) acceptable in a v1 release are not tolerable when they are still unaddressed a long time later. The only part I disagree with above is that we don’t care. I took this job in part because the company recognizes that we’ve underinvested, is strongly committed to changing this, and has asked us to do what’s needed to make sure we have a very compelling offering in the embedded market. In addition, we have people in our team (Andy Allred is a great example) who have cared deeply about this for along time, who have been frustrated that we haven’t been doing more, and who are now thrilled to have the opportunity to change this.

At a high-level, I have 4 priorities right now – I’ll share them here and I’d love feedback.

1) Build the team’s capacity. Obviously this isn’t going to directly fix any of the problems people point out. But the reality is that we have far more to do than we are currently capable of. To get out of this situation we are growing the team significantly. Again, to the “MS doesn’t care” argument, I’ve been told to go build a great team – in the near term I have all the mgmt support I need and my greatest constraint is finding/bringing on board people who will help us go fast soon. (We’re hiring in all disciplines – dev, test and program management – if you know someone who should be here fixing that problem you are yelling at us about, please point them my way. :-)) We’ve made good progress in just these three months, but we have a ways to go still.

2) Make existing customers successful in 2006. Another way to say this - address major pain points that are limiting people’s success today. Given that we’re still building the team, we can’t bite off a lot. But I do want to identify a set of things that, for example, make XP embedded systems more expensive to deploy than they need to be, and then address them. The area where I’ve heard consistent feedback is around post-deployment – managing, servicing, upgrading, etc.. These are problems that can cost customers' customers a lot of money. Upstream in the lifecycle, we’ve gotten a huge amount of feedback about the tools (what prompted the newsgroup thread). I’ll admit to being schizophrenic about these problems. On one hand, some are truly embarrassing and it’s lame not to just fix them. On the other, when I ask customers “which is more important – these tool problems, or these embedded-features or these management issues?” the answer I typically get is “tool improvements are always helpful, but the reality is we are used to them and can live with them – what our business needs is these other things”. Andy pointed out today that there is a difference between some bigger customers who are driving volume, and smaller customers who do a lot of prototyping. It’s an important distinction which we need to remember. My hope is that we can take some small steps on the tools – even just fixing a small number of “top bugs” people complain about would be a start.

3) Respond to Vista. Our mission is to ensure that enhancements in the desktop flow through to embedded customers. For Vista, the plan of record is to ensure that all Vista technology that is backported to XP works well for XP Embedded. In this way, our embedded customers can benefit from at least some of the Vista improvements. We’re in the process of working through the details of what’s in Vista, what Vista components will be available on XP, what functionality those components deliver on XP, and how to get embedded customers what they need. There are a number of options we’re considering – more on that in the future when know enough to share the details.

4) Build a long-term roadmap for success. I view the 3 things above as somewhat reactive – we underinvested for a while, so now we need to do the work that didn’t get done during that time. We have to start there and that work alone will keep us busy for a while, but we eventually want to get ahead of the game. There are number of things we want to accomplish in the long run. To pick a couple: We are going to work with Windows to increase the alignment between the mainstream release and the embedded version – my nirvana here is that the embedded build comes out of the build lab simultaneous with desktop Windows, and ships the same day. (We took steps towards this with Vista but unfortunately did not get as far as people hoped at the beginning of the release). Another big long-term goal is to have great tools – I expect we’ll do a significant rework at some point, addressing the problems that people are pointing out. The tools we have today have bugs/productivity issues, but beyond that I think they don’t tackle some problems we need to tackle (e.g. someone mentioned the fallback scenario where you have an upgraded component but need to work with the old version). That’s just two areas - there is so much we could do. That’s why this is a fun job.

I took this job knowing that in the short-term I’d be hearing a lot of (valid) criticism and that it would be some time until the delay between “Customer points out an issue” and “We address the issue” is reasonable. The thing we have going for us is a lot of mgmt support, and a (growing) number of people in the team who are very excited to be able to show that we do care, that we are capable of delivering a high-quality experience, and that Windows is a great technology base for the embedded market. I realize that it will be some time before we prove ourselves. The one thing I ask until then is that people keep giving us the feedback because we’re finally setting ourselves up so we can address it.

Speaking of which, we do plan to bring over some of the things from DevDiv – for example Product Feedback Center-style feedback (which puts the feedback out in the public, lets customers vote on what is most important, and forces – through internal tools - us to respond with an outcome) or the Forums (which don’t scroll away as newsgroups do and which have some other strengths). Look for that to start happening in the next few months – in my last job I found out you can’t just turn those on, the team has to be ready to do the work. In the mean time, you can reach us through my blog here, through the XP Embedded Team Blog, or via wecrt_nospam@microsoft.com. Or the newsgroup that got this started :-)

Every day I come to work thinking “There are 100 things we really should do today – which one thing should I focus on so that it happens?” And to some extent the answer every day is “Hire one more person into the team”. But with every person we hire, our ability to do work that customers see increases. Look for that to start being visible soon.

Rumors of my demise…

markcli — Sat, 15 Jan 2005 09:22:00 GMT

AKA "I’m not dead yet!"

Thanks to androidi for prompting action on one of my New Year’s Resolutions. See the end of this post for some introspective blather on why I haven’t posted since June. The real reason is that I’m lame.

I’m still at work on customer connection in DevDiv, and now in other parts of the company. Three quick highlights:

1) There’ve been recent conversations with Soma about how to do transparency right – what helps customers, what makes it harder for them, how do we make sure that in trying to shed “the light of transparency” we don’t unintentionally burn the house down (to mix some metaphors). I’m really excited about Soma’s active participation in the blogging community, and that’s prompted another conversation about whether more senior people in the division and company SHOULD have a blog. In the case of mgmt I'm more interested in blogs as a tool for customers to find and contact “whoever is in charge here”, although I’m sure our managers also have interesting things to say. When I checked about 8 months ago we weren’t doing so well here. This will be a focus in coming months.

2) Josh and I and others have recently spent time with others doing similar customer connection work in the community. It’s pretty obvious that if DevDiv does a better job of being open about features, status, etc., but there are not comparable efforts elsewhere in the company, it will be frustrating for customers at best. Stefan Weitz recently took a job similar to mine in Windows Server and it’s great having him help with these efforts, not only for the benefit of Windows Server customers, but also internally in rationalizing some of the chaos. It would be great if things like feedback mechanisms were relatively similar across products, in part so customers know what to expect but also in part so we can make one thing great vs. having several mediocre things.

3) I recently read this great post from Rick Strahl on his perceptions of CTPs, MSDN Product Feedback Center. Josh responded to many of the specific points but Rick’s post kind of woke me up about our need to do a better job explaining externally what we are doing. As Josh said, I don’t necessarily agree on some things but I do not disagree that Rick’s perceptions are 100% valid and reasonable based on what we’ve done and communicated. (One of the challenges I haven’t figured out how to handle it when people I work with here have done something badly and we’re actively working with them to change it, but don’t have a solution yet. It won’t help anyone to be open about that, and there are times where even I’m astonished at our inability to make what seems like a simple change. Silence is not a bad option, although maybe a quick "yep, we recognize we have issues and are working on them, stay tuned". The trick is what to do when I have no clue about the delivery date - I've made that mistake before.) Fundamentally things like CTPs and Product Feedback Center are all about giving people more control over their experience with Visual Studio and the .NET FX (and ultimately other products). I can see – based on our recent execution and our past history – how it would be easy to perceive otherwise (e.g. a marketing program) and I recognize that it will take time and good execution to change that. But the commitment is consistent from my team up to Soma and even beyond to Eric Rudder and Jim Allchin.

Enough for a Friday evening…

The Promised Introspection:

I love getting comments back from blog postings. Being public/visible is not my – uh – calling. I hated drama in grade school. But I think the real issue is that the energy I use for blogging seems to be the same energy I need with my kids and I haven’t quite figured out how to do both consistently and at the same time. I read an article or letter somewhere recently (Fast Company? Harvard Business Review?) where a dad said that it becomes exponentially harder to balance work and family as kids grow older. It’s hard to argue with that, at least based on my experience. Anyway, I’ll be the prodigal son here, admit to straying from the Blog Path, and get back on the straight and narrow in the weeks ahead.

Whidbey leak late last week

markcli — Wed, 30 Jun 2004 01:57:00 GMT

As many of you know, late last week we sent mail out to Whidbey alpha participants alerting to them to the announcement this week. I was particularly excited to send that mail because we had some real news in the mail about the Express skus - not tons of detail and not lots of advance notice but at least we made an effort to give the early participants a heads up. Lastly we asked people to keep it quiet until the announcement.

As Robert McLaws talks about, the news was very quickly leaked. I want to thank the people who honored our request - I know almost everyone on the list kept it quiet, and we will to continue to push to be open with this sort of information. For the one or two of you who weren't able to honor it, well, it has definitely prompted some discussion and “management visibility” here. It has also given us an experiment - did the leak matter? We'll look at this in a couple weeks, trying to assess whether it had a negative impact on our ability to communicate the information in a way that was most helpful to customers. If the mail-and-leak caused a lot of confusion then we'll probably think very hard before doing that again. With respect to the leakers themselves, I'm actually interested in feedback on what to do. On the off-chance we find out who did it, we'll likely reconsider their involvement in the alpha. For Whidbey this is moot - everything about Whidbey is pretty much public at this point. But we'll keep the names around and look for them as we're pulling together names for future alphas. I'm not 100% comfortable with this approach but I'm not sure there's an alternative and to every extent possible I want to ensure that we optimize for the 99.9% of people who will honor our requests to stay quiet for a few days. Another approach is to wait and send the mail the moment we're rolling it out to the press, but that's hardly more than sending the press release directly.

Robert raises another point that has always bugged me - we often take our most enthusiastic customers, tell them our plans, get them excited, and then put an NDA gag on them. We have some ideas around this that we'll pursue in the next version, ranging from more radical (do an early non-NDA design review and invite people based on their ability to be a conduit for information/feedback to/from the broader community) to less radical (assume news will trickle out during the NDA and provide clear guidance/process through which well-intentioned people can make appropriate judgement calls without feeling like they are at risk of getting in trouble).

Lastly, as I said in an earlier post, a lot of this is about the transition. When we give less information out and then give a little bit, it's news and someone can gain brownie points by leaking. If we consistently do a better job, my guess is that people will mostly ignore the articles based on incomplete leaks and either look for the news directly from us or read the articles based on more comprehensive research.

Whidbey: opaque -> transparent

markcli — Fri, 25 Jun 2004 08:13:00 GMT

Stuart commented on my last post - explaining how in the long run people will adjust to the new way and we'll still get to have our cake (be much more transparent) and eat it (get some bang around milestones like beta release) too. What he said is key and is the thing I keep in mind every time some “bad” thing happens because of transparency: the challenge is the transition. When people are used to our old beta cycle and they get a mostly-untested community drop, they expect it to work like a beta. When we don't reveal much information about schedules, it's a big deal when we suddenly start dropping bits of info about our schedules. In the long run people will start having more accurate expectations. In the short-term though each change is an opportunity for a disconnect between what we do and what people are expecting.

Of course, this doesn't absolve us from having to do a better job with each of these things. I really want to make the community drop experience better. When we talk about schedules, we have to go overboard in explaining them so we don't accidently leave room for misinterpretation. Etc..

Gotta run - a lot of release prep going on right now and I want to make sure we're ready when the bits are ready. :-)

Upcoming Whidbey beta

markcli — Thu, 24 Jun 2004 00:43:00 GMT

Eeesh, more than a month since I last posted.

I got feedback in the past couple days ago that while the community drops/more frequent builds are nice, what many people (especially non-MSDN subscribers) want is access to any build period. The feedback is to work on that first, and more specifically to get our Whidbey beta out and give the public access to it.

That is actually what we are doing right now. The Whidbey beta is close - our mktg folks will announce it when the time comes but it's not far off. There will be a public fulfillment mechanism - details will be part of the announce. There are some other good things coming as well that will help address the pent-up demand among people who don't have it already.

It's an interesting time that shows we haven't gotten the transparency thing totally mastered yet. We still want the big press bang that comes with announcing a beta. To get that it means that people like me have to be kind of cagey about what's going on. I don't love it and I think in the long run we'll move towards being more open about schedules and status. I've had a conversation with both Channel9 and our mktg team about tracking the beta2 ship push on Channel9 - publishing bug charts (big numbers!), talking about status, etc.. People are in favor of doing this even though it means people will be able to infer the beta2 release date pretty accurately long beforehand. This idea came from looking at beta1 - we're doing better at being open with technical info and direction but for no good reason we have not been as open about dates/status. Some of the other comments to my blog and elsewhere have pointed out specific examples of how this causes problems for people. It's great to have good press buzz but you also want to keep enthusiastic customers enthusiastic by not making them guess so much.

Followup to post re: community preview philosphy

markcli — Thu, 13 May 2004 03:43:00 GMT

Thanks for the comments/feedback I received. A few additional comments (I'll try to keep this one short, or at least shorter).

Josh Ledgard questioned the need for every/daily builds. I use the word “every” to make a couple points, one internally and one externally.

The internal point is that “shipping” these builds - with all necessary extra info like content and quality - has to be automatic/free. If it’s not, i.e. we have to decide which build, or do extra work to “polish” the build before it ships, there will be resistance to doing them - the daily build is pretty ingrained into how we do things and keeping forks (and dev machines set up) for previous builds adds lots of work. We also get into gnarly conversations about whether to fix some hideous bug in one area before we ship it - those are gnarly conversations because up until we get close to shipping, there's almost always at least one hideous, embarrassing bug in every build - the choice is really to lockdown the build while we fix the hideous bugs, or fix it in the next build and risk that the new crop of hideous bugs is worse. The longer we stay locked down, the longer most devs go without having their work subjected to the next testing gauntlet – they can’t check in, can’t build against all the other changes, etc.. Someone commented that Whidbey is complex – that is absolutely reflected here at the daily, product development level.

Externally, I want to make sure that we set expectations that the build from today is not necessarily better than the build from yesterday and that the casual observer should think twice before installing. Our builds do not have a consistent incremental increase in quality. There are spikes and valleys. There are valleys even late in the release cycle, e.g. we send a beta out, get feedback, check in improvements across the product, and break things (which we then fix). I want customers to see lots of builds coming out so they understand the risk, vs. seeing fewer builds and thinking (erroneously) that each build is good and “guaranteed” to be better than the last one.

All that said, I’m more committed to frequent, automatic, and public, than “every build”. I just think aiming for every build is the best way to get there – otherwise we’ll think incrementally. It's easy to scale back, e.g. from every day to once a week, or to whatever we end up seeing customers do.

I do not mean to imply that there is zero order within our teams. Over time the builds absolutely get better, we have a set of build verification suites that are run for every build, we have tools in place to help make “breaking changes” easier to manage across the division, and there are always people looking at process/efficiency improvements. In many ways, what our teams do is amazing. But the complexity is very high, the % of tests we can run against each build is relatively small, and bad bugs get through.

The feedback about “must always pass” tests is good. That is what our build verification tests are meant to show, but reading this prompted me to think about three things: 1) maybe we’re not looking at the right things in the bvt’s. 2) maybe we need a more comprehensive set that takes longer to run but which still completes within a day or so. 3) can we get community help here? i.e. perhaps we augment what we’re doing with a suite that the community provides. That would be a great rallying data point for the division – do our daily builds at least reach the bar for what customers are willing to install? I’m not looking to pass work off on others, just thinking about where I can blur the line between internal and external so that everyone ends up better off in the end. I don’t remember if I’ve written about this before, but I’m also interested in the possibility of developing many test cases in public workspaces so people can at least see them. Maybe bvts are the pilot we’ve been seeking.

Thanks for the comments re: encouraging people to try this. That’s exactly where we are (although we’re also lucky to have strong support from Eric Rudder on down to help the division get started and survive through the inevitable glitches). People are pretty excited about being more aligned with customers’ needs and wants. I haven’t met a person who thinks this is a bad thing. The concerns I have heard are very legitimate: build quality is variable, how do we do this in a way that doesn’t cause customers to waste their time or become dissatisfied, it’s extra work to gather/publish some of this information. Those are all things to work through, vs. reasons for not doing it.

Long-term goals/challenges of community tech previews

markcli — Sat, 08 May 2004 00:56:00 GMT

I was forwarded a great email thread from VB customers. Here are some excerpts (from multiple people):

"I'm beginning to think "Community Editions" of products as large and complicated as Whidbey are NOT a good idea. Yes I said, NOT.

Reason? I believe the average non-NDA, Joe Blow developer (esp. those

new to .NET or those who've not upgraded yet) out there get the wrong

impression. These editions are just not up to the quality-bar that the major betas. And *I* understand that. Most of us do. I just wonder if they don't hurt the products.”

“I'm disappointed in how Microsoft is handling this "release". Each time they've done it (MVP summit, Dev conf in Orlando, Whidbey preview

workshop this week), I've counseled them to tell the developers that

anyone installing this software must be prepared to format their drive

and re-image. Invariably, they agree but say nothing.

This can only cause hard feelings as more "Technology Previews" arrive. Fewer developers will dare to try them--simply because they don't know how to protect themselves.”

“Why not offer these drops in the form of a VPC image with the build

installed? Not only would this give the community a safe way to work

with the builds, it would also let you get up and running in 2 minutes

instead of the 20-40 it takes to get a preview installed.”

“I agree that VPC images would be a _lot_ more compelling. Unfortunately, there are lots of issues with distributing VPC images.”

It’s great feedback, partly because it shows that we need to explain our long-term intent better. I’ve touched on some of this before, but I’ll try to elaborate here.

Our ideal is that every build is publicly available. Another way to say it is that we want to enable a scenario where a customer finds a bug, reports it, the team/developer gets the report essentially immediately, sees it’s an easy fix, fixes it, resolves the bug, the customer gets notified, and with no extra effort from anyone (beyond the customer installing) the customer can install the fixed version within a few days. I have tried to do that within our existing processes and it requires a significant amount of extra effort to get the fixed build distributed to even one customer. There are similar (but different) scenarios around getting feedback on features before the features are so baked that it will be a whole product cycle (!) before the feedback can be addressed. Frequent public builds are a means to the end of enabling us to be much more responsive to some kinds of customer input.

We’re in a very significant transition phase that will be challenging due to people’s expectations (both internally and externally). For a long time we’ve released builds very infrequently and gone to great lengths to make sure “builds work as well as possible” by having long lockdown/test periods before each beta. (More on “builds work as well as possible” below) This March Tech Preview is a step towards much more frequent builds, but the transition is causing turbulence because of past expectations. Once we’ve been putting builds out frequently (think weekly or every day) for a while, there will be less confusion about which ones have been through comprehensive testing – many people will learn to ignore most and look for the Beta indicator of quality, while bleeding-edge customers have the option of being more involved. I’d love input on how to get through the transition phase better.

I put “builds work as well as possible” in quotes above because the downside to the long lockdown periods is that we’d regularly get customer feedback several weeks prior to the release and decide not to fix it simply to avoid the risk of destabilization. That has been a source of great frustration for many of us for prior releases – we’d get builds out to some customers at the earliest possible moment but we’d already be well into lockdown so feedback we got prior to beta N often didn’t get addressed until a beta N+1 many months later. This is the tension – minimizing change so we don’t break things that worked for customers yesterday, or addressing the feedback they gave us based on their experience yesterday. Community drops are about getting feedback in time to address it.

We’ll be releasing another community technology preview in a few weeks. You will probably find that the quality will be significantly better than the March on. This is a function of where we are in the release cycle. The March one happened very shortly after a significant number of changes went into the build. Since then we’ve fixed a huge number of bugs, done extensive testing and not added/changed functionality. We will explain this to people. There will still be significant bugs – otherwise we’d call it a beta.

The feedback above on messaging at conferences, etc., is great – I’m following up on that. We did put a fairly dramatic warning on the cd envelope - I know that had some effect because someone in my management chain asked a VSLive attendee whether they’ve installed the build, and the person said “Are you kidding?!? Have you read the warning?” We’ve also had relatively limited downloads on MSDN (in the thousands, vs. many times that for betas). Some obviously got the message but again, given expectations, we have to do stellar here to avoid problems and we’re not stellar yet.

In the long run we need to do five things well for this to really come together:

First, we need to solve the distribution problem. We’re looking at several different options – shipping media, VPC images, the p2p solution suggested earlier. The one I’m most intrigued by (and probably the farthest from reality today) is incremental updates. Anyway, we need to figure out how to get a large quantity of bits to customers in a quick, reliable, and cheap way, otherwise it’s impossible to do this frequently.

Second, we have to expose what we know about the build’s quality. Internally we survive by running build verification tests on every build. At a minimum we can start exposing that level of information for these builds. Part of the problem is that the information is very hard to digest, e.g. it comes out as 83% of the tests passed, but what you really want to know is that you can’t open a project or debug because there are significant failures in those areas, i.e. the test results can look good when any real user would say it’s broken.

Third, we need to be smart about letting the community help in communicating build quality. I described this in an earlier post, but next to the “Download this build” button, I would love to show how people who’ve installed so far feel about the build. If I’m a customer about to download the bits and I see that 21 out of 23 people who’ve installed so far say “It’s great” (or “it sucks” as builds sometimes do), I can make much better decisions about using my time to download and install. There’s an opportunity here to promote and benefit from community self-organization.

Fourth, we need to be more transparent about where we are in the dev cycle, what features are coming in now, etc.. We’ve made some progress here, but we need to do more. One thing about the March tech preview was that it happened pretty quickly and we missed opportunities to explain where we were in advance, to set expectations better. In the coming months expect us to do things like cover the project status on Channel 9 and be even more explicit in blogs. We’re doing a review of the March tech preview now to figure out what to change and what specific steps we can take today to help with the next one.

Lastly, we need to give customers a dependable, effective mechanism for submitting feedback and finding out what we’ve done with it. Few customers have this today, and historically we’ve done a poor job responding (although we’ve done much better with feedback on the Whidbey alpha from last summer). If customers can submit feedback, get a response within a few days, and see the results a few days later, my guess is they’ll be more interested in tracking our progress more closely.

I think if we do those things – make the logistics easy, make it easier to assess quality, make it easy to understand whether the builds have features/bug fixes that are interesting enough to me – then much of today’s pain will go away (or be offset by benefits). The problem right now is that none of these things are automatic. The many many month release cycle has gotten us in the habit of doing little of it as we go, and then trying to pull a bunch of stuff together at the last minute – it’s pretty classic for us to send out a beta with 0 guidance on what new features there are or what has been fixed. The current community drop approach is still too much like infrequent betas – but we are starting to figure out how to build it in and make it part of what we do every day.

In the end, if we do all this and 99% of installs still happen with betas (but 1% continue to make the effort to get more frequent drops), we’ll be satisfied because two things will have changed:

- We will have shifted much of the control/decision-making over how involved customers can be from us to our customers.

- It will enable every person within our division to be much more responsive to customer feedback, without a lot of incremental extra effort.

That’s the big picture. I’m very excited about frequent community/public builds not because people are clamoring for them, but because of the implications of making them work well for us and for our customers. There is some skepticism internally about these – no one will install, etc.. And there are stages in our product cycle where builds generally are bad. But even scoping this down to just the time between a beta1 and RTM would be a huge improvement on what we do today. The great thing is that that period is coming up soon and it's a great chance to figure it out, so that when we get ready for the next product cycle we have enough of the infrastructure/processes/philosophy in place that we can continue.

Thoughts?

MS Openness and MVPs/RDs/trainers/authors...

markcli — Fri, 30 Apr 2004 01:00:00 GMT

I just happened across this blog post from Randy Holloway (http://weblogs.asp.net/rholloway/archive/2003/11/04/35707.aspx)

I’ve been thinking about this from a slightly different perspective. As we do a better job in being open with information about upcoming products, responding to feedback, getting bits out earlier, I wonder how it impacts our “special” customers – MVPs, RDs, writers, trainers.

The perspective I’ve been thinking from has to do with “privileges”. Think of a spectrum with MS-internal at one end and the public at the other. As you move from internal to public, you have fewer privileges. Internal people can read/write product functionality from day 1, the public gets to read it historically very late in the dev cycle. Same with things like specs, bug reporting, etc.. “Special” customers (sorry, I can’t think of a better word – best isn’t right, top – vs. bottom?, influential – sometimes) fall somewhere in the middle of the spectrum, with more privileges than the public but less than most internal people.

In many ways what we’ve done in recent months is taken the privileges that “special” customers had, and make them public. We’re being much more open about product features, we’re moving towards making pre-release builds publicly available, etc.. And I don’t think we’ve replaced what we’ve “taken away” from this group with new privileges that only internal people had before. (Although there are a few examples, for example there are C# MVPs contributing to the C# FAQ on the MSDN Dev Center).

I’m currently assessing our community priorities for the next year, and I think addressing this “special” class of customer is one of them. Some of the ideas we’ve talked about include:

- moving more internal privileges over, e.g. maybe MVPs can see test cases (or if everyone can see them, MVPs get read/write access)

- being more responsive. Presumably, as we give people better mechanisms for contacting us, submitting bugs/suggestions, etc., we’ll get to a point where we have to prioritize/triage who we respond to first. We could consciously be more responsive for this group.

- Giving people roles in implementing some of the more challenging ideas. For example, I posted earlier about possibly having MVPs vote on daily builds, to give customers more information when trying to decide whether builds are worth installing.

These ideas are mostly MS-centered though – how these people can be involved in what we’re doing. I'm excited about them, and we’ll do at least some of them. But Randy’s post helped me recognize how MS-centered I was being. What are the ways that we can help these customers be more successful? These are incredibly capable people and I want to make sure they continue to have opportunities around MS products and technology (so they don’t go Elsewhere). Here’s one random idea along those lines – they get to have MS personnel accompany them on a sales call (or at a training), no-questions-asked, once a year. I’m sure people outside can come up with much better ideas than I can though, hence the post.

So, at a broad level, here are the two questions:

- has the openness (what Randy called The New Microsoft) been uniformly positive, or have there been unintended/unforeseen negative consequences?

- When bits/info/contact info are generally available, what other things can we do to make MVPs/RD/trainers/authors successful?

P.S. I apologize if you’ve tried to contact me in the past few weeks and I’ve been unresponsive. We went on a two week driving trip down to Joshua Tree National Park, and I’m just digging out. That’s a very cool place, and really fun after my sons realized that stepping out of the car didn’t mean instant rattlesnake bite/scorpion sting. I’d love to spend a couple months climbing there sometime. This time I got to do just enough to get scraped knees and remember how important getting my forearms in shape can be. I also stumbled upon the Gram Parsons “memorial”. I knew it was there somewhere but I literally happened upon it wandering around one of the rock formations. I wonder how many grievious angel fans are reading this - can't be many :-) ...

Tech Preview stability, risk of wasting time, and CDs

markcli — Sat, 03 Apr 2004 02:01:00 GMT

I had the following exchange with a customer via email earlier today. I thought it would be of general interest (thanks to Richard for letting me post this). It explains some of the behind-the-scenes thinking around the Tech Preview, how it relates to our internal builds and external betas, where we want to go with these, and how we might enable customers to make a more informed decision on whether to spend time installing these Tech Previews. I'm about to be out-of-office for a bit - sorry in advance if I'm slow to respond.

From: Richard

Is there a suite of CD’s that could be rushed to me?

I greatly appreciate your help in advance.

From: Mark

We do not have CDs available for that build. We will make CDs available though for our beta. For this tech preview we went with just DVD, partly for the sake of simplicity and partly based on community input. We made this decision for other developer events recently too. Unless you are an MSDN/U subscriber, there is no download mechanism.

If you look at this as a mini-beta, there are obvious holes. But these tech previews are not meant to be that – they are meant to be a way that bleeding edge customers can get greater visibility into our progress and give us feedback earlier so we can do more with it. There were plenty of challenges with this first one, so we’ve been simplifying where possible – dvd only, no obvious public download mechanism (that’s reliable for such a large download) within MS so we’ll skip that this time, etc.. We’re learning a lot about what’s really necessary, and we’re working on solutions for the holes so that there are fewer each time we do this.

To get back to your specific question – if you really want to install this tech preview, you need to have a DVD reader. In the long run I really want a public download mechanism in place so that I never have to tell someone “sorry, you can’t participate in our pre-RTM program unless you jump through a lot of hoops”. Unfortunately, that’s what I have to say this time. I’m sorry about it, but I hope you at least understand some of the rationale behind why it’s this way now

From: Richard

Since I do wish to continue active participation in the preview program, I will purchase a DVD reader so that I can use this build.

From: Mark

Good luck! It’s a little funny – I’ve had a few people tell me “I really want this but can’t get it, and other who say “wow, this is a lot buggier than the alpha – more features but buggier”. It shows the difference between weeks of testing (which betas get) and essentially one night, which this got. It’s a glorified daily build. My hope (and I have someone looking at this) is that some day all daily builds will be available, but right now this is a transition in expectations for everyone.

From: Richard

I am a bit confused by your comments. Would you recommend that I stay with my older alpha version since you indicate that this new version has many bugs? I would hope that this new version has repaired bugs that were reported in the older alpha version. Of course as new features are added, so are new bugs. Please clarify and give me your recommendation.

From: Mark

Sorry to confuse you. The quality does not continuously increase during the product cycle. In fact, right after big checkpoints it tends to go way back down. Before the release we stop changing things, test for a few weeks and focus on fixing only showstopper bugs. Right after the release there’s a lot of pent-up change that pours in and quality is very bad – things that worked break, etc.. It stays pretty bad all through the coding period. It gradually creeps up after that – with significant setbacks along the way – and then rapidly increases as we get close to the next release. The bits on the dvd are from somewhere towards the end of the “gradual improvement” phase. This means that some features you used in the alpha will be less stable now than they were. I hope that an indirect effect of releasing more frequently is that the quality stays better throughout the cycle – internally we can put up with a fair amount but we hate sending out things that don’t work.

It all depends on what you want to do. If the alpha is working for you and you don’t want to find yourself broken, you probably shouldn’t install the tech preview. If you have a spare machine and want to see the new stuff go for it. There’s a gray area in-between where I can’t make a recommendation. It gets back to keeping the quality higher throughout the cycle. That’s just hard to do with a big complex product and significant changes/fixes that have division-wide impact. The one thing I do know is that starting sometime after beta1, the quality should steadily get better as we drive towards RTM. These pre-beta tech previews are something of a mixed bag.

That’s probably clear as mud, but accurate. Let me know if you have additional questions. I’m oof for two weeks starting tomorrow but if you have questions Josh can probably help you.

From: Richard

Thanks for this help. I just purchased a DVD reader (have not installed it yet). I will keep my current alpha version and install the new version in a separate area (if this is possible). If not, I can always go back and re-install the alpha version.

Please confirm that I can have both versions simultaneously installed (Whidbey and the new release). Thanks.

From: Mark

Gosh, you’re really motivated and I keep having to disappoint you. We can’t commit that side-by-side install (SxS) works. The way we ensure SxS works is fairly exhaustive testing where people across the division install, lots of lab tests, etc.. We don’t do that every night, mostly because we have a lot of dedicated machines. It gets a ton of focus before major milestones go out though. Obviously it has a lot of impact on users’ ability/desire to install an interim build. For this release, we were serious about that disclaimer on the dvd that says in essence this release might melt your machine.

One of the things I’ve asked my team to figure out is how to improve customers’ ability to make better decisions around whether to use their time on builds. I think there are at least a few customers who have a spare machine and will roll the dice on anything. I don’t know if this will work but I’d like to see some kind of public categorized voting system where those pioneers can log their experience for the rest of the world. All you have right now is a DVD and hope – sort of like Russian roulette only perhaps with worse odds. If you instead had the dvd, perhaps our nightly build results (which are currently inscrutable to all but a few), and the results of 100 customers voting on whether the build is worth getting (perhaps with some subcategorization – vb, SxS, etc.) you could make a much more informed decision. Do you think that would work? I don’t know but I want to try that – it can’t be any worse than what we provide now.

P2P redistribution of Technical Preview

markcli — Tue, 30 Mar 2004 01:03:00 GMT

A couple people suggested p2p mechanisms as a way to enable downloads for anyone, and asked about restrictions on copying in the EULA. Offhand, P2P seems like a great solution here, and I've started a thread with our legal department about making it so the EULA doesn't preclude it. It's going to take a while to resolve - you can imagine that a request to enable massive copying of software is somewhat unusual :-) In any case changes would not show up in the EULA until the next technical preview drop, which means we still don't have public access for this one. As I said before, we don't have all the answers yet but we do want to enable participation for anyone who is interested. We took a step forward this time but we're not finished yet.

March Technical Preview and non-MSDN subscribers

markcli — Mon, 29 Mar 2004 20:45:00 GMT

I received a question asking how/if people who aren't MSDN subscribers can get this build. Unfortunately, we do not yet have a way to enable this. The issue is really one of not figuring out the logistics yet - we want to do this.

There are two (obvious) approaches - some kind of media fulfillment process (e.g. pay a small shipping/handling charge and we'll mail it to you) and downloads. The issue with the former is that when/if we drop builds more frequently the manufacturing/distribution cost and the delay end up being significant for everyone. The issues with the latter are that we apparently do not have a reliable mechanism for non-MSDN subscribers to download something as large as Whidbey, and I saw a piece of email indicating that the bandwidth consumed serving up the download costs us more than making media. We're in the process of sorting through the issues. I'm 99% certain that the Whidbey beta will have media fulfillment for non-MSDN subscribers. But we need a definitive answer for these non-beta technical previews as well.

This technical preview is available to a lot of people, relative to what we've done before: alpha customers, MVPs, VSLive/PDC/MDC attendees, and MSDN subscribers. But if you aren't in one of those groups I'm sure this step forward in availability offers no consolation. Please bear with us as we work through this - a lot of our tools/processes assume infrequent/small numbers and we're having to change them to do what we want.

Visual Studio 2005 Technical Preview March 2004

markcli — Sat, 27 Mar 2004 02:26:00 GMT

... also known as Whidbey. People are starting to get this - some got it at VSLive/MDC, MSDN subscribers can download it now, and we're in the process of manufacturing and shipping media to alpha participants, MVPs and PDC attendees (should start showing up over the next week).

This is not beta-quality. We run weeks of tests prior to shipping a beta, and fix very little during that time other than major showstoppers. This build in contrast has roughly one night of testing on it - we know it's not toxic but you'll definitely see ugly bugs. We're trying to balance multiple considerations here - we don't want to stop the team just to get a build out, we do want to give people more frequent access, and we don't want customers to waste time installing a build that doesn't work at all for them. For now, we think customers should install this only on a spare machine and only if you're ok with a “work-in-progress”. We'll do more of these and as we get closer to RTM the quality will go up. But for now, be warned :-)

http://msdn.microsoft.com/vs2005 has some additional information on Whidbey, and we'll be posting more information specific to this preview in the coming days. If you have comments/suggestions, please let me know. Thanks!

Microsoft Values

markcli — Fri, 19 Mar 2004 15:47:00 GMT

(Warning: This is squishy and abstract and has nothing to do with developer technology.)

I just came back from a management development course called “Values-based Leadership”. The premise of the course is that by understanding one's personal values and melding them with the company's values, one can provide more credible and effective leadership. I had a few thoughts and questions afterwards - I figure I'll put them out to the world and see what comes back...

First of all, in case people don't know Microsoft actually has company values :-), they are:

Integrity and honesty.
Passion for customers, partners, and technology.
Open and respectful with others and dedicated to making them better.
Willingness to take on big challenges and see them through.
Self-critical, questioning, and committed to personal excellence and self-improvement.
Accountable for commitments, results, and quality to customers, shareholders, partners, and employees.

These are listed at microsoft.com, but I was a little surprised just now to see that they aren't explained in more depth. There's more information/depth internally.

It was pretty interesting watching ~40 managers from around the company tackle something as squishy as personal values. There are lot of pretty analytical people in the company and at times we'd fall back to business questions, probably because we felt more comfortable there. On the other hand, we did the classic MS technique - make a list, prioritize it, and then draw a line - to identify which values were important to each of us personally.

During the class we assessed how we are doing in terms of understanding and living these values. Our view wasn't particularly surprising - we felt we do great on passion for technology and big challenges, but not so well on open and respectful or the ones around customers and partners. There was a great discussion around how our customers/partners would rate us, i.e. were we being honest enough with ourselves. This is one of my questions - how do you think we're doing? I was eager to ask this but when I saw how little external information there is about these I realized you may not have enough to have an opinion. Open and respectful is an interesting one - the part that people sometimes struggle with internally is respectful. Probably both are an issue externally.

In general I think the values are great, but there are two areas I struggle a bit with. First, I don't think passionate is the right word with respect to customers and partners. It reminds me a bit of a short-term affair - lots of passion, no commitment in the long-term. The value is meant to get at things like caring deeply about understanding exactly what the customer need is and doing a great job addressing it. But it misses - for me - the long-term commitment necessary to truly serve customers. There is the value around accountability for commitments, but that's sort of transaction-oriented - we're accountable for the commitments we make, but that doesn't mean we're committed overall. Going back to the premise of the class, I guess this is an opportunity for me to meld my personal value (commitment) with the company value (passionate) and provide leadership. Here's a question - passionate or commited?

The second thing is a bit harder to describe (and maybe related to passionate/commited) - I'd like to see something around commitment to people. Open and respectful partly gets at that - when you are truly commited to someone, you have the hard conversations and do it in a respectful way. Accountability to the various stakeholders partly gets at it. But as a manager there's a situation I sometimes face that the values doesn't get at - a person does great work for a while and ok work now but they've reached their potential and maybe I can replace them with someone with more potential. Viewed through a commitment to people/individuals, I start thinking about the person's family, the relationships our customers have with that individual, the wisdom the person can provide to those around him, etc.. Stating a value is never going to make that decision easy, but it would acknowledge that those other considerations are relevant.

Do the company values make sense? Did you know that the Evil Empire has values? Do we walk the talk? Is the trend headed in the right direction?

Any timing on the Whidbey Beta yet?

markcli — Thu, 11 Mar 2004 22:44:00 GMT

I got this beta timing question a few days back. I'm sure you've heard this before, but we don't announce beta/release dates mostly because we don't know exactly when the development/testing stars will line up.

While I can't tell you when the beta will be, I can tell you that we will be releasing a non-beta-quality build in a matter of weeks. This build won't be for everyone - it will have very limited testing, it may/will have significant bugs, it probably won't install/side-by-side/uninstall cleanly, etc.. But for someone with a spare machine and the patience/time, it will be an opportunity to see Whidbey's progress since last fall. We'll distribute this to alpha customers, mvps, pdc attendees, msdn subscribers, and attendees at some events in the near future. In the long run we want to make these “community” builds available broadly/publicly, but we don't have those logistics worked out yet and so we're starting with what we know we can do. I've mentioned before that we're going to do more frequent drops - this is the first one. We will also continue to do less-frequent beta-quality drops - with full testing, clean install/uninstall scenarios, etc.. -which will be intended for all customers.

More details to come in the near future.

A Whidbey Wiki?

markcli — Thu, 26 Feb 2004 22:57:00 GMT

There's a fair amount of interest in wikis in the Developer Division. Several of us have been bouncing ideas around for where we could experiment with wikis in our interactions with people outside the company. I wanted to put one idea out and see if there's interest (or if it Just Happens :-)) - a Whidbey wiki along the lines of Wikipedia only focused on the Whidbey release. I can see it starting as a simple dictionary/glossary, describing terms or new features, but it could expand in a lot of ways - detailed descriptions of how things work under-the-covers, some higher-level explanation of the user scenarios Whidbey enables (including gaps/issues in those scenarios). I would expect a significant portion of the content would come from the community, but I know there would be numerous people inside MS contributing as well (and I would encourage the rest).

Would people see value in this? Is there anyone out there who is eager to put this infrastructure in place? If so I'd love to work with you to make this successful. There is the potential for making this happen inside the company, but lots of people get involved, they have (legitimate) concerns about things like vandalism, there are questions about whether wiki is the best approach, etc.. Your feedback will be helpful in figuring out how much energy to spend on this, and maybe it will come from the community instead - like pdcbloggers or longhornblogs.