release_team's WebLog

The RC: What works with what?

2005-09-28T23:11:00Z

Soma received a few questions on his blog from folks who received the Release Candidate at the PDC about what works with what. The PDC was tricky because we wanted to have a set of bits that teams from Microsoft could standardize on AND we wanted to hand out the latest and greatest bits to let people try out the Release Candidate. We knew that these two worlds would collide, but decided that getting the latest build into everyone’s hands to find those last show stopper issues outweighed the confusion that might arise.

Jason pulled the questions out of the comments and answered them in Q&A style here. Thanks Jason!

Q. I got the VS RC1 bits at PDC…yea! But am really interested in getting the Biztalk 2006 Beta 1 working with it...which the doc's say currently work with VS Beta2.

A: BizTalk 2006 Beta1 does in fact require the use of VS Whidbey Beta2. However, the exciting news is that BizTalk 2006 plans to release their Beta2 later this year. The BizTalk 2006 Beta2 will require VS Whidbey RTM! You can also find out more about the compatibilities of the PDC bits here: http://channel9.msdn.com/wiki/default.aspx/Channel9.PDCTheGoods

Q. Will there be updated versions of various .NET Framework 2.0 dependent products like WinFX Runtime Components which work with RC1 or do we have to wait for the final release of .NET Framework 2.0?

A: Each of the various technologies announced at the PDC that leverage the .NET Framework 2.0 have plans to provide updates of their technologies that work with Whidbey RTM. Specifically, BizTalk 2006 and WinFX Runtime Components should have externally available releases (Betas) by the end of the year.

Q. FAQ: How do I know which version of VSWhidbey to use with the technologies from Microsoft that also leverage the .NET Framework v2.0? And why do I have to know this?

A: Many of the technologies that leverage the .NET Framework v2.0 specify in their installers or readme documentation which release of the .NET Framework they require; be it Beta2, August CTP, or the September CTP (aka PDC build).

There have been some community efforts to help developers and other enthusiasts to better understand which builds of the .NET Framework v2.0, SQL Server 2005, VS 2005 all work together. You should check out the Channel9 site that can help detect which version you have installed and suggest combinations of technologies that work together: http://channel9.msdn.com/ctpmadness/Default.aspx.

So, why do you have to know all this? The simple answer is that while the .NET Framework v2.0 can be installed side-by-side with previous major releases of the .NET Framework (v1.1 and v1.0), it cannot be installed side-by-side with other interim releases of v2.0. In other words it’s not side-by-side with itself.

Additionally, the .NET Framework v2.0 has had some great innovation in it over above v1.1 (and v1.0). Some of this innovation has been iterative and included the feedback from you, our customers. These iterations have sometimes resulted in “breaking changes” that prevent applications that were written to work with .NET Framework v2.0 Beta2 to stop functioning on .NET Framework v2.0 RC1/RTM.

We’re putting together documentation that will help developers understand the ‘breaking changes’ between Beta2 and RTM. This will be available at the same time that .NET Framework v2.0 is released…RTM’d!!

Whidbey RTM Escrow

2005-08-26T07:45:00Z

As we begin to close down on Ask Mode for the Platform and Design Time teams we enter the final stage of the release, Whidbey RTM Escrow.

DevDiv Whidbey RTM Escrow – the phase before the completion of the RTM milestone where the product goes through a period of bake time. No additional work is planned for the Whidbey RTM ENU product during this period of time, “we are done”. All QA teams have completed their final test passes and have signed off on the product, including all Whidbey release criteria. The only work left to complete will be media verification.

Once the Platform teams meet the set of entry criteria required for entering escrow we will complete our final long-haul stress runs and turn our focus to locking down the Design Time teams. When the Design Time teams meet the set of entry criteria for escrow we will turn our focus towards meeting our escrow exit criteria. We will build what we expect to be our final build and QA will begin media verification.

Escrow Entry Criteria

0 recall class bugs left to be fixed for Whidbey RTM
QA teams have completed their Final Test Pass and signed off on the release
Final release criteria met/signed off

Escrow Exit Criteria

Media verification completed
No Bugs marked as Escrow Reset under investigation

What will people be doing during escrow?

Long-Haul Stress – we will continue to run Long Haul Stress to confirm that we have not encountered any new recall class bugs
App Building – we will continue to do App Building to confirm that we have not encountered any new recall class bugs
Media Verification – during the first week of Design Time escrow we will complete media sign-off
DevDiv shiproom will continue to meet daily while we are in real-time bug mode (24-hour triage), where all new escrow bugs filed must be triaged within 24 hours.
Investigating issues and regression testing any changes taken during escrow

Where is the triage bar? What would reset escrow? During escrow only issues deemed “recall class” by shiproom meet the bar and would reset escrow. This would have to be a serious enough issue requiring us to recall the media, such as an MSRC or GDR class security issue.

Would we restart the escrow clock? Yes, depending on where the fix is we will reset escrow based on QA’s expected Whidbey Response Time for verifying and regression testing a given fix. The expectation is that if we take a serious enough escrow resetting fix that we would need a maximum of 2 weeks from the completion of the fixed build to ship RTM.

We are almost there!!!

-Mike

Whidbey RTM Release Criteria – Dogfooding Team Foundation Server

2005-07-25T22:38:00Z

During Whidbey Beta 1 we built a web application for reporting the division release criteria called DDMetrics. This tool served its purpose for Beta 1 and Beta 2. Now... it is time to take it to the next level and start using an incredibly awesome tool built by the Team Foundation Server team (dogfooding)!

Starting this week we will be using “Team Foundation Server work item tracking” for reporting on our Whidbey RTM release criteria. I just wanted to take the opportunity to say the more time I spend in Team Foundation Server the more excited I get about the awesome features/tools that we are building everyday in the Developer Division. We are very fortunate that we have such a great opportunity to thank the VSTF team by getting some real time usage.

Just some of the TFS Capabilities…

Users of DDMetrics during the previous milestones are going to be thrilled when they see the capabilities in Team Foundation Server... below I will just name a few that I think are very cool:

Customization – we were able to define our own WIT (Work Item Type) to collect only the data necessary, we don't have the 60+ fields you have in Product Studio just because some info is required for other types of bugs. We only have what is necessary!
Querying – I already created some template queries for people to use that are grouped by AssignedTo, PU's, or by criteria. In addition to queries based on all criteria that are marked as “Meeting”, “Not Meeting”, “On-Track”, “No Data”, etc. We had some of these capabilities with the old web app, but nothing like we do now. This is a 100 times better...
Excel and Project integration – ok this is cool... you can run a query for work items assigned to you, your PU, or as a criteria owner, open the results in Excel, make your modifications and publish. BAM!!! All your work items have been updated in Team Foundation Server.
Alerts – you want to receive email when work items change? Imagine if you are a single criterion owner and you want to know when we complete a sign-off. Well, now you can.
Much much more... I haven't even touched on the reporting (SQL Reporting Services or simply through Excel), I will save that for next time or better yet my daily status mails!

The Team Foundation Server team is working hard refreshing their Dogfood servers. Once that completes I will be sending out instructions to the necessary folks for how to access the DDMetrics RTM project.

-Mike

Whidbey Community Technology Previews (CTP)

2005-07-20T22:18:00Z

Most of you are already aware of the Visual Studio 2005 Community Technology Previews (CTP) and are familiar with the philosophy and benefits of the CTPs – you get working bits more frequently, great opportunity for feedback from you to the product teams (using Ladybug), potential of finding issues earlier than RTM, ability for you to verify requested fixes and lots of other good stuff. We are working on striking a balance between releasing CTPs frequently, the amount of time we spend verifying their quality and their effect on the main product schedule.

We have now released the July CTP, which is available to our MSDN subscribers. The SKUs already released in the July CTP includes the VS Pro, VS Std, VSS, VSTO, VS SDK and VS PPE. VSTS and VSTF will be released next week.

So, what really happens behind the scenes before you get the bits? Basically, a group of people work together and start cranking the CTP machine J Just to give you an idea, let me walk through my life as a PM on the release team before the release of Phase 1 of the Visual Studio 2005 July CTP, which was scheduled to release in 3 phases to account for the large number of SKUs:

Phase 1: VS SDK, VS PPE, VS STD due on July 8th

Phase 2: VS Pro, VSS, VSTO due on July 15th

Phase 3: VSTS, VSTF and VS SDK (compliant with the phase 3 bits) due in late July

June 28^th- 29^th:

Phase 1 work starts a week in advance. I started to scout the Main branch to select a build that would be released as our July CTP. Some people ask “A week early? Can we not just wait until July and then pick a build a day before it is released?” While that is the ideal case, we want to make sure that our customers get a minimum quality of bits. To do this, we need to allow sufficient time (as you will see in the rest of this blog) for install and uninstall testing as well as media verification (For example, virus scans need to be run so that you don’t get viruses as part of the download). To gauge quality, I look at each build to see if it had passed the Integration tests that we run daily on all builds. In this case, Murphy’s law prevailed - the Main lab, which was doing just fine the week before, suddenly had some build issues so I had to wait for those to be resolved and wait for a new build to be generated.

June 29^th- July 1^st:

- Selected a build as the CTP build since it had passed all the Integration tests.

- Created a save request for the build as new daily builds routinely replace older builds.

- Discussed SQL Server Express dependency with the Yukon team to determine the build of SSE to ship with the July CTP.

July 2^nd:

Came into work (thanks to an issue with my smart card not allowing me to VPN into the corporate network) to request actual media to be burned for the CTP. This will ensure that the requests are seen by whoever first walks into the burn lab on Tuesday. Media needs to be burned for purposes of QA who test whether the images on the bits (actually being posted) match the bits of the CTP build on the build drop server.

July 5^th:

10:00 AM – So, our detection logic is causing SQL Server Express installation to look like it failed (as the post-install check looks for the version the build was generated with). Decide to push this into Known Issues that accompany the download as the install actually succeeds but only looks like it has failed. Also, its too late now as the media is being burned as we speak.

July 6^th:

-Quickly hand over all the media to the test team and create media verification requests for all the SKUs.

-4:00 PM - Test verification finds that the VS Pro un-install fails! After remembering to breathe, I find the Setup devs - Is there a fix? Can we investigate and fix it fast enough to turn around and ship this thing? So, I move the Pro SKU to ship with Drop 2 so that we can get the fix into the build.

- Request MSDN to stage the bits so that the bits will be available when we want to go live as this step usually takes 2-3 days for all the bits to propagate.

- Created a request to post the public symbols on the Internet server so that customers can debug their applications

July 7^th:

Create internal CTP site so that folks have one place to view everything about CTPs and their schedule. Provide MSDN with all the documentation they need and the marketing blurb and give them the go-ahead to post the bits.

July 8^th:

The bits are live on MSDN! Drop 1 goes out to our customers right on schedule and I periodically check MSDN to ensure that the bits have been replicated. But, what is this – the content looks a little different than what I sent out. Where are the Known issues? I picked up the phone and called the MSDN contact and turns out the content was indeed different. A quick change and a couple of emails later, the content looks good. I then send out a status email to the whole Division informing them of the same. Start to work on Phase 2………

Known issues (July CTP - Drop 1 and 2):

While we haven’t released an official “Known Issues” document and these are also published as Notes in the download location, I wanted to post these here:

1) At the end of the Visual Studio 2005 installation, it might indicate that SQL Server Express is not installed, even though SQL Server Express was installed successfully. Look at Add/Remove Programs to confirm that the SQL Server Express has been installed.

2) Installing on Win2k3 gold will fail until you install MSI 3.1

As you can see, there is still a long way to the realization of our vision where we are releasing builds on a daily basis. We are constantly thinking about ways in which we can streamline these processes and automate as much as of them as we can. But, we are constantly improving on releasing bits “over the wall” to you in a predictable, consistent manner.

Thanks,

Kavitha

Whidbey RTM Tell Mode!

2005-06-22T01:11:00Z

On Monday (6/20) the Platform teams (those teams shipping in the .NET Framework Redist) entered DevDiv Tell mode, I see this as a very exciting time of the product cycle! It is one where you can see the light at the end of the tunnel, while at the same time every decision/bug fix gets greater scrutiny as we approach shipping a great product with Whidbey RTM. Tell mode is the first of four stages at the end of the RTM milestone (below I have broken out each stage). Each stage has a higher triage/bug bar than the previous stage as well as increased process and QA coverage in an effort to prevent regressions.

Stage 1: DevDiv Tell Mode: At the division level we will run a daily DevDiv shiproom. Tell mode is going to be 4 – 6 weeks. The focus of tell mode will be in establishing the triage/bug bar, understanding team’s bug trajectory, and moving teams towards a single branch. During shiproom we will be using the triage/bug bar as a guide, which we will continue to refine as we progress through Tell/Ask mode. Some of the data we will discuss at DevDiv shiproom:

Examples of approved/rejected (including tough calls and/or border line bugs) bugs. The focus here centers on the customer scenario and the fix in order to help level set the triage bar for teams across the division.
Have there been any major regressions from previous milestone or from Everett?
Team’s trajectory
- RTM Active bugs
- Bug fix rate (how many bugs were approved for checkin today)
- Incoming/resolved bug rates
- Weekly step down goals
- Bugs resolved but not yet closed

Stage 2: DevDiv Ask Mode: DevDiv shiproom will continue to meet daily. During Ask mode we will further define and apply an even higher bar for the bug fixes we take into Whidbey RTM. All checkins to Whidbey RTM must be approved by DevDiv shiproom.
Stage 3: DevDiv Endgame: This is where the bar is raised to recall class bugs only.
Stage 4: DevDiv Escrow: This is a period of bake time, there should not be any Whidbey work or QA planned during this period of time. To be exact, “we are done with the product” and QA has signed off on the release

As we reach Ask mode I will provide more details on how we are progressing and the focus for the other 3 stages.

Thanks!
-Mike

Whidbey Security: Pen Testing Engagement

2005-06-04T00:01:00Z

I mentioned in my last blog that we’re heading towards a first step in the "Final Security Review" for Whidbey. A couple of weeks ago, our central security team finished a 9 week long Penetration Testing (pen-testing) effort of Whidbey. They had 8 people working just on our product. We began by doing "education" sessions for the security testing team, spending 1-3 hours on each area of the product. Based on these sessions and their expertise and experience, the testing team chose a number of areas to focus on. With a product as large as ours, it was impossible to cover everything. Remember that this is somewhat of a safety-net review. We have done extensive training and invested a lot of time in testing and reviewing our product ourselves.

During those 9 weeks, the test team sent out a report each week with the areas they have covered, and the issues they logged. They defined severities for issues they logged, 1 being "Critical" and 4 being "Low". Each issue has a description of their concern, and was assigned to the owner team. The agreement was that if they hit multiple critical issues in a component, they stop their testing and require that component to re-confirm their security work. The good news is that this didn’t happen. Not only that, we had no critical issues logged at all!! Now, obviously, this is not a full proof guarantee, but it’s another confirmation that we’ve done the right things.

Overall, they logged 13 "important" issues, 8 "moderate" issues and 9 "low" issues. Right now, the teams are analyzing the issues and forming their recommendations. But again, I’m really happy with this result. On a product the size and complexity as ours, these are really good results.

Our next step is to apply for the SWI Final Security Review signoff. This is being done through a sort of documentation tool. I will use the tool to document that he have met the requirements of SDL (and a few others that we imposed on ourselves). The documentation involves supplying the team with "proof" like threat models, results of tools run on our product etc. One thing to remember is that throughout the engagement with SWI, we had a member of their team be a virtual member of our security group. This helped us be always in sync with the requirements and make sure we don’t miss any requirements. Right now, I fully expect us to pass with flying colors.

In parallel to this sign off, I’m working on publishing our release criteria for security. For the most part, it is the same as our Beta2 criteria. Since Beta2 allowed customers to "go-live" with it, we could not afford to hold off any security work for post that release. We do, however, are constantly improving our tools, and from time to time we identify new potential attacks that we want to ensure we are not vulnerable to. So I have a couple of tools (FxCop and PreFast) that we’ve improved that I need to set the bar for. There were also another three requirements that our SWI buddy sent us that we have to factor in.

Another effort that I’m working on separately is getting our division to establish a dedicated security team. Basically, replicating a little SWI of our own. If we had dedicated pen-testers, we could go into a rotation model and ensure coverage for the whole product. We could also have them participate in the early phases of design to provide expert advice on security. I’m getting support for this, now we just need to find the right people.

I think my next blog on this will be when we have FSR sign off. I can tell you all about how we managed the late-coming changes and what lessons we’ve learned.

Whidbey Beta 2 “Broad Go-Live”

2005-04-18T18:02:00Z

One of the things we are doing in Whidbey Beta 2 is providing a broad Go-Live License agreement. Some of you may remember we provided an ASP.NET Go-Live License with the Beta 2 release of the .NET Framework 1.0, from an old blog post by Scott Guthrie. We view this as a terrific way to help enable our customers to get Whidbey now. This has already been a long product cycle, and our customers don’t want to wait any longer to get their hands on Whidbey. With the broad Go-Live we are giving customers that opportunity!

What products have Go-Live rights?

Visual Studio 2005 Beta 2 family (excluding Visual Studio Team Foundation Server)
.NET Framework 2.0 Beta 2
.NET Compact Framework Beta 2
SQL Server 2005 Express Edition April CTP
(a complete list of products will be included in the Go-Live License).

What does Go-Live mean? It basically breaks down like this…

Internal Deployment, a customer can build an application based on Whidbey Beta 2 and deploy that application in an internal production environment
Web Applications, a customer can create a web application targeting Whidbey Beta 2 for the purposes of hosting that application accessible by a third party
ISV deployment, an ISV can build an application with Whidbey targeting the .NET Fx redist, J# redist, the .NET Compact Fx, or SQL Server 2005 Express Edition (important note, we are not giving ISVs redist rights as part of their applications). In order for a customer to use that ISVs application they will need to agree to the Go-Live License agreement and get the redistributable from Microsoft.
Native & Managed, the Go-Live covers both native and managed applications.

Important Notes:

Does Go-Live mean I have redistributable rights? No, as part of the broad Go-Live we are not providing external redistributable rights in Whidbey Beta 2. Although, an ISV can create an application targeting… say the .NET Framework 2.0 Redistributable…, provide that application to a customer, and inform their customer that they need to go to Microsoft to accept the Go-Live License and download the required redistributable in order to be able to run that application.
Will I get product support? No, Microsoft will not be providing support for Whidbey Beta 2. Although, informal community support is available through online forums, such as newsgroups, blogs, etc. A good resource for this information is http://msdn.microsoft.com/community/.
Will I get servicing support? No, Microsoft will not be providing any patches, updates or other fixes.
When/how do I Sign up for the Go-Live License? The Go-Live agreement is available now: http://lab.msdn.microsoft.com/vs2005/golive/.

Whidbey Beta 2 Product Metrics/Release Criteria sign-off!

2005-04-18T17:58:00Z

Last I blog’d I talked to the Product Metrics/Release Criteria… and now we’ve released Whidbey Beta 2! It has been a long but satisfying road… As you can see from Chris blog below “In the home stretch…” he mentioned stress was going to be a wild card. He was right, which is why Beta 2 took just a little longer than expected.

Over the final few months of Beta 2 we met on a daily basis in shiproom to discuss release criteria, bugs, fixes, stress, builds… With Beta 2 having a “go live” license there was no way we wanted to release Beta 2 before it’s time, before meeting our quality goals. I think that was the right choice and I hope are customers agree!

As we roll from Beta 2 I am in the middle of reviewing our Whidbey RTM release criteria. I will again be holding reviews with all the release criteria owners. The main topics this time around will be about how we did in Beta 2, defining any necessary changes, setting checkpoints, and building a great quality RTM product!

Now go download and start using Beta 2!

In the home stretch....

2005-03-15T20:30:00Z

Wow! It’s hard to believe but we’ve completed 5 weeks of “Ask Mode”. Every day for the past 5 weeks, representatives of each product team in the Division have met daily for 2 to 3 hours reviewing each change that is going into the product. We look first at the customer scenario that is being fixed, we look at why the bug happens (regression, test hole, coding error, etc.), and then we look at the source code changes. It feels like we’ve read every line of source code (Visual C#, Visual Basic .NET, C/C++/MC++, and even a little Assembly for fun!) across the CLR, Framework, and Visual Studio. The teams have done a great job keeping the quality of the Beta 2 tree high and we’ve had excellent build just about every day for the past 5 weeks.

We’ve got about 3 more weeks in March and less than 400 Beta 2 bugs to manage. For a product this large, a few hundred bugs is a very small number if you consider 30+ teams and each investigating (but not necessarily fixing) around 10 apiece. It is all about maintaining stability for Beta 2 right now and driving our Stress programs to hit their criteria. We are moving into “recall class” only bug fixing this week, except for Stress, which we want to continue to take fixes for. Once we do the final build we will need a little time to produce media and verify it across the various SKUs, formats (DVDs, downloads, etc.) and platforms.

Overall, I feel good about our ability to get Beta 2 out by the end of the month, with Stress being the "wild card". At the same time, my Inbox is full of questions about the InfoWorld article last week on the “slip” of Whidbey Beta 2. Unfortunately, it looks like the Microsoft person who made the comments about Whidbey was misquoted. He stated that Beta 2 was on track for the end of March, with general availability in early April, which is completely accurate. Once you produce the media, it then takes time to duplicate it and get it out to customers, which means the first week or so of April. The Express SKUs and .NET Framework redist will be generally available within a day or so of signing off on Beta 2.

Again, things are looking up, but we're still not hitting our Stress criteria yet, which we will hold the release for until we do. More updates the closer we get to shipping Beta 2...

Starting Ask Mode!

2005-02-08T01:56:00Z

Check off another mini-milestone for Whidbey! Beta 2 Tell Mode is over, and we started "Ask Mode" today. Over the course of the last week, we finished up a round of what we called “aggressive” integrations of the source code branches and are currently in the process of moving to a single Beta 2 branch (we had a tremendous group of dedicated developers in the build lab all weekend). Once all the labs are up into the Beta 2 branch, we begin the stabilization process there and begin fixing bugs in earnest on the live branches.

To recap our Beta 2 schedule (based on Fridays):

ZBB: 11/19/2004
Security Push: 11/15/2004 through 01/14/2005
Tell Mode: 01/17/2005 through 02/04/2005
Ask Mode: 02/07/2005 … until we meet our release criteria, which defines all of the various quality measures for Beta 2.

To be exact, we finished Tell Mode on Friday, but have two teams that need through Wednesday of this week to reduce their bug backlog a little more. So, everyone went into Ask Mode on today except these two teams. We’ll take one last integration of their sources for Beta 2 on Wednesday evening, and then they will be in Ask Mode too.

So starting today, every bug that a team wants to fix for Beta 2 needs to be brought to ship room for approval. The first question asked is “what is the customer scenario?” and if we decide that the benefits sufficiently outweigh the risks, we’ll ask the team to go and check in the fix. Otherwise, they’ll move the bug to the next milestone and fix it in the live trees. We’ll probably batch up a few fixes early this week as we get the Beta 2 branch building. We typically lock the trees from any check-ins so we can stabilize off a non-moving base. Once we open the trees, we’ll take all the approved fixes into Beta 2 and begin building every night. My expectation is that we’ll have solid Beta 2 builds that we can hand off to QA by Thursday or Friday of this week.

Whidbey Beta 2 Tell Mode

2005-01-31T10:13:00Z

Two weeks ago, we entered a phase of the Beta 2 product cycle we call “Tell Mode”. This is the point which marks the transition to the end game for Beta 2. Here’s what has happened so far, and an idea of what’s to come for Beta 2.

ZBB: 11/19/2004
Security Push: 11/15/2004 through 01/14/2005 (rest assured, these are not the only weeks we thought about Security in Whidbey! We spent this time doing concentrated efforts and implementing new processes which are now a part of our daily activities. The push let us take Security to the next level.)
Tell Mode: 01/17/2004 through 02/04/2005
Ask Mode: 02/07/2005 … until we meet our release criteria which defines all of the various quality measures for Beta 2.

Tell Mode is when the Division begins to meet daily. Every team reports on the number of bugs fixed the previous day, with those fixes to be available in the current day’s build. The idea behind Tell Mode is to start the shut down process. Teams move into a centralized triage process, where every bug that comes into the product team is scrutinized as to whether or not is should be fixed for Beta, moved to the next milestone, or punted from the project completely. Over time, we start to look at samplings of bug fixes and begin to set the bar, ensuring that teams are fixing the right set of bugs and that we're positively affecting the quality of Beta 2. This additional process slows the churn on the product because the more fixes we take, the more chances of regressions or build breaks. Our goal is to slow the number of check-ins, stabilize, take the necessary fixes to meet our release quality criteria, give the product a little time to “bake”, and then ship it.

Tell Mode is scheduled to last three weeks, through 02/04. We expect to then move into Ask Mode, where every bug that we want to be fixed has to be approved by our centralized ship room. The first question we ask on every one of these bugs is “what is the customer scenario?” Every fix needs to have a positive, tangible, and critical customer impact or else we don’t take it in Beta 2 at that point. Once we are in Ask Mode, we’ll do another test pass and we’ll finish driving up our quality criteria such as our Stress pass rates.

So far, we’re doing pretty well. We’re running a little hotter than I would like, but all the right things are happening. The next week will really give us a good indication on how well we’re tracking towards our goals.

Chris

Whidbey Beta 2 Tell Mode

2005-01-31T10:13:00Z

ZBB: 11/19/2004
Security Push: 11/15/2004 through 01/14/2005 (rest assured, these are not the only weeks we thought about Security in Whidbey! We spent this time doing concentrated efforts and implementing new processes which are now a part of our daily activities. The push let us take Security to the next level.)
Tell Mode: 01/17/2004 through 02/04/2005
Ask Mode: 02/07/2005 … until we meet our release criteria which defines all of the various quality measures for Beta 2.

Chris

Whidbey Security Push Progress

2005-01-10T21:53:00Z

The past few weeks have been pretty busy for me. In my last blog I mentioned I was working on planning our "Security Push", which is a time we allocate in our product cycle where the top priority and focus is security. Having this kind of time is very valuable because you can get everyone working on this at the same time, and it builds great momentum. We started out with a big list of work items we wanted to complete, some of which were brand new, bleeding edge, using tools we had just finished developing. Talk about dogfooding…

After a wobbly start, everyone really kicked into gear and we started making good progress. This is the fifth week now out of a total of six, and things are looking great. We found a few areas where we needed to change the design to make the product more secure, and we fixed issues reported by scanning tools like FxCop and PreFast (if you want to try this one out, check out the December CTP). We also updated our threat models to include the latest changes, and we’ve reviewed a ton of source code.

In a month or so, our central security team (for the company) will be engaging our team for our "Final Security Review" (see Soma’s blog about the Security Development Lifecycle). Every product shipped by Microsoft needs to go through one of these and can not ship without passing it. The central security team is a group of security experts who verify that a product meets the security bar defined for MS products (listen to Mike Howard talking about this in his MSDN TV episode). During February and March, they will be focusing on testing our product. It’s always useful when outside eyes are looking at your product, they may find things we somehow missed.

My next blog will be soon after they complete the testing of our product, I’ll keep you posted how that went.

-Natalie

Yukon Bug Progress…

2004-11-16T09:47:00Z

SQL Server 2005 (code named “Yukon”) is built on the Whidbey platform. The integration of the Whidbey .Net Framework with the SQL server engine is well known; this provides the “SQLCLR” functionality that enables managed code stored procedures, user defined types, aggregates and functions. What’s less well known is that the Yukon administrative and business intelligence (BI) tools are largely written in managed languages and heavily leverage the Whidbey .Net Framework. In addition, the Yukon tools incorporate VS Whidbey components and several of them are implemented as VSIP applications that plug into VS. Yukon developers are daily users of VS Whidbey and the SQL buildlab utilizes Whidbey compilers and tools to build Yukon on a daily basis. It would be fair to say that the SQL division really cares about Whidbey progress and quality.

With so much exposure to Whidbey, it should come as no surprise that the SQL division reports Whidbey bugs. For much of the product cycle we’ve seen a slow but steady decline in the total active Yukon originated Whidbey bug count. The incoming rate varies with SQL's transitions to newer Whidbey builds and QA activities (bug bashes, stress runs, etc.) and we’ve been consistently resolving bugs at a slightly higher rate. Whilst the total active number was never very large, as we close in on Whidbey Beta2, our fix rate has increased and we’re down to a small number of remaining Yukon originated bugs. Most of these are SQLCLR stress related bugs, which is not unexpected at this point, with teams focusing on meeting stress and other release criteria goals. We know we can count on a bug flow from stress runs to Whidbey teams until release criteria are met and Yukon is ready to ship

Yukon's Whidbey exposure is clearly good news for customers. Having a Microsoft division the size of SQL pounding on Whidbey and probing its limits helps smooth the way for customers to extract great results from Whidbey when it ships next year.

-James

DCRs and Breaking Changes

2004-11-13T01:35:00Z

Have you ever been asked to change something when you felt you were done? It happens; customers provide feedback, security reviews reveal vulnerabilities in a design…there’s always a reason for a change. A “DCR” is a Design Change Request and are designed to track these types of change requests to the original design. Often DCRs will come in after features have been coded and when teams dive into their test passes and get lined up for release.

As we get closer to release (either Beta2 or RTM) we look more carefully at these DCRs to ensure that the reason we’re doing them is a good one and worth the risk to our deliverables and our dates. When I say “our,” I mean “our” in terms of a group, not just a single feature team. This is where the second topic comes in, “breaking changes.” Breaking changes are typically something that customers are concerned with when a new version of a product comes out; say the migration from v1.1 to Whidbey. However, within Microsoft, we have similar issues moving from build to build. Intra-cycle (Whidbey -> Whidbey) breaking changes are always a concern as teams react to the changes, but they are particularly risky during the push to a Beta or RTM, where the focus should be on solidifying the product (fixing bugs) and not reacting to unnecessary changes (e.g. an API or function behavior change.)

As part of our DCR process, we’ve included a check for intra-cycle breaking changes. Making certain that these changes are necessary helps us to deliver the right product, with the least risk to the project and with the maximum benefit to the team involved. Also, by performing reviews on these we’re able to track them better for our customers. Understanding what is changing between “drops” of the .NET Framework allows our partners to better anticipate and plan for any impact associated with moving to a new version and (hopefully/ultimately) allow them to focus more on delivering their feature and value, rather than deal with an integration that is more expensive that in needs to be.