Randy Miller's Blog

Managers and Makers

RandyMiller — Thu, 17 Dec 2009 15:14:00 GMT

A little over 20 years ago, I started working for the other large software development company. They had recruited me from college and I was honored to join them. At the time, the perception at my school was that no one got to work for them because everybody wanted to. In fact, many of my fellow students said, don't even submit your resume because it will end up in the circular file. I was thrilled when the company interviewed me and even more thrilled when they hired me.

Life at that large company was interesting. I started working on ECF, a mainframe to PC data transfer system written in OS/370 and later in a dialect of PL/1. When I started, I was clearly out of my league. The veteran programmers schooled the young college graduate (me) and I worked really hard to keep up with them. At some point though, working through lunch and into the night seemed to have paid off. I was one of the few "kids" on a really tough project but the veterans started to give me cudos for my work.

On my next project, we had a project manager who liked to meet to discuss status. It didn't matter whether it was 15 minutes or two hours, the meetings always seemed to interrupt when I was debugging a vital piece of code or writing something complex. Additionally, this large company had a lot "all hands" meetings which always seemed to come right before my deadlines. I would work late into the night to make up for a day where executives would spend the day BSing. Most of the time, they would tell us things like “If you don’t like working for this company, leave!” I was too busy building products to think about these things but the meetings started to give me ideas.

But the schedules had to always be made even if I lost an entire day to one of these meetings. I realized that as a coder, my priorities and focus was different than theirs. Peter Coad gave me the first clue when he said that every interruption to a coder required, on average, fifteen minutes for a programmer to get back “into the flow”. In other words, interruptions impact programmer productivity. And they impact the productivity greater than the length of the interruption.

Last night, I was talking about this with Brian Crissup. As a veteran programmer, I get back into the flow much quicker than I did in my youth (seems counter-intuitive). But he turned me onto the idea of makers and managers. The idea is that a maker's schedule is divided into large chucks of time of being “in the flow” to create a tangible element such as software.

A manager’s schedule is divided into one hour chunks (meetings) where information is communicated but nothing tangible is delivered (unless your company sells meeting minutes). When a manager injects an interruption (meeting) into a maker’s time block, it can have an impact far greater than that one hour block. It can destroy the entire large chunk that a maker needs to accomplish something tangible. Thus makers will often work into the night to get that block back while the manager goes home.

Depending on the urgency of the project, this can either be annoying or downright maddening. You see, the maker is responsible for the delivery of a tangible item so they are “on the hook”. While a manager just reports the lack of progress by the maker. As a manager, you get what you need through your meeting and leave the makers to pay the price.

One of my beliefs as someone who is now both a maker and a manager is to endeavor to make reporting painless to the makers. As someone who participated in creating TFS, I had always hoped that work items would allow makers to make and managers to manage without a cost to either one. The idea behind TFS work items and reports was exactly this ideal.

How can the United States (and other countries) stay technically literate?

RandyMiller — Sat, 14 Nov 2009 18:04:00 GMT

Thanks to plastic bricks from Denmark, kids in the United States (and all around the world) are learning to program at age 8. I had the priviledge of coaching a young team of NC kids (9 to 11 years old) to win the NC State Championship in FLL in three years. The Microbots were a dedicated group of kids, none of whom knew how to program when they started FLL. One of these kids quit football to dedicate himself to programming. I tell folks about this program all of the time. If you have a child, aged 8 to 13, and you don't think that he/she will be a pro-football player (wow - the odds are 1:a gizillion) when he or she grows up, get them on the leading edge of technology (oh and he/she can play with plastic bricks too).

OK - so you have an older kid? There is a corresponding program for high school kids. Microsoft has graciously sponsored Team Awkward Turtle, a high school team from Southwest Wake County in NC (four local high schools and the home schoolers too). Check out this robot! High school kids built and programmed this! I spend a fair amount of my time training the next generation to be great!

I recently showed pictures of the team to a researcher from China. He said that they didn't have anything like this happening in their high schools. I say this not to knock on China. I am so sure that it is just a matter of time before they are doing this too. My point is, your sons and daughters will be competing with these kids (from all over the world).

I love watching football and look forward to spending my Monday nights watching the stars but I will feel much better about the world at large if more people were doing this on their weekends like I am.

What is Shadow Architecture?

RandyMiller — Sat, 14 Nov 2009 17:36:00 GMT

Shadow Architecture is method of communicating architecture to others on your agile team. The idea is that components and classes cast "shadows". These shadows are simplified views of the code base. UML is a type of shadow. We can look at a UML model of a piece of code and we can say some things about the code but we know that it isn't "the code". Sometimes though, when we want to teach a new team member where things can be found in our ever growing code base, we draw a quick UML diagram. Anyone see BDUF in this example? No, good.

Another example of where "shadows" come in handy is when we are making an architectural change during an iteration. We draw out what we want the architecture for our code base to look like at the beginning of an iteration (or during it) and refactor the code accordingly. Multiple developers can work on these architectural changes simultaneously since they are all on the same page. But they can't unless they really are on the same page and a "shadow" gives us a place to start from. Of course, constant communication during these refactorings is also a must. You might redraw the architecture when you encounter problems or learn more. Common sense, right?

So we call the first example, the one where we have an existing code base and describe it using a modeling language, a trailing shadow. Trailing shadows are used to communicate what exists in our code. The second example is a leading shadow. We use leading shadows to talk about what we want the code base to look like. Both shadows are descriptions of the code base and should be generated (by hand or machine) in no more than a couple of minutes. Certainly not BDUF.

OK - so here is the most important part. At the beginning of an iteration, we can talk about refactoring the code base architecturally to allow us to add additional functionality (remember Ward's definition). We would quickly provide a leading shadow and get to work, adding the new functionity! As the sun sets on the iteration, the leading shadow should be gone and all that remains is the trailing shadow. Definitely not BDUF if every architectural change that you make spans the scope of an iteration.

To give credit where it is due, I collaborated with Bill Gross (of VSTS Architect) on Shadow Architecture. I believe that he came up with the name. MSF v.4 had this concept (perhaps convoluted as that much of the process seems to have been in retrospect). I also have to give cudos on this to Scott Ambler because it is definitely an extension of Agile Modeling.

Agile Architecture

RandyMiller — Thu, 05 Nov 2009 14:22:00 GMT

In the mid-1990s, the prevailing trend in the industry was to begin a software development project by building elaborate models of the system. The idea was that writing the code was simple once the architectural structure and requirements were in place. However, many software development projects spent large amounts of time building these models only to find that they became quickly outdated as soon as coding began. This practice became known as “big design up-front” (BDUF).

The impact of BDUF was felt on many fronts. First, it alienated developers of the system. After all, coding was merely an afterthought and many leaders in the BDUF community suggested that writing software would eventually require lower levels of skill. Model-Driven Architecture (MDA) promised to make developers obsolete by creating models that would generate systems with absolutely no coding at all. While technological advances were made in MDA, the full promise of "codeless software" was never realized.

Many BDUF projects also failed because these projects expended large amounts of time building the models and left little time for the actual coding. Since the models provided no mechanism for feedback, it was difficult to tell how good they were. Interaction diagrams such as message sequence charts could generate lower level coding structures such as methods and classes but a fully working system could not be created from their outputs. In contrast, code can be compiled to form a system that can be tested and verified.

The agile movement started as a backlash against BDUF. The first agile software development process, Extreme Programming (XP), forbade the use of modeling on projects that used it. Software modeling was considered “un-agile”. Kent Beck, the founder of XP, was determined to break the grip that the BDUF community had on the creation of software. The agile movement was very successful in this endeavor.

Software modeling slowly came back in vogue with the Agile Modeling (AM) movement by Scott Ambler. Scott argued that modeling should be done on whiteboards to communicate architectures and designs during software development. The idea behind Agile Modeling was that models were not high-end software artifacts that should be kept around forever but low-cost work elements that reflected the rapidly changing code base.

As Extreme Programming evolved, Kent Beck realized the value of architecture and found a very simple modeling language that he called Simple Modeling Language (SML) that was similar to the subset of UML that Agile Modeling used. This language consisted of boxes and lines that were used to reflect the components and/or classes and their relationships, the basic construct of object-oriented languages. This language was the simplest thing that could possibly work as a modeling language.

AM and SML identified lightweight approaches to describing the designs of systems but there is so much more to architecture than design. George Fairbanks defines architecture as “The architecture of a system is the design decisions made by system developers to achieve their top-level intentions, and the designs that follow directly from those decisions.” This implies that design elements such those described with AM and SML might be descriptions of the decisions but won’t reflect the alternatives that were discarded. In other words, through a modeling language, we can understand the consequences of the decisions but not the reasons that the decisions were made.

Another definition of architecture comes from Ward Cunningham through conversations that I had with him when he worked for Microsoft. Ward told a story where his brother was building video editing software before Movie Maker and the Adobe ilk. At some point in time, he realized that he could add no more functionality without stepping back and refactoring the system. Ward argued that this was architecture. It was the point in time when a developer had to look at the system as a whole and move it to a desired state where it could be extended.

I have given several “Agile Architecture” presentations including a panel at this year’s OOPSLA in Orlando. In these presentations, I describe my first experience with the planning game. Our team met with the customer to “price” the user stories. The first user story was read and two developers came up with different numbers of story points. We agreed that we would use the larger as a baseline. As we continued, the story point numbers continued to be “all over the place”. In fact, the developers began to disagree with each other in front of the customer.

It turned out that each one had a different architecture in mind and that they would never agree on how hard each story was until they agreed on some of the system characteristics. I use the word “some” deliberately because they have to know just enough to “price” the stories. In other words, the planning game for nontrivial systems (and maybe for trivial systems too) requires that developers be on the same page with “how” the system will be implemented at the highest level. This is NOT BDUF.

I would argue that this understanding is necessary also throughout the development of the system especially when multiple teams are involved (team of teams). I have thus looked at expanding AM and SML through a technique called “Shadow Architecture”.

Programming with Kids and Adults

RandyMiller — Tue, 10 Feb 2009 19:14:00 GMT

It has been five years since I joined Microsoft and started writing MSF. There are a couple of milestones that this anniversary brings that I would like to share with you. First, the MSF book is not published and you have a right to ask why. Second, the robotics team that I have been mentoring (all 11-13 year olds) won the NC State championship and will advance to the National one. All of these kids can program extremely well as they have been working on building and programming robots for three years straight. Third, I continue to see the same negative patterns in (recovery) projects and I have been wondering if there is a way to process some of these patterns out of existence.

Some of you may not be aware that I have been working with two robotics team, one younger and one older. As I was working with the older Robotics team (high school kids), I happened to see a problem that I frequently encounter on projects. We were trying to test the robot in a museum on a simulated ice floor. The code for the drive control system was being finished on the way to the museum which is similar to the way many adult projects are done. When we got to the museum, we couldn't download the code to the robot.

Now, it turns out that we had fried the drive controller by static electricity from the simulated ice floor so it didn't matter. But as I thought about the current project that I was working on, it occurred to me that it had the same problem. One fellow who was working on it was out for the week and had no realistic backup. He had changed some of the code and it wasn't working as intended. We hadn't found the problem until he had left for vacation. It was an honest mistake (and one where he will have a pair from now on). But the delay cost the project.

A stitch in time saves nine.

I look back on the original manuscript for the MSF book and I don't like it. The whole introduction about second generation agile processes needs to go. My thinking was clearly flawed (some would say on several aspects :-)).

We are rewriting a piece of one project that I am working on because we now know more about this area than we originally did. The area works but it isn't elegant and certain bugs are unfixable. We have learned by doing once and then doing right.

But sometimes, like good wines, things have to age.

What has this got to do with programming kids, the MSF book, my projects, and my anniversary? I suspect that all have been influenced by stitches in time and aging. But sorting out the things to age and the things to rush is always the hard part. I suspect that we should always strive to do something and then evaluate how well we did. That is far better than waiting until we get it 100% right.

Where have you been?

RandyMiller — Thu, 28 Jun 2007 03:10:00 GMT

For those of you who do not know - I have left the MSF team and joined the Project Recovery Team at Microsoft (back in January). At some point, I'll explain what we do in Project Recovery but for now, I want to talk about VSTS. I recently posted the following information to a question regarding my experience with running a project and using VSTS. It's one thing to work inside Microsoft on projects where everyone knows how to use VS and TFS. Outside of the hallowed halls, there can be learning curve to using it. Don't let that stop you. The benefits more than outweigh the costs.

Here is my post and my recommendation on how we got started:

"I run a development team at one of the government agencies and just happen to be a former VSTSer. Getting folks up and running is not a trivial task but we held a quick 2-hour 'how-to' session when we started working with VSTS. Most of our developers were able to get going with CM just based upon this session. The trouble came when we had new folks enter the project. Each one requires a twenty minute one-on-one workspace creation/get/edit/check in explanation. I haven’t met a person yet who has experience on VSTS/TFS coming in.

Getting TFS up and running took roughly half a day but the use of multiple domains (many reports do not have the fully qualified domains in the urls) made us hold off on using the full capabilities initially. We had started development at the same time as standing up TFS which was an interesting process. You look at the work of VS differently when using TFS (get all of those slns out of the documents and settings folders). We restructured our TFS tree a couple of times to put all of the pieces in the right place. Your architects should learn to use the command line.

Build can take some effort especially for more sophisticated apps (we are composite with everything from web/Biz Talk/VSTO/C#/VB script/C++/DB Pro). I’d recommend simple scripting to augment team build. Some of these areas required a few of us to dedicate a couple weeks to get all of the respective components built. It looks easy but gets harder as you get into all of the things that VS can do. Work item migration is still in-progress as our test folks had been using another bug tracking system. Many projects start with WIs first."

Writing Good Scenarios

RandyMiller — Wed, 03 Jan 2007 18:32:00 GMT

Discipline	Product Management
Role(s)	Business Analyst
Activity	Write Scenario Description

Short Description

Scenarios are used to define the services that the software will provide to the user. A scenario can be thought of as a path through the system and can be represented by use cases or collections of user stories. Most of the scenarios for a system are written at the start of the project to help scope the work and identify the most important features or services of the system to be developed. It is not uncommon, however, that new scenarios are identified or existing ones modified or deleted as the development of the system proceeds and iterations are completed and previewed by users.

Scenarios are user and product, not technology focused. They are written in the language of the user and can be as short as a couple of sentences or as long as a few pages. Each scenario must stand on its own. Scenarios can be written as a collection of steps that the user performs when interacting with the system and the resulting services that the system performs for the user. When writing scenarios, the prerequisites for a given scenario should be spelled out ahead of time. When writing a scenario, it is often best to name the scenario as something that either describes the intended result or the problem the user wants to solve.

Workproduct Inputs

Vision statement or document
Access to domain expert or end users of the proposed system
Personas of expected users of the system

Workproduct Outputs

A description of a path through the system to be developed
An initial understanding of what the end user for that path expects or needs from the system

Checklist(s)

Writing Good Scenarios, items to consider

Check	Description
□	The language of the scenario is that of the target consumer/end user of the scenario, free of technical jargon that is not related to what they will expect and experience.
□	The scenario is concise and deals with a single path through the system. Variations and alternate flows can be discussed but the scenario is not meant to be a complete documentation of the system.
□	Scenarios are used to form a basis for agreement between the software development team and the end users of the system. They should represent a clear picture to each party of what the system is expected to do, and what business limitations the system should enforce for the particular path through the system
□	You will need to define multiple scenarios to describe a single system and should start with the most important ones to the end user. Not all aspects of a system will be described in the collection of scenarios developed. For example, there may be performance or scalability requirements on the system not reflected in any scenario.
□	Scenarios are used to scope the system and help identify requirements of the system. They do not document all of the requirements for the system. Scenarios are used as an input to the requirements gathering and documentation.

Size and number of scenarios

Check	Description
□	Scenarios can be as short as a few sentences or as long as a few pages. Any prerequisites that must be satisfied for a scenario to be performed must be specified in the scenario.
□	Scenarios can be written in a narrative form, as a collection of user stories, or as a numbered set of steps. They should clearly state the purpose or end goal the user has in mind when performing a scenario.
□	Not all aspects of the system will be documented in scenarios and more scenarios may be identified as the development proceeds. You should start with the key scenarios that will define the success of the system.
□	When possible, it is best to name each scenario with the goal that the user wants to achieve when performing the scenario.

New MSF Class Available For Partners

RandyMiller — Mon, 27 Nov 2006 22:17:00 GMT

Title:

Managing your teams and business process using Microsoft Solutions Framework (MSF) as a template

Abstract:

Microsoft® Solutions Framework (MSF) is a deliberate and disciplined approach to technology projects based work and organization based on a defined set of principles, models, disciplines, concepts, guidelines, and proven practices from Microsoft, our clients and our partner community.

This class introduces MSF concepts, provides an overview of its foundational principles, core models, and essential disciplines and then focuses on how their application contributes to the successful running of a services business.

What is Microsoft Solutions Framework (MSF)?

Creating meaningful business solutions on time and within budget requires a proven approach. Microsoft Solutions Framework provides an adaptable framework for successfully delivering information technology solutions faster, requiring fewer people, and involving less risk, while enabling higher quality results.

MSF helps teams directly address the most common causes of technology project failure in order to improve success rates, solution quality, and business impact.

Created to deal with the dynamic nature of technology projects and environments, MSF fosters the ability to adapt to continual change within the course of a project. MSF is called a framework instead of a methodology for specific reasons. As opposed to a prescriptive methodology, MSF provides a flexible and scalable framework that can be adapted to meet the needs of any project (regardless of size or complexity) to plan, build, and deploy business-driven technology solutions.

During this 2-day session, we will discuss how to apply the MSF team and process models to your partner organizations to drive better quality, efficiency and profit

The core focus areas are:

1) Building a team

2) Definition and adoption of the MSF process (delivery) model

3) How to properly manage risks

4) Organizational Maturity overview

5) Tools and processes to help drive better profits and reuse

Agenda:

Day 1:

1) MSF Overview

2) MSF Process Model Overview and its application

3) MSF Team Model and its application

Day 2:

4) Risk Management

5) Organizational Maturity Model (Carnegie Melon SEI)

6) Tying it all together

Class Duration: 2 Days - (9am to 5pm) each day

Target Audience: Microsoft Partner Community

Prerequisites: NONE

Registration: http://www.msreadiness.com/IL_abstract.asp?eid=5014632

Dates:

Date/Time	City/State	Partner Price
Dec 11 - 12, 2006 9:00AM-5:00PM	Irving, Texas	$150.00
Jan 17 - 18, 2007 9:00AM-5:00PM	Orlando, Florida	$150.00
Mar 6 - 7, 2007 9:00AM-5:00PM	Irvine, California	$150.00
Mar 19 - 20, 2007 9:00AM-5:00PM	Atlanta (Alpharetta), Georgia	$150.00
Apr 16 - 17, 2007 9:00AM-5:00PM	Chicago, Illinois	$150.00
May 1 - 2, 2007 9:00AM-5:00PM	Denver, Colorado	$150.00
May 14 - 15, 2007 9:00AM-5:00PM	Waltham, Massachusetts	$150.00
Jun 4 - 5, 2007 9:00AM-5:00PM	Bloomington, Minnesota	$150.00

MSF for Agile Software Development and MSF for CMMI Process Improvement Version 4.1 Available

RandyMiller — Wed, 15 Nov 2006 19:02:00 GMT

Yesterday, we released brand new versions of MSF for Agile Software Development and MSF for CMMI Process Improvement. These new versions create the first commercial agile and CMMI-based software development processes with support for database unit tests and refactoring. The release of these processes coincided with the release of our tooling support for these practices, Visual Studio Team Edition for Database Professionals. These new versions also contain several bug fixes and clarifications.

If you are using the Visual Studio Team Foundation Server, here are the instructions to update the MSF versions.

Instructions

If you want to view the guidance

1. Download and extract the contents of the zip file to your local machine. (select Use folder names).

2. In the extracted files, review the Readme.

3. To view the content, open \Process Guidance\ProcessGuidance.html.

If you want to update the guidance in a process template

1. Download and extract the contents of the zip file to your local machine.

2. If required, customize process guidance using instructions available at http://msdn2.microsoft.com/en-us/library/aa730855(VS.80).aspx.

3. Download the process template you want to update guidance on from your TFS server to your local machine.

4. Replace “Windows SharePoint Services” folder in the process template you downloaded from TFS with the one you obtained from the web download.

5. Upload the updated process template back to TFS.

6. When you create a new project using this updated process template, the portal will show the updated guidance.

If you want to replace the guidance already on the SharePoint server

1. Download and extract the contents of the zip file to your local machine.

2. Go to the home page of your project portal.

3. Click Documents and Lists, and then "Document Libraries" in the left menu.

4. Replace all files and folders in the following Document Libraries on SharePoint site from their corresponding folders under "Windows SharePoint Services" folder in the process guidance you downloaded.

1. Project Management

2. Requirements

3. Security

4. Test

5. For the "Process Guidance" Document Library, replace the following files and folders with the new content

1. Folder – Supporting Files along with all the subfolders and files

2. File – ProcessGuidance.html

3. File – version.txt

Changes to MSF for Agile Software Development

RandyMiller — Wed, 11 Oct 2006 18:57:00 GMT

A common question that I get is "What has changed in the latest version of MSF for Agile Software Development?" There are two reasons that this question is asked. First, people want to know why they should update their projects and secondly, they want to know what files have changed. This blog entry will attempt to answer both questions.

To start, we have tried to clarify a number of the concepts that were being misconstrued by poor wording or organization of the content. These concepts include the role of the Project Manager in iteration planning (they are a facilitator) and the use of Test Driven Development (earlier versions slated this as optional).

As for which files have change, the answer is all of them have. However, only a dozen or so have changed in a meaningful way. The rest have the new copyright and release information. Choose from the list below based upon you needs. I recommend adding files for the Plan an Iteration Collaboration. Check out the files from your project portal and replace them with these new ones.

Plan an Iteration Collaboration

architect.htm (clarified collaboration on plan an iteration workstream)

architect_plananiteration.htm (new file - clarified collaboration on plan an iteration workstream)

architect_showall.htm (clarified collaboration on plan an iteration workstream)

businessanalyst.htm (clarified collaboration on plan an iteration workstream)

businessanalyst_plananiteration.htm (new file - clarified collaboration on plan an iteration workstream)

businessanalyst_showall.htm (clarified collaboration on plan an iteration workstream)

developer.htm (clarified collaboration on plan an iteration workstream)

developer_plananiteration.htm (new file - clarified collaboration on plan an iteration workstream)

developer_showall.htm (clarified collaboration on plan an iteration workstream)

dividequalityofservicerequirementsintotasks.htm (clarified how the division gets done)

dividescenariosintotasks.htm (clarified how the division gets done)

plananiteration.htm (new collaborations)

tester.htm (clarified collaboration on plan an iteration workstream)

tester_plananiteration.htm (new file - clarified collaboration on plan an iteration workstream)

tester_showall.htm (clarified collaboration on plan an iteration workstream)

workstreamsindex.htm

Test Driven Development

fixabug.htm

Broken Link Fixes

brainstormqualityofservicerequirements.htm (fixed broken link to ‘howto’)

brainstormscenarios.htm (fixed broken link to ‘howto’)

buildaproduct.htm (fixed broken link to ‘Team Build’)

closeabug.htm (removed extraneous links)

createorupdateunittest.htm (add link to TDD information)

developlefestylesnapshot.htm (fixed broken link to ‘howto’)

fixabuild.htm (fixed broken link to team build)

mitigatearisk.htm (removed link to MSF home page)

qualityofservicerequirementlist.htm (fixed broken link to actual work products)

reviewobjectives.htm (fixed broken link to scenario list)

scenariolist.htm (fixed broken link to actual work products)

selectandrunatestcase.htm (added link to test case management)

startabuild.htm (fixed a broken link to help system)

testaqualityofservicerequirement.htm (added link to test case management)

testascenario.htm (added link to test case management)

Added Queries

cycles_dailybuild.htm (added queries for this area)

cycles_iteration.htm (added queries for this area)

projectmanager_guideiteration.htm (added queries for this area)

projectmanager_plananiteration.htm (added queries for this area)

projectmanager_showall.htm (added queries for this area)

tracks_build.htm (added queries for this area)

Additions to the glossary

glossary.htm

New Mindsets

mindsets.htm

principles.htm

teammodel.htm