Software Engineering, Project Management, and Effectiveness
This conversation (er, debate) comes up a lot. What's the difference between performance, load and stress testing? I'm sure there's tons of *official* definitions. At the end of the day, I think about them like this:
I could say more, but sometimes less is better. If you want to read more, check out our patterns & practices Performance Testing Guidance Project on CodePle or browse through Scott Barber's exhaustive collection of performance testing articles.
The Team Foundation Server Branching Guidance whitepaper is now available! It's a comprehensive whitepaper that covers strategies, patterns and anti-patterns for branching and merging with TFS. You can view the branching guidance online or you can download the PDF version.
Branching Guidance Index
My team works closely with Graham and Mario of the Branching Guidance team as we build out our patterns & practices VSTS Guidance Project. We're sharing learnings and synchronizing recommendations as we go along. We'll be adding more cross-references to the Branching Guidance Project and VSTS product documentation on MSDN from our guidance so you can easily hop for more information.
Here's some quick blogging tips I shared with a colleague, that they found helpful:
Today I was reminded of the powerful scenario for building a custom set of guidance on the fly using Guidance Explorer. One of the scenarios for Guidance Explorer that's probably not well known, is the ability to generate MS Word documents. You can also save to HTML or export to XML. The idea was that you could build a custom set of guidance by grabbing the guidance modules you wanted.
In my case, I needed to quickly create two documents -- a set of ASP.NET 2.0 security guidelines and a set of ASP.NET 2.0 security checklist items. To do this using Guidance Explorer, I took the following steps:
This gave me a single Word document of the ASP.NET 2.0 security guidelines with an index up front and the details in the doc. I repeated the steps to create a set of ASP.NET 2.0 security checklist items.
If necessary, I could have tailored the guidance before creating the document. Another feature that's not well known is that you can use Guidance Explorer as an authoring tool. You can quickly modify the content of guidance modules and then save to one of your read-write libraries.
IT Security posted their list of The 59 Top Influencers in IT Security. They say their list includes "...most influential security experts of 2007 - from corporate tech officers and government security types, to white hat hackers and bloggers." I don't get caught up in whether it's the right list or complete list. Instead, I use the list to look for names that I don't recognize to see who might have new ideas or thoughts I should explore.
On the Microsoft side, I like to browse the following lists to see what our security-minded community is up to:
Building software involves a lot of communication. Behind this communication, lies perspectives. These perspectives often get lost somewhere between initial goals and final product, which can lead to failed software. I found that by using a simple Perspectives Frame, I improve my chances for success.
In PracticeI could easily over-engineer it, but in meetings and hallways, this quick, memorable frame of four categories helps. OK, so it looks simple enough, but how do I use it? Here's how I use it in practice:
This perspectives frame becomes even more powerful when you combine it with MUST vs. SHOULD vs. COULD and What Are You Optimizing.
Whether I'm dealing with software requirements, or I'm prioritizing my personal TO Dos, I think in terms of MUST, SHOULD, COULD. It's simpple but effective.
Here's an example of some scenarios and usage:
It's easy to get lost among SHOULDs and COULDs. I find factoring MUSTs from the SHOULDs and COULDs helps get clarity around immediate action.
Here's an example of a mistake I made tagging to illustrate how I'm now thinking about tags. I originally created the following three tags for my team system nuggets: Visual Studio 2005, Team System, and Team Foundation Server. I did this because I first did my research on Visual Studio Team System tags and found that's what Technorati was using. I figured by covering my bases, users would find what they might otherwise miss.
The problem with this approach is I no longer had a single big bucket to show me all Visual Studio Team System posts. This approach is also error prone (tag a post with two out of the three buckets). Worse, I was cluttering my tag cloud, without adding value.
I then adopted the approach of thinking in big buckets first. For now, all my team system related posts will go into my Visual Studio Team System tag. If I need to chunk that up, then I'll add a tag for my team foundation server posts. If I need to further divide then I'll add tags for sub-buckets like source control, reporting, work items ... etc. I'll also draw from my research on Visual Studio Team System tags.
I should point out that while now I think in terms of big buckets to small, I like the fact that I can also jump right to the smaller bucket. However, I also know that I can count on one big bucket to contain all my smaller buckets.
I'm not satisfied with the browsability of my blog. While I can get to a lot of the nuggets I need, sometimes I have to dig.
My initial reaction was that I just need to throw all my nuggets into a Wiki and do what I do best. Then I realized, no, I'm making a very basic mistake. True, blogs are oriented around time, but there's a lot I can do with tags. I simply need to make the most out of what I've got, before I take another path.
There's a few things I need to do:
I do have a few other rules of thumb that guide me. Rather than make a bunch of buckets up front and then wonder how to fill them, I prefer to make them as I need them. Also, even though I'm adding a little more focus to being able to walk my categories, I realize there's many paths, and part of the power of tags is more about "related item" discoverability, than actual hiearchy.
One of the metaphors I use to explain the distinction between documentation and guidance is Driver's guide vs. Owner's Manual. While I could go into the finer details, it's a good starting point. From an owner's manual, I expect to see how things work and how they're intended to be used. From a driver's guide, I expect "how to get the most out of it."
I see the two bodies of information as very complimentary. I also see them as distinct. I wouldn't want to mix my driver's guide with my owner's manual. However, I do want to be able to seamlessly go from one to the other, when I need to. I also want my owner's manual written by the people that built it and I want my driver's guide written by the people who use it in action.
In practice, I use my owner's manual when I care and tune my RV. When I take a cross country trip, I use my driver's guide. Knowing this distinction helps me choose the right tool (information set) for the job, as well as set my expectations about the type of information I'll find.
I think finding the right metaphors is important because it helps illustrate a distinction that's not always obvious or hard to explain. I don't think guidance is yet a pervasive part of our technical landscape, and yet I see it as a key differentiator between success and failure. By pervasive, I mean I can use any product or technology and easily find the driver's guide. I mostly see owner's manuals.
How do you share code in Team Foundation Server? That's what our team is working through at the moment. We're looking at what's working, what's not working, and what should customers be doing.
Here's how we're basically thinking about it so far:
Here's what seems to be our emerging guidance:
The problem with workspace mappings is that they're developer specific. Each developer will need their own mapping. You'll also need to lock down permissions to avoid accidental changes. Branching has the advantage that you can be explicit about taking changes, so you have stable builds but with the overhead of merging. You can branch within the same project or cross-project. A separate project might make sense if you have multiple projects consuming the code.
I need to still look across more customer sets, but so far I mostly see binary reuse.
I'm particularly curious in any lessons or insights those of you would like to share. I think this is an important area for effective source control practices.
I'd like to share some of the insights that others have shared with me over the years about choosing paths. My favorite insights have always been guiding questions that help me choose my own adventure.
As more folks ask me about their careers, I've found myself talking about three things
I met with Anil John today since he's in town for the 2007 MVP Global Summit. I always like talking with Anil because he asks the tough questions, he has thoughtful feedback and he keeps things real.
Anil's first question for me was why are there three different threat modeling approaches (SWI, ACE, and patterns & practices). This was easy for me since, I used to get asked this fairly regularly. Rather than focus on the implementation deltas, I focused on the context that shaped them. SWI threat modeling was born among our Microsoft product teams. ACE threat modeling was born among our internal line of business applications. patterns & practices threat modeling was born among an external set of customers, dominantly corporate line of business applications and vetted by some agile practitioners. They all work, so the trick is to figure out which one fits your scenario best.
Next, I shared my secrets for project management and personal effectiveness. It was nice to be able to finally walk Anil through some real examples and use the whiteboard as needed. Some concepts are easier to show and tell, then they are to write about in a way that sticks. (that doesn't keep me from trying!)
Over lunch, we reflected on career paths and stories. One point that really hit home was how small the world really is. We both noted that throughout our paths, there's always been a set of people that tend to show up time and again. One more reminder, not to burn bridges!
One of my readers asked me if I could provide a bit more insight on branching. I think the best thing I can do here is summarize a few tips and then point to some useful resources.
Here's an example starting point.
In this case, Main is your main source tree and project assets. Development is a root level folder for isolating your features or teams (branched off your Source folder in Main).
I got some face time with Rudy Araujo today. I always enjoy our meetings because we talk about anything from security to personal productivity to future software trends.
Today, we bounced around topics including compliance, agile, systemic problems, software scenarios, security guidance for business, mash ups, virtualization, blogging practices, password management, MindMaps, and managing action.
The highlight for me was that Rudy shared my belief that businesses need more help rationalizing how to bring security into the picture. While there's a lot of technical guidance available, there's simply not enough prescriptive guidance for the business stack. I've talked to analysts and customers about incrementally adopting security, but I think it's time to make that information more broadly available.
Graham Barry, one of the key VSTS members helping us pave our paths through Team Foundation Server, shared a model he likes to use. He uses multiple solutions, but with a flat project list:
It's flat, but effective. You get flexibility and you avoid baking in a folder structure than can be tough to change. You can also spin up solution files for presenting different working groups of the projects as you need them.
I've seen a few customers asking how to structure projects for Team Foundation Server. I don't blame them. Finding a structure that works well for you can be tricky, particularly if you don't have the benefit of hind-sight or a bunch of reference examples to draw from.
My team spent some time this past week evaluating various approaches and lessons learned from various projects We boiled it down to something that seems to be working well, and distills what's worked for some teams. With the caveat that we're still evaluating, here's what we learned ...
SolutionLocal File System
Source Control (Team Foundation Server)
Key pointsHere's a few highlights about this approach:
Repro StepsHere's a brief walkthrough to test using a file-based Web:
Verify your folder structure on your File System:
Adding to TFS
Verify your folder structure in Source Control Explorer
More InformationYou should know that while I talked through the single solution scenario, there are additional patterns. Here's the key patterns we see:
You can find more on these patterns at Team Development with Visual Studio .NET and Visual SourceSafe. You should also know that we have a few How Tos on structuring your projects coming your way. We'll post them to our VSTS Guidance Project.
Share Your StoryIf you've got lessons learned the hard way or practices to share, I'd like to hear them. Now's a great time to share since we're actively building guidance. Either comment here or write a post and leave a link.
Web 2.0 means a lot of things to different people. For me, it means moving from a read-only, one-way Web to a read-write, conversational Web. I think the most exciting part is the shift from sites serving content to humans driving the experience. The machine is us! Where will we take us next?
For a short, thought-provoking video, watch Web 2.0 ... The Machine is Us/ing Us.
The next time I need to get a set of requirements, I'm simply going to start with roles with goals.
I made the mistake of asking for a set of scenarios. What I ended up with is a list of user tasks. Tasks aren't the same as goals. Flying a kite is not the same as having a goal to discover electricity. One is what you're doing, the other is what you want to accomplish.
Since I didn't have the benefit of direct customer interaction for this case, I'm using that as a challenge. What is the most precise, reliable way to get the information need? Roles with goals. I realized, iterating on a list of roles with goals, would get me closer, faster. I can always drill into the stories or scenarios from the goals. The reverse doesn't seem to be true.
David Anderson distills his software management learnings down into four bullets:
He names this set a recipe for sucess. Here's what the recipe mean to me.
Focus on Quality. For me, focusing on quality means pushing quality upstream, and choosing quality over scope or quantity. It also means quality isn't somebody else's job.
Reduce work-in-progress. Reduce work-in-progress really hits home. Carrying too much in-flight work leads to excessive task-switching and administrative overhead. In the buffet of work to be done, frequent small plates, wins over the single, large plate that over-floweth.
Balance capacity against demand. Thinking in terms of demand, helps me right-size execution. Specifically, it helps me think in terms of frequency (how often do I need to deliver value), size (how big does the value need to be), and quality (what does good enough look like now versus perfect down the road).
Prioritize. There's always more work than we can possibly do. To prioritize problems, I think about pervasiveness and impact. Sometimes I need to prioritize a lesser problem that hits more people, over a bigger problem that hits few. I also keep my teams focused on value delivered vs. work completed. It's a subtle shift in mindset, but it makes all the difference.
In my 30 Day Improvement Sprints post, a reader asked, what tips do I have to make 30 days sprints more effective. Here's my short list
Few problems withstand sustained focus. There's a bit of captive genius in everyone that just needs to be uncorked. 30 days of focused improvement seems to be a great way to pop the cork. I'm finding improvement sprints refreshing because I now have a schedule for exploration. I can rotate through more interests. Most importantly, rather than tackle everything all at once, I just wait for my next 30 day focus. It's easier to put something aside for the moment, if I know I'll have a chance to immerse myself in it in the future. If I enjoyed something so much and I want to continue, I just do another 30 days.
Hope that helps!
Here's a simple set of perspectives I use for rationalizing requirements:
Believe it or not, simply identifying these perspective helps a lot. You'd be surprised how many debates happen simply because nobody explicitly identified the source or perspective of the requirement. You'd also be surprised how quickly this helps you make more deliberate decisions and understand what you trade.
How does this help? Your business goals might be a n orders per hour. Meanwhile, your user wants sub-second response time. Well, that's what your users want, but what will they tolerate? Can your business afford the resources for an idealistic experience? Design is always about trade-offs.
In practice, I generally see industry/standards trump business trump expert/technical, trump user. Unfortunately, a lot of software has unconsciously catered to the expert/tech/business at the expense of it's users -- making a lousy experience. On the flip side, some software has catered to users at the expense of the business or industry goals. See the dilema?
You can make a difference. If you build software, know the perspectives, know the goals, and know the scenarios. In some contexts, some tech/expert reccomendations simply do not make sense. Know what you're optimizing. In some scenarios, industry/standards reign, while in others user experience is king. Make sure you have the right representation at the table. Don't ask your patient to prescribe the medicine, or your doctor to rate the taste.
I'm using 30 day improvement sprints as a way to sharpen my skills. I pick a focus to work on and I committ to improving it for a 30 day timebox. Committing to 30 days of improvement in a focused area, is easier to swallow than changing for life. However, improving an area for 30 days, is actually life changing.
With 30 days, persistence and time are on my side. It's a big enough time box that I can try different techniques, while building proficiency. Using 30 days makes working through hurdles easier too. A lot of the hurldles I hit in my first week, are gone by week 2. Little improvements each day, add up quickly. I look back on how many things I tried for a week and stopped thinking I hadn't made progress. The trick was, I didn't get to week 2 to see my results. Lesson learned!
Prashant Bansode has stepped up to the blogging plate. He's been a core member of my team for a few years. I first met Prashant, when he reviewed our Improving .NET Performance and Scalability. I was impressed by his attention to detail, focus on the customer, and keeping things real. Next, we teamed up on .NET 2.0 Securirty Guidance, exploring new models for more consumable guidance. During this adventure, I found that Prashant was a developer who could do it and write about it -- a trait I value. Our next adventure was Guidance Explorer -- an R&D experiment around building tooling to help change the guidance game. We learned a lot about innovation, distributed teams, scenario-driven engineering, and shipping software incrementally and iteratively. Our latest adventures include learning how to build guidance projects on CodePlex: Performance Testing Guidance and Visual Studio Team System Guidance.
Prashant has worn many hats throughout our adventures together -- developer, tester, writer, mentor ... you name it. He's a true case of student becomes the teacher. Be sure to stop by and say Hi to Prashant at his new blog: PAB's World (Prashant uses his initials).