Software Engineering, Project Management, and Effectiveness
Several years back, I did an exercise in mapping out families of application architectures and application types. It was an extensive archeological expedition.
Key Goals / Outcomes There were several goals of the exercise:
The exercise fed into a number of later works years later, including:
Key Activities Some of the key activities of this early exercise included:
Keep in mind, that going into this, I already had the benefit of doing more than 650 customer architecture and design reviews -- yet still, this was an overwhelming exercise. It forced me to find new ways to deal with large bodies of data and information, and somehow turn them into shared maps, browsable, reusable knowledge nuggets, and backdrops for deeper conversations and elaboration.
Lessons Learned from Architectural Exploration Some of the surprises for me or things that I learned that I didn't expect include:
In retrospect, the simplicity and common denominators of CRUD make a lot of sense. It's all about interacting with information at a fundamental level. People are just trying to get things done and there's only so many things you can do with information -- find it, browse, save it, share it, transform it, etc. as part of your workflow.
Early Map of App Types I included one of the many early maps of the application types that helped us figure out what to throw out and what to keep as we moved forward. One of the key distinctions that Ward Cunningham helped me figure out was to distinguish between the shape of the application architecture and design, and the actual “purpose” of the application. Some purposes were more business-oriented, while some were more technically oriented, and this helped me find and distinguish different families of apps.
It's circa 2004, but the irony is how timeless the backdrop really has been. It’s rough and raw, but like I said, it’s just one sampling of the many braindumps behind the scenes of mapping out app types.
Cloud security has been a hot topic with the introduction of the Microsoft offering of the Windows Azure platform. One of the quickest ways to get your head around security is to cut to the chase and look at the threats, attacks, vulnerabilities and countermeasures. This post is a look at threats and countermeasures from a technical perspective.
The thing to keep in mind with security is that it’s a matter of people, process, and technology. However, focusing on a specific slice, in this case the technical slice, can help you get results. The thing to keep in mind about security from a technical side is that you also need to think holistically in terms of the application, network, and host, as well as how you plug it into your product or development life cycle. For information on plugging it into your life cycle, see the Security Development Lifecycle.
While many of the same security issues that apply to running applications on-premise also apply to the cloud, the context of running in the cloud does change some key things. For example, it might mean taking a deeper look at claims for identity management and access control. It might mean rethinking how you think about your storage. It can mean thinking more about how you access and manage virtualized computing resources. It can mean thinking about how you make calls to services or how you protect calls to your own services.
Here is a fast path through looking at security threats, attacks, vulnerabilities, and countermeasures for the cloud …
Overview It is important to think like an attacker when designing and implementing an application. Putting yourself in the attacker’s mindset will make you more effective at designing mitigations for vulnerabilities and coding defensively. Below is the cloud security frame. We use the cloud security frame to present threats, attacks, vulnerabilities and countermeasures to make them more actionable and meaningful.
You can also use the cloud security frame to effectively organize principles, practices, patterns, and anti-patterns in a more useful way.
Threats, Attacks, Vulnerabilities, and Countermeasures These terms are defined as follows:
Cloud Security Frame The following key security concepts provide a frame for thinking about security when designing applications to run on the cloud, such as Windows Azure. Understanding these concepts helps you put key security considerations such as authentication, authorization, auditing, confidentiality, integrity, and availability into action.
Threats and Attacks
SDL Considerations For more information on preferred encryption algorithms and key lengths, see the Security Development Lifecycle at http://www.microsoft.com/security/sdl/ .
What is the success pattern for leading a successful v-team? I asked one of my mentors, a seasoned softie for his take on what are the keys to a successful v-team. Here is what he had to say:
This is a walk down memory lane. It’s an idea I was circulating circa 2005 so the words will seem stale now. Periodically I like to flip through my old ideas to see how I’ve used them, what impact they made, and what I learned in the process.
When I originally pitched the idea, there were two basic reactions – either “this will never fly” or “we have to make this happen.” The funny thing was, when Amazon launched its Mechanical Turk, several months later, some of the original naysayers, came back to me and said, “Oh, I guess you were on to something.”
I wasn’t passionate about the idea, so I didn’t pursuit it. I was also busy with other game changers. However, I did share the idea with some key folks, and then I wrote it up in a simple way so that whoever might need to see it, could quickly understand the scenarios and the value. However, the thinking behind this did end up shaping some interesting efforts. It also helped me articulate a principle that kept showing up – Human Shepherds and the Law of Relevancy.
As an aside, the big thing I learned from having clusters of ideas at a time … (I filed several patents that year, created a few key software prototypes that internal/external customers built offerings around, and helped shape some key product-line paths) … is that you can actually get ahead of the innovation eight-ball, if you know how to combine key principles and patterns with the trends and the maturity cycles of markets. The patterns and principles help you see things that might not otherwise be obvious. There are even tricks to getting ahead of the mainstream market. Someday I’ll share the patterns and practices for innovation that I’ve learned from the school of hard knocks (part of our group acted like applied research so we have the battle scars of making innovation stick against real-world scenarios.)
Here is the original idea of “Short-Burst Work”, in raw form. Note that one of the other things I learned since then is that choosing a sticky name for an idea, makes all the difference in the world – alliteration and fun, helps a ton!
Idea: Short Burst Work
Elevator Pitch ”Connect short-burst human power to the relevant short-burst work opportunities ...”
Summary Here's an idea that could be a game changer ... What if you could connect a world-wide market of job seekers to short-burst opportunities through "adverstising" in the corp space? (consumer works too, but let's play w/corp scenarios for now) This is nearly the "long tail" of job opportunities issue. Imagine all the short-burst work opportunities in your day job. What's interesting is this model is happening on SecondLife, with real people and real exchange of money. Imagine the world of pooled resources, available 24x7. Imagine the world of editors, coders, great thinkers …. etc. Imagine how many former dot-Com'ers that would love to lend short-burst help. Imagine all the great professors of the world you can suddenly involve when you can virtualize your work items and collaborate with a simple platform.
Recommendation From a mental model, think of it as a highly relevant "match" service for short-burst work opportunities and resource pools.
Corporate Scenarios Imagine live services to connect your short-burst work items to:
Why is this feasible? It's happening on SecondLife. Folks around their world are selling their services (scripting, image consulting, music dj …. You name it) It's happening because the barrier to entry is low:
To sell your services/skills:
To consume the services/skills:
There's reasons why existing services like this haven't taken off. Connecting the consumer/provider to the "long tail" of the short-burst opportunities is a relevancy algorithm issue and a platform hurdle (services platform with good consumption/provider models). Your engine can help folks find the short-burst services they need. And when the relevancy is beyond organic search, you broker a "concierge" type service (Live Concierge) to connect a consumer to a provider … or a provider to a consumer. It's a game changer. How to make it happen?
Imagine you the user wants to connect to "live" help. Think (Google (relevancy) + Monster (labor pool) + Temp Agencies (labor pool and short-burst work) + (plug in your favorite Web 2.0 model)
I use scenarios all the time for anything from designing a user experience to evaluating architecture. Scenario is an overloaded term though. There are lots of types of scenario tools. If you know the types of scenario tools, you can use the right one for the job. For example, exception scenarios are useful for assessing robustness. Misuse cases are helpful for figuring out potential threats and attacks.
In the book Scenarios, Stories, Use Cases: Through the Systems Development Life-Cycle , Ian F. Alexander and Neil Maiden write about the types of scenarios and when to use them in software development.
Here is a brief explanation of each:
Workshops are one of the best ways to rapidly brainstorm ideas, capture scenarios, mine for patterns, etc. The challenge can be designing an effective workshop.
In the book Scenarios, Stories, Use Cases: Through the Systems Development Life-Cycle, Ian F. Alexander and Neil Maiden present The Six P’s Workshop Framework. Here is a summary of the approach:
As you can see, the six P’s (Purpose, Participants, Products, Place, and Process) create the overall frame. Each category within the frame has a driving question. For example, “Purpose” contains the question, “Why are we doing the workshop?” Each category within the frame also has a list of “Concerns.” For example, “Purpose” contains the concerns, “goals", “need”, and “motivation.”
By addressing the question and the concerns, you can design and shape a more effective workshop.
While working with a customer at our Microsoft Executive Briefing Center (Microsoft EBC), one of the things we need to do was find a simple and effective way to partition SQL Azure.
The problem with SQL Azure is that depending on your partitioning strategy, you can end up allocating increments that you don’t need and still pay for it. The distinction is that for Azure Storage, you pay for storage used, but with SQL Azure you pay for the allocated increments. See Windows Azure Pricing.
After checking with our colleagues, one of the approaches we liked is to partition by customer ID ranges. By adding this level of indirection, you can optimize or reduce unused increments because you already know the ranges and usage, and you can change the ranges and allocations if you need to. Here is a visual of the approach:
Thanks to our colleague Danny Cohen for the insight, clarification, and suggestions.
What is a PM (Program Manager)? While the Program Manager role seems unique to Microsoft, in general, when you map it to other companies, it’s a product manager, or a project manager, or a combination of the two. At Microsoft, there are various flavors of PMs (“design” PM, “project” PM, “process” PM, etc.) and the PM discipline can be very different in different groups. I’ve also seen the PM title used as a general job title, in the absence of something more specific.
At Microsoft it’s a role that means many things to many people. In general though, when you meet a PM at Microsoft, you expect somebody who has vision, can drive a project to completion, can manage scope and resources, coordinate work across a team, bridge the customer, the business, and the technology, act as a customer champ, and influence without authority. From a metaphor standpoint, they are often the hub to the spokes, they drive ideas to done, they take the ball and run with it, or find out who should run with the ball. Some PMs are better at thought leadership, some are better at people leadership, and the best are great at both.
Here is a roundup of some of my favorite points that elaborate, clarify, and distill what a PM is, what a PM does, and how to be one.
Attributes and Qualities of a PM Here is a list of key attributes from Steven Sinofsky’s post -- PM at Microsoft:
Here is an example of PM qualities from Sean Lyndersay’s post -- Exchange team defines a Program Manager:
Microsoft Careers site on Program Manager Here is a description of a Program Manager from the Microsoft Careers site: “As a Program Manager, you’ll drive the technical vision, design, and implementation of next-generation software solutions. You’ll transform the product vision into elegant designs that will ultimately turn into products used by Microsoft customers. Managing feature sets throughout the product lifecycle, you’ll have the chance to see your design through to completion. You’ll also work directly with other key team members including Software Development Engineers and Software Development Engineers in Test. Program Managers are advocates for end-users, so your passion for anticipating customer needs and creating outside-the-box solutions for them will really help you shine in this role. As a Program Manager you will have the ability to lead within a product’s life cycle using evangelism, empathy, and negotiation to define and deliver results. You will also be responsible for authoring technical specifications, including envisaged usage cases, customer scenarios, and prioritized requirements lists.”
Chris Pratley on Program Manager Here are some points on Program Management from Chris Pratley’s post -- Program Manager:
Here are the stages of your first year as a PM, according to Pratley:
Joel Spolsky on Program Manager Here are points from Joel’s post on How To Be a Program Manager:
Ray Schraff on Program Manager Here are point from the comments on Chris Pratley’s post, Program Management:
Sean Lyndersay on Program Manager
Steven Sinofsky on Program Manager Here are points from Sinfoksy’s post on PM at Microsoft:
My Related Posts
The Microsoft patterns & practices Windows Azure Architecture Guide - Part I is now available on MSDN.
This guide focuses on a migration scenario. It walks through migrating a custom expense tracking and reimbursement system to the cloud.
While working with a customer last week at our Microsoft Executive Briefing Center (Microsoft EBC), one of the things we need to do was find a simple way to whiteboard the architecture. We walked through a lot of variations of application patterns, but the customer ultimately wanted to see one code base running on Azure that could support multiple clients (such as desktop, phone, browser, etc.) We found a model that was pretty straightforward and allowed us to easily map the customer’s scenario onto the Azure platform.
Once we had a model, it was a lot easier to drill down on things such as why SQL Azure vs. Azure Storage or how to flow claims or how to partition the database. It’s like a backdrop that you can overlay or drill down on security, performance, manageability, etc. It helped us quickly explore paths without getting into weeds and keeping the big picture in mind. It also helped us figure out where target some of the key architectural spikes.
Here are some of the models we walked through on the whiteboard:
Azure Storage Example
SQL Azure Example