Most of people around are not doing eXtreme Programming, should I try it?

If billions of people believe something, does that alone make it a justified true belief?

Please, try for yourself eXtreme Programming techniques or whichever else design technique you found, and bring your findings and insights for discussion.

Also please, disregard the frequent plea for “the real world”, the real world is what we all do, do something different and you are essentially changing the world.

"How do I sell my executive team on doing this stuff? Don't. Just do it. They don't know what you're doing anyway" -Jim Highsmith

"Never be afraid to do something yourself. Remember - amateur built the Ark, professionals built the Titanic" -Gov Maharaj

"They believed that the structure of the universe could be worked out by sheer thought. And so they decided there were five elements, earth, water, air, fire, and ether. And that the universe was constructed of spheres made of these elements. Any element moved away from its sphere tried to get back to its sphere. Thus, stones (earth) fall, smoke (fire) rises, etc. It all made sense. It was all wrong" -Unknown

Paving the history of our trade

Which could possibly be the most common mistake on the history of the adoption of iterative and incremental development?

Craig Larman proposes an answer to such a question in his book Agile and Iterative Development: A Manager's Guide:

“Sure, we don't apply the waterfall—everyone knows it doesn't work. We've adopted <iterative method X> and are into our first project. We've been at it for two months and have the use case analysis nearly finished, and the plan and schedule of what we'll be doing in each iteration. After review and approval of the final requirements set and iteration schedule, we'll start programming.

This profound misunderstanding, still superimposing waterfall-inspired, big up-front analysis and planning (predictable manufacturing) values onto iterative methods, is one of the most common mistakes that new iterative and agile method adopters make.”

Based on the number of times and frequency I have found such a misunderstanding, certainly, make it a well positioned candidate for such a place in the history of our trade.

The new hells have been nurtured, by now they are alive and well

“Want me to adopt your solution? Let’s clarify first which are the new problems it brings on” -a conscious customer

The very presence of a solution to a problem brings on new kind of problems, ad infinitum. That is another way to say that there always will be room for improvement. That is why, when offering a contribution as a solution to a problem, there is need for a higher level of integrity like the one described as scientific integrity by Richard P. Feynman¹, one that let the buyer beware or Caveat emptor².

Problems sometimes are known by kind of a nickname with the word hell in it. For example, Microsoft Windows’s COM components used to be global for the local computer just because there is a single Microsoft Windows Registry by operating system installation. The problems associated with centralized and globally accessible information, in this case, took the nickname “DLL Hell”. Of course, the Component Object Model was the heaven —the solution— for certain kind of problems and we adopters were glad to pay the costs (consciously or not) of the correlated new problems it brought with itself, up to some —subjectively defined— point. The information needed to determine where that point is for any given subject is what proponents of the solution must provide in order to aspire for a behavior with scientific integrity.

Do you see the relationship between science and engineering? Do you see why software development is not a full-fledged branch of engineering discipline hitherto (by far)? By the way, maybe you are not conscious of this yet but, you don’t really want software development be a branch of engineering discipline in its entirety (more on this later).

Another example³ of heaven/hell dualism —what can now be called “Framework Hell”— is that of maximized flexibility or extensibility in object-oriented frameworks and over-designed helper⁴ libraries (delivered as unsupported, or hopefully supported, source code) lacking the properties of Application Programming Interfaces.

First, the general theme with those is the intention to provide a single design decision to be suitable for all unforeseeable singularities down the road in the evolution of an application’s design. They try to do that by means of over-generalized and premature abstractions introduced “for the needs of the future” that ultimately become more hindrance than help when disruptive singularities are discovered on the way to materialize the original intent of such frameworks or “helper” libraries.

Second, designers seem to be unaware of economical properties of software designs. They do not realize that each new line of code must be kept accurate and supported for the rest of the design’s lifecycle. So, the cost of a given application that makes use of a source code library increases in relation to the size and complexity of such a “helper” library. Those costs can be mapped to the costs of warehouse or inventory of goods in the manufacturing industry. Sadly enough, often the supposedly benefits for such a warehouse of flexibilities never are delivered because the “just in the case” needs never came.

When introducing the concept of scientific integrity there are, of course⁵, comments pleading to “the real world”; it is curious how, what is Kansas for somebody, could be Oz for somebody else, and vice versa⁶. For those with the spirit of continuous learning in general and software design in particular, please consider serious works on design activity, like what Donald A. Schön has published⁷; along with a seasoned advice by longtime consultant Peter Block⁸:

“Better to define our task as a process of discovery and dialogue more than as an act of diagnosis and prescription”

¹ The Pleasure of Finding Things Out: The Best Short Works of Richard P. Feynman by Richard P. Feynman

² Caveat emptor concept graciously referred to the author by colleague Jorge Eduardo Barsallo Caballero

³ The Legacy and Liability of Object Technology The Dark Side of OO by Dave Thomas

⁴ “Everyone may be trying to help, but good intentions are no substitute for competence” -Dan Starr

⁵ “Unanimity of opinion may be fitting for a church, for the frightened or greedy victims of some (ancient or modern) myth, or for the weak and willing followers of some tyrant. Variety of opinion is necessary for objective knowledge” -Paul K. Feyerabend

⁶ “There are people who prefer to anchor themselves in the comfort of a limited level of knowledge. They consider themselves practical and 'real-world'-ish” -Andrei Alexandrescu

⁷ Reflective Practitioner: How Professionals Think in Action by Donald A. Schön

⁸ Flawless Consulting: A Guide to Getting Your Expertise Used by Peter Block

The interplay of art and science in software

I have found an article with a sound explanation of the interplay of art and science with software development, enjoy:

The interplay of art and science in software
Terry Bollinger
IEEE Computer
Volume 30, Issue 10, Oct. 1997 Page(s):128, 125 - 127

Will mankind exist on Earth 100,000 years from now?

It is hard to say based on self-destructive trends in us. Help are available in several forms. Science is one, not as a system of beliefs taken religiously but as an attitude of the mind. Taken this way, I appreciate the following quote in full:

"One thing I have learned in a long life: that all our science, measured against reality, is primitive and childlike -- and yet it is the most precious thing we have" -Albert Einstein

A more attainable question would be: Are the current schooling systems really educating in the scientific attitude of mind?

Convention on the Rights of the Child

Adopted and opened for signature, ratification and accession by
United Nations General Assembly resolution 44/25 of 20 November 1989
entry into force 2 September 1990, in accordance with article 49

Article 28

...

3. States Parties shall promote and encourage international cooperation in matters relating to education, in particular with a view to contributing to the elimination of ignorance and illiteracy throughout the world and facilitating access to scientific and technical knowledge and modern teaching methods. In this regard, particular account shall be taken of the needs of developing countries.

Who is an architect?

While the question ‘Who or what is an architect?’ could be popular nowadays, the important for a project and for an organization is the act of architecting, the continuous care of all-encompassing properties that give the illusion of simplicity to customers and end-users. That is what is important. That is an essential ingredient for project’s success.

Discussing the establishment of a single individual as the prima-donna decision-maker is foolish, is more about that “taking the form but not the substance” unwise behavior typical of overenthusiastic and blind advocacy of the youth (see: Discussing uncomfortable questions).

Some people believe an architect does not need to get dirty with technical details but also believe they must be in charge of important technical decisions, at the same time. I think doing so is to keep this guy in a position of incompetence.

I think architect is a collective role, not an individual. if we think —on one side— of an architect as a role, not as an individual, whose key aim is to describe an abstract, coherent and conceptually integrated system clearly to all the team and, if we think —on the other side— of a developer as a role, not as an individual, whose key aim is to provide feedback about the technical feasibility for each architectural decision, then both roles could be fulfilled by the same professional or group of professionals (like a team in charge of accurately delivering the right system into the hands of customers and end-users).

What do we –really– mean by 'coding'?

Suppose a young member of the developer role in your next project team approaches to you (member of the architect role in the same project) and said: —I will be coding as part of my role in our project and I am looking for further understanding of how my work will be related to actual business value. Could you –as seasoned software professional– please share your insights with me?

What would be your answer?

1. Young subordinate: your main task —as a necessary evil— will be mechanical and repetitive because all intellectual and engaging artwork will be exclusively done by me and my peers; please be prepared to eagerly follow our direct commands and to strictly follow our plans, doing whatever required extra effort in order to hit established milestones on time. Your association with business value is that and the same of the cheap labor force in well established economies of scale, that is, the execution of industrialized tasks by expendable and interchangeable human resources. You know, it is the real world and nothing can be done about it. No problem at all, it’s my pleasure to clarify.

2. Young peer: first of all let me thank you for the opportunity to share my perspective on such an important question of yours; second, let me please clarify that coding is –actually– another name for a design activity which is part of a continuous flux of learning experiences shared by all members of the team. You, along with the rest of project team members, will be doing software design which is a cooperative endeavor to tackle the complexity coming from a number of dimensions like people, process, product and technology; we need all your intellectual power, critical thinking and energy to keep a balanced mix of problem-solving, creativity and tactical execution as the project unfolds. Join us, we are about to review current business priorities with the customer...

3. Please excuse me but I’m really not the right person to answer your question because I’m not in a position that has exposure to such level of fine-grained details so I don’t know what you are going to be doing and how that is related with business value.

4. Next possible answer...

Now, please assume that you are fully aware of a secondary purpose for the approaching developer in your project, who also is a senior researcher, to report back to general scientific community a current, actual, reliable and justified characterization of the ‘coding’ activity.

Please re-read the proposed answers and re-think your own answer so far but now with this new perspective in mind.

Would you change your answer? Which would it be now? Why?

What the role of an architect really wants to be?

After re-reading sections about architecture in The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition (2nd Edition) by Frederick P. Brooks, Jr. I am wondering if what the role of a nowadays architect really wants to be is that of the old notion of data processing analyst.

The key deliverable from data processing analysts was written functional specifications; the best of those analysts described an abstract, coherent and conceptually integrated system —yet to be implemented— and all their description was from the standpoint of the business and end-user. Later (and sometimes), the senior developers took the functional specification and delivered a technical specification of how the system’s internals would be “constructed” and then —in all waterfallish fashion and most often than not— the “dime-a-dozen coders” took the technical specification and “built” the system. It all made sense. It was all wrong (see Writing 2 here).

Data processing analysts did not “code”; very, very far from doing that. Isn’t what many are saying about architects?

Many years, colossal amounts of human effort and vast amounts of money have been wasted for our industry to learn that waterfall and sequential thinking —like the one described a paragraph ago— for the realization of complex computing systems just don’t work for most of the situations. Why would we want to bring back the notion of data processing analysts? Which could possibly be the advantages of doing that?

Now, if we think —on one side— of an architect as a role, not as an individual, whose key aim is to describe an abstract, coherent and conceptually integrated system clearly to all the team and, if we think —on the other side— of a developer as a role, not as an individual, whose key aim is to provide feedback about the technical feasibility for each architectural decision, then both roles could be fulfilled by the same professional or group of professionals (like a team in charge of accurately delivering the right system into the hands of customers and end-users).

What is the role of an architect? Sorry, I don’t get it, architect *is* already a role, not an individual. Should an architect design? Yes, the mere essence of architecting if that of describing —from the standpoint of the business and end-user— an abstract, coherent and conceptually integrated system that solves the business problem.

Should an architect code?

For those interested, the role of an architect is –also- being discussed in MSDN, here. My first reply next:

The answer depends on what do you mean by “architect” (noun) and also by “code” (verb).

What seasoned designers talk about when discussing architecture is so all-encompassing and important for the final outcome that makes me wonder why a project team member wouldn’t need to fully master the actual concretization of such concepts at all levels? I suspect the answer is related to the sad state of practice in many organizations due to the faulty belief that the main task in writing software is a mechanical and repetitive task that can be industrialized with expendable and interchangeable human ‘resources’.

See: Discussing uncomfortable questions

Pursuing a clear definition of a concept like architecture is good because the related thinking-researching cycles give opportunity to reach sound conclusions (by ‘sound’ I mean -inferences with scientific properties like provisional and falsifiable). I have not yet found a satisfactory definition for what architecture in computing really means, far less a definition of what the role of an architect could possibly be. By now, the proposal by Frederick Brooks, Jr. for the concept of architecture is arguably the most sensible definition I have found:

“Computer architecture, like other architecture, is the art of determining the needs of the user and then designing to meet those needs as effectively as possible” –Oxford English Dictionary

Why? Because it explicitly includes three key concepts: art, user needs, and design.

A discussion about the design concept would be in order to answer the question: “Should an architect code?”

So, if by code, do you mean executing a mechanical and repetitive task that can be industrialized? Then the answer is ‘no’ (and also ‘no’ for any self-respecting software professional by the way).

On the other hand, if by code, do you mean the act of expressing design intentions and the execution of models to corroborate hypothesized ideas with contextual data? Then the answer is ‘yes’ (and also ‘yes’ for any self-respecting software professional by the way).

Next, I quote a thinker whose name I don’t know (if the author is listening, see: What type of person should design software? ).

"There are certainly people who think more about high level structure than they do about low level details. This is a valuable talent, and not one that should be scoffed at or discounted. Unfortunately, in some circles, it has become popular for people with these talents to divorce themselves from code. The term "architect" takes on an aura of authority and power, whereas a coder is deemed a lowly laborer, a dime-a-dozen worker bee.

In reality, while the difference in talent is real, the polarization into different roles with different authorities is harmful. Those folks who are good at modeling and architecture must still remain grounded in code. For their skills to remain relevant and useful, they have to keep in touch with the medium they are trying to create structures for. Likewise, the programmer who divorces himself from concerns of architecture harms himself and his team by abdicating responsibility for something he is in an intimate position to control. There is a way for these two talents to work well and closely together, but they have to put forth the effort to understand each other."

More notes about the practice of architecture:

Software architecture is much more than structure

Architecture is not about scalability, not even about user, it is all about usage

Discussing uncomfortable questions

For those with critical thinking habits, let’s cogitate about this:

If day after day goes by with nobody discussing uncomfortable questions like these, won't the good people of my country be guilty of making things worse?” —Donald E. Knuth

Could this type of questions be translated to the lesser sphere of software development and the quality of what we deliver to customers and end-users?

Let’s think, for example:

Could it be possible that the notion of division of labor in industrial settings just does not deliver the same benefits in software development?

I have seen and heard that many so-called ‘software architects’ lack the ability to write shippable software by themselves (with their own hands) and that they are taking key architectural decisions based on hypothesized ideas not actually corroborated with the results from execution of models. After taking architectural decisions this way follow the expenditure of vast amounts of human effort and financial resources to carry out a given project only to find out —most often than not— that most or all of that expenditure have been wasted. Serious postmortem analysis could revel that part of the root cause are those architectural decisions taken prematurely. Why should software consumers accept increasing levels of ineptitude in software development teams?

Could it be that software development industry has got wrong the practice of architecture?

Could it be that software development industry has got wrong the very essence and purpose of modeling (in reference to modeling as in engineering disciplines)?

Has the “taking the form but not the substance” behavior gone too far among us members of software development industry?

What kind of work could be expected from developers and their managers if their view of software development is based on the belief that the main task is a mechanical and repetitive task that can be industrialized with expendable and interchangeable human ‘resources’?

If day after day goes by with nobody discussing uncomfortable questions like these, won't the good people of our software development industry be guilty of making things worse?

Update on what is software development -art or science- debate

This is an update on my internal inquiry: What is software development, art or science?

Not surprisingly, the act of programming digital computers is both, art and science. Now I know. Actually, the simple question “what is software development, art or science?” is a malformed or ill-formulated question because it presupposes the subject matter can be or an art or a science as a whole.

Software development is art because the essence of the root meaning of art includes the notion of skill, dexterity, ability. That is to say, in order to practice software development there is need of –unavoidable- certain skill set.

Software development is science because the essence of the root meaning of science includes the notion of reliable knowledge. That is to say, in order to practice software development there is need of –unavoidable- reliable knowledge.

In both —art and science— we, as guild of professionals, need to improve badly.

For further reading, read Computer programming as an art by Donald E. Knuth.

Belief and behavior

A belief is —for practical purposes— something that we thought is true. The incredulity or disbelief is a case of belief where we thought of a belief to be false. To doubt about something means to keep the related true-false judgment in suspended or pending state. To ignore is to have a complete absence of that judgment. To know something or to acquire knowledge is the outcome of a process that serves as basis for a special belief, one that is properly justified.

A religious belief is part of just one more type of belief within the extensive spectrum of belief types any given person could uphold. In current sociology of software development, many people uphold beliefs very similar in nature as religious beliefs, e.g., "X will remove all our problems" or "This important project will succeed because we have adopted X"; where X is any fashionable trend or buzzword blindly followed.

Most of the time, the conduct of a person can be explained in terms of the beliefs this person upholds in the occurrence context of the behavior in question. The context can include social aspects like office, school, home, church, etc. Our beliefs have great power to influence our behavior and while our beliefs remain our behavior will do so.

The correlation between the chosen conduct and its consequences (costs and benefits) encompasses the fuel that feeds the process to improve our beliefs, and therefore, our behavior. This correlation is not the only input for an improvement process, it is only one that is usually more affordable in comparison with other inputs as can be a tragic or catastrophic personal experience, also it can be the increase in conscience because of the understanding of a justified true belief about the same subject matter.

A key question is then: Which process have you chosen for the improvement of your beliefs?

The "What's coming after X?" question

What could be say about the question: What's coming after X?
Where X could be:

• Object-orientation

• Software engineering best findings

• Agile development

The context for the question:

• Asked by blind advocates of X

• Usually young people (mental youth mainly)

• Looking for the next sensational fad

The question remains me about the question "What is coming after object-oriented programming?" to an object-object oriented practitioner and author and his reply: "Our industry needs to understand object-orientation in the first place before thinking in what is next". Or the question "Which mayor techniques do you see coming in the horizon of software engineering" to Dr. David Parnas and his reply "The mayor techniques for software engineering have been already here and for a long time, but have not been fully understood".

Learning items in software development

A subjectively created list of books about software development grouped in the following categories:

Category I: Practitioners sharing their hard-won and thoughtful experiences.

Category II: Foundational knowledge.

Category I: Practitioners sharing their hard-won and thoughtful experiences

Agile Principles, Patterns, and Practices in C# by Robert C. Martin, Micah Martin

The Art of Agile Development by James Shore, Shane Warden

Implementation Patternsby Kent Beck

Extreme Programming Explained: Embrace Change (2nd Edition) by Kent Beck and Cynthia Andres

Object Solutions: Managing the Object-Oriented Project by Grady Booch

Organizational Patterns of Agile Software Development by James O. Coplien, Neil B. Harrison

Agile and Iterative Development: A Manager's Guide by Craig Larman

Agile Software Development: The Cooperative Game by Alistair Cockburn

Agile Software Development With Scrum by Mike Beedle, Ken Schwaber

The Enterprise and Scrum by Ken Schwaber

The Craft of Software Testing: Subsystems Testing Including Object-Based and Object-Oriented Testing by Brian Marick

Test-Driven Development: A Practical Guide by David Astels

Concurrent Programming on Windows Vista: Architecture, Principles, and Patterns by Joe Duffy

Multi-Paradigm Design for C++ by James O. Coplien

Working Effectively With Legacy Code by Michael Feathers

Windows Internals: Including Windows Server 2008 and Windows Vista, Fifth Edition by Mark Russinovitch, David A. Solomon

Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans

 Agile Project Management: Creating Innovative Products by Jim Highsmith

Agile Estimating and Planning by Mike Cohn

Software Pioneers by Manfred Broy (Editor), Ernst Denert (Editor)

I. M. Wright's Hard Code by Eric Brechner 

Pragmatic Programmer: From Journeyman to Master by Andrew Hunt, David Thomas

Software Craftmanship: The New Imperativeby Pete McBreen

Category II: Foundational knowledge

Software design by David Budgen

Software for Use: A Practical Guide to the Models and Methods of Usage-Centered Design by Lucy A.D. Lockwood, Larry L. Constantine

Sketching User Experiences: Getting the Design Right and the Right Design by Bill Buxton

Object-Oriented Analysis and Design with Applications (3rd Edition) by Grady Booch

Practical Guide to Structured Systems Design (2nd Edition) by Meilir Page-Jones (Author)

Structured Design: Fundamentals of a Discipline of Computer Program and Systems Design by Edward Yourdon

Structured Analysis and System Specification by Tom Demarco and P. J. Plauger

Structure and Interpretation of Computer Programs - 2nd Edition by Harold Abelson, Gerald Jay Sussman

Instructor's Manual t/a Structure and Interpretation of Computer Programs - 2nd Edition by Julie Sussman

Computer Science: An Overview (9th Edition) by J. Glenn Brookshear

Generative Programming: Methods, Tools, and Applications by Krzysztof Czarnecki, Ulrich Eisenecker

Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili by Barry Boehm

Message to the whole body of management teams in software industry

To all management teams in our industry:

Please consider doing more of this: Increasing the minimum level of knowledge and modern actual practice required from software development sales personnel in order to negotiate projects with customers/users. Most of current sales personnel have notions, beliefs and premises no longer valid in the software development high-tech market in general. For example, that “analysis” or “design” or “programming” are separated stages in the software development process; they are not stages, they are concurrent activities. And many more decayed beliefs which could seriously decrease the chances for those projects to reach better business results and better customer satisfaction.