CUSEC 2010 Keynote: Reg Braithwaite - “Beautiful Failure”

Here’s the third in my series of notes taken from keynotes at CUSEC 2010, the 2010 edition of the Canadian University Software Engineering Conference. These are from Beautiful Failure, a keynote given by my friend Reg “Raganwald” Braithwaite, who’s forgotten more about combinators than I will ever learn.

My notes from his keynote appear below; Reg has also published his slides online.

• I gave a talk at Stack Overflow DevDays Toronto in which I was thinking out loud about programming about programming
• I was trying to rewrite the way we program
• The language we use for coding guides the way we think about the program and the solutions
• When you write things to change your programming language, you change the way you think

Thinking About Programming About Programming

• I often get called in by clients to automate a process
• Often, during this process, they want to change the process that I’m supposed to automate
• Automating a process forces you to think about it
• The very act of thinking about how you do things helps you understand what it is you do
• The exercise of thinking it through is useful, even if it fails or you don't end up using it
• Languages and frameworks come and go, but everything you to do fix what's between your ears stays with you forever
• Programming languages are just a notation for the way we think

• Some people try to do things like add a "sum" method to Ruby’s Enumerable mixin
• What happen when you try [[1, 2], [3, 4], [5, 6]].sum?
• [He showed two implementations of a “sum” method:
  • One by “Alice”, which when applied to [[1, 2], [3, 4], [5, 6]], yielded 21,
  • and one by “Bob”, which when applied to [[1, 2], [3, 4], [5, 6]], yielded [1, 2, 3, 4, 5, 6]
• With “monkeypatching”, it’s possible for two different modules to implement Enumerable#sum, and then for someone else to import both modules.
  • In which case, which version of sum will get called? It depends on the load order of the module
  • But what if these were written as gems? Then there’s trouble
• To solve this sort of problem, I decided to steal extension methods from C# and add them to Ruby [Joey’s note: extension methods are a C# feature that let you add methods to an existing class without subclassing]
• It works, but what’s wrong with what I’ve done?
• My extension methods for Ruby are a hack…
  • to fix Alice and Bob's hack…
  • of Matz's (Yukihiro Matsumoto) hack of Smalltalk's hack (i.e., Ruby)
• It eliminates the annoyance without solving the core problem
• Do extension methods reengineer the way we think about problems? Or do they simply deal with an annoyance?
• Do they reengineer the way we think about programs?

• Take the Single Responsibility Principle (SRP)
• When you write an extension method, you break SRP
• When you monkeypatch, you violate SRP
• Is that bad? I don't know
  • C# breaks SRP with extension methods
  • Rails "runs roughshod over it"
  • If two popular languages break SRP, maybe SRP isn't all that
  • What does the sum method tell us?
  • Why is this a beautiful failure?
  • Maybe we've gone beyond the class --- Ruby is not C++ or Smalltalk
  • Hacks like this scratch an itch and suggest a flaw -- what else is flawed?
• You have an advantage over me
  • I have this ball and chain of experience
  • I've been fucking with computers for almost 40 years
  • They way I’ve been doing things has made me a living; I’m not incented to change the way I do things
  • You’re not tied down
• So now I present a few ideas that have occurred to me -- think about them!
  • I don’t have the answers

• Unit tests tell us that compilers are flawed
  • If we need them, what is wrong with our programming languages and compilers that requires us to step out of what we're doing to implement them?
  • Why do we need to take a great language and bolt something onto the side?
• Github tells us that our existing idea of a program is flawed
  • Most people think of programs as static things
  • In Github, there is no "program" -- there are branches, forks and tags
  • Languages themselves have no notion of what a version is
  • Looking at the way we actually use tools shows that there's a disconnect between our toolsets and the way we write code
  • Are Github commits congruent to objects?
    • If you change 4 classes in a commit, there must be something they have in common, but that’s not apparent from the way we write them
• Do we manage work the way we manage code?
  • Project management seems awfully disconnected from our tool chain
  • Consider the complete disconnect between issue tracking and time tracking
  • Maybe not so important in your company, but more important for personal projects
  • Git and Lighthouse -- “like two cups connected by string”
• Do we manage object versions the way we manage API versions?

• "Do not follow in the footsteps of the sages, seek what they sought."
• What I think is particularly cool and interesting is…but to me
• Think about what your heroes were trying to achieve using the tools available to you today
• An example of following blindly in the footsteps of sages:
  • In November 2002, I attended Paul Graham's “Lightweight Languages 2” conference in Boston
  • The morning keynote was by John Armstrong, who presented Erlang, which today is considered an important language for concurrent programming
  • The afternoon keynotes was Matz, who presented Ruby, one of the most influential dynamic languages that soon after enjoyed a meteoric rise in popularity
  • Many people in the room, die-hard Lisp-heads, were shouting them down because their languages didn't have macros [Joey’s note: Macros are a Lisp feature that smug Lisp weenies often use in the never-ending “Why my language is better than your language” argument]

Four Ugly Failure Modes and How to Avoid Them

Confusing correlation with causation

• I think it's one of the most prevalent diseases in the business world
• Ruby is not a silver bullet
  • Was the success of many Ruby projects [such as Rails and Twitter] because of Ruby the language?
  • Or was it that smart people who could get things done were picking Ruby at a given point in time?
• Agile is not a process
  • It's a set of values
• Here’s how many companies fail:
  • They start a little consulting company
  • They enjoy some successes, which leads to more business
  • As a result, they hire people and the company grows
  • But they can't hire smart people faster than the work is coming in
  • So in order to hire people to meet the demand, they start hiring people who aren’t as smart
  • That's when things go downhill
  • Who here doesn't think this isn't standard for any consulting company?
• Toronto Agile User Group recruiting process
  • In our field, "best practices" are cow patties
  • I’ve gone to many companies where they combine "best practices" simply by smooshing them together
  • I’ve been to many Toronto Agile User Group meetings where very few attendees work at companies that even practice agile
  • The important thing is that the people there are attending because interested in finding a better way of doing their work – those are the people you should be hiring!
• The plural of "anecdote" is not "data"
  • Greg Wilson will talk about this in his keynote later today – listen to him!
  • Problem: Talks are given by narcissists (or masochists)
  • When you read something in a blog, see something on TV or buy a book, you're not getting a large enough sample, and the content is biased
  • Another problem is that history is written by the survivors
  • People write about really notable successes or failures

Confirmation bias

• "Most of you will be immune to this, because you're all sensible people"
• You might fall victim to confirmation bias if you have an overly-inflated (or under-inflated) ego
• You might also fall victim to it if your worldview is too narrow
  • If you're a Ruby developer, you probably don't read C# blogs, and vice versa
• Seek out more representative info; not just the stuff that confirms your opinions

Local maxima

• The innovator's dilemma
  • If you have customers, they will trap you in a local maximum
  • They're not trying to be mean, they're trying to give you money
  • You might end up optimizing to serve your customer base while the rest of the world (and eventually your business moves on)
• The Principle of Least Surprise is a trap!
  • Familiarity comes from doing the old things the old way
  • This doesn’t apply to just UI, but also naming variables or coding styles
  • Once in a while, you should say "Maybe this one time, we should do things differently"
• Iterative anything is a trap
  • It's hill climbing
  • Sometimes you have to leap
  • It's supposed to be bad to "go dark" in development for a longer period rather than go through many small iterations, but sometimes it's the only way to make a great leap
  • You can't climb a big mountain if you do things in small increments

"A Market for Lemons"

• What happens when you sell to people who don't fundamentally understand what they're buying?
• If customers don't understand what they’re buying, they make their decisions based on easily differentiable features
• One example is buying a house, which you’re not going to do very often in your life, so most people know very little about it
  • As a result, they focus on easily differentiable features like square footage, number of rooms, and other features that can easily be picked out
  • But it’s better to focus on whether the house’s design makes it more liveable, which is harder to suss out
• Another example of this is feature checklists on the back of product boxes
• Gresham's Law -- “bad money drives out good” -- applies to talent: When you have good currency and bad currency in an economy, the bad currency drives the good currency out
  • This happens in Cuba, where the good currency – black market US dollars – gets hoarded while the local currency gets spent
  • It also applies to information: people put the crappy information out, and it drives the good information down
  • It also applies to talent: headhunters, not knowing what sort of people to look for, end up grabbing the people who put the most buzzwords on their resumes
• You don’t want to be one of those buyers

At the end of the presentation, posted a slide dedicated to his late friend, Sam Roweis (1972 – 2010).

[This article also appears in Global Nerdy.]