As I alluded in my last post, I've officially swallowed the Red Pill...at least this month.  For the 3 developers who haven't seen the Matrix--the red pill pops you out of the Matrix into reality, the blue pill returns you to blissful ignorance and a life of computer-generated fantasy...the latter much like owning an XBox.

I want to emphasize some core philosophical points I re-learned on my last project:

  • use supported technologies/products
  • focus on the final goal, not the path to it
  • spend engineering effort where it helps the business
  • when you pay for expert advice, listen to it

Use supported technologies/products:  Open source tools like NUnit, CruiseControl, NDoc, etc. are just super.  They're indispensable to me.  But you'll notice, they're not really embedded in the actual output; they're tools we use during development.  The flip side is code that'll show up in release builds; there's a different animal.  How much do you truly trust what you've included?  How bulletproof is it?  Chances are, you really don't know.  I know how hard it is to truly test code; I mean TEST it!  Beat it up.  Run it for 72 hours at absolute maximum load.  Gather perf-counter and exception data for that whole time.  Verify accuracy of results during that run.  Black-box testing--positive input, negative input, boundary input, garbage input, etc.  Does it leak handles or memory?  Does it degrade over time?  Is it fragmenting the heap(s)?  Is it chewing up threadpool threads?  Is it hiding exceptions but spewing out garbage? 

It's really hard to honestly test this stuff and trust completely in the outcome...I mean, trust to the extent that I'll gladly say "Why YES, Ms. CIO, you can confidently put your $25million/day website on this application.  No really."

I've done this extent of testing, front-to-back, honestly about five times in the past five years.  It's expensive, time-consuming, and many customers just won't buy off on it or won't dedicate the hardware and people to make it happen.

When you go plugging random Mom & Pop LLC code into your code, you should know if they've done this testing, or do it yourself. 

On the other hand--look at quasi-products like PAG's EnterpriseLibrary.  It is BULLETPROOF, and whenever it's discovered it's not, it's fixed.  You have a "neck to wring"--Microsoft's.  It has huge community support--extensions, bug fixes, discussions, and its developers' blogs.  It's had the horrendous brutalizing testing done already--in fact you can SEE the unit tests right there, and read the tester's articles on MSDN. 

Focus on the Final Goal:  Too often on a big project we (developers+architects) get lost on the design, the abstractions, the "frameworks"...in other words, navel-gazing academic purism of the worst degree.  The fact is, every design will evolve during development.  Period.  There is NO AMOUNT of pretty UML and exhaustive user interviews that will change this--which is why I'm so in love with XP's pragmatic weekly iterative approach.  Every day, ask yourself:  "Did we do something today that contributes directly to the end goal?"  Or "Did we implement a feature?  Did we fix a bug?"  At the end of the week, when you do an interim gold release, will the users see measurable progress and no regressions?

I FREAK OUT when I see folks designing "frameworks", without an application inside it that serves as a weekly user-visible benchmark of actual progress.  You're asking for trouble with that old-school waterfall mentality; you can't possibly anticipate every need for the framework unless you are simultaneously loading it with an application.  Much worse, if you're developing a "framework" with multiple apps in mind.  The specifications become so vague, the abstractions so broad, and the timelines so long that they basically never get done.  What have you saved?  You may have reuse, but the effort to get there was more than the time you would have spent adapting smaller, more modular "application blocks" which were more focused on a particular problem.

I guess that last part means:  think first about pattern, template, and process reuse; then about modular "application block" reuse--like CryptoUtility, or all the awesome PAG blocks; THEN maybe think about reusing an entire framework...

Spend Engineering Effort Where It Helps The Business:  Stop!  Why are you writing a thread pool?  Sure it's fun.  I've written one or two of those; and other meaningless things like workflow engines.  But unless you're a software house, it's unlikely you're doing anyone a favor with such a duplication.

By the same token, don't do too many "smart" things--I kick myself routinely looking back on a piece of over-engineered code that gets called maybe once a day.  "do the absolute simplest thing that could possibly work".

But DO put your engineering best into TESTING, then more TESTING.  The bulk of your work comes AFTER you are code- and feature-complete.  Have you load tested?  Stress tested?  Soak tested?  I'm assuming of course you already had thorough TDD-style unit tests which were run every time you automatically built, at least once a day.  This stuff really takes time because it depends on other groups' cooperation.  You have to interface with the infrastructure folks, with management; you're asking for an isolated test-network, with lots of machines and software and setup.  You'll have devs and testers hanging out looking glazed poring over perf-mon logs and trace files and dumps.  You'll have bleary-eyed coders who aren't used to using LoadRunner writing scripts--and wondering if they're representative of real-life data.

And before all that--you've added testing hooks, right?  You can turn on and off detailed tracing?  You have performance counters, dozens of them?  You have MSI installers that go BANG and put it in right?

That's where the real engineering effort goes.  It's not algorithms; those things you find in books for the most part.  It's the hard tedious stuff, that stuff above, that counts.

When You Pay for Expert Advice, HEED IT:  I'm astonished at how many companies will gladly shell out $10,000 a week for consultants, and then blandly ignore their advice; that's one of those "ya outta be fired" sins.  Let's pretend there's an adequate screening process for said consultant, and they know what they're talking about--as proven by a track record of customers willing to give testimony, and a list of successful projects.  They work for a reputable and preferably the vendor company. Probably would be nice if they're published, too.  And be sure they're in their area of expertise--and if they're not, they're willing to admit it and bring in other expertise.

Those things said, you now have an expert.  And you're paying for it.  Will you listen?  People on the team may question the advice, and that's good; the consultant should hear it and ably defend his/her thesis.  But when push comes, chances are they're right.  If they're not why are you paying ten grand a week??

The best part about a vendor consultant--the company's reputation is on the line, not just his/hers.  And that company will move mountains to see a good outcome; use that to your advantage!  Chances are they actually know how to get the best out of their own products....funny how that works :)

 

Man.  Couple more posts like this and I'll just be moving right into management.  Think I'll plug that jack-thingy back into my head now and jump a couple of skyscrapers....