Patrick comments on my When To Automate post:
I am confused by your third bullet in that unstable/newly developed/high change code should have automated tests written for it.
It has been my experience that test automation is a great deal of work to maintain and that the more changes there are to an application under test the more automation maintenance work is required. It almost never pays off to write automation against an application that is undergoing major changes to its UI, structure, or functionality. Having to rewrite large portions of the automation code and cases on every build often takes longer than it would take to perform those tests manually. Additionally it is of paramount importance that a human being’s intuition is leveraged on very new code to account for that which had not yet been considered. I have had very good luck using automation for regression testing or getting lots of boundary cases executed, but only on code that was pretty well established.
Am I missing something?
That said, certainly there are cases where automation is not worth the trouble. (See my post I Want Testers, Not Automators for example.) And I completely agree that automation does not take the place of real live testers poking at code. My goal in writing automation is to free this real live tester from doing one set of test cases over and over and instead let me spend time banging on my app in new and devious ways. Of course, automation has all sorts of side benefits, such as big-time payoffs come sustained engineering time, and automatic checking for regressions. But freeing me from drudgery is the main point as I see it.