Random Disconnected Diatribes of a p&p Documentation Engineer
Listening to the radio one day this week, I heard somebody describe golf as being "a series of tragedies obscured by the occasional miracle". It struck me that maybe what I do every day is very similar. If, as a writer, you measured success as a ratio between the number of words you write and the number that actually get published, you'd probably decide that professional dog walker or wringer-out for a one-armed window cleaner was a far more rewarding employment prospect.
Not being a golfer myself (see "INAG"), I'd never heard that quote before. However it is, it seems, quite well known - I found it, and several similar ones, on various golf Web sites. Including a couple that made me think about how closely the challenges of golf seem to mirror those of my working life. For example, "Achieving a certain level of success in golf is only important if you can finally enjoy the level you've reached after you've reached it." How do you know when you've reached it? Or can you actually do better next time? Or maybe you should just assume that you're doing the best you can on every project? That seems like a recipe for indolence; surely you can always get better at what you do? But if you keep practicing more and more, will you just end up creating more unused output and reduce your written/published ratio?
Or how about "Golf is the only sport where your most feared opponent is you"? I find that writing tends to be a one-person activity, where I can concentrate without the distractions of the outside world penetrating the whirling vortex of half-formed thoughts and wild abstractions that are supposed to be elements of a carefully planned and managed process for distilling knowledge and information from the ether and converting it into binary data. I always assumed that professional developers tended to have the same issues, so I have no idea how they can do paired programming. Imagine two writers sat side by side arguing about which words to put where, and if that should be a semi-colon or a comma, while trying to write an article.
I've always maintained that the stuff I create should, by the time it actually pops up in the Inbox of my editor and reviewers, be complete, readable, as free of spelling errors and bad grammar as possible (unlike the subject of one of my previous posts), and - of course - technically accurate. OK, so you can't always guarantee all of these factors, but making people read and review (and, more to the point, edit) stuff that is half-baked, full of spelling and grammar faults, and generally not in any kind of shape for its intended use just seems to be unprofessional. It also, I guess, tends to decrease the chance of publication and reduce your written/published ratio.
Ah, you say, but surely your approach isn't agile? Better to throw it together and then gradually refactor the content, modify the unsuccessful sentences, and hone the individual phrases to perfection; whilst continually testing the content through regular reviews, and comparison with reality (unless, I suppose, you are writing a fantasy or science fiction novel). Should "your most feared opponent" be the editor? I'm not sure. When it comes back from review with comments such as "This is rubbish - it doesn't work like that at all" or "Nice try, but it would be better if it described what we're actually building" you probably tend to sense a shift in most-feared-opponentness.
I suppose I should admit that I once tried writing fiction (on purpose), but every page turned out to be some vaguely familiar combination of the styles of my favorite authors. Even the plot was probably similar to something already published. Thankfully I gave up after one chapter, and abandoned any plans to write the next block-selling best-buster. And I couldn't think of a decent title for it anyway. Written/published ratio zero, and a good reason to stick with my proper job of writing technical guidance for stuff that is real. Or as real as a disk file full of ones and zeros can be.
And while we're talking about jobs, they have a great advert on one of our local radio stations at the moment. I've never figured out what they're trying to sell, but it does offer the following useful advice: "If you work on the checkout in a hand-grenade shop, it's probably best not to ask customers for their PIN". However, in the end, I suspect that none of the quotes can beat Terry Pratchett's definition of the strains of the authoring process: "Writing is easy. You only need to stare at a piece of blank paper until your forehead bleeds".
I've been trying something new and exciting this week. OK, so it's perhaps not as exciting as bungee jumping or white-water rafting, but it's certainly something I've not tried before. I'm experimenting to see if I can use Team Foundation Server (TFS) to monitor and control the documentation work for my current project. As usual, the dev guys are using agile development methods, and they seem to live and die by what TFS tells them, so it must be a good idea. Maybe. But I suppose there's no room in today's fast-moving, high-flying, dynamic, and results-oriented environment for my usual lackadaisical approach of just doing it when it seems to be the best time, and getting it finished before they toss the software out of the door and into the arms of the baying public.
So, dive into the list of work items for the current iteration and see if I can make some wild guesses at how long the documentation work will take for each one. Ah, here's a nice easy one: fix some obscure bug that hardly anybody was aware of. That's a quarter of an hour to add a note about the fix to the docs. But it seems like I can only enter whole hours, so I suppose I'll have to do it slowly. And here's another no-impact one: refactor the code for a specific area of the product. And these three are all test tasks, so I don't need to document them either. Wow, this is easy. It looks like I'll only have three hours work to do in the next fortnight. Plenty of time to catch up on the gardening and DIY jobs I've managed to postpone for the last year or three.
Next one - completely change the way that the configuration system works. Hmmm, that's more difficult. How many places in the 900 pages will that have an impact? And how long will it take to update them all? Oh well, take a wild guess at four days. And the next one is six completely new methods added to a class. That's at least another three days to discover how they work, what they do, and the best way to use them. And write some test code, and then document them. After a few more hours of stabbing in the dark and whistling in the wind, I can add up the total. Twenty three days. That should be interesting, because the iteration is only two weeks. Looks like I need to write faster...
Now skip forward to Friday, and go back to TFS to mark up my completions. How do I know if a task is done or not? Will the code change again? Will changes elsewhere impact the new updates to the docs? When will test complete their pass on the code so I can be sure it's actually stable? And do I have to wait for test to review my docs? Or wait for the nice lady who does the English edit to make sure I spelt everything right and didn't include any offending letters (see Oending Letters). I guess I've finished my updates, so I can mark them as "Done". But does that mean I need to add a task for review, test, and edit for my updates? Surely they won't want to work through the doc until it contains all of the updates for that particular section of the product?
So this isn't as easy as it may have seemed at the beginning. In fact, I've rambled on in the past about trying to do agile with guidance development (see Tragile Documentation). I can see that I'll be annoying people by asking them to test and edit the same doc several times as I make additional changes during the many upcoming iterations. Perhaps I should just leave them all as "In Progress"? But that will surely mess up the velocity numbers for the iteration. And they'll probably think I went off on vacation for the two weeks. Not that the sound of the waterfall in my garden pond and the ice cream van that always seems to go past during the daily stand-up call won't tend to reinforce this assumption.
Still, it will be interesting to see how it all pans out. Or whether I spend more time fighting with my VPN connection and TFS than actually writing stuff...
Isn't it funny how - after a while - you tend not to notice, or you ignore the annoying habits of your closest colleagues. As I work from home, some 5,000 miles away from my next closest colleagues, the closest colleague I have is Microsoft Vista (yes, I do lead a sad and lonely life doing my remote documentation engineering thing). I mean, I've accepted that sometimes when I open a folder in Windows Explorer it will decide to show me a completely different view of the contents from the usual "Details" view I expect. I suppose it's my own fault because I happen to have a few images in there as well as Word documents, and Vista thinks it's being really helpful by telling me how I rated each one rather than stuff I want to know - like the date it was last modified.
But worse of all is the search feature, or perhaps I should call it an unfeature. In XP, I could select a folder and enter a partial file name then watch as it wandered through the subtree (which, with my terrible memory of where I put stuff was often "C:\"). It told me where it was looking, and I knew it was just looking at filenames. If I only wanted to search the contents of files, I could tell it to do that. In Vista, I type something in the search box and get a warning that the folder isn't indexed, then a slow progress bar. I've no idea where it's looking, or what it's looking for. And neither does it by the look of the results sometimes.
It seems to decide by itself whether to look inside files (so when I search for some part of a filename I get a ton of hits for files that happen to contain that text), yet it seems incapable of finding the matching files by name. I have to either wait till its finished or open the Search Tools dialog before I can get at the advanced options to tell it what kind of search I want and if I want all subfolders to be included. And when I do look for something in the contents of the files, I get either 1,000 hits or none at all. In fact, I've actually resorted to using TextPad to search for strings in text files recently. And after all that, I have to go clicking around the folder tree (while trying to cope with the contents oscillating widly from side to side as I open each one) to get back to where I was because it helpfully moved the folder view to the very end of my long and complicated list of folders.
I can see that the Vista approach may be easier and quicker for simple searches, but I can't help feeling that it often just gets in the way by trying to be too clever and "usable" (something I've grumbled about before - see Easter Bonnets and Adverse Automation). Maybe some of the problem is that I'm continually creating and deleting folders and moving stuff around as I gracefully slither between projects and my other daily tasks. I've tried setting default folder and search options, but I guess Vista can't cope with my indecisiveness. Perhaps I should just keep everything in one folder called "Stuff". But then I'd need a really good search engine...
Probably a lot of this ranting comes about because of the totally wasted day spent trying to get some updated software to run on my machine. The software in question uses a template and some DLLs that get loaded into Word, some other DLLs that do magic conversion things with the documents, and some PowerShell scripts that drive the whole caboodle. So after the installation, PowerShell refused to have anything to do with my scripts, even though I configured the appropriate execution policy setting. Finally I managed to persuade it to load the scrips, but all it would do was echo back the command line. In the end, I copied the scripts from the new folder into the existing one where the previous version was located, and the scripts ran! How do you figure that? Is there some magic setting for folder permissions that I have yet to discover?
And then I had to run installutil to get the script to find some cmdlets in another assembly, and delay sign a couple of other assemblies that barfed with Vista's security model. After about 6 hours work, it looked like it was all sorted - until I went back into Word to discover that the assemblies it requires now produced a load error. In the end, the only working setup I could achieve was by uninstalling and going back to the previous version. And people wonder why I tend to shy away from upgrading stuff...
At least there is some good news - the latest updates to Hyper-V I installed that morning included a new version of the integration components, and (at least at the moment) I've still got a proper mouse pointer in my virtual XP machine (see Cursory Distractions). So I guess the whole day wasn't wasted after all.
Footnote: Actually it was - my mouse pointer has just gone back to a little black dot...
It's a good thing that Tim Berners-Lee is still alive or he'd probably be turning in his grave. I was hoping to find that my latest exploration of Web-based Interfaces for Kommunicating Ideas would lead me to some Wonderfully Intuitive Kit Intended for sharing knowledge and collecting feedback, but sadly I'm Wistfully Imagining Knowledge Instruments that should have been around today - and aren't. And, yes, I'm talking about wikis.
As an aside, you've probably seen those word ladder puzzles in the Sunday papers where you have to turn one word into another by adding one letter at a time. Seeing as how I talked about Wii last time, and wiki this week, maybe I can continue the pattern. Any suggestions of a five-letter topic that contains the letters w, i, k, and i are welcome...
Anyway, coming back to the original topic, it could all have been so different. Instead of the awful and highly limited format capabilities, and the need to spend an inordinate amount of time creating conversion tools, we could have had a ready-built, comprehensive, easy-to-use, and amazingly less grotty technology than wikis if we hadn't let some guy get in the way some time back in the mid nineties. Mind you, it's probably not wholly fair to blame it all on Marc Andreessen and Netscape; Microsoft followed the same path and I guess are equally guilty. I suppose the drive for world-wide adoption, the opening of the Web to the unwashed public, and commercial factors in general were the real reason behind it all.
You see, when our Tim and his team invented HTML and the associated server-side stuff, the intention was that it would be a collaboration and information sharing mechanism. User agents (what we now call browsers) would fetch content from a server if the user had read permission and display it in a documentation format using markup to indicate the structure and type of content it contained. Elements such as "p" (paragraph), "strong", "ul" (unordered list), "ol" (ordered list), "dl" (definition list), and the "h(x)" (heading) elements would indicate the type of content contained, not the way it should be displayed.
But, more than that, the user agent would allow the user to edit the content and then, providing they had write permission on the originating server, update the original document with their revisions and comments using elements such as "ins" and "del". However, as we've seen, the elements in HTML have come to represent the displayed format rather than the context of the content, and browsers are resolutely read-only these days. Of course, more recent mechanisms such as CSS and XML transformations allow us to move back to the concept of markup indicating context rather than display attributes. But if you want to see what it should have been like, download and install the W3C reference browser Amaya and see how it allows you to edit the pages it displays.
So, instead we had to invent a new way to do collaboration, and wiki caught on. OK, it's probably fine for quickly knocking up a few pages to allow users to edit, review, and provide feedback. But it's seriously broken compared to doing anything sensible like you can with an XML-based format (which includes HTML 4.x). It's a kind of "Web for dummies" approach, where the concept of nesting and formatting content consists of a few weird marker characters that easily get confused with the content - even to the extent that you need to "escape" things like variable names that start with an underscore.
I guess this railing against technology comes about because I just spent two days building a tool to convert our formatted Word docs (which use our own DocTools kit to generate a variety of outputs) into a suitable format for Codeplex wiki pages. I even had to build another "kludge" tool to add to our growing collection - it's the only way I can find to do the final content tweaks. All I can say is, whoever dreamed up this format never tried to do stuff like this with complicated multi-topic Word source documents...
And, worse still, you have to add each page to the wiki project site individually and then attach the image files to it. OK, so the tool does give you a TOC and text files you can copy, but it sure would be nice to have a way to bulk upload stuff. My current test document set has 59 pages, so I can see I'll be spending a whole day clicking Save, Attach, and Edit.
But maybe that has some advantages. I'll have less time to spend inflicting the general public with Wild and Incoherent Komplaints and Insults in my blog...