Random Disconnected Diatribes of a p&p Documentation Engineer
At one time you had to work in a museum to be a curator, but the wonders of information technology mean that now we can all exhibit our technical grasp of complicated topics and elucidate the general population by identifying the optimum resources that help to answer even the most complex of questions.
I'm talking about the new Curah! website here. The idea is simple: a resource that gathers together the questions most commonly asked about computing topics; each with a carefully and lovingly crafted set of links to the most useful blogs, reference documents, tools, and other information that offers a solution to the question.
Anyone can register and create a curation, and the site is optimized for search engines to make it easy to find answers. It's still in beta as I write this, but already has hundreds of answers to common questions. The great thing is that the curations are not just a set of links like you'd get from the usual search engines, which tend to optimize the list based on keywords in the resources, the number of links to them from other pages, and the newness of the content. None of these factors can provide the same level of usefulness as a list compiled by an expert in the relevant topic area who regularly creates and uses information that provides the maximum benefits.
My interest in the Curah! site also comes about partly because I am part of the group that defined the original vision and got it started. I've also added a few curations of my own, which are centered on the topic area that I now seem to have been permanently assigned to - Windows Azure application design and deployment. My regular reader will probably have noticed this from the rambling posts on this blog in the past.
However, one point that concerned me was that, having created my own curations, I am now responsible for maintaining them. As I plan to create more in the future, I was beginning to wonder if I would end up spending all of every Monday just checking and updating them as the target resources move, disappear, or I discover new ones. What I needed was some type of automated tool that would make this job easier. So I built one.
The CurahCheck utility is a simple console-based utility that will check one or more views on the Curah! site by testing all of the links in each curation ID you specify. The curation title and the linked page titles can be displayed to ensure that it is valid and that all of the linked resources are still available. It can also be run interactively, or automatically from a scheduled task.
The utility generates a log file containing details of the checks and any errors found. It can also generate an HTML page for your website that shows the results of the most recent check and the contents of the log file. If you have access to an email server, the utility can send warning email messages when an error is detected in any of the views it scans.
If you are a Curah curationist you can download the utility from here, and use and modify it as you wish. The source project and code for Visual Studio 2012 is included. Before you use it, you'll need to edit the settings in the configuration file to suit your requirements - the file contains full details of the settings required and their effect on program behavior.
Of course, the usual terms and conditions about me not being responsible for any side-effects of using the program, such as your house falling down, your children being eaten by a dinosaur, or your computer bursting into flames, still apply...
Having run out of ideas, and given up Binging a solution for my intermittent connectivity problems around Windows 8, IE 10, and Outlook, it was time to stop playing nicely. Time instead for a day with my head in the server cabinet, a handful of network cables, some sticky labels, and decisive action.
The problems documented over the past few weeks (see All I Want For Christmas) were still intermittently annoying, as well as being annoyingly intermittent. I was totally unable to track down any DNS problems, despite many hours experimenting with different forwarders, root entries, test scripts, clearing caches, and more.
I'd logged all dropped packets in ISA server for a day, but there were none related to the problem from the machines under test. Though ISA's performance monitor was consistently reporting an average dropped packet rate of 0.3 per second and I wasted half an hour tracing these back to my wireless access point. Even though all of its fancy USB, printer connection, and media sharing features are turned off it still insists on sending out a network discovery packet every three seconds. Highly annoying.
Then I read more on the ISA Server blog sites about how using the proxy client changes the behaviour of machines connected to ISA. Of course, I haven't actually installed the proxy client directly since XP days. I just assumed that some clever mechanism in Windows Vista, 7, and 8 used the Gateway/Router setting specified in DHCP to find the proxy server and set themselves up for it automatically.
What I read suggested that ISA itself is doing DNS lookups in response to requests from clients, whereas a ping or nslookup on the client uses the network DNS server or does its own DNS lookup. So trying to track faults with nslookup after a connection failure was a waste of time. By now I was rapidly tiring of trying to be a network administrator, and I didn't bother following this up any further to see if it really is the case.
All of which prompted the decision to perform some radical surgery in the server cabinet, and get rid of ISA Server altogether. It's a Hyper-V VM, so it won't reduce the physical server count - but it will simplify administration and backup tasks and, hopefully, resolve my connectivity problems. I replicated all the ISA outbound rules in the firewall of my load-balancing router, which sits between the ISA server and the two ISP modems. A day monitoring the router logs and fine-tuning rules suggested this would work fine.
Reconfiguring the network was deceptively easy. Simply power off the ISA VM, change the IP address of the router to the same as ISA (the address already specified in the DHCP scope options), and run some network penetration tests. Instantly everything seems to be faster, web page loads are snappier, and no sign of smoke or loud bangs. And if it all goes wrong, or turns out to be a mistake, I still have the ISA Server VM so I can easily revert to the previous configuration.
But will my simple load-balancing router be able to cope with all that extra firewalling and packet shifting load once I start hammering the network with my usual working-week vigour? It's only an old LinkSys RV042 with a 100 Mbs Ethernet port. Do I need to upgrade to something like the new RV320 instead? I guess I'll soon find out.
And, of course, the question now is what will I do next if my intermittently annoyingly connectivity problem is still annoyingly intermittent...?
No I'm not talking about the clever "please mend my computer" tools that you can run on Microsoft's website. I'm talking about my regular tasks trying to fix all the things that break in our house, seemingly one per day at the moment. It's usually a wide and assorted selection of tasks; this week comprising a table lamp, a DECT phone, my fishpond filter, SQL Database, and Windows 8.
Invariably my fixit jobs fall into two categories: "not worth the effort" and "fingers stuck together with superglue again." The second category tends to be associated with jewellery and ornament repairs (the latter typically being my fault), but the table lamp repair was in the first category - an IKEA lamp that originally cost twelve pounds new, but came from a jumble sale at a cost of two pounds. My wife had bought six new 12V halogen bulbs for it that cost more than the lamp did originally, but it refused to work. The transformer is a sealed unit, and the wiring is sealed inside the lamp. Do I spend three hours breaking it just to see what's wrong?
But I had better luck with the DECT phone, a couple of new triple-A rechargeable batteries brought it back to life. And I managed to fix my fishpond filter using the typical handyman technique of a few bits of bent wire. These are the satisfying kind of repair jobs where you can carry around a big toolbox, and look like you know what you're doing. It works even better if you can arrange it so you didn't shave for a couple of days before.
What's most annoying however, and has little capacity for appearing butch and manly, is the so-far-unmentioned third category of repair jobs: broken software. A year or so ago I parcelled up my locally-hosted websites and dispatched them to heaven - or, to be more precise, to live in the clouds of Windows Azure. Amongst them is our local village website, which is reasonably complex because it handles news, events, photos, and has a user registration facility.
Yes, I know I should have adapted it to use claims-based authentication and Windows Azure Active Directory, but I just never got round to it; instead it has an ageing "aspnetdb" database sitting in SQL Database. There's only one role instance, and it works fine. Well, almost. Yes I did do comprehensive testing after deployment before the site went live, checking that I could add and edit all the items on the site, sign in and change my password, view lists of registered users, and see the error lists in the admin section of the site.
I even made sure the site could create and remove users, and allow them to edit their details. But it turns out that my test coverage was a little less than perfect. For the first time since deployment I needed to use the functions of RoleManager to change the roles for a registered user. And everything broke. Even going into the SQL Database management portal and deleting the row in the data view of the table failed. As did executing a SQL DELETE statement in the query window.
It took some searching based on the error message about SQL collation to find the answer. And the fix is so simple that it should painted in six inch high letters on a big piece of wood and nailed to the Azure portal. Simply open the stored procedure named aspnet_UsersInRoles_AddUsersToRoles and insert the text COLLATE Latin1_General_CI_AS into the DECLARE @tbNames... line as shown here, and then do the same with the aspnet_UsersInRoles_RemoveUsersFromRoles stored procedure.
To get to the stored procedures, open the database management page from the main Windows Azure portal and choose the Manage icon at the bottom of the page. Sign into the SQL Database server and choose Design in the left-hand navigation bar, then choose Stored Procedures in the tabs near the top of the page. Choose the Edit icon next to the stored procedure in the list and do the edit. Then choose the Save icon on the toolbar. Repeat with the other stored procedure.
Meanwhile, ever since I upgraded my trusty Dell E4300 to Windows 8 I've been plagued by wandering cursor disease. I'll be typing away quite happily and suddenly the letters appears in the middle of the next paragraph, or halfway along the next line. It's amazing how much something likeum. this can screw up your finely crafted and perfectly formatted text. It really is a pain in the b
Of course, the answer is the same as most Windows 8 problems with older hardware. The jazzy new all-singing and all-dancing hardware drivers that come with Windows 8 don't always do the same as the wheezing and arthritic ones that came on a disk with the computer when you bought it. Thankfully, plenty of other people are having the same variable input position issue as me, and their posts led me to the updated Alps touchpad driver on Dell's website.
Not that it fixes the problem - my touchpad still seems to think it's morphed into an X-Box Kinect - if I wave a hand anywhere near it the insertion point cursor leaps madly around in my Word document. But the Dell driver can detect when you plug in a proper mouse, and disables the touchpad automatically. Problem solved.
Now I just need to prise my finger and thumb apart so I can mend a pair of my wife's earrings...
Agile development is an important technique here at p&p; and throughout much of Microsoft. However, I'm yet to be convinced that it's a good way of creating user guidance and documentation. It seems to me that the process often gets in the way more than it helps to produce a great final product.
I've rambled on many times in the past about agile documentation. Most specifically in Can Writers Dance The Agile? and other posts here. Yet I keep thinking it needs deeper investigation - especially as the group I'm officially assigned to, CSI, insists we keep prodding it to see if it works.
Note: Here at Microsoft, CSI is "Content Services & International". It's probably a bit less exciting than doing clever stuff with fingerprints and DNA, but we do have fancy computers that nearly come close to those you see in the TV versions of CSI. Maybe we should call ourselves "CSI Microsoft," wear white coats, and walk around with a flashlight held over our head.
Anyway, coming back to the point, we've had another go at agile docs recently. Instead of a solid and agreed structure plan and detailed implementation notes of what we wanted to achieve, we started off with a "vision" (but see this post from a couple of weeks ago) and "brainstorming" to get a list of topics. Then we "asked the audience" with developer and advisory board surveys to see which of the topics they liked most. Next, we threw together some rough notes about each topic and then produced a first draft of each topic document.
The next stage was multiple reviews. After each review we restructured the content, and often the whole document, to get it closer to the ideal. But, because it's agile, we often changed our mind about what the doc was trying to achieve (remember, there's no detailed implementation notes to guide us here) and completely rewrote it. And then did this again after the next review pointed out the holes left by (or introduced by) the previous reworking. Some documents went through four or more complete restructurings, and several were rewritten twice.
The agile process also resulted in some of the "envisioned" topics being abandoned, often after they'd been rewritten several times, because - after investigation - they were too hard to define accurately. Or because they turned out to be not really relevant or practical. And as there is no overall structure plan, it was hard to see which topic should contain what section of the content, and how it related to other sections. It also meant that the focus could only ever be at the individual document level rather than the entire guide level, because that isn't defined yet.
But what we could be sure of was that each individual topic was precise, accurate to the nth degree, and compact with no irrelevant content. This is, of course, extremely important; especially if the content will be used as reference information. But is it still "guidance?" I guess that's the core of the problem. What exactly should "guidance" look like?
Typical equivalents of the word "guidance" include the obvious ones such as "help" and "advice." However, there are broadly two categories of meaning: "leadership" and "assistance." These almost seem like opposites - one leading from the front and the other pushing from the back. Yet the sub-meanings according to my thesaurus include "direction", "support", "management", and "control." Some of these seem more like they are related to aiding through understanding, whereas others are more related to despotic regulation. I'm going to take a guess that we're aiming for the first of these.
So did we end up with what we wanted, and does it aid through understanding? It's not completely finished yet and it will be a while before we see any user feedback. And there's no doubt that the content will be extremely useful for the specific users and use cases it addresses. But it still seems like we missed opportunities. The agile process narrowed the focus and transformed the content based on individual reviews of segments, and forced additional depth of detail. It also removed a lot of the general "understanding" content because it already was familiar to the experienced reviewers. Most of all, it resulted in huge amounts of extra work writing and repeatedly updating (and then sometimes discarding) the content.
Without a predefined (if flexible) structure and an overall feel for how it will all fit together and appear to the readers, there is nothing to prevent this wandering. As a writer, I'm lost if I can't see the finished thing in the back of my mind. It's like driving through a city in your car while blindfolded, and navigating by reversing and choosing a new direction at random every time you hit something.
Maybe agile is good thing that can help to focus guidance more accurately. Or maybe it just allows the guidance to wander away from the original vision and risk irrelevance. If the original plan was to provide guidance around X and you end up with fabulous documentation of Y instead, did you do a good job?
So I continue to battle with Windows 8.1 and Outlook 2103 on my nice big Dell workstation. Our IT support department have given up on me, saying it's obviously a problem with my own network configuration. And it looks like they are correct. It's just a shame they can't tell me what the problem is.
For a long while Outlook has been doing strange things. It had a few days of keeping count of how many messages I sent during the day (see Downwardly Upgraded) but that problem seems to have gone away again. It also regularly loses its connection to the mail server and then restores it - sometimes immediately but at other times it takes several minutes. And, best of all, it waits about ten minutes before displaying the Windows 8 desktop notification of new emails. Usually I've read and deleted them by the time it pops up.
The same issues occur on other computers as well, both Windows 8 and Windows 7, but only when connected to my internal domain and going out through my ISA proxy server and load-balancing router to one of my ISPs. Bypassing all this, and plugging directly into the back of the ADSL modem, seems to solve the problem. So it's increasingly looking like an internal network issue.
I've checked all the DNS servers I use as Forwarders and they resolve fine. There are no event log warnings in any of the servers. The ISA proxy server log reveals no denied requests to my email host, and only one or two to anywhere else - certainly not enough dropped packets to justify the problems with Outlook. I turned on logging in Outlook and used the new Microsoft Message Analyser to read them, but I can't make any sense of the contents. I tried network packet sniffing, but that revealed nothing useful from the few bits I could decipher.
And then there's browser. Occasionally it has a spell of not being able to find sites. Today it couldn't find Bing for about five minutes yet other sites worked fine. Then it couldn't find the MSDB Blogs site. Other days it can't find anything for several minutes, then it all starts working again. Yet everything else seems to work just fine. My internet radio plays radio, Lync links, Team Foundation Server serves, and News has the up to date news.
I've tried disconnecting the modem for the cable ISP connection and just using ADSL to a different ISP, and vice versa. I've run network diagnostics and DNS validation checks. I've monitored the performance of the ISA 2006 server, and double-checked all the rules. I've played with the routing tables in the separate hardware load balancer. I tried specifying the proxy server settings manually in the browser. All to no avail.
Maybe my network is just too complicated. It's left over from the days when I was an IT consultant (well, jumped-up writer and occasional conference speaker actually) when I needed lots of infrastructure for developing and testing the few applications I built for customers. And, I guess, because I enjoyed playing with hardware. Perhaps it's time to review that decision. Do I actually need:
...just to use Word, Visio, and Visual Studio? Probably not. And all of a sudden I can see why my electricity bill is so high.
Perhaps my Christmas present to myself will be a nice hardware firewall that I can just plug everything else into and forget about it...
So I've temporarily escaped from Azure to lend a hand with, as Monty Python would say, something completely different. And it's a bit like coming home again, because I'm back with the Enterprise Library team. Some of them even remembered me from last time (though I'm not sure that's a huge advantage).
Enterprise Library has changed almost beyond recognition while I've been away. Several of the application blocks have gone, and there are some new ones. One even appeared and then disappeared again during my absence (the Autoscaling block). And the blocks are generally drifting apart so that they can be used stand-alone more easily, especially in environments such as Azure.
It's interesting that, when I first started work with the EntLib team, we were building the Composite Application Block (CAB) - parts of which sort of morphed into the Unity Dependency Injection mechanism. And the other separate application blocks were slowly becoming more tightly integrated into a cohesive whole. Through versions 3, 4, and 5 they became a one-stop solution for a range of cross-cutting concerns. But now one or two of the blocks are starting to reach adolescence, and break free to seek their fortune in the big wide world outside.
One of these fledglings is the block I'm working on now. The Semantic Logging Application Block is an interesting combination of bits that makes it easier to work with structured events. It allows you to capture events from classes based on the .NET 4.5 and above EventSource class, play about with the events, and store them in a range of different logging destinations. As well as text files and databases, there's an event sink that writes events to Azure table storage (so I still haven't quite managed to escape from the cloud).
The latest version of the block itself is available from NuGet, and we should have the comprehensive documentation available any time now. It started out as a quick update of the existing docs to match the new features in the block, but has expanded into a refactoring of the content into a more logical form, and to provide a better user experience. Something I seem to spend my life doing - I keep hoping that the next version of Word will have an "Auto-Refactor" button on the ribbon.
More than anything, though, is the useful experience it's providing in learning more about structured (or semantic) logging. I played with Event Tracing for Windows (ETW) a few times in the past when trying to consolidate event logs from my own servers, and gave up when the level of complexity surpassed by capabilities (it didn't take long). But EventSource seems easy to work with, and I've promised myself that every kludgy utility and tool I write in future will expose proper modern events with a structured and typed payload.
This means that I can use the clever and easy to configure Out-of-Process Host listener that comes with the Semantic Logging Application Block to write them all to a central database where I can play with them. And the neat thing is that, by doing this, I can record the details of the event but just have a nice useful error message for the user that reflects modern application practice. Such as "Warning! Your hovercraft is full of eels...", or maybe just "Oh dear, it's broken again..."
According to Readers Digest, there's a dyslexic agnostic insomniac out there somewhere who lies awake all night pondering on the meaning of dog. Thing is, it really should be "doG", not "dog". But it seems that, according to our most recent style guide here at Microsoft, capital letters are fast becoming obsolete, although it's probably not so that jokes like this will be more accurate.
The aims of the style guide updates are constructive and practical. We should provide help and guidance only where it will be useful, and phrase it in such a way that it appears friendly, open, and easy to assimilate. All very sensible aims, though I've yet to discover where simplifying content so that it makes it harder to use, or sprinkling it with exclamation marks and apostrophe-shortened words (such as "don't" and "shouldn't") is advantageous.
But behind all this is an undercurrent of practical mechanisms for actual word styles and structure. For example, in the previous paragraph I committed the sin of "excess-hyphenation" (or, to be more exact "excess hyphenation"). I'm sure Microsoft style gurus aren't aiming to purge documentation of all hyphens, though it sometimes feels like it. I now have to use "bidirectional" instead of "bi-directional", "rerouting" instead of "re-routing", and "cloud hosted" instead of "cloud-hosted". Though it seems I can still get away with "on-premises", as in "deployed to an on-premises server". Perhaps "onpremises" is just one step too far. Though I do like that they ban the use of "on-premise" because a premise is, of course, a proposition upon which an argument is based or from which a conclusion is drawn.
And then there's the race to remove capital letters from as much of the content as possible. Things that used to be a "Thing", such as "Windows Azure Storage" are now just a "thing", such as "Windows Azure storage". Tables are now tables, and Queues are now queues. I did think that "BLOB" might survive the cull because it's an acronym for Binary Large Object, but sadly no. It's now a "blob" in the same way as a patch of spilled custard is.
Mind you, there are other rules that make it hard to create guidance that reads well. I still haven't found a solution for the repetitiveness in phrases such as "stored in an Active Directory directory", or the fact that the Access Control Service (ACS) is no longer a service; it's just "Access Control" and can't even have "service" after it. So "authenticated by ACS" now becomes "authenticated by AC", which seems too vague so I end up with "authenticated by Windows Azure Access Control" and repetitive strain injury.
I get that over-capitalization and excess hyphenation can make the text harder to read and assimilate, and that we need to provide friendly guidance that uses familiar words and styles. And thankfully I have wonderful editors who apply all the rules to my randomly capitalized and hyphenated text (though, sadly, not to my blog posts). But I wonder if we'll soon need to start writing our guidance in txt speak. imho well mayb need to 4go sum rules b4 its 2 l8...
I discovered this week why builders always have a tube of Superglue in their pockets, and how daft our method of heating houses here in the UK is. It's all connected with the melee of activity that's just starting to take over our lives at chez Derbyshire as we finally get round to all of the modernization tasks that we've been putting off for the last few years.
I assume that builders don't generally glue many things together when building a house - or at least not using Superglue. More likely it's that "No More Nails" adhesive that sticks anything to anything, or just big nails and screws. However, the source of my alternative adhesive information was - rather surprisingly - a nurse at the Accident and Emergency department of our local hospital.
While doing the annual autumn tidying of our back garden I managed to tumble over and poke a large hole in my hand on a nearby fence post. As I'm typically accident prone, this wasn't an unexpected event, but this time the usual remedy of antiseptic and a big plaster dressing didn’t stop the bleeding so I reluctantly decided on a trip to A&E.
Being a Sunday I expected to be waiting for hours. However, within ten minutes I was explaining to the nurse what I'd done, and trying to look brave. Last time I did something similar, a great many years ago and involving a very sharp Stanley knife, I ended up with several stitches in my hand. However, this time she simply sprayed it with some magic aerosol and then lathered it with Superglue. Not what I expected.
But, as she patiently explained, they use this method now for almost all lacerations and surgery. It's quicker, easier, keeps the wound clean and dry, heals more quickly, and leaves less of a scar than stitches. She told me that builders and other tradesman often use the same technique themselves. Obviously I'll need to buy a couple of tubes for future use.
Meanwhile, back at the hive of building activity and just as the decorator has started painting the stairs, I discover that the central heating isn't working. For the third time in twelve years the motorized valve that controls the water flow to the radiators has broken. Another expensive tradesman visit to fix that, including all the palaver of draining the system, refilling it, and then patiently bleeding it to clear the airlocks.
Of course, two of the radiators are in the wrong place for the new kitchen and bathroom, so they need to be moved. Two days later I've got a different plumber here draining the system again, poking pipes into hollow walls, setting off the smoke alarms while soldering them together, randomly swearing, and then refilling and bleeding the system again.
But what on earth are we doing with all this pumping hot water around through big ugly lumps of metal screwed to the walls anyway? Isn’t it time we adopted the U.S. approach of blowing hot air to where it's needed from a furnace in the basement? When you see the mass of wires, pipes, valves, and other stuff needed for our traditional central heating systems you have to wonder.
Mind you, on top of all the expense, the worst thing is the lump on my arm the size of a tennis ball where the nurse decided I needed a Tetanus shot...