Since life can't be all Team Foundation Server and rainbows I've started getting serious about learning more about Ruby and the Rails framework. To keep the TFS focus on this blog I've started a new blog dedicated to non-TFS tech topics http://www.GhostOnThird.com.
When installing a service path on Team Foundation Server one of the first things that needs to happen is that TFS is "quiesced" - which basically means "to make quiet" - i.e. it's running but it's not accepting new client connections. The tool that does this during the SP installation is called 'TfsQuiesce.exe".
This blog post is to explain what you can do when you see the following in your log file after a failed upgrade:
TFSQuiesce - Team Foundation Server Maintenance Tool
Copyright (c) Microsoft Corporation. All rights reserved.
Using workflow file from location exe.
Executing workflow 'Quiesce DT'...
Disabling SQL Jobs for databases TFSActivityLogging,TFSBuild,TFSIntegration,TFSVersionControl,TFSWorkItemTracking,TFSWorkItemTrackingAttachments,TFSWarehouse
Blocking service account from accessing database TFSActivityLogging
No associated service account for role TFSEXECROLE.
Executing workflow 'Unquiesce DT'...
Enabling SQL Jobs.
Unblocking service account from accessing database TFSActivityLogging
Unblocking service account from accessing database TFSBuild
Unblocking service account from accessing database TFSIntegration
Unblocking service account from accessing database TFSVersionControl
Unblocking service account from accessing database TFSWorkItemTracking
Unblocking service account from accessing database TFSWorkItemTrackingAttachments
Unblocking service account from accessing database TFSWarehouse
Workflow 'Quiesce DT' failed! ExitCode = 8000.
10/31/07 12:17:15 DDSet_Status: Process returned 8000
10/31/07 12:17:15 DDSet_Status: Found the matching error code for return value '8000' and it is: '29206'
10/31/07 12:17:15 DDSet_Error: 8000
10/31/07 12:17:15 DDSet_CARetVal: 29206
10/31/07 12:17:15 DDSet_Status: QuietExec returned 29206
Notice the bolded line "No associated service account for role TFSEXECROLE" - this is the key line.
This error is indicating is that one or more of the databases have no members in the TFS exec role. The reason we are running this check at all is to ensure that we are not running the upgrade as the service account user. Let’s start by figuring out if this is a problem with all seven TFS databases or just one of them.
Run the following SQL statement 7 times – each time changing the “use TfsVersionControl” to one of the other TFS databases:
use TfsVersionControl
SELECT dpMember.name
FROM sys.database_principals dp
JOIN sys.database_role_members drm
ON drm.role_principal_id = dp.principal_id
JOIN sys.database_principals dpMember
ON dpMember.principal_id = drm.member_principal_id
WHERE dpMember.Type = 'U'
AND dp.name IN ('TFSEXECROLE')
I expect one or more of the result sets to be empty.
Ultimately the role can’t be empty. When you find the database(s) with the empty TFSEXECROLE role you should add the appropriate service account user to the role for that database(s).
Re-run the SQL snippet to ensure the fix was made properly and now running the installation should past this error.
At this time 98 people have downloaded the TFS Migration and Synchronization Toolkit. Ok - some of you are saying "100? What's the big deal? A picture of a rabbit with a pancake on it's head gets more downloads every hour!"
And they would be right.
But think big picture - what if 50% of the downloaders actually open the zip file?
And what it 50% of them actually compile and run the product?
And what if 50% of them start looking into the source code and trying to write their own converter?
And what if 50% of them actually succeed?
And what if just one of those people decides that their converter is worth sharing with the world? How many people will that help? Hunderds? Thousands? More? Or maybe they decide to use the tool for consulting engagements - how many successful careers could be made using that tool?
In the migration space it does not require big numbers to make a big impact.
We've already impacted 100 people. That is exciting to me.
How many more will download it tomorrow? And the next day?
Have you downloaded it yet?
Interested in writing a migration tool targeting (or synchronizing with) Team Foundation Server version control or workitem tracking?
Then check out the pre-release of the TFS Migration and Synchronization Toolkit on CodePlex!
http://www.codeplex.com/MigrationSyncToolkit
This drop is the complete source code for the migration toolkit and sample converters. The VC sample is an application that will synchronize WSS document libraries with TFS version control. Developers can check in doc changes with their code and managers can see the updated docs on the WSS server after the mirroring is done (and it works the other way too).
Oh - and this is still under active development (and has known issues) so if you have any feedback it's still early enough for it to be considered for inclusion in the final drop.
And to quote Matt ...
"Please note that this is a prerelease version of the Migration and Synchronization Toolkit. Several features are not complete and as testing has not completed, the code has the potential to contain bugs and overwrite data stored in TFS. Please be careful to use this only in a testing environment and not on live production data."
As Matt noted on the migration blog a working copy of the toolkit specification has been released. You can download it and leave feedback from this link:
http://msdn2.microsoft.com/en-us/vstudio/aa948851.aspx
If you have feedback on migration scenarios for version control, work item tracking or other products (test, office, web, file systems, etc) please check it out and let us know. We are still actively developing the toolkit and your feedback is very important to us!
When running the VSS to TFS converter (VSSConverter.exe) one of the most common questions I hear is how to create a UserMap.xml file for users who are no longer with the company or have had their accounts renamed.
My first suggestion is that any customer who has to deal with this issue (or any user mapping issues – or users in a workgroup configuration) should start by getting the hotfix provided in KB 928828. This can be gotten through your CSS (support) contact.
This hotfix changes the way that user authentication is done and is an important fix. It is compatible with both RTM and SP1 however it was not included in SP1 (it was made after the SP1 cut-off date and does include all the VSSconverter fixes already in SP1).
That said - what I’m going to describe is the same in both the RTM and SP1 versions as well as in KB 928828.
The correct way to deal with the users may be as simple as doing nothing. Let me give you an example.
To test this out I created a sample VSS repository and added two users “User1” and “User2”. These were in addition to the default users of “rhorvick” (me) and “Admin”.
I connected as “User1” and created some content. Then as “User2” and created more. And finally as “rhorvick” and created even more.
I then ran the converter in analyze mode. This created the following UserMap.xml file:
<?xml version="1.0" encoding="utf-8"?>
<UserMappings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<!--
This file is automatically created by VSS Converter. You can optionally use the file to map
a VSS user to a Team Foundation user. For example, <UserMap From="Jane" To="MyDomain\Janep"></UserMap>
This mapping causes all actions logged by VSS user “Jane” to be changed to Team
Foundation user “ MyDomain\Janep ” during migration.
-->
<UserMap From="USER1" To="" />
<UserMap From="RHORVICK" To="" />
<UserMap From="USER2" To="" />
<UserMap From="ADMIN" To="" />
</UserMappings>
At this point I just ran the conversion – I did not change the mapping file in any way. Please note that User1 and User2 are NOT existing accounts on my TFS server.
With the QFE the conversion created the following history in TFS:
Changeset User Date Comment
--------- ------------- ---------- --------------------------------------------
10837 rhorvick 1/4/2007 {1/4/2007 3:47:26 PM}added by rhorvick---- D
10836 rhorvick 1/4/2007 {1/4/2007 3:47:22 PM}added by rhorvick
10835 rhorvick 1/4/2007 {1/4/2007 3:47:14 PM}
10834 User2 1/4/2007 {1/4/2007 3:47:00 PM}added by user2
10833 User2 1/4/2007 {1/4/2007 3:46:18 PM}
10832 User1 1/4/2007 {1/4/2007 3:46:00 PM}user 1
10831 User1 1/4/2007 {1/4/2007 3:45:12 PM}added by user1
10830 User1 1/4/2007 {1/4/2007 3:43:52 PM}created by user1
10829 rhorvick 1/4/2007 ----------------VSSConverter Project Mapping
So the migration did properly migrate the changes and attributed them to the proper authors. I don’t know that I’m explaining this correctly – perhaps someone from Admin/Ops can explains it better – but when a checkin occurs and the author the change is being attributed to (which is not necessarily the account making the checkin call) is a non-existing user and the account is not a domain name or a TFS account/group name then a special identity is created which can only be used for the purpose of attributing changesets to (i.e. an audit-only account, not one that can be logged in as).
Now – had I wanted to attribute the changes to a real user I could have altered the mapping file like so:
<?xml version="1.0" encoding="utf-8"?>
<UserMappings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<!--
This file is automatically created by VSS Converter. You can optionally use the file to map
a VSS user to a Team Foundation user. For example, <UserMap From="Jane" To="MyDomain\Janep"></UserMap>
This mapping causes all actions logged by VSS user “Jane” to be changed to Team
Foundation user “ MyDomain\Janep ” during migration.
-->
<UserMap From="USER1" To="northamerica\rhorvick" />
<UserMap From="RHORVICK" To="northamerica\rhorvick" />
<UserMap From="USER2" To="northamerica\rhorvick" />
<UserMap From="ADMIN" To="northamerica\rhorvick" />
</UserMappings>
And this time when I migrated I see the following history:
Changeset User Date Comment
--------- ------------- ---------- --------------------------------------------
10846 rhorvick 1/4/2007 {1/4/2007 3:47:26 PM}added by rhorvick---- D
10845 rhorvick 1/4/2007 {1/4/2007 3:47:22 PM}added by rhorvick
10844 rhorvick 1/4/2007 {1/4/2007 3:47:14 PM}
10843 rhorvick 1/4/2007 {1/4/2007 3:47:00 PM}added by user2
10842 rhorvick 1/4/2007 {1/4/2007 3:46:18 PM}
10841 rhorvick 1/4/2007 {1/4/2007 3:46:00 PM}user 1
10840 rhorvick 1/4/2007 {1/4/2007 3:45:12 PM}added by user1
10839 rhorvick 1/4/2007 {1/4/2007 3:43:52 PM}created by user1
10838 rhorvick 1/4/2007 ----------------VSSConverter Project Mapping
SharePoint does not have persistent hyperlinks for all document revisions.
More specifically – SharePoint has two methods for linking to documents: the canonical path and the revision paths.
The canonical path is the latest copy of the document at the time of the download request. Every time a new revision is created the contents of the canonical path change. Canonical paths look like this:
http://<servername>/Shared%20Documents/document.doc
Revision paths are paths to historical revisions of the document. More specifically they are paths to the non-tip (latest) version of the document. The revision path exists under the _vti_history location and have the version number in the path. They look like this:
http://<servername>/_vti_history/7/Shared%20Documents/document.doc
That link is for revision 7 of the document.
When I first saw this I assumed that all documents were accessible via revision paths, and the canonical path was just an abstraction over the latest revision document. I was incorrect.
You can prove this to yourself by taking any document in a WSS repository and trying to request the latest version by the revision path that would contain its content. For example if you add a new file it has only one revision. So its revision path (if it could have one) would be:
http://<servername>/_vti_history/1/Shared%20Documents/document.doc
Requesting this fails with a HTTP 400 response (HTTP BAD REQUEST). Requesting the canonical path works as expected. If a new revision of the document is created the revision path for version 1 becomes valid and the canonical path now refers to version 2 (and as before – the revision path for version 2 is not valid).
Why is this a problem?
Enumeration of the document library is slow. The time between when we analyze the changes to migrate and the point where we actually perform the migration could be seconds, minutes or hours apart. During that time the document may have been updated. When that happens the version we cared about is demoted from the canonical path down to a revision path. If we downloaded the canonical path we would end up getting the wrong content.
At first my plan was to store the canonical path and then to check during migration to make sure that canonical path is still the version we want before downloading. But that just narrows the window on the race condition – it does not eliminate it.
I ended up doing the following:
When analyzing an item we know if we are looking at the latest version (canonical path) or a historical revision (revision path). We store this fact as a boolean along with the path necessary to download the document (the canonical or revision path) and the version number of the item. During migration we download the item from the stored download path. Next, if the document was the latest version (i.e. a canonical path) we now double check that the canonical path refers to the document version we needed. If it does we are done. If it does not then we enumerate the revision history to find the revision we want. If it exists we download the specific revision and continue. If it does not then the revision was deleted and we handle that condition.
I believe this approach guarantees that the content we end up is always the proper content and does so in the minimum number of round-trips to the server.
All because canonical paths are not just an abstraction over revision history.
Lesson 6: The conceptually simplest operation can end up being incredibly complex to perform correctly.
Lesson 7: When representing data (such as document revisions) make all the data available through a uniform mechanism. Shortcuts to the latest revision of the data should be an abstraction over that mechanism.
In my previous rant I made this statement (quoted out of context):
Migrating WSS content to TFS? Piece of cake. Just enumerate each WSS change and replay the action in TFS. If you are doing an incremental migration then just enumerate those changes that have occurred since the last time you checked for changes.
There are two base actions we need to support:
1) Enumerating all revision history for a document library
2) Enumerating all revision history for a document library after a specific point in time
At the end of the day these boil down to the same requirement (#2) with a default start time that precedes the first revision in the document library.
This should be easy, right?
Now in my mind I think that this should not be a big deal. All of this data is stored in SQL. There is a single SQL query that can be crafted to find this information very efficiently. Even if the document library had millions of items and was located on a slow and unreliable connection … this should be able to run quickly and without a lot of fuss.
Let’s find out!
So let’s go back to the WSS Web Service documentation and start looking for something that looks promising. “Document Workspace” (DWS) is the obvious starting point. When you visit the DWS front page it is sparse to say the least. It includes 5 words. The title “Document Workspace Web Service” and a link “Methods”. No summary of usage or sample code. Not an auspicious start.
So I click on “Methods” and this page has one-fifth the content of the previous page. It simply reads “Methods”.
So here is the first rant.
What the $*#@??!!?? “Methods”? That’s it? Someone decided to publish this? Is this a joke? I’m sure the answer has something to do with automatically generated content and blah-blah-blah.
So I used the search box. I entered “Document Workspace Web Service” into the MSDN search box and the first result in the search engine was this. This looks more promising. We’re up to nearly 20 words now. And that hyperlink looks good.
Lesson 4: MSDN has multiple active pages of documentation for the same logical thing. Some pages are horrible. Some are reasonable. Some are great. When in doubt, search.
Anyway – long (even by my standards) story short … the DWS web service was not what I wanted. It includes methods for creating, deleting and modifying document workspaces. I want to enumerate the contents of one.
Eventually I discovered that what I want to do is use the Site Data web service which has the SiteData class that has a EnumerateFolder method. With the results of all that I could use the Versions web service which exposes a Versions class whose GetVersions method returned what I wanted.
It didn’t take a lot of time to whip up a sample that enumerated the contents of a WSS service. The tedious part was parsing the XML responses (it turns out that parsing responses is a reoccuring theme in this project).
So now I’m sitting here with a code snippet that allows me to enumerate each revision of each item in a WSS document library. But this isn’t really what I wanted. I wanted to know every revision after a specific point in time. I spent a few hours searching the web, reading documentation and talking with folks more knowledgeable than I about WSS, and what I discovered was this…
There is no way to do this in WSS 2.0. WSS 2.0 does not include the search web service (which apparently can be jury-rigged into doing this using query constraints – but I never researched it to prove that). So my options were limited.
"I still haven't found what I'm looking for."
- U2 (The Joshua Tree)
Every time I want to identify changes to migrate I have to enumerate the entire tree structure and prune those items that are older than the range start time.
So while we’re enumerating things let’s enumerate a few problems with this approach:
1) Enumerating is a somewhat expensive operation that will result in something like 2*N round-trips to the server each time the migration process begins (where N is the number of versioned items in the WSS repository).
2) Enumerating is a slow operation (at least we're doing the expensive thing slowly ... right? Or are we just doing the expensive thing for a long time? You decide.).
3) The contents of the server are changing during our enumeration. Remember that we want to identify everything that has occurred since the last migration time. What if the enumeration process takes 8 minutes – should we consider documents added during those 8 minutes? It turns out we should not. If we do, we run the risk of missing or repeating revisions. If you don’t get why – leave me a comment and I’ll explain it in more detail. So really we need to filter based on items that occurred after the previous run and no later than the start of this run.
4) WSS uses one minute timestamp granularity. Why is this important? If three documents were added during the same 1 minute window we need to make sure we either record none of them or all of them. We won’t know how many to capture until the entire minute is over. This means that when we enumerate documents we don’t filter the based on the start time through to the current time – we filter based on the start time through to one minute prior to the current time.
Let 3 and 4 sink in for a minute.
They are a pretty big deal.
We can never know when we are done. The act of looking for a change takes so long that another change may have occurred during that time and we could have missed it. So we need to look again. Which takes a long time. Lather well. Rinse and repeat as necessary.
In summary…
So let’s review our original goal:
“Enumerating all revision history for a document library after a specific point in time”
Which has now become:
“Enumerating all revision history for a document library since a after point in time but before one minute prior to the start of the enumeration period.”
Lesson 5: Just because something can be done efficiently in the native storage system does not mean that an efficient means of doing it will be exposed to the user.
This is a good lesson for the TFS APIs too. We have several areas where it is more difficult than it should be to do things that seem like they should be trivial (searching changeset comments and annotations are two that come to mind quickly).
Is it just me?
When I’m writing code that feels wrong I tend to believe that I am the problem. That I haven’t invested enough time into learning how to use the API effectively. That may well be the case. I’m flying by the seat of my pants here. Am I missing the boat on this? Is there a “EnumerateAllChangeSince” method I didn’t see in the wsdl? Is there a better way to figure out the contents that are interesting for migration? And most importantly – is the criteria I’m using to figure out the interesting items correct?
Happy New Year and all that. I spent the long weekend at Myrtle Beach soaking up rain or sun - depending on the moment you checked.
I’ve been out of Fargo for a year now, after nearly 30 years living in Minnesota and North Dakota. I wasn’t timid about the winters growing up. I’ve slept outdoors in the dead of winter (in an igloo and not), ice fished in 40-below temps and done the whole “run from the sauna into the snow drift” thing (poor-man’s polar bear club) more times than I can recount. Growing up Minnesotan was all about snow, ice and fun.
But this year I spent New Years in a pair of shorts walking the beach and playing mini-golf with the kids.
I’m hooked.
The Indigo Girls said it well … “When god made me born a Yankee he was teasing”.
So if you didn’t enjoy your New Years weekend as much as I did … why not consider a change? We’re hiring. (If the direct link does not work – visit http://members.microsoft.com/careers/search/default.aspx and search for the “NC – Raleigh” location and the “Visual Studio” product. Leave the other defaults and click “Search Jobs”).
[My regularly scheduled rant is coming shortly]
If I could do it all over again I would take auto shop class in high school. I didn’t know it at the time but I really enjoy automotive technology. I have bought several text books used in automotive tech classes and read them cover to cover several times. I have a small library of books on the repair and restoration of Chevy trucks and engines, and Chilton manuals for vehicles I'll never own.
The Chevy Advanced Design (’47-’54) trucks are my hands-down favorite (with the ’51 being my favorite of the mix) – but I’d “settle for” a Judge if the opportunity presented itself.
A few years ago I went and bought a 73 Chevy K10 pickup. It was a good truck. A manly truck. A lime green truck. I intended to get the vanity plate "SOYLENT" but the plate would have been worth more than the truck and nobody seemed to get the joke anyway. I even finished a few projects on it (these should serve as evidence that I’m not a very good mechanic). But when we moved from the Fargo area to North Carolina I sold the truck since moving it was not practical.
For a few years I had been reading all about how to repair a truck. Before buying this truck I mentally rebuilt the Chevy 350 hundreds of times. I had taken countless “end of chapter” tests on identifying and resolving automotive problems. The only thing missing was the truck to prove how much I'd taught myself.
Got an electrical problem? Well that’s simple … consult the wiring diagram to identify the relevant wires and fuses. Check the fuses and connection ends, make sure you are getting power and have a good ground, and aren’t shorting out. If all of that is good then start looking at the component that is not working (e.g. the wiper motor).
But the books never mentioned that the previous owner may have re-wired the system and not color coded the wires properly. Or maybe the ground was cut when the previous owners kid tried to install a new stereo. Or perhaps there is a build-up of rust that is preventing the body from grounding to the frame properly. Or that 30 year old wires fall apart when you touch them.
It seemed so simple on paper. It always does.
Migrating WSS content to TFS? Piece of cake. Just enumerate each WSS change and replay the action in TFS. If you are doing an incremental migration then just enumerate those changes that have occurred since the last time you checked for changes.
So what is the method to enumerate all of the items in WSS? Hmm … wait … how do I even talk to WSS?
And so begins my first rant on WSS.
The WSS converter has a few simple requirements:
1) It will be able to run on a machine other than the TFS application or data tier and it will be able to run on a machine other than the WSS server (which is usually the TFS application tier).
2) It will use the WSS provided APIs to interact with the document libraries.
3) It will not access the WSS database directly.
Prior to this experience what I know about programming SharePoint could have been summed up in one word: “nothing”. So I grabbed a book and hit MSDN. I learned all about the SPWeb class and how to enumerate SPFile objects and man-oh-man life was good. This was going to be easy.
I setup a WSS server on my development box and started writing code. The tests were working great. I was just about to go grab my PM to show him a quick demo of my progress when I decided that it would be much cooler to demonstrate it by grabbing content from the production WSS site. I updated the configuration to point to the remote machine and hit "F5"...
All of the sudden my code, my beautiful code, which had worked so well just moments earlier was dead in the water. The WSS site couldn’t be found. The document libraries didn’t exist. The files were all missing. How could this be? I was pointing at the right machine. The document library existed. I had the proper access. Why wouldn't this work?
I spent almost a full day debugging this before I found this web page on MSDN – http://msdn2.microsoft.com/en-us/library/ms954094.aspx. Please note these two really important phrases:
“You can use the Windows SharePoint Services object model in code running on the server to access and work with list or Web site data and templates, to customize Web Part views of list data, or to manage a top-level site or a subsite.”
“The Windows SharePoint Services Web Service provides interfaces for communicating with the server running Windows SharePoint Services from a remote computer to work with list and site data.”
Remember requirement #1?
I literally yelled. In the event that children are present I won’t say what I yelled. But I’ve heard that it is still hanging in space over Lake Michigan.
A week of learning and I was back to square one.
I took a walk (this was a twelve thousand step day), had a Diet Pepsi and start reading all about the SharePoint web services.
Three important lessons were reinforced from this experience.
Lesson #1: If you need to support running on remote machines then do your testing on a remote machine from the beginning.
Lesson #2: Read the API overview pages on MSDN very carefully before investing time in reading the API.
Lesson #3: When in doubt, double-check that you have a good ground.
As Matt has mentioned on the TFS Migration blog we’re working on a migration toolkit and, specifically, a sample converter tool to keep WSS document libraries in sync with a location in the TFS source control server.
One of the most common questions I get when people hear this is “Why a WSS converter? Are people really asking for that? Why not CVS or PVCS or something?”
Well – there are a few reasons. Some good. Some not so much.
1) This is a sample application to demonstrate how to write a converter on the toolkit. It is not necessarily meant to be useful to a broad group of people but rather useful as a code sample.
2) We did not want to create a converter that relied on technology that not every TFS customer would have access to or that would require them to install additional software. We can be pretty well assured that all TFS customers have access to TFS and WSS. This reason gets the response … “then why not TFS to TFS? That would be useful to many people!” … to which I say:
3) Because it is a sample application we needed to pick something that was approachable. WSS doesn’t have concepts like branch and merge (and it barely has rename) so that lowered the bar significantly (or I thought, anyway).
4) There is actually a somewhat cool scenario that this converter opens up – you can check in your document changes at the same time as your source code (to TFS) and it will be mirrored to WSS. Then your PM can make document updates in WSS and the change will be mirrored to TFS. That’s kind of cool … right? I think so anyway.
5) Writing a more complex converter would add months onto the delivery time.
6) I already had a WSS->TFS prototype so the bar seemed like it would be lower than any other option.
Ultimately it was the decision we made. Sorry if you’re not happy. Life isn’t always butterflies and unicorns. Sometimes it’s a WSS converter. I think it was the right decision.
Why am I sharing all this?
Over the next few blog posts I’m going to be sharing some of the problems I’ve run into while working with WSS. This isn’t just to rant. Don’t get me wrong. I fully intend to have these posts included in the Rant category … but I have ulterior motives.
I would love to hear feedback on the issues I describe. Is there a better solution? Did I do something horribly wrong? Stuff that would be great to know BEFORE releasing the code.
Also as I share the problems it is inevitable that I will share some of the design (I’d like to share it all – we’re trying to figure out how to do that). The design isn’t in stone yet. If I’ve done something dumb you can yell at me and I can fix it. More of that “stuff that would be great to know before releasing”.
This week is the one year anniversary of my move from the Fargo, ND office and the Business Framework team to the Durham, NC office and the Team Foundation Server Version Control team. I’ve spent the last few days reflecting on the last year. This is one of those reflections. It was written for myself but I thought I’d share it – my blog is a glorified backup system if nothing else…
My work habits are somewhat reflected in my writing style. I am overly verbose, I get off topic quite frequently, and once I can understand how the problem will be solved I get bored and want to move on to the next problem.
Oops– just did it again. The last part has nothing to do with my writing style. Or perhaps everything to do with it.
The point is that I ramble.
I’m like a hound dog chasing its nose through unfamiliar woods. I end up miles from where I should be and I'm not always sure how to get home.
My rambling is not limited to the world of writing or conversation. When I do something I go all the way. I also ramble around the office.
In the Fargo office the building layout is your typical cube farm. Hundreds of people crammed into 8x6 cubicles in the most economical means possible. It was too bright, too noisy and there were too many interruptions. I would solve this by going on walks. Long walks. Thinking walks. I might put on four to six thousand steps a day on my walks (I often wear a pedometer to work). On these walks I would think about the problem I was trying to tackle. I would think about schedules and estimations and meetings.
Basically it ruined walking for me.
But it became a habit. When I have a problem I think it through on a walk.
After moving to the Durham, NC office things were different. I have a private office so I can control the light levels and noise. I can close my door when I want to be left alone, and I can leave it open when I don’t mind hearing the exterior conversations. But what didn’t change was that I still took long walks. In Fargo you could not leave the building for half the year – it is simply too cold. But here I can go outside and walk around. Ridiculously long walks. I once came home with more than 12,000 steps on my pedometer (though I did think my way through a rather complex problem so it was worth it).
The problem is that I sometimes was walking when I shouldn’t be. Any gap in my day was suddenly filled with aimless rambling. I started keeping track and discovered I was leaving my office for a short walk (7 to 10 minutes – two or three laps around the second floor) more than 6 times per day. I’d leave the building for a long walk twice a week on average.
Add on to that the trips to the restroom and break room and I was leaving my office upwards of 20 to 25 times per day.
This was keeping me from getting into the zone. When I’m not in the zone I’m not productive. When I’m not productive I fall behind. When I fall behind I get stressed. When I’m stressed I can’t get into the zone.
What’s a rambling-addicted fellow to do?
I took a piece of yellow business paper and a black dry-erase marker. I wrote the words “Where are you going?” in huge print and I hung it on my door in such a way that I could not leave the office without seeing it.
I can’t tell you how many times I stood up to go on a walk, without even thinking about it, and seeing the paper made me think for a few more seconds before walking out the door.
Is this walk going to be productive? Can it wait? Do I really need a thinking walk or do I just need to stretch? What is the goal of this movement?
After a few weeks of having that sign hanging the number of times I leave my office has been cut by more than half. I still take think walks but I go on them with a goal so I know when I’m done walking and can measure if it was successful or not.
I am spending less time rambling and more time in the zone where I get my best work done.
This week is the one year anniversary of my move from the Fargo, ND office and the Business Framework team to the Durham, NC office and the Team Foundation Server Version Control team. I’ve spent the last few days reflecting on the last year. This is one of those reflections. It was written for myself but I thought I’d share it – my blog is a glorified backup system if nothing else…
It seems that every work-day ends just a little too soon. I’ve spent all day getting into that zone where I’m the most productive. By now my door is closed and Molly Hatchet is pouring out the speakers. Inevitably I’m about an hour from being “done” when quitting time rolls around. The desire to finish the task is strong but I’ve finally hit that point where I’ve come to understand that it’s important to leave on time when you can – there will be plenty of times when you can’t.
The problem with breaking away early is that it took half the day to get into that zone. My mind works differently when I’m in the zone than when I’m not. My understanding of the code is fresher and more accurate. I have ideas coming into my mind that seem like they should be easy to remember. I’ve mentally mapped out the next 45 to 90 minutes of work – I’ve written the code in my mind but my fingers haven’t caught up yet.
I used to just walk out of the office knowing that the ideas would retain in memory until tomorrow. Surely when I look at the code again I will remember where I left off. Right?
Wrong.
About 35 seconds after leaving the office my mind is a blank slate. I start trying to figure out if I should take I-40 or US-55 to get home. Is there an accident on Davis today? Does it look like the freeway is moving slowly? Why doesn’t 96 Rock give me the traffic report every 5 minutes instead of every 10? Oh – and the kids! How do I get Big B to scouts at 7 when Little B needs to be at Taekwondo at 6:30 and Big C needs to be at school by 6? What about dinner? And homework. And did I remember to setup the DVR to record Heros? Oh shoot... Now I’m stuck in traffic. I knew I should have taken Davis to 55.
I’ve been out of the office for all of two minutes and I’ve already forgot every detail of that algorithm I was working on.
When I arrive the next day I go through some email, pick what will be my life’s sound track today, check for a quick update on that news story I’ve been following … pretty soon 30 minutes has passed. I know I should be working on something but I can’t remember exactly what. So I open the source code editor and build. It succeeded so I run a test pass. If something breaks – that must be what I’m working on. If nothing breaks I get the latest code and try again. Still nothing broken? Then I get to decide – bugs or features? What am I working on today?
I played this game for years. I would wager that over the last 7 years I’ve wasted at least 10% of my time by not knowing exactly what I should be working on because I didn’t have a system to remind myself. That doesn’t even begin to account for how much time I’ve lost by not being able to quickly get back into my effective work “zone” – that mental state in which I am most productive.
But now it’s different. I’ve developed a few simple techniques that make it so I can leave on time, forget everything I know and still get back into the zone quickly – picking up where I left off the night before.
It’s a simple three step process that I perform before I leave:
1) Write a brief summary of what you are working on before leaving.
a. I open notepad and write up a few sentences about what I am working on, what the core problem(s) is and what the next steps are. I think differently when in the zone then when not. I need to capture not just the problem but also the next steps otherwise I will approach them differently tomorrow.
b. I maximize notepad on my primary monitor (it has to be primary to ensure I see it if I connect from home using remote desktop).
c. I do not save notepad. I want it to complain it I try to close it so that I don’t accidently dismiss it.
2) Email that summary to myself with a red flag or high priority.
a. Most get deleted without being read. But sometimes at night I want to see the list. If I emailed it to myself all I need to do is check my mail – 15 seconds of time. Otherwise I’d need to use the VPN which means about 15 minutes and that means I won’t do it. The only thing I’ll remember the next day is that I had an idea – not the idea.
b. Sometimes SMS patches my machine and reboots me. I want to make sure I’ve captured the notes in a way that I will notice. By emailing it to myself I will notice a new email and read it.
c. &