when setup isn't just xcopy : context

Conflicts of Interest?

robmen — Tue, 10 Oct 2006 18:55:01 GMT

Nicholas Carr has a great blog entry about conflicts of interest in blogging. It got me thinking about a silent policy I've always had for myself when blogging here. I don't blog about the many vendors in the MSI creation space except the ones that I am affiliated with. Namely I write about the WiX toolset (because I lead that effort) and sometimes VisualStudio's Setup Projects (because I work for Microsoft and I'm often asked to relate it to WiX ). I don't write about InstallShield or Wise or the many, many others.

My question to those of you who read my blog regularly is, "Do you think this policy is a good one and should I keep following it?" Please, leave a comment below or send me a comment through the contact link.

The Windows Live Writer team rocks.

robmen — Thu, 05 Oct 2006 18:00:00 GMT

I installed Windows Live Writer 1.0 Beta a while ago to test a project I was working on. I didn't actually boot the application or anything, just installed it. I think this proves how much of a setup geek I am. <smile/>

Well, as you can see in my previous blog entry, I actually tried using Windows Live Writer. I must say, in the short time I've used it, I'm impressed. The UI is very intuitive. The HTML generated is very clean. The ability to preview the post using the styles from my blog is sweet. I love the way that "Recent Posts" and "Drafts" are tracked seamlessly.

That said, I have had some small issues with cut and paste not doing exactly what I expected (whitespace just disappeared). Window Live Writer also doesn't provide an easy way for me to add code snippets to blog entries (awesome if it was supported just like blockquote). But I can edit the HTML directly to add PRE elements and Writer seems to handle that pretty well.

But this blog entry isn't supposed to be about Windows Live Writer application. Instead, I wanted to talk about the Windows Live Writer team.

I met three of the developers on the Writer team a month or so ago: Bonnie, Joe and Spike. We had a great discussion for almost two hours after work one Thursday. One of the things I remember most is when we were discussing Beta feedback Joe commented sheepishly, "Probably the biggest mistake we made was not including the word 'blogging' in our dictionary. Lots of people complained about that." I chuckled when I posted my last blog entry because I did have to add "blogging" to the dictionary when the spell checker complained.

But even more important than the interesting conversation I had with Bonnie, Joe, and Spike, I saw something I hadn't seen inside Microsoft for so long I had almost forgotten to look for it. This team is hungry.

They aren't just building a simple blogging tool. They are building a blogging tool that beginners can be comfortable with while meeting advanced users basic needs. Then they released an update to their Beta based on user feedback. Next they made their application a platform and opened up an untold number of new uses. And most importantly they didn't blink when it was suggested Windows Live Writer wasn't necessary because Word 2007 would support blogging too (as Chris Corio might say, "The two apps are in completely different markets, yo!").

It wasn't until I was driving away that evening that the last piece of the puzzle clicked in my head. Windows Live Writer was created by the same team that brought us Onfolio. This team of hungry developers was acquired from the "outside".

That realization eventually crystallized into something that I have been feeling for a year or more. It feels like many developers at Microsoft are "playing to not lose". What I saw in the Windows Live Writer team was a small group of developers "playing to win". I loved that. It's past time for me to do the same.

Restarts Revisited: Windows isn't that different than UNIX

robmen — Tue, 17 Jan 2006 11:20:59 GMT

I've been meaning to follow up on my blog entry about the Restart Manager and reboots on the Windows platform in general. I saw a lot of incorrect conclusions being drawn from my blog entry. So, I want to quickly revisit the issue and hopefully clear up the confusion. Hopefully, this time I'll communicate more clearly.

Jonathan Bastien-Filiatrault's blog post goes right to the heart of the incorrect conclusion drawn by many. He states the following:

One thing that makes Unix in general not need this is that if a file is unlinked, as long as there are open file descriptors to this file, that file is not removed from disk (even if it cannot be referenced from the directory hierarchy) and the memory maps from an executable or library can be maintained.

I didn't clearly note in my last blog entry that my experiments with Reid's PowerBook proved to me that this is true. You can delete files even if they are in use on UNIX. That's nice but it still doesn't solve either of the two issues that are the real reason a reboot is required:

1. The old file is still running (i.e. you are still vulnerable to any security exploits)

2. New executions of the file get the updated version while the old executions of the file keep using the old file (i.e. you may have inter-/intra-process communication problems).

On Windows, replacing a file that is in use takes more steps (rename, replace, delete) than UNIX (delete, replace) but on either operating systems you need to stop the updated process for the update to truly take effect.

Jonathan also states:

When that file is unlinked, one can sneak a new library into place atomically.

I'm not sure what he means by "atomically" but unless the file system supports true ACID-like transactions then an "rm oldfile" "cp newfile oldfile" is not an atomic operation. The next version of the NT file system (shipping for the first time with Windows Vista, I believe) supports transactions such that "ren oldfile tempfile" "copy newfile oldfile" can actually be done atomically. But that is a conversation for a different day.

Anyway, I just wanted to be a bit more clear on the topic and note that fundamentally, file-in-use handling on the two operating systems aren't really different and that the Restart Manager functionality would be useful on any operating system.

Blogging comments the "Scoble Way".

robmen — Sat, 07 Jan 2006 20:00:41 GMT

Usually, when I find comments about the WiX toolset on other people's blogs I would put comments there. However, after watching Robert Scoble for a while, it seems like there is more utility in writing up your comments as a blog entry on your blog and linking over to their blog. I also found that sometimes my comments were as long as some of my blog entries. So, I'm going to try linking and trackbacking for a while (hopefully Community Server's trackback functionality will actually work. The software has not been behaving well lately, IMHO).

Anyway, expect to see more entries that are comments, questions, or rebuttals to stuff I've seen on other blogs and web sites. Some of the blog entries or sites I point at may be very old for a while since I have a back log of collected data that I haven't posted.

PDC '05, I'll be there. Will you?

robmen — Thu, 01 Sep 2005 20:07:00 GMT

Well, it looks official. My features are in and my bugs are under control. I'm going to PDC '05!

No, I'm not a part of the Windows Installer presentation on the last day at the last hour of the PDC. No, there isn't a Birds of a Feather (BOF) session on the WiX toolset. No, I'm not dressing up like an Imperial Stormtrooper.

So what am I doing at PDC'05? Well, I am slated to be part of the Ask the Experts session on Thursday evening. I will also be hanging around, attending shows, and writing WiX code (probably in the "Bean Bag" or "Fundamentals" Lounge... whichever has the best atmosphere).

But the real reason I'm writing this blog entry is that I'm thinking about putting together some ad hoc meetings to talk about the WiX toolset or setup in general. If you're going to PDC and think that would be interesting then drop a comment below. For example, I was thinking it might be easy very easy to create an unscheduled WiX BOF session on Thursday evening after the Ask the Experts session.

I'll post more blog entries (even during the conference) if there are particular gatherings times/points that look fruitful.

Personal musings on shipping the first version of software.

robmen — Sun, 29 Aug 2004 07:56:00 GMT

I find it amusing that even though Peter and I have been on very different schedules as of late (and thus haven't run into each other around the house much) his latest blog entry captured very much what I've been focused on as of late.

I have been neglecting regular updates to my blog for the last month or so because I've been coming home completely mentally fried. At work, I have been struggling with many of the questions that Peter described in his blog for a particularly difficult feature in the product I'm currently working on. I seriously spent a few days just trying to figure out what a user would expect to see and be able to do with this one particular facet of our product. I then spent two weeks brainstorming solutions with everyone else on the team to come up with a list of six or seven options. Next, the dev team went through and "costed" (calculated how long a feature would take to get working) each option. Finally, we went through and listed all of the side-effects each option had. After about a month total of data collection and analysis, it became fairly clear which options were feasible (note, we didn't have all the data like test costs, or localization hit. those costs weren't necessary to see which options were expensive and which weren't).

Late last week, we met with the other teams that would be affected by the option picked and made a decision. We picked "Option 1". Option 1 was "Cut the Feature". You see, we're closing in on Beta2 and many of best options just couldn't be considered because there isn't enough time to get all of the existing work done and add this new work on top. Sometimes it can be very frustrating to watch a feature fall on the floor because we miscalculated something(s) in the beginning and don't have time to go back to fix it correctly. Other times it's just frustrating trying to figure out what to do with such problematic features and cutting them is something of a relief even if you now have to worry that you've cut too deep. Ultimately, I think cutting this particular feature was the right thing to do but I do wonder how users will interact differently with the product without this feature.

I obviously can't go into details about the exact feature here (otherwise I would have already) but I wish I could. I often wish that my day job project could operate like my Open Source project. The feedback loop is so much tighter when the community is built into the project. Maybe after we ship I'll discuss it, but I doubt it.

Finally, first version projects (like what I'm working on) are notoriously difficult to wander through because quite often you're building something for an audience (community?) that doesn't exist yet. Each feature is vetted against what you imagine your customers are going to need because your customers don't exist yet. In my humble opinion, that is why everyone says you should wait for the third version of a product. It usually takes a couple versions of customer feedback to be incorporated into the product before users find the products truly usable. This is yet another example of why I enjoy working on my Open Source project; the feedback loop is much tighter.

On a brighter note, it looks like all of this struggling will be behind me in the next week or two. Then I'll be off to Italy for a week. When I get back from holiday, I'll actually get back to write code again. Yes, for the last few months I've been of working on a non-coding feature. I can't tell you how much I am looking forward to writing code again. I always thought I could be a PM (I know I'd be a lousy Tester) but now I know better. I'm a Developer. I write code.

Creating a Microsoft Open Source project on SourceForge.

robmen — Mon, 09 Aug 2004 11:04:00 GMT

Ever have one of those days where you find yourself staring down all the little tasks that you really should get done but have absolutely no desire to start? Instead, you spend most of the day lying around resting and maybe reading some inconsequential fiction. Today was one of those days for me. Fortunately, I usually "go nocturnal" after such a day and can still get most of the software projects complete. Tonight I'm working on a blog entry I promised a week ago then I'm going to drop into some late night SDM modeling.

A number of people at Microsoft have asked that I'd write a step-by-step guide for creating a project on SourceForge based on my experiences with the Windows Installer XML toolset. So, let's jump right in:

0. Get your Open Source project approved. Work with your business development team and the Shared Source Initiative team to have your project approved to be released as Open Source. It is important to work with your business team because there are many business angles that the typical geek doesn't necessarily think about. Don't just post your code to SourceForge because you think it is a cool idea, that's a great way to end up in some seriously hot water.

1. Create a SourceForge account. This process is very simple and usually takes less than an hour to be good to go.

2. Register your project with SourceForge. This is a multi-step process that can take about twenty minutes to read all of the licensing information and answer all the questions. It will also take a couple days for the project to be approved and created so be sure to leave extra time if you have a deadline for project creation. Finally, be sure to have the following pieces of information ready before going through the registration process. You should have all of these answers from Step 0.

a. Project name. 3 - 15 lowercase alphanumeric characters that are used to uniquely identify your project everywhere. Choose carefully because you'll never be able to change this name once you create your project.

b. Project title. This is a bit more descriptive than the name. For example, "Windows Installer XML (WiX) toolset" is my personal favorite.

c. Short description. 10 - 255 characters of marketing goo that makes your project sound cool.

d. Business justification. 200 characters - 10 KB that explains why your project should exist.

e. OpenSource License. For example, the WiX toolset uses the Common Public License.

3. Configure Trove categorization. This categorization shows up just under your project's short description and provides a lot of useful metadata about your project. If you don't post this quickly, you'll get a lot of requests to have it filled in when your project gets more popular.

4. Configure the rest of your project. There are many little things you can tweak in your project. Read about each feature (trackers, file releases, forums, mailing lists, etc.) and decide how you want your project run. Personally, I highly suggest creating three mailing lists: "project-users", "project-devs", and "project-commits". The "users" list is where general questions can be sent and answered. The "devs" list is where bug reports, fixes, and new features can be discussed. The "commits" list is where all CVS information is sent for those that want to monitor the continual status of the project. I also turned off the forums for my project because it was time consuming to track the mailing lists and the forums where the essentially the same questions were being asked. I chose to consolidate everything in the mailing lists.

5. Download PuTTY. Since almost all raw project communication with SourceForge is via SSH we need a Windows client that handles that. PuTTY is recommended by SourceForge and has been pretty friendly once I quit trying to be smarter than it was. My suggestion is to just download the .zip file and extract it into a C:\Program Files\PuTTY directory.

6. Generate a SSH key. Those SourceForge steps are pretty good if you just follow them step by step. One of the important pieces to get right is in "step 6" there is a picture that shows the "Public key for pasting.". Be sure to copy the entire block of text in there. Some of the earlier (or later) SourceForge instructions weren't clear that everything (including the funky little "ssh-dss" text and all of the new lines) was important. Oh, and put the SSH keys somewhere you won't lose them. You can re-create keys but life is just easier if you don't have to.

7. Wait. The SourceForge site says that it only takes 10 - 30 minutes for the SSH keys to be updated, but I had a terrifying early late night experience with this when trying to post WiX for the first time. I started late the Saturday night before WiX was to go public and was up until 4:30 AM Sunday morning trying to get SourceForge to recognize my SSH key. I finally went to sleep exhausted, hoping that I would wake up in the morning and everything would just be okay. I was up at 6:30 AM (not a great night of sleep) and magically all my SSH commands were working. I was so relieved.

8. Play with PAGEANT. PAGENT is a cute little tool that simplifies using SSH tools, but isn't terribly discoverable. The SourceForge documentation about this fact is easy to skip over but once you read the docs (hopefully my link helps find them) all should be clear. Before you try to do any SSH communication with SourceForge, be sure PAGEANT is running (you'll see a little computer con with a black hat in your system tray) or SSH commands will fail.

9. Punching through the firewall with PuTTY. One of the trickiest things to get right was how to talk to SourceForge through Microsoft's firewall. There is a short blurb about that topic in the SourceForge documentation, but here's a step by step.

a. Start putty.exe from the location you downloaded it (in my case, C:\Program Files\PuTTY)

b. In the "Session" node of the navigation tree

Host Name: "PROJECTNAME.cvs.sourceforge.net"

Port: "443"

Protocol: "SSH"

c. In the "Connection" node of the navigation tree:

Auto-login username: <your SourceForge account>

d. In the "Connection->Proxy" node of the navigation tree:

Proxy hostname: proxy (or any other Microsoft proxy server)

Port: 80

e. Back to the "Session" node of the navigation tree:

Saved Sessions: "cvs-ssh"

Click "Save" button

Click "Open" button.

Note: You should see an SSH session start up with SourceForge. If you see something that says, "Welcome to cvs1.sourceforge.net" then get disconnected (because SourceForge does not allow direct access to CVS) then you're good to go. If not, go back and make sure you've configured everything above correctly.

10. Download a CVS-client. The first time through this process, I downloaded WinCvs and just used "cvs.exe" from the command-line because the UI is so atrocious (don't even bother booting the UI). I just added the "C:\Program Files\WinCvs 1.3\CVSNT" directory to my PATH and forgot that there was ever a UI for the cvs.exe. Also, TortoiseCVS looked cute but I don't use Explorer for anything. The command-line is where it's at for me. I recently realized it was possible to get CVSNT without WinCvs and that works just as well.

11. Configure CVS-client to talk SSH via PuTTY. It took a little while to finally get cvs.exe talking to SourceForge. Turns out all you need are two simple environment variables. I added these two variables to my user environment (Start -> right click "My Computer" -> Advanced Tab -> Environment Variables button -> top "New" button) so I wouldn't have to remember them all the time:

CVSROOT=:ext:cvs-ssh:/cvsroot/PROJECTNAME
CVS_RSH=C:\Progra~1\putty\plink.exe

Note: that you need to replace "PROJECTNAME" above with your project name from Step 2 above. Of course, you'll also want CVS_RSH to point at your copy of plink.exe that should be found wherever you extracted PuTTY in Step 5 above. Finally, the string "cvs-ssh" is tied back to the "Saved Sessions" entry in Step 9 above. Those strings must match for PuTTY to connect all the dots.

12. Import your license into CVS. My suggestion is that your first check-in to CVS be a copy of the license you're using for your project. This is a really safe way to get started with CVS. So create a directory where you want your project to be stored and put the license text file and only the license text file in that directory. In my case, I created a new directory for my project called "wix_public" and copied the CPL license into a CPL.TXT file there. Then you should be ready to do your first submission to SourceForge:

cd projectdirectory
cvs import -m "Initial license import." projectname Microsoft start

Note: that you should be in the directory that contains only the license text file when you execute this command, so you'll need to replace "projectdirectory" with the correct directory (in my case, "wix_public"). You'll also want to change "projectname" to the name of your project.

13. (optional) Checkout a copy of your CVS configuration (CVSROOT). This step is optional but will enable you to do the advanced configuration of your CVS repository in the following steps. I highly suggest reading all of the SourceForge documentation about these advanced topics before continuing. I personally like to check out CVSROOT to a directory named "projectname_cvsroot" right next to your project. To do so, just enter the following command from the directory that contains your project directory:

cvs co -d projectname_cvsroot CVSROOT

14. (optional) Send mail on all commits. This is an optional configuration step, but I highly recommend it. The SourceForge documentation is pretty good, so I'd suggest just following it step by step.

15. (optional) Configuring granular ACLs on your project. I highly suggest reading this topic and decide if this applies to your project. I decided to enable the cvs_acls script because I wanted to have developers that could do documentation and project maintenance and still be able to review all changes before modifying the source code. It's up to you.

16. Add your files to CVS. While you can have all of your files added during the import in Step 12, I highly suggest adding them by hand (after loading the license text) the first time you go through the process. Also, if you have any binary files be sure to read the SourceForge documentation about the "cvs -kb" switch. Oh and for those of you who are used to Source Depot, CVS requires you to add directories before you can add files to the directories.

17. Wait. Somewhere SourceForge mentions that it can take a couple hours for your changes to CVS to propagate to the anonymous CVS servers that the Web based UI reads from. So, don't worry if your changes aren't showing up through the Web UI for a while. When I was posting the WiX toolset the Monday morning it was to go live, I was sitting around from 4:30 AM to about 6:30 AM waiting for all the CVS servers to catch up. Fortunately, I had already had my scare with the SSH keys in Step 7 and was pretty sure SourceForge would just "Do The Right Thing" (tm). So, over those two hours waiting, I drafted the blog entry that announced the WiX project.

That's it. Hopefully, this blog entry will help those of you taking care of projects that will follow WiX and WTL onto SourceForge. Have fun with it!

Balloons, fishing poles, and personal empowerment.

robmen — Fri, 18 Jun 2004 18:31:00 GMT

I'm hanging out in Long Beach with a bit of Jenny's family for a short holiday (Thursday through Sunday) to celebrate her graduation. We're staying with her brother who has two little guys, one 3 years old and the other 1.5 years old. Last night the kids came home from their grandparent's (who were acting as babysitters for the evening) house with a couple helium-filled balloons. Of course, in the morning the balloons were no longer flying high (I can't tell you how much that used to frustrate me as a little kid). So, instead of tying things to the balloons to see how much weight could be added before the balloons would no longer stay afloat, the oldest boy began batting one of the balloons around in a sort of one-man volleyball game.

After an hour or so, the balloon floated a bit more out of control than previous hits and it got stuck on top of the TV. The 3 year old couldn't quite reach it and started pleading for support. Now, I believe strongly in personal empowerment so rather than free the balloon for him, I suggested he use a nearby toy fishing pole to help himself. After a few more denied requests for me to use the fishing pole to free the balloon for him, he started poking at the balloon with the toy fishing pole. After three or four swipes the balloon was free again. Cheers went up around the house and he seemed quite happy with himself.

What does this story have to do with anything?

Well, at the same time the 3 year old was learning how to use tools in his surroundings to solve his problems; I was answering a couple emails from people trying to use the WiX toolset. More accurately the authors of the email were pleading for someone to provide step-by-step instructions how to solve their individual setup authoring issues. The emails sounded amazingly similar to the little boy trying to get his balloon back.

So, I responded to the emails in much the same manner I did the three year old. "With every drop of the Windows Installer XML toolset, there is a documentation file called WiX.chm. You should read through that to get started. You should also read through the Windows Installer documentation to get a solid understanding of the platform you're building on." Consider that documentation to be the toy fishing pole near you to free your balloon of WiX toolset questions.

Somewhere in all of the above the old saying, "You can free a boy's balloon and he can play for another hour, or you can teach him how to use a toy fishing pole to free it himself and he will play all day" seems appropriate. But I'm bad with fables, parables, and catchy sayings in general. So instead all I really hope is that next time you have a question about the WiX toolset (or any other technology, really) you'll think of the 3 year old boy who made the effort to free his balloon all by himself and do a little more reading.

The questions Microsoft employees keep asking about Open Source.

robmen — Thu, 10 Jun 2004 09:18:00 GMT

Almost two months ago OSDir.com interviewed me about the Windows Installer XML (WiX) toolset. It was my first interview ever and I learned something interesting. As the interviewee there is a very good chance you will not get to talk about the things you think are interesting. That is, of course, unless you ignore the questions being asked and that's not generally considered polite.

In my case, the interviewer was obviously looking for a story where I was some radical agent that snuck the source code out the firewall in the depth of the night without anyone noticing... or something like that. More than once, I answered a question with "See my above answer" because I had already said, "Yes, my managers knew what was going on." Seriously, I had no intentions of getting fired over some tool that I work on in my free time.

Anyway, as I mentioned in the interview other Microsoft employees have asked questions about releasing code as Open Source projects. Recently, an employee sent me several questions in an email (just like my interview for OSDir.com) that I thought made great interview questions. So, rather than answer his email privately, I decided to post the questions and my answers here so other Microsoft employees can read them.

Q: Why did the WiX project choose SourceForge (over something like GotDotNet.com)?

I answered this question in the previous interview so I'll repeat myself here and add a bit of detail I didn't have originally. SourceForge has many tools and applications for developers building other tools and applications for the Windows platform. The Windows Installer XML toolset is another one of those tools for Windows developers. When the CPL was chosen as the license for toolset, it became possible for us to post WiX on SourceForge where the toolset would be visible to a large number of developers. Essentially, I believed that those developers checking out GotDotNet.com were likely to also be checking out SourceForge so the WiX toolset would be found either way.

A second benefit of SourceForge that wasn't immediately apparent but saved me a lot of attempts at explanations later was that SourceForge only accepts projects that use an approved OSS license. The fact that WiX toolset was being hosted on SourceForge meant that the project must be using an approved OSS license. I remember seeing comments somewhere (probably Slashdot) where someone questioned whether the WiX toolset code was really truly released under an OSS license. Several people silenced such questions by correctly noting that SourceForge only hosts projects under a true OSS license. That helped reduce some of the "Fear, Uncertainty, and Doubt" being spread about Microsoft's first official foray into Open Source.

Q: You have mentioned that Microsoft employees should access the internal drops of the toolset instead of downloading the bits from SourceForge. Why?

Participating in any Open Source project as a Microsoft employee is still in its infancy. Even Open Source projects that originated in Microsoft have interesting legal complications that have not been well tested, yet. My intention is to reduce the chances that a Microsoft employee inadvertently makes a mistake with severe legal repercussions. The last thing I want to have happen is an employee lose his or her job because I wasn't clear how to interact with the WiX toolset in his or her day to day job.

In particular, I am the gatekeeper that carefully reviews all changes accepted into Microsoft from SourceForge. If there is anything questionable in a change then I am responsible for ensuring the change does not get into Microsoft without appropriate review from legal. For example, no change can be accepted into the WiX toolset until the developer of the change signs the assignment agreement (see my previous blog entry for more information). If a Microsoft employee goes straight to SourceForge then he or she is bypassing the "safety check" I am charged with providing. I take my responsibilities very seriously.

Now, to head off any conspiracy theories, I want to be very clear that all changes made to the Windows Installer XML toolset inside Microsoft do flow out to SourceForge. If you watch the project closely, you'll notice that I push any changes made inside Microsoft to the toolset up to SourceForge on Thursdays. For "outgoing changes" (Microsoft to SourceForge) I try to ensure that the WiX toolset operates correctly before committing. For "incoming changes" (SourceForge to Microsoft) I review all submissions and ensure they appear "good".

Q: How did you find the internal process for creating a SourceForge project (i.e. did you get approval from legal)?

I definitely did get approval from legal before releasing the WiX toolset to SourceForge. As I mentioned in my blog entry about the release of the WiX toolset I actually started with legal and they pointed me over to Stephen Walli. Stephen had been looking for a project to release as Open Source from Microsoft and knew exactly what had to be done to get the project released. Overall, the process was extremely simple but did take longer than I anticipated because there were quite a few people that had to sign off that this was a "good thing" to do.

My understanding is that the process is now faster. For example, the WTL project was released to SourceForge only a month or so after the WiX toolset. However, even if it takes four or five months (like WiX did after I was in contact with Stephen) the process is really quite simple.

The hard part is building the community once you release.

Q: What has been the internal and external reaction for your project's appearance on SourceForge?

From Microsoft employees I have received nothing but positive comments and a desire to understand more. I received quite a bit of email from people that basically said, "I read on Slashdot about what you did. That's cool. Good job." Throughout the day of the release people would send email to me pointing out all the articles talking about the WiX toolset release. That was incredibly encouraging because the initial external response was this amazing onslaught of "You are a @#$@% horrible person! Microsoft is @#$@% evil! I $@#%@ hate you!" I am not exaggerating. Lots of messages filled with foul language were sent to me via my blog early in the morning on Monday, April 5th.

As the day progressed, more rational comments started showing up and some people I've never interacted with even started to defend the release of the WiX toolset. I was seriously overwhelmed for the first couple days since there was so much attention to the project and a lot of the attention was trying to spin what I was doing as "evil". That was very disenchanting since I was really just trying to make life better for setup developers.

Q: Would you do it again?

Absolutely, for two reasons. First, I have believed for years that Microsoft needed to be mature in its dealings with the Open Source community. I ran the Windows Installer XML project as an internal Open Source project to provide an internal demonstration how Open Source can work. I never would have passed up an opportunity to improve Microsoft's dealing with Open Source. Second, I wanted the WiX toolset out there to make developers' lives better. Just the other day, I found this quote in a blog entry "Amazingly, for an installer builder, WiX is turning out to be almost entirely pleasant." Statements like that make the release completely worth it for me.

Q: If someone else internally wanted to create something as a SourceForge project, would you recommend it?

If the project was appropriate to be Open Source then I would definitely recommend releasing as Open Source on SourceForge. Note I am not someone who believes all software should be free and that all source code should be open (I don't understand the economics of it). However I do believe there are many opportunities for projects to be released as Open Source like WiX and WTL.

If you are a Microsoft employee and have any questions, feel free to contact me. If I can't answer the question, I can point you to someone who can.

Local Installation Source Tool (listool.exe) released to web.

robmen — Fri, 28 May 2004 19:03:00 GMT

K informed me yesterday that the Local Installation Source Tool (listool.exe) has been RTW (Released to Web). Terri (my PM on LocalCache) always called this thing the "Cache Movin' Tool" in her specs. Anyway, listool.exe was an intern project last summer that Terri and K pushed out the door. Finally.

So what does it do? The listool.exe does three very useful things that I wish we would have been able to ship on the Microsoft Office System 2003 CD from day one:

1. Enable the LocalCache - this feature will direct the Office Source Engine (codenamed "injector") to ensure the products you choose are cached. My suggestion is that if you are having any problems patching that you run this option for all your products.

Note: listool does seem to require your original source media even if you already have everything cached. K, why doesn't listool direct injector to only cache files that are missing or corrupt? That seems unfortunate.

2. Move the LocalCache - this feature will move all the cached files from one volume to another. There was a lot of debate early in the LocalCache feature design whether the user should be prompted to pick the volume where files should be cached or not. I'm not one for hiding behavior from users and suggested the user should be presented a combo box (or something, I don't do UI) that displayed the available volumes for caching. However, Terri had a very good point that it is difficult to explain to novice users what to do when presented with such a combo box. In the end, I was overruled but Terri added a section to one of the LocalCache specs (there were 8 or so, I had all kinds of crazy ideas) about the "Cache Movin' Tool". So here we are.

Note: you don't get the option of caching individual products to different volumes because the test team ran out of time to verify that LocalCache worked properly in those cases. I can't tell you how frustrating it is to have a feature coded to completion and then have it get cut because testing can't get to it in time. Anyway, injector supports caching to separate volumes but all of the tools (Office's setup.exe and now listool.exe) ensure all products are cached to the same volume.

3. Disable the LocalCache - this feature obviously directs injector to uncache all of the files that were LocalCached and ensures they are never cached again. If you turn off this feature then if the Windows Installer ever requires the original source media you will be prompted for your CD (for example, during patching). I am admittedly biased, but I highly suggest you not disable the LocalCache.

Note: Throughout this blog entry, I've referred to what the Office Resource Kit calls the Local Installation Source or LIS as LocalCache.

My philsophical musings about building setup for software.

robmen — Mon, 17 May 2004 00:13:00 GMT

Mike Gunderloy, who has written a couple articles about the WiX toolset, posted a comment on a previous blog entry. In the comment, he suggests that splitting a setup project into fragments only moves the problem but doesn't solve it. Thus, he argued that adding a tool to generate the fragments has value. I still didn't agree. Then I realized that I disagreed because I have some philosophies about how to build software and setup for software that I've never posted here. So, I thought I'd share those philosophies today. Please note that these are guidelines that I use when discussing setup build processes with other people not hard rules.

The developer that wrote the feature knows best what needs to be authored into setup.

This part of my philosophy is based on the fact that the person that knows the most about a particular resource is the developer that wrote the resource. Seriously, if the developer who wrote the code doesn't know where his or her files need to be installed or doesn't know what registry keys are necessary to make the feature work (or whatever other resources are necessary) who does?

Yes, there have been (too many) cases in my career where the developer who wrote the code said, "Uhh, I don't know what my code depends on." However, in those cases it was pretty easy to look at him or her (or his or her manager) and ask, "Well, uhh, shouldn't you?" In every case, they went off and figured out what was necessary to get their software "in the box".

Fundamentally, if developers don't know what their dependencies are there is very little chance their project will have a solid security, performance, or deployment story. Area experts may be necessary to help developers work through complicated issues but, in general, developers must be aware of what their software is doing.

Setup authoring is a part of the development process.

Every team I have interacted with (the grand majority of Microsoft and some smaller companies), the developers on the team are expected to add their source files to the makefile before checking in new files to the project. Yet, many of those teams have people who are solely responsible for adding new files to the setup project. In many cases, one set of developers write registry keys into the code for SelfReg (this is evil, and I'll explain why one day) and the setup developers have to reverse engineer the registry keys out of the built executables to author setup. Why didn't the original developer just author the registry key into setup in the first place?

There was a point in time where it was arguably difficult to distribute the authoring to all developers when you had to buy custom tools that didn't fit well into the development process. However, today there are a few alternatives out there (like the WiX toolset) that allow setup to be treated like source code. Now there are no excuses I've heard that hold water why setup authoring for the majority of the resources in setup cannot be distributed across the developers in the organization. Doing that distribution leaves only the "look and feel" and integration between the individual pieces of setup to the core setup developers. All other excuses have always been just whinging (as Peter might say).

This part of my philosophy and the one above are the reasons I disagree when Mike says that "breaking up the package into multiple source fragments pushes the problem back one level, but doesn't necessarily solve it." Breaking up your setup in to multiple text files enables developers to maintain setup authoring the same way they maintain all the other code that is part of a project. And that brings me to my final point.

Text files should be the only inputs into the build process.

Over the years, developers have created fairly significant processes for tracking, merging, and reverting changes to text files. For example, Concurrent Version System (better known as CVS) is used heavily by SourceForge.net can show you the individual lines changed in a text file over time. For a demonstration, take a look at Compiler.cs from v1.6 to v1.7 in CVS at SourceForge.net for WiX. It is not generally possible to visualize the differences between two revisions of a binary file.

Every input into the build process that is manipulated by a human will eventually hose the process at least once. Being able to text search (using even the simplest tools like grep) for the exact change that caused the problem significantly improves build throughput. Binary files usually hide the information by requiring custom tools to be opened and queried to find the break.

Of course, once you find the break it is really nice to be able to fix the problem by launching your favorite text editor and tweaking the line(s) of code with the fault. Requiring a tool to be installed on the build machines increases your impedance to fixing the issue.

Ultimately, keeping binary files out of your build process simplifies your life. That fact is why I disagree with Mike when he suggests it makes sense to "put one more tool into the chain, something with a friendly interface that could spit out the source fragments as needed? I could see doing that with Access/Excel, among other things."

Note I do think it is reasonable for developers to use tools that help generate text files. Those text files then can get checked into source control and built as part of the standard process. However, the tools need to generate human friendly text files. Text files that can only be modified with a custom tool are only half a step better than raw binary files.

This is the philosophy that I developed during my tenure in Office and have promoted across the company for a couple years now. Many teams have had an incredible amount of success with their setup processes when following these guidelines. There have been many cases where I was asked by other teams to talk about the philosophy. At the end of my talk I always left them with, "If you have someone resisting change, have them contact me. I'll be happy to talk to him or her to discuss how successful other teams have been with this process and see if there isn't some way to address their concerns." I'd be happy to do the same for you.

PS: This philosophy has worked out extremely well for me, but if you have a unique situation that you think wouldn't work using these guidelines, I'd be very interested to hear about them. I'm always looking for alternative views (as long as people aren't just whinging <smile/>).

Checking in on the Windows Installer XML (WiX) toolset development environments.

robmen — Fri, 14 May 2004 18:49:00 GMT

After getting the latest round of bug fixes posted in the WiX 2.0.1713.0 release yesterday morning, I decided to check in on the other projects on SourceForge to create development environments for the WiX toolset. When the WiX toolset was first released, I used to check in on these projects all the time. However, after a couple weeks of absolu tely no activity I feared they were all DOA (Jenny real ly enjoys that show, by the way).

I started with WiXStudio because the project admin, Terry, was very gung-ho about the project. We even exchanged a numb er of email messages right after the WiX toolset was released. Terry dug i nto the object model pretty quickly and determined that the XmlSerializer classe s provided by the core wix.dll weren't going to provide a friendly enough progra mming model. So, I suggested considering creating a CodeDom of sorts for WiX source files. Terry jumped on the idea and promptly disappeared. Well, it turns out Terry has a blog and has posted an entry about the current progress of WiXStudio . I've subscribed to the RSS feed in hopes more progress appears soon.

Since I felt caught up on the status of WiXStudio, I went back to SourceForge 's search and found the other three WiX development environment projects. Terry's blog mentioned talking to the WiXGUI project members so I clicked on that one first. While it appears the project has had more page hits that the WiXStudio project, there is absolutely no other progress displayed on SourceForge. It appears the onl ine forums have even been disabled.

I then checked out the . NET Installation Generator project. This project had even fewer hits t han the WiXGUI project and the same amount of progress (i.e. none). If any one has any news on either of these projects, please feel free to leave a commen t below. I'm curious if things are progressing silently behind closed door s or something.

Anyway, I was a little disappointed no one had anything working yet so I had low expectations when I went to check the last tool on the list, WiXTool. Needless to say, I was qui te pleased to see that it actually had files to download. I downloaded the MSI file (the source code is unfortunately under GPL so I'll be staying away fr om that) and tried to install it. The package wouldn't install so I pulled it up in Orca and ran basic validation. There are several serious authoring errors in there (which is unfortunate since it seems the MSI was created by the WiX toolset, guess there are places to tweak down) including missing platform inform ation in the _SummaryInformation stream. I tweaked the _SummaryInformation stream, tried installing again, and got the executable installed on my machine (but no shortcut since that contained one of the more serious validation errors). The UI looks pretty basic and straight-forward. None of the graphical editing seemed to work but the XML editor looked pretty decent. I didn't play with it much after that (just uninstalled the tool), but it was nice to see one of the WiX development environment tools on SourceForge making progress.

Anyway, I will continue to keep an eye on things to see if anything develops. Hopefully one of these projects will take off. Lots of people have asked for a decent development environment for the Windows Installer XML toolset and I personally have very little interest in doing any UI-related work. The WiX toolset keeps me busy enough as it is.

Updated: Fixed a typo in URL and removed a couple invalid spaces

Windows Template Library (WTL) has released as Open Source on SourceForge.net

robmen — Wed, 12 May 2004 17:13:00 GMT

The Windows Template Library (WTL) appears to have finished propagating around the SourceForge.net CVS servers, so I can finally post this blog entry. Many people have pointed at my interview with O’Reilly’s OSDir.com where I noted that other employees in Microsoft have come to me looking for advice to releasing code. Nenad was one of those employees and it looks like everything has finally worked out.

From what I understand WTL picks up where ATL leaves off and adds a bunch of useful things for doing GUI stuff on Windows in C++. I don’t do GUI so I can’t talk much more about WTL but a search shows plenty of resources to get you started if you’re interested.

Anyway, now that WTL has been released under an Open Source license the community that has built up around the code will finally be able to contribute back (should they desire). Is there more to come? I don’t know (I hope so) but I can tell you people around here are definitely still watching WiX and WTL.

So you want to be a Windows Installer XML developer?

robmen — Wed, 14 Apr 2004 11:21:00 GMT

People have started expressing interest in joining the Windows Installer XML toolset development community so I figured I should get some administrative details out of the way. If you are interested in contributing code to the Windows Installer XML toolset, it is very important to read through all four of these topics.

1) The Windows Installer XML toolset copyright is held by Microsoft.

I want to be very up front about the copyright of the Windows Installer XML toolset and how it affects us as developers. Microsoft is the sponsor of the Windows Installer XML project. Before a contribution can be accepted into the WiX project, the lawyers have asked that we assign our rights to those contributions to Microsoft. By having developers sign a copyright assignment agreement, Microsoft can maintain single legal control of the project. That single legal control enables Microsoft to best defend the project in the future if there was ever any sort of legal challenge.

Before jumping to any conspiracy theories, please note that this copyright assignment is exactly the same process the Free Software Foundation has you go through if you work on a project they sponsor. Also, in Clause 5 of the Windows Installer XML assignment agreement your rights to your contribution are explicitly granted back to you. If you would like a copy of the assignment agreement, please contact wixadmin@microsoft.com.

2) The Windows Installer XML project is a benevolent dictatorship.

In order to ensure consistency in the schema and maintain the quality of the tools, the Windows Installer XML project’s CVS tree is locked down. In other words, commits to the code-base by the general populace are prevented. If you attempt commit changes, CVS will inform you that you have "Insufficient Karma to complete the task."

To have your contribution submitted to the project, please submit an assignment agreement as described above (you only need to do so once) then send your code diff to WiX-devs@lists.sourceforge.net. The developers there will review the changes and someone will apply them to CVS as quickly as possible.

3) The Windows Installer XML community is a meritocracy.

Those individuals in the community who demonstrate an understanding of the code base by actively participating on the Windows Installer XML mailing lists and consistently submitting high quality diffs will be given a “Karma boost”. With enough Karma you will earn the ability to commit changes directly to the Windows Installer XML project’s CVS tree.

Commit privileges should not be taken lightly. It is very important that the WiX toolset maintain a high quality bar because many people depend on the tools working properly. Very few developers earn these privileges. In fact, in over four years of development, only five developers have earned commit privileges to the internal Windows Installer XML project.

4) The Windows Installer XML developers are all volunteers.

Everyone (to the best of my knowledge) that works on the Windows Installer XML toolset does so in his or her free time. Please keep that fact in mind when asking for help, submitting code diffs, or interacting with any members of the project. We all want to help to make the Windows Installer XML toolset as solid a tool as possible, but sometimes “real jobs” and “significant others” have to take a higher precedence.

If worse comes to worse, you have access to the source code. Try reading for a while. :)

Open Source Business Conference and green light means go.

robmen — Tue, 16 Mar 2004 10:20:00 GMT

I just got back from a successful trip to San Francisco and I'm very glad to be home again. However, I really wish I could attend the Open Source Business Conference 2004 in downtown San Francisco. It'd be really interesting to listen to speakers address the issue I've highlighted from their introduction:

[read more]