HD DVD / Randomness... : Office

Code Repurposing and Untrustworthy Data

ptorr — Fri, 03 Sep 2004 14:22:00 GMT

This is just a generic launching place for four other blog entries, since I seem to send them to people on a regular basis and sending one URL is easier than four :-)

Code repurposing

· http://weblogs.asp.net/ptorr/archive/2003/10/16/56270.aspx

· http://weblogs.asp.net/ptorr/archive/2003/10/21/56296.aspx

Untrustworthy data

· http://weblogs.asp.net/ptorr/archive/2004/04/12/111342.aspx

· http://weblogs.asp.net/ptorr/archive/2004/04/13/112404.aspx

A useful regfile for VSTO

ptorr — Fri, 16 Jul 2004 10:11:00 GMT

Here's a quick post with a regfile you can use to help you test your VSTO projects.

Cut and paste the text below into a text file (be careful of line wrapping) and save it with a reg extension. Then open up regedit (as a member of the Administrators group) and select File -> Import... from the menu and navigate to the file you just saved (you could also just double-click on the reg file in Windows Explorer).

This will give you two new context menu items on DLLs, EXEs, and Folders. The first one will add a URL membership condition to user-level policy to fully-trust the file (or folder), and the second one will remove that entry from policy. Note that these shortcuts are pretty dumb, they won't actually "untrust" the file or folder (they just remove the "explicit" entry required by VSTO), they could completely destroy your computer and the surrounding countryside, they should be used at your own risk, etc. etc. etc.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\dllfile\shell\FullTrust]

@="Trust assembly"

[HKEY_CLASSES_ROOT\dllfile\shell\FullTrust\command]

@="C:\\WINDOWS\\Microsoft.NET\\Framework\\v1.1.4322\\caspol.exe -q -u -ag 1 -url \"%1\" FullTrust -n \"%1\""

[HKEY_CLASSES_ROOT\dllfile\shell\UnTrust]

@="Remove assembly trust"

[HKEY_CLASSES_ROOT\dllfile\shell\UnTrust\command]

@="C:\\WINDOWS\\Microsoft.NET\\Framework\\v1.1.4322\\caspol.exe -q -u -rg \"%1\""

[HKEY_CLASSES_ROOT\exefile\shell\FullTrust]

@="Trust assembly"

[HKEY_CLASSES_ROOT\exefile\shell\FullTrust\command]

@="C:\\WINDOWS\\Microsoft.NET\\Framework\\v1.1.4322\\caspol.exe -q -u -ag 1 -url \"%1\" FullTrust -n \"%1\""

[HKEY_CLASSES_ROOT\exefile\shell\UnTrust]

@="Remove assembly trust"

[HKEY_CLASSES_ROOT\exefile\shell\UnTrust\command]

@="C:\\WINDOWS\\Microsoft.NET\\Framework\\v1.1.4322\\caspol.exe -q -u -rg \"%1\""

[HKEY_CLASSES_ROOT\Folder\shell\FullTrust]

@="Trust folder"

[HKEY_CLASSES_ROOT\Folder\shell\FullTrust\command]

@="C:\\WINDOWS\\Microsoft.NET\\Framework\\v1.1.4322\\caspol.exe -q -u -ag 1 -url \"%1\"\\* FullTrust -n \"%1\""

[HKEY_CLASSES_ROOT\Folder\shell\UnTrust]

@="Remove folder trust"

[HKEY_CLASSES_ROOT\Folder\shell\UnTrust\command]

@="C:\\WINDOWS\\Microsoft.NET\\Framework\\v1.1.4322\\caspol.exe -q -u -rg \"%1\""

VBA Take Two: Responding to some comments

ptorr — Sun, 25 Apr 2004 05:37:00 GMT

The other day, Karl Levinson added a comment to my previous entry about the Outlook OM. He raises some interesting points, so I thought I'd reply here. (Karl, please don't take any of this personally; I hear the same arguments from people all the time, and it's something I believe very strongly in -- we're not going to make the world a better place until we start focusing on the right problems to solve).

A quick opening comment: I am in full agreement that (in an absolute sense) a computer without VBA on it is "more secure" than a computer with VBA on it; the same can be said for almost any piece of software. The question is whether or not VBA should be singled out as the "bad guy" and special-cased for removal when you take a look at it from a value proposition / risk assessment perspective.

So here it goes (Jeff, I promise this will be a short entry! Honest! Ha ha ha):

I don't see how you can argue that technologies like VBA and WSH don't present a very compelling attack surface, given the billions of dollars and system availability that have been lost combating Office macro and .VBS viruses over the years.

Certainly VBA and WSH have been unwilling participants in a large number of viruses over the years; nobody can deny that. But ask yourself "Would the world be virus-free if WSH and VBA were never invented?" and of course the answer is "No."

Those viruses were not caused by the presence any particular tool; they were caused by someone with malicious intent taking advantage of weaknesses in the perimeter defences of the user's system (historically, the combination of e-mail clients not blocking potentially dangerous attachments, and users' willingness to execute dangerous attachments after being socially-engineered into doing so).

On a technical note, nobody really attacks VBA or WSH; there have been very few security problems with each of those technologies, and none of them have been abused to the best of my knowledge. They are not an "attack surface" per se any more than the GNU C++ compiler is an "attack surface" -- they are just tools that can be used by evil miscreants to help do their deplorable deeds.

Returning to the burglar analogy again, the rack of knives in your kitchen does not present an attack surface; it is the wide-open front door and your soft, fleshy exterior that are the attack surfaces. You can remove the knives to help mitigate any damage that an attacker might do if they break into your house, but:

1) They've still broken into your house and can use any other kind of weapon to attack you (including ones they brought with them!); and

2) Now you no longer have the utility of the knives at your ready disposal (maybe that's a price you're willing to pay)

Have any of the recent virus outbreaks (Slammer, Blaster, MyDoom, NetSky, Beagle, Witty, Klez, etc.), actually taken advantage of WSH or VBA or the Outlook object model? No; that proves that neither is necessary for the propagation of viruses. And have any of my machines (which have WSH and VBA on them) ever been infected with a virus? No; that proves that neither is sufficient for the propagation of viruses. It takes something else (a naive, curious, or malicious user; a buggy, poorly-designed, or out-of-date product; physical access to the machine; etc.) to propagate a virus, and the part played by VBA or WSH is more or less replaceable by any other technology.

And now MSH is coming down the pipe.

We've had batch files since the DOS days; should we rip out the batch processor from Windows? On an ironically related note, many people complain about the lack of a good scripting solution on Windows; they point to the (ultra-secure, of course) *nix variants with bash and ksh and perl and so on. How come nobody asks those guys to remove the features from the respective OSes? It's because the people running those platforms tend to know what they are doing and would not execute arbitrary code from unknown locations. (At this point, someone will no doubt pipe up: "But you have to chmod +x a file before it will run on *nix!" to which I reply: "Yes, and you have to chmod +x <insert favourite application here> to install it, too!" -- if I have the skill and authority to download, install, and execute CoolApp then I also have the skill and authority to download, install, and execute NastyVirus. But I've been over that argument before).

XP SP 2 does *not* fix the problems with VBA and WSH viruses... precisely because it does not disable the technologies in question.

You are correct, but you are correct because (brace yourself for even more controversy) there are no problems with VBA and WSH viruses to "fix!" There are only problems with users (unwittingly) downloading and executing malicious code. And SP2 tries to address that by locking down Internet Explorer (LONG overdue; you have no disagreement from me there) and by providing better blocking of attachments for applications that request it (see below).

But seriously, how would you "fix" VBA? Disable it? Great! No more VBA!

Now ask yourself, "What did that buy me?" All things being equal, you're still just as susceptible as you ever were to all the viruses and worms I listed above, plus plenty more. All you've done is removed your ability to record macros or run other useful software applications on your system.

As I understand it, SP2 adds the AES API to block attachments and integrates this with OE and Windows Explorer. Unless I'm mistaken, some of the features of AES may not protect users of other non-Microsoft software, email clients, P2P file sharing clients, etc.

(Note to readers: in this context, AES refers to Attachment Execution Services and not the Advanced Encryption Standard)

Correct; applications have to know to call into the new API to take advantage of it, but that's nothing new. Any time Windows adds a new API, applications have to be modified and re-released to take advantage of them. I seriously doubt that any file sharing client would ever use AES though -- how would you download your warez from KaZaA if it blocked EXEs? (Yes I know there are legitimate uses of P2P software... I'm just being facetious :-) ).

Also you can't expect Windows to blanket deny access to all EXEs or other file types; Windows doesn't know why an application requests access to a specific file; it just checks "Is this user allowed to have access to the file?" and if so, grants it. This gets better with partial trust in the CLR because applications can have fewer rights than the user running them... but I don't want to get into that right now. Hackers aren't going to write managed apps (which are subject to stringent security checks) while they can still write native apps (which are not).

You don't explain why it's a good security practice [dare I use billg's words, "secure by default"] to (1) leave these technologies enabled on, say, home computers, and (2) give the user absolutely no way to disable unwanted technology such as VBA despite numerous user requests.

That's a good question, and I can't really give you an "absolute" answer because security is about risk management, not risk avoidance, and since there are a lot of unknown variables involved it is not an exact science. "Secure by Default" is part of the SD3+C campaign and revolves around disabling (or not even installing) features if they present a high risk to the safety of the user's PC or data. For example, a service that runs as SYSTEM and accepts unauthenticated packets from the network clearly represents a high risk and should be disabled unless it is absolutely critical for the health of the system. But what about the "Letter Wizard" in Microsoft Word? (I just picked a random feature I've never used) It's not critical to the health of the system, and I bet most users never need it, but it doesn't represent a high risk because it is not remotely accessible and a bug in it wouldn't allow for elevation of privileges (it runs in the context of the user accessing it). So it is left on by default.

VBA and WSH clearly fall somewhere between these two extremes, but IMHO they are much closer to the Letter Wizard than they are to the SYSTEM service. Neither of them is remotely accessible or would allow an elevation of privilege if it were buggy. It requires explicit user action to invoke either of them (opening a file) and, in the case of VBA, it is already shipped in a pretty locked-down mode (no unsigned code will run from documents). You could argue that double-clicking on a JScript or VBScript file should open it in Notepad by default... but then what about EXE files -- should they open in Notepad too? And screen savers? And Control Panel applets? And let's say that we did this, and everyone learns that to execute a file you no longer have to double-click it, but instead you have to right-click and choose the "Run" command. What's going to happen when the CelebrityNaked.exe virus comes around? People will right-click it and choose "Run!"

This is a phenomenon that I have witnessed many times over -- the idea that script files and executables are somehow inherently different and should therefore be treated differently. It's OK to execute an EXE if I double-click on it, but it's not OK to execute a VBS file if I double-click on it. Hmmmm, why is that so? They're both essentially the same thing. They can both do equivalent amounts of damage. In fact from the recipient's perspective there is no discernable difference between them (and that is, funnily enough, by design)!

One thing that we do hear though is that "customers know EXEs are dangerous" and so they are less likely to double-click on an EXE than they are to double-click on a "known to be safe" (ha!) file type such as .TXT or .DOC, or on an unfamiliar file type like .VBS or .PIF. That may be true, but those users are fooling themselves. Even a text file can contain a virus! The basic idea is to trust no-one, especially not Prince Whatsumacallit from Nigeria who wants your help in liberating $10,000,000 and will give you a healthy cut of the deal if only you'll give him your bank account details and pay hundreds of thousands of dollars in expenses. (My apologies to any non-419-scamming Nigerians who may be reading this blog). And don't assume that just because you've never seen a .FOO file before that it will magically be safe! :-)

I'm not asking for Microsoft to get rid of VBA or WSH or MSH... just recognize that these are proven virus platforms, and that we should have an easy way to disable them if we want, or even consider the security benefits of making them disabled by default.

As I have tried to present, VBA and WSH are not "virus platforms;" they are computer languages / runtimes.

But here I present three easy ways you can disable WSH if you so wish (ha ha, pun intended :-) ):

· On NTFS-based systems, ACL cscript.exe, wscript.exe, or any other files of your choice so that they are not accessible to the user. Lots of legitimate things may break if you do this though ;-)

· Modify the registry keys in HKLM to map the "Open" verb for the various script extensions to run Notepad

· On Windows XP or Windows Server 2003, use Software Restriction Policies to block the execution of unsigned scripts

VBA is disabled for most scenarios out of the box anyway; just leave it at "High" mode (or the new "Very High" in Office 2003), uncheck the "Trust installed templates and add-ins" setting, and remove all the "Trusted Publishers."

You state that disabling WSH and VBA would just "make you less vulnerable to the more "popular" attacks." To me, that's like saying "you shouldn't run a firewall, because you'll still be vulnerable to viruses." Yes, the virus authors would probably start using other attack vectors... at which point we would want to take steps to reduce the risk from THAT new vector.

I would never recommend anyone not run a firewall :-) And I now realise I was actually a few years out of date by referring to VBA and WSH viruses as "popular," but eh -- I've never claimed to be up with the latest fashions ;-)

Firewalls perform a very legitimate task by reducing your attack surface and providing a first layer of defence against malicious code attacks. They are the fortified castle walls that protect your soft fleshy body from the daemons of the night. But, as you note, they are not a panacea. Nothing is a panacea. Not firewalls, not partial trust, not digital signatures, not limited user accounts -- nothing. It comes back to risk management again -- you could disable WSH and VBA and bask in the (small) additional protection you got by not being vulnerable to viruses that party like it's 1999, but you'd still be vulnerable to all the others and you would have lost the functionality that those features provide. Enabling a firewall, on the other hand, gives a significant amount of additional protection that (for many users) imposes no undesirable restrictions on their computer usage.

Also, making a truly secure by default computer does not mean secure yourself just from the most popular viruses. Even if few people are writing Word macro viruses nowadays, you're still at risk from a teenager from Iraq writing one up to get into your nation's infrastructure.

Yes, you are at risk from anyone from any country in the world sending you malicious code. I return to my point above -- why focus on script (or VBA) as being different from any other kind of code? In an age where we have point-and-click virus creation tools and exploit testing tools, the argument that "it's too easy to write script" doesn't seem to matter much anymore.

Blaming the user here doesn't increase security much, not when it's a sure bet that at least 1 in every 100 users will execute an attachment, and you only need 1.

I don't want to blame users; I want to educate them :-)

That one user will click on the attachment whether it is a VBS, an XLS, or an EXE. Heck, they'll even open up a ZIP file, type in some blurry password from an attached image file, and then open the EXE inside. And if they were running on MacOS or *nix, they'd do whatever was necessary to get the file to run on that platform, too.

Many of the people that run these latest viruses WANT TO RUN THE CODE -- they just don't know (or don't care) that the code they are about to run is malicious. Maybe they think it's a pornographic picture, or a cool joke, or a cracking tool for some hot new game. Maybe they've been told their internet connection or their bank account or some other valued resource will be cut off if they don't run it. I don't know. But the newer viruses get executed not because of any flaws in the system, but because there is a person at the other end who has been tricked into doing something they probably shouldn't have done.

Nobody argued for disabling RPC in Windows because Microsoft programmed RPC into many other products that Microsoft Windows customers are also running. Besides, there's already a way to disable RPC/ DCOM if you wish.

WSH and VBA are also used by many products that customers want and need; in fact many companies run their business on solutions built on top of VBA or script.

There's no way to disable VBA. I've asked.

Surprise! I have an early Christmas present for you :-) VBA has been an optional component of Office for (at least) the last two releases!

· Select Start -> Run -> %comspec%

· runas /user:Administrator "control appwiz.cpl"

· Select "Microsoft Office Professional Edition 2003" and click "Change"

· Select "Add or Remove Features" and click "Next"

· Select "Choose advanced features" and click "Next"

· Expand "Microsoft Office Shared Features" and de-select "Visual Basic for Applications"

· Click "Update"

So you can completely un-install VBA if you don't trust the "High" or "Very High" modes with all the other settings cranked way up... but things will break. For instance, you will probably be unable to install or use any 3rd party add-ins, formula libraries, etc. and of course you won't be able to record and run macros.

We wouldn't have had an ILOVEYOU virus if Microsoft had simply changed the default action on .VBS and other files from Execute to Edit.

See comment above -- this would NOT have stopped ILoveYou-like viruses at all. The author would just have picked a different route, or a different author with more skill would have come along instead. MyDoom and the other recent viruses have done HUGE amounts of damage and do not rely on VBS. The fix to all these problems was to stop users from accessing unsafe attachments (which of course annoyed those of us that knew what we we're doing and actually had legitimate reasons for receiving JS or VBS attachments... like say the Program Manager for JScript or the main dev for VBScript ;-) ). And possibly forcing the display of the real extension so that it says .TXT.VBS instead of just .TXT. (Yes Mr. Word Grammar Checker Sir, I know that's a sentence fragment, but I like it that way!!!)

Can you give me a good reason why this still hasn't been done on the "secure by default" XP SP 2 and Windows Server 2003? Will you trudge out the old excuse that this would somehow "break functionality?"

Yes, it's probably for backwards compatibility, and they probably did a risk assessment and decided it wasn't worth the effort. But I don't have anything to do with the groups that produce those products; you should ask Michael Howard. Nevertheless, whilst the OS teams probably could make some changes to the way WSH worked if they wanted to, the OS team isn't in the business of making changes to VBA, which is a feature of the Office System and a bunch of other 3rd party applications (I can imagine the uproar from our ISV partners if Windows XP SP 2 broke all the 3rd party applications out there that relied on VBA because of an "Outlook virus problem").

Here's another side to the equation: Let's say the Windows team decided to disable WSH in the next version of Windows. That's probably a 1-line change to some metadata file that goes into the Windows build process (to flip the reg key from cscript.exe "%1" to notepad.exe "%1") but as Eric Lippert has pointed out, that's not the end of the story! It would probably take -- no kidding -- a month or more to run all the necessary tests on this to make sure it was an "OK" fix. Hundreds or even thousands of 3rd party applications would be tested, and many of them would break in weird and wonderful ways. We'd have to co-ordinate with them, and where applicable we might make special shims for specific applications. And then when we did ship the product, PSS would be busy answering calls from customers asking how to turn it back on (and then there would be KB articles written about the problem). And all the 3rd party vendors would have to update their software and re-ship it to customers. And all their customers would have to install the upgrades. I don't know about you, but I'd rather all those resources went into making RPC more secure, or building a better firewall, or answering customer support calls about real security issues. And the customers would not be happy about the cascading re-installs, all for very little real benefit. As Raymond says, No code is an Island.

Running as a non-administrator does not stop viruses. A non-admin user can still execute a virus and access the TCP /IP ports necessary to spread an RPC or email worm.

Exactly! I get a wonderful happy feeling deep down inside every time I meet someone else who appreciates this fact (which of course means I get a horrible sinking feeling whenever I read Slashdot :-) ). In fact, in a comment to a previous blog I said just about the exact same thing to another reader :-)

My understanding is that Linux prevents non-admin access to certain TCP /IP ports, but Windows does not.

...and this will have no material impact on the security of Linux if it ever gets wide-spread consumer adoption in the way Windows has. Why? Because consumers will want to install iTunes or KaZaA or Trillian or Unreal Tournament 2037 or any number of other applications that will require access to the internet. And either those applications will use the non-locked-down ports (in which case malicious code will use those ports, too) or those applications will require the user to run them as root (in which case the user will run the malicious code as root, too) or they will require punching a hole in the firewall (which the user will do for the malicious code, too).

Thanks for writing an interesting and thought provoking article. It's interesting to see the thought process from the other side.

Agreed. Like I said, please don't take this personally. And I do appreciate the other points of view -- in fact if it served my purpose to do so, I'd probably make exactly the same arguments you are making ;-). (Shhhh, don't tell Eric!)

Believe it or not, I am considered the tin-foil-hat wearing, ultra-paranoid, the-sky-is-falling security guy in my team. They think I'm radical for running as a normal user, reading my e-mail as plain text, running IE in "High" security mode, and insisting on draconian mitigations for "theoretical" attacks in our product design. And here I am spending my Saturday afternoon arguing loud and clear for the existence of programmability features such as VBA and WSH and why they don't represent a real security risk. (Perhaps that speaks more about my work/life balance than it does about my attitude towards security ;-) ). It's a strange world ;-)

The fact is that while we're busy building secure designs, locking things down by default, giving guidance on secure deployment, and providing timely and clear communications about security issues, we still have to provide a compelling, useful experience for users. Otherwise they'd all be running unpatched versions of Office 97 and IE 5.0 on Windows 98 Gold and the viruses would never stop. Oh wait... :-)

P.S. Karl, can you please make your website work without script enabled ;-)

Why does Outlook have an OM?

ptorr — Sat, 17 Apr 2004 06:46:00 GMT

This one could be controversial ;-)

In a recent comment, Edd James (note to Edd: that link gives a 403) asks why Outlook and Excel "need this ability to run scripts/macros[?]"

First I want to clear up a common misconception about Outlook: Despite what the endless ill-informed posters on Slashdot might claim, no recent version of Outlook (or recent update to an old version of Outlook) is designed to run code out of e-mail messages in the default configuration. Every once in a while someone finds a bug in the IE rendering engine or in Outlook that enables such execution, and that bug is fixed. Customers who want "dynamic" e-mails can still enable the feature through the Outlook security settings, but it is not the default and is not at all recommended.

But moving on, let's turn the question around: why shouldn't Outlook have a rich object model? I challenge you to give me a sound answer to that question based on security concerns (I can understand why you might not want the feature for "code bloat" reasons, etc.)

The obvious answer is that having an object model in Outlook makes all those mass-mailing viruses possible. Apparently anyone who uses that argument hasn't heard of the recent viruses going around; that latest "virus technology" doesn't rely on Outlook to do its dirty work. It scans files on your hard-disc to scavenge e-mail addresses, and then it uses a built-in SMTP mailer to send out the mails. If you are running Lotus Notes or Pine or Eudora or Mozilla Mail or any other e-mail client and you execute a MyDoom-like virus program, you are in trouble. (At this point, someone may point out that their e-mail program of choice is not susceptible to the virus de jour because the virus only understands the Outlook address book file format. True, but that has nothing to do with whether or not Outlook exposes an object model, and everything to do with the size of the installed user base).

The next answer is that having an object model in Outlook makes all those mass-mailing viruses easier to write. It is too easy for a "script kiddie" to cobble together some VBScript code and take over the world, so the argument goes. But the term "script kiddie" doesn't necessarily literally refer to "kids" writing "scripts." It refers to relatively unskilled people (of all ages) downloading sophisticated attack tools (written by the "real hackers") and then using them in some possibly-automated fashion. I doubt the average "script kiddie" has enough m4d c0ding 5ki11z to even write "Hello World" in VBScript, let alone craft something sophisticated like MyDoom. What the kiddies can do is surf around on #hacker IRC channels, download pre-canned exploit code from hackers, double-click on the icon on their desktop, and then brag to all their other 1337 friends. Basically, the really bad people (the criminals) who write real viruses don't need the Outlook OM to do their dirty work; sure if it is there they might choose use it, but they don't need it.

Of course the other point is that having an OM makes it easy for legitimate developers to write applications that better meet their customers' needs, and that is A Good Thing. We don't want to make it arbitrarily hard for "the good guys" to build solutions using our technologies, and in the end it won't really buy us anything since the bad guys are more determined than the good guys and so they will persevere with writing their malware whether we "help" them or not.

In fact, by making it hard for ISVs to write against the Outlook OM, you can argue that the world has gotten worse because customers now typically install 3rd party applications such as ClickYes or Redemption to re-enable access to Outlook... and those programs may have security bugs of their own!

The next answer (and a slightly better one) is that many people don't need the object model, so in order to reduce the attack surface of Outlook it should not be installed. The same argument has been used for WSH (Windows Script Host) as well, and you could make it for all sorts of other features, too; installing anything on your system (even security software) increases the attack surface of your system in one way or another. But the fact is though that these kinds of features don't actually present a very compelling "attack surface" as we usually define it. The person making this particular argument is probably missing the point. Once the malicious code is running on your system, it's game over man, GAME OVER!

I think this calls for an analogy :-)

One day a burglar breaks into your house. You surprise them in the kitchen, so they grab a sharp knife out of the drawer and stab you with it before running away. In an attempt to prevent this from happening in the future, you banish all knives from your house when you return from hospital. Sure, it might be hard to cut your steak from now on, but that's the price you pay for security. (Thanks to Randy for pointing out that "cheese" was not a good choice of words here for an American audience... :-) )

A few weeks later, the burglar breaks into your house a second time, and you surprise them in the kitchen again. This time they grab a big saucepan from the drawer and hit you over the head with it before running away. After returning from the hospital, you remove all saucepans and frying pans from your kitchen to improve the security of your house. No cheese and no fried food might be good for your waistline, after all!

A week or two goes by, and the burglar strikes again. This time you catch them in the bedroom, and bereft of cooking implements they pick up a shoe that is lying on the floor and throw it at you. One trip to the hospital later, you have no choice but to ban shoes from your house as well. Hmmm, this could make going out for a walk a bit tricky, but hey, you have to protect yourself, right?

What's the point of this analogy? Well, you have failed to perform a root cause analysis of the problem. The problem isn't that you have knives or saucepans or shoes in your house; it's that the burglar keeps getting inside! If only you'd invested in a good-quality front-door lock (and possibly a guard dog or an alarm) none of this would have happened. And to appease the "attack surface reduction" argument, removing knives and saucepans isn't a very effective technique because the burglar will just start packing their own weapons or perhaps they'll come in at night when you're fast asleep. A good attack surface reduction technique in this scenario would be to permanently seal the back door (so there's only one main entrance to the house) and to place bars on all the windows (so there's less room to crawl through).

In the same way, removing WSH or Outlook or any other piece of "end user" code on Windows doesn't really help with attack surface reduction, and won't improve the real security of your machine one bit (it simply makes you less vulnerable to the more "popular" attacks, which is cold comfort indeed). Now don't get me wrong -- attack surface reduction is a great thing and we should be doing more of it. But a better example of attack surface reduction is the disabling of unneeded services, or the blocking of dangerous attachments in e-mail messages (the thing most responsible for the drop in e-mail viruses, until ZIP files became the infection vector), because both of these represent possible attack vectors for malicious code. Once malicious FullTrust (native) code is running on your system, it doesn't need any help from installed applications. As an extreme example, imagine that Outlook completely dropped its object model in the next version, and imagine further that nobody else in the world knew how to write a program to send e-mail. Mass-mailing viruses would still be possible; they'd simply bundle an old copy of Outlook '97 into their virus payload and use that to do their dirty work!

This is where threat modelling comes into play. If you yourself are writing some software and are worried about exposing features to COM or .NET clients because of the security implications, then think of it this way. If your threat is:

· User downloads malicious full-trust code that uses your application's OM to do harm

Then you also have the threats:

· User downloads malicious full-trust code that sends windows messages to your application to do harm

· User downloads malicious full-trust code that patches the binary (or in-memory image) of your application to expose an OM and then uses it to do harm

· User downloads malicious full-trust code that duplicates the functionality of your application to do harm

What is the root cause here? It is User downloads malicious full-trust code and that is the thing that you (or rather we :-) ) should be trying to address with things like firewalls, proxies, virus scanners, attachment blocking, Software Restriction Policies, and so on. We put a better lock on the front door so that you don't have to throw out all your kitchen knives.

Obviously this simple threat model doesn't apply if you have an ActiveX component marked "Safe for Scripting," or if you develop managed code that allows partially-trusted callers, or if you accept any kind of untrusted communications from other machines across the network. In those cases you have much more complicated threat models and you do have to start worrying about what happens when someone with restricted permissions talks to your application's OM, and it would be great for you to look at how you can reduce your attack surface. But for a rich-client full-trust-only application like Office 2003, these kinds of threats simply don't apply.

Finally, if you're still not convinced, then think of it this way: Outlook is nothing special. Sure, it's a great e-mail client and I spend most of my day using it, but at the end of the day it's just a piece of software. Whenever a destructive virus comes around, you don't see the whole world demanding that Microsoft remove the DeleteFile API from Windows because it makes it easy to write file deletion viruses; that would be ludicrous. It's the same with viruses like Blaster or Slammer -- nobody asked Microsoft to pull networking from Windows just because there were viruses that propagated across the network (although some people have asked for "raw sockets" to be pulled, even though they are a standard part of most modern operating systems).

Another angle: A recent InfoWorld article shows the prevalence of spyware and adware on users' machines; a lot of that stuff is "willingly" installed by the user as part of some other application (typically file-sharing software), although a lot of it is also "accidentally" installed by users who do not read the (very poorly designed) IE download dialogs (which will thankfully be fixed in SP 2). These programs are probably more "destructive" to the average user than any e-mail virus, since they completely violate your privacy (monitoring web surfing, stealing passwords and credit card numbers, etc.), can include "backdoors" to allow later access and complete compromise of the system, and often cause instability of the OS.

And none of this has anything to do with Outlook, or, indeed, any kind of e-mail program. It has to do with the users' basic right to install software on their own computer and make bad trust decisions in the process. This is the problem that we need to fix long-term; not the ability of software to expose powerful, flexible object models that can be freely and productively used by suitably-trusted clients to build great customer-focused solutions. And of course, since trust is inherently a social problem, not a technological one, so the best we can do is educate users and guide them into making good decisions; we cannot make decisions for them.

There is no silver bullet.

Follow up to "Don't trust that data"

ptorr — Tue, 13 Apr 2004 23:19:00 GMT

Eric makes some good points in a comment to my last post. Nevertheless, the forces of evil within me compel me to respond anyway. (You should have blogged it, Eric ;-) ).

Eric's main point is that the employee doesn't need to use formulas in order to fool the expense report system -- he can simply redirect the TotalExpense named range to point to some arbitrary location that his boss will never look at. That would be correct in an automated system, but the supposition in the first example was that there was no code involved in the scenario; it was all based on people looking at the expense report and following a manual process. Hiding the column or re-directing the named range doesn't make much sense, because the payroll clerk will see the same column that the manager sees ($100).

Hiding / moving a named range (or any other kind of UI spoofing attack) will typically only work when a human makes a decision that a computer then acts upon (because the computer "sees" a different value to the human). You must understand your threats (or your opportunities... mwhahahahahahaaaa) in order to successfully protect (or exploit) a system.

Eric also points out that hacking the cached data blob is probably the hardest attack of all to mount, but that just means developers will be least likely to deal with it! Nobody expects the Spanish Inquisition! If I know you are passing the data to some unmanaged component, for example, maybe I can trigger a buffer overflow by fiddling with the bits. Or perhaps I can just break some of your other assumptions in the code by inserting too many (or too few) rows of data, etc. I just don't want developers to fall into the trap of believing (incorrectly) that the data cache always holds what they are expecting it to hold. We've seen far too many web developers fall into that trap and get themselves (and their customers) into all manner of nasty problems.

You cannot trust anything that was under the control of the attacker.

Using protected documents might help somewhat against a causal attacker, but you need a whole lot of infrastructure to set up IRM, and the other kinds of protection are trivially broken. Also it should be noted that IRM is not a security technology! It is not a foolproof way of thwarting all attacks by well-skilled evil doers; it is a technological measure to encourage users to adhere to existing corporate policies (such as "don't forward confidential e-mails outside the company").

Obviously these kinds of threats will not exist in the vast majority of cases -- most employees are not going to spend time hacking into your Excel based solutions in order to cheat on their expense reports; they're just going to try and get their jobs done. But you should be aware of such possibilities so that you can weigh up the costs of adding in additional protection (in terms of increased development time, reduced productivity / usability, more help desk calls, etc.) against the risks / likelihood of employees rorting the system (if you are a large bank or a secret government agency, the risks might be pretty high).

Oh and this is nothing unique to Office -- if you built a custom WinForm application (or even a Java application!) and used it to connect to a server, I would be giving you the same advice; you would be asking for serious trouble if you blindly accepted all data coming from those clients and acted upon it without first doing some kind of verification. Just as the employee can dork with the spreadsheet in order to send you fudged data, so too could they dork with the client application (or just write their own!) and use it to connect to your server and send you bogus data.

Don't trust that data!

ptorr — Mon, 12 Apr 2004 07:04:00 GMT

A while ago I wrote a couple of blog entries on code repurposing and some mitigations, and one of the main causes of that problem is that developers inherently trust data. The text box caption says Name, so it's always gonna contain the user's name, right? Nobody is ever going to put a SQL query or a JScript statement in that field... are they?

But I want to talk a bit today about users inherently trusting data, and how it's just as bad. I'll eventually talk about Excel and some of the cool new stuff we're doing with VSTO 2.0, but let's start a bit smaller than that.

And a bit less techy.

Imagine you're a caveman (or woman) and you've just discovered how to cultivate crops. You have two fields of corn ready for the winter, and one day your neighbouring caveman comes by and says "Ugh, me see sabre-toothed tiger nearby. Ugh. You hide in cave, ugh, and me go fight tiger. Ugh-ugh?" Being very afraid of sabre-toothed tigers, you are more than happy to let your neighbour go off and fight it while you roll a big rock over the entrance to your cave and hide away in fear. The next day you roll back the stone and emerge, only to find that all your crops have been pillaged and your neighbour is no-where in site. Congratulations! You just fell for the world's first matchstick man!

Fast-forward to the (very) late the 20th century. You're sitting at work looking through your e-mail, when the subject line "HOT STOCK TIP!" catches your eye. You've always wanted to make a killing on the stock market (and quit your crappy job) so this could be the answer to your prayers! You read the message and jump onto your E*Trade account to buy up as much of the stock as you can. Three days later the stock is unlisted and you're left penniless on the street. Oops!

There could be an infinite number of examples here -- the basic problem is that you are acting on information furnished to you by people whom you should not trust. I gave these examples (and will give a few more) mostly to placate any fears that what I am about to describe is a new problem that we are introducing with VSTO 2. Nothing could be further from the truth; it is just that VSTO 2 will provide many amazingly cool features that, like all features, can be used for good as well as for evil.

You have been warned :-)

Let's talk about using Excel to implement an expense report (a common scenario we use here at Microsoft) and how that can be abused by untrustworthy employees. We'll start off with an expense report that doesn't use any kind of code (VSTO, VBA, whatever) although it does use formulas. The expense reporting process is as follows:

1. The employee fills out an expense report and e-mails it to their manager

2. The manager approves the expense report and e-mails it to the payroll department (obviously they could also choose to reject it)

3. The payroll department receives the report and reimburses the employee with their next pay cheque

Note that in these scenarios I will only focus on the employee trying to abuse the system to get more money than they should; in a real system you would also think about all the players that could be trying to abuse the system -- the manager, the payroll employees, the vendor who wrote the payroll system, etc. -- and all the things they might try to do -- steal money, block legitimate payments, and so on.

The problem in this case is that the manager accepts an expense report from the employee and approves it or denies it simply by looking at the values in the cells. But those cells were under the control of the attacker (the employee)! Let's say the employee recently took a client out to lunch and is claiming a fairly reasonable $100 for it. This sounds good to the manager, so she approves the request and sends it on to the payroll team. Unbeknownst to her, the $100 in the expense report was not a static value added by the employee, but rather a formula that would change to the fraudulent amount of $1,000 when viewed by the payroll employees. The payroll guys see the value for $1,000, verify that it was approved by the manager, and promptly over-pay the employee to the tune of $900.

Note that using digital signatures or other "security" technologies wouldn't have helped here; the manager would just have signed the spreadsheet containing the formula. In fact it may have increased her liability because she can no longer claim that someone spoofed her e-mail account and sent the dummy report -- after all, it was signed with her private key!

About the only thing the manager could do in this case (short of performing a full audit on the spreadsheet) would be to take a new, known-trustworthy expense report template and manually re-key the employee's data into the spreadsheet so as to ensure no trickery was underway. This of course is a colossal waste of time, so nobody is going to do it. Of course they could also copy the entire spreadsheet and paste it back on top of itself with the "values only" option, but then it might break other parts of the spreadsheet (like the =SUM() field at the bottom). Basically, it's a big hassle.

(Oh, and in case you think this is a problem with using Excel and it's auto-magic formulas, imagine that the expense report is a plain text file written in Notepad. The manager gets an expense report from her employee and sees the single line item "Lunch with client: $100" and sends it on to payroll. Unbeknownst to her, the employee simply added fifty blank lines after the first item and added "Ticket to the Caribbean: $5,000" to the end, knowing full-well that the payroll system will not be fooled by blank lines and will pay out for both line items).

The reason I brought up VSTO 2 earlier on was that Eric and Eric (and the rest of the team) have been making data-binding and data-centric programming and server-side access to data so easy and powerful in Excel that I fear people will throw themselves into this cool new technology head-first and never stop to realise all the horribly bad assumptions they are making. Does the cached data your server-side component "see" have anything to do with the spreadsheet itself, or did the user hack it with a binary editor before e-mailing it to you? Does the TotalAmount named range still refer to $C$10, or did some nefarious employee move it to point to $D$20 instead? Has the user filled the "real" worksheet with bad data, hidden it, and then replaced it with a spoofed (look-alike) worksheet with benign data intended to fool other users? Did the user open your spreadsheet without the managed code executing, thereby bypassing any client-side validation functions you used to vet data before submitting it to a server system?

The solution to the problem is, of course, to ensure that the only thing the employee is in control of is the data, not the way it is presented or the behaviour of the program. And thankfully the great work being done on VSTO 2 helps you out here; you just have to know how to use the tools effectively. There are two fairly obvious solutions to this problem of the employee being in control of the cells in the spreadsheet:

1. Utilise a trusted third party (often a server) to perform the "copy and paste" operation noted above

2. Utilise a trusted UI (not under the control of the attacker) for displaying and confirming the values in the spreadsheet

You can probably think of other ways, too. (Note that here we assume the attacker does not have any control over the code you are executing on your machine; they only have control over the spreadsheet).

The first solution is (to me) the coolest, and it uses the VSTO 2 technology quite well. Instead of the 3-step process above, we build a more complicated (but less prone to abuse) process that uses a web site to help "cleanse" data:

1. The employee fills out an expense report and submits it to the server

2. The server strips out the data from the expense report, stores it in a database, and sends a notification to the manager

3. The manager clicks on a link to the server, which extracts the data from the database, shoves it into a brand new expense report template, and serves it up to the manager

4. The manager approves the expense report and submits it back to the server

5. Repeat steps 2-4 for the payroll people

In this scenario, the manager (and the payroll people) are guaranteed to see exactly the same data that the back-end processing system will see, because the Excel spreadsheet (which in the past may have held nasty formulas, hidden sheets, re-directed named ranges, etc) is never propagated from one user to another. The employee can dork with the expense report all they like, but they will not be able to get away with the same attack; when they submit the report to the server, they will get an error if they have placed formulas where numbers are supposed to be, and no matter how much they try and spoof the UI of the spreadsheet to make it look like it is for $100 when it is really for $1,000, the manager will see the true value of $1,000 and not approve the report (and hopefully fire the employee). You might realise that this is the way most web sites work, and you'd be perfectly correct; we're simply using the power of the Excel client to make the data entry and data viewing experiences better. It breaks down if you don't have a trusted server, or you need off-line support, or if for whatever reason your current process inherently relies on people e-mailing stuff to each other.

The second solution can help here. At its heart, this solution leverages the rich Excel user interface for the data entry portion (the employee), but completely bypasses it for the validation / approval portion (the manager / payroll clerk). The process is modified thusly:

1. The employee fills out an expense report and e-mails it to their manager

2. The manager opens the expense report, reviews the data inside a custom-built dialog box, approves the expense report and e-mails it to the payroll department (obviously they could also choose to reject it)

3. The payroll department receives the report and reimburses the employee with their next pay cheque

In this scenario, when the manager indicates their desire to approve the expense (by clicking a button, etc.) the solution gathers up the data from the spreadsheet (the same way that the server would do in the previous example) and shows it to the manager in a "trusted" user interface such as a data grid inside a modal dialog, or (dare I say it?) inside the "Document Actions" pane. The manager then ignores what is in the Excel cells and makes their decision based on the numbers inside the trusted UI. (They will most likely look at the original expense report anyway, just to see what the expenses were for, but they need to make their decision on the value shown in the dialog, not the one shown on the spreadsheet). Just to make doubly-sure there is no deception going on, you could require the manager manually insert the total amount into a "Verify amount" field on the spreadsheet before submitting it.

Update 12-04-04

Having a policy such as "No direct manager can approve expenses over $500" would also help here, because even though the manager would see an expense for $100, the system would see $1,000 and flag it as a policy violation. Now of course the manager would then complain about the stupid computer system messing up again, but hopefully someone would track down the discrepancy, ferret out the fraudulent employee, and fix the system so that the same kind of thing didn't happen again.

What this shows is that technology is not a panacea to solving security issues. Technology has no concept of morality and can be used for good as well as for evil. Having solid designs for your solutions and building quality threat model for them will help you way more than throwing random technology buzzwords at a solution. User education and having good policies & procedures goes a long way, too.

Oh, and hiring trustworthy employees (and keeping them trustworthy by treating them well) is also incredibly important.

Balancing Security and Usability

ptorr — Sun, 21 Mar 2004 05:58:00 GMT

I'm often tempted to write about viruses and what I think the next "innovation" might be, but then I get scared that I might get put in jail (or deported) should any of my ideas ever see the light of day. (Not that I think the virus writers need any help coming up with new ideas, but you know what I mean). Anyway, one thing I have been meaning to talk about is how I approach this problem for VSTO solutions, and since Raymond just blogged about some new shenanigans in this area I thought I'd do it now.

Aside: I say "how I approach this problem for VSTO," but it's not as if I am the only person thinking about this problem at Microsoft. Many smart people are thinking about it both from the security and the usability side of things, and we have spent (and will continue to spend) a metric boatload of time going over many different ideas and designs for the secure sharing and deployment of Office solutions. Hopefully, together we will come up with something that strikes the right balance between security and usability and lets the good people get their work done while keeping out the bad people. But this is my blog and I don't want to speak for any other people. I know what my motivations / goals / guiding principles are, and I'll try to share them with you.

I have a very simple set of metrics for deciding if a method of code installation meets my personal "it's secure enough" bar -- it has to be harder than double-clicking on an e-mail attachment, and easier than copying and pasting the source code into a new VBA project. (I'll probably intermingle references to VSTO and VBA here because whilst I work on VSTO, it is not an end-user accessible technology whereas VBA is).

Let's look at the current "state of the art" in viruses. Most new e-mail viruses like MyDoom rely on user interaction to spread (although they are often incorrectly termed "worms" -- true worms like Blaster require NO user interaction). Running MyDoom "apparently" (I've never done it ;-) ) takes about three atomic actions:

1. Open the attachment

2. Accept the security warning

3. Open the EXE inside the attachment

(You could say that the first two are one and the same, and I'll argue for that case below, so really there are only two user actions).

Clearly requiring only three (or two) basic user actions to go from "clean machine" to "virus infected spam bot zombie" is a little too easy. Now let's look at the typical "Linux virus" approach as applied to VBA: you get an e-mail message with some source code in it, and instructions similar to the following:

1. Open Microsoft Word

2. Hit Alt+F11

3. Copy the code below into the clipboard

4. Paste the code into the VBA editor

5. Hit F5

Now five user actions isn't a huge step up from three, but to most users all but the first step is unfamiliar, and it is very easy to mess up step #3 if you copy too much or too little from the e-mail into the VBA editor. The point being though that any security mechanisms we put in place that require more than five user steps are a waste of time -- the hackers will just go with the copy-the-source-code route, which will still be easier for the end user than installing "legitimate" software through the imaginary 10-step "secure" process.

So we want a process that is harder than two trivial actions, but easier than five non-trivial actions. Time to focus our energies!

One important thing to note is that there are two entities involved in software distribution -- the publisher (developer) and the installer (end user). In an ideal world we would want to make the distribution experience as seamless and easy as possible for both parties, but in a hostile, broadly-connected world that doesn't cut it any more. Nevertheless, we can still try and simplify things for the developer; there's no point in making it really hard to develop solutions while keeping the barrier for installing them very low -- the bad guys don't mind doing hard work to write their viruses, and the good guys will give up and go do something else if it's too hard (and thereby not reach their full potential). And the end users will happily click on the attachments until the cows come home.

Instead, we want to make it as simple as possible to develop solutions, as simple as possible to install solutions from the "right" people, and virtually impossible to install solutions from the "wrong" people. Unfortunately, figuring out who is good and who is bad is the one thing in this equation that software alone can't do. Once you know who's naughty and who's nice, it's a Simple Matter of Programming:

if (publisher.Diposition == Disposition.Good)

InstallAndRunCode()

else

CallTheFBI()

But the burden of figuring out who is good and who is bad falls on the user of the computer (or perhaps their administrator). And, as history has shown us, users are less interested about who sent them the solution than they are about what the solution claims to do. It doesn't matter if an e-mail comes from a random e-mail address with a random subject line and some random text inside it -- if there's an attachment entitled BritneySpearsNude then a significant number of users will open it and disregard any warnings. Even if the attachment is DoNotOpenMeBecauseIAmAVirusAndWillDeleteAllYourFiles, some people will open it just out of curiosity or because they think it is a joke.

Another aside: Modal warnings are horrible. It's what we use today, but they are truly useless. I open BritneySpearsNude from my inbox. Clearly my intention is to open the attachment, but my e-mail program helpfully asks, "Are you sure you want to open this attachment?" OF COURSE I WANT TO OPEN IT! I would not have opened it if I did not want to open it!! Stupid computer!!! Stupid Microsoft!!!! Now where are my pictures of Britney????? And where have all my documents gone?!?!?!?!!!!!!

It also doesn't help that you get redundant dialogs. I open a Word document from Outlook, and it tells me "Hey, y'know this could have a virus in it, don't you?" even though the Word document may not have any macros in it. Then it opens in Word and (depending on my security settings and whether or not the document really has code in it) Word might warn me again. Now there's a good reason for this -- we call it "defence in depth" -- since either of the two mechanisms could fail. Outlook is being ultra-paranoid: maybe you have your Word security settings at "Low," so all macros can run. Maybe there's a problem with the way Word checks for macros (we have issued at least one patch for just this problem in the past) and it won't protect you from this particular virus. Maybe there's a buffer overflow in the normal parsing of Word documents that doesn't rely on VBA code to perform the exploit. You get the idea.

But of course the plain and simple truth is that Outlook is dumb and it just knows that .doc files need to prompt the user before being opened. It doesn't know why, it just follows orders. Now if we could have Outlook, Word, Windows, Windows Update, Office Update, the virus scanner, the virus scanner's update server, the firewall, and maybe a managed PC service provider all working together behind the scenes, maybe we could do away with the prompts altogether (or at least only show one prompt, and only when it was really necessary). But that's a pipe dream right now.

That's our goal -- run good code and block bad code. But we can't really get there without massive user education and a lot of infrastructure. So our next-best approach is to make it hard for end-users to run solutions. The idea here is that if it's too hard to open the BritneySpearsNude attachment, then most people will give up and get on with their job. At the same time though, if it's too hard to open Budget.xls with some cool Excel user-defined functions (UDF) in it, then people won't be able to get on with their job.

Stalemate.

Rant: I just tried looking for a good hyperlink to Microsoft documentation on UDFs in Excel. Could I find one? I'll leave that as an exercise to the reader (thankyou Google). Anyway, when we did customer research on how people use VBA with Excel, we found that most people don't know half of the things they can do. Workbook events -- what are they? Worksheet functions -- what are they? We have all these cool technologies and nobody knows about them. We suck. We need to do much better in the future, both in terms of increasing discoverability and in terms of documenting this stuff. And people say there's no reason to keep building new versions of Office!

So anyway, back to where we left off the conversation... if it's hard to install all solutions, then it's equally as hard to install the good ones as it is the bad ones (remember, we can't tell the difference between them). So the next best thing is to front-load all the pain of installing good code and require some kind of pre-arranged client setup (or, in the case of an enterprise, perhaps some domain-level policy) that will help to ease the installation of good code.

In this case, maybe we can require more than five non-trivial steps to setup the machine as long as those steps are done out-of-band and in a way that makes it very clear you should not do them as a normal part of your daily computer use. After all, you only need to do it once, and then all the good code runs with little or no fuss. You might note that not only am I a big fan of killing modal dialogs, I'm also a big fan of "out-of-band" activities for security-related actions. Don't let somebody trust a piece of code while they are in the process of trying to run it! Force them to have thought about it before hand. That is what policy is all about -- having a set of well-thought-out, generally applicable rules, and sticking to them. Don't make ad-hoc decisions at the worst possible time!

And I'm going to stop here. Not because I don't have anything more to say -- I have lots to say -- but because this entry is already quite long, and after all, I only ever said I'd tell you how I approach the problem, not how we solve it. We're a long way from having a good solution to this problem at this point in time, although we're all full of ideas.

Final aside: Regular readers will know I'm a huge fan of the Pet Shop Boys, but I'm also a big fan of Howard Jones. I think his song Someone You Need from the Perform.01 or The Very Best Of CDs is one of the most romantic songs I've heard in a long while. The lyrics are very simple, but it gets to the heart of the matter. Thanks Howard!

Andrew Whitechapel's blog

ptorr — Sat, 28 Feb 2004 02:54:00 GMT

Laugh-a-minute Andrew Whitechapel has started a blog at http://blogs.officezealot.com/whitechapel/

Andrew (like the other Andrew) hails from the UK, and even though he likes the Pet Shop Boys he promises to try very hard not to mention them. He should, nevertheless, have some great info on managed code, Office, VSTO, etc. from a "real world" perspective.

Oh and if you ever catch him laughing, be sure to snap a photo and send it to Siew Moi.

Don't use ApplicationClass (unless you have to)

ptorr — Thu, 05 Feb 2004 14:10:00 GMT

A comment on Mike Howard's blog exhibits a common problem that I see time and time again: developers are creating instances of Word.ApplicationClass or Excel.ApplicationClass in their projects.

Even though it's the wrong thing to do, I don't blame them for doing that. I blame IntelliSense.

First things first: What's the right way to do it? Well, just use Word.Application or Excel.Application (or any other type that follows the same Thinggy / ThinggyClass pattern).

But IntelliSense doesn't show me that as a valid option!

That's because IntelliSense is not as intelligent as its name might suggest. IntelliSense has a simple rule that says "after the user types the new keyword, show the user a list of new-able things." (Note I may be simplifying things here as I don't work on IntelliSense... but it explains how the system works. Feel free to correct me if you work on that team ;-) ). IntelliSense believes that the only things you can new are concrete, visible classes with one or more visible constructors. And it's almost right.

In general, you cannot new an interface because interfaces have no implementation. But the default interfaces on COM CoClasses are a little different; the CLR knows something that IntelliSense's parents forgot to teach it. Crack open the Excel PIA using ILDASM and have a look at the Application interface. Among the gobbledy-gook, you will see the following:

.custom instance void [mscorlib]System.Runtime.InteropServices.CoClassAttribute::.ctor(class [mscorlib]System.Type) = ( 01 00 2F 4D 69 63 72 6F 73 6F 66 74 2E 4F 66 66

69 63 65 2E 49 6E 74 65 72 6F 70 2E 45 78 63 65

6C 2E 41 70 70 6C 69 63 61 74 69 6F 6E 43 6C 61

73 73 00 00 ) // ../Microsoft.Office.Interop.Excel.ApplicationClass

This tells the CLR that when someone wants to create an instance of type Application, it should really go ahead and create an instance of ApplicationClass.

Another tip: In Microsoft Word, you may want to handle the Quit event to do something when the user closes down the application. But the Word.Application interface defines Quit as a method, not an event. What to do?

Well, you cast it to a Word.ApplicationEvents4_Event (you could find this out by using the Object Browser):

Word.Application app;

app = new Word.Application();

app.Visible = true;

Word.ApplicationEvents4_Event appEvents = app;

appEvents.Quit += new Word.ApplicationEvents4_QuitEventHandler(QuitHandler);

Both of these are due to the way COM works (again I am going to gloss over the details here, so feel free to correct me or point readers to a more accurate, in-depth blog if you have one). COM doesn't really have the notion of a 'class' as a first-class (ha ha) object; everything to COM is an interface. Now of course in order to get a real live implementation of an interface in your hot little hand, you need to be able to instantiate a concrete implementation of the interface through a function such as CreateObject or CoCreateInstance. But these things simply use the ProgID or ClassID to look up the implementation provider in the registry, and from then on you're pretty much dealing with interfaces. The COM coclass Application gets turned into the managed class ApplicationClass and a new interface Application is created to represent it and given the custom attribute you see above. You should always bind to this interface. I believe this is for versioning reasons, but I can't really remember now (this topic discusses the versioning problem when exposing .NET objects to COM, but we're doing the reverse here).

The reason you have to have the funky cast to get the Quit event is that COM has no notion of overloads and, indeed, no native notion of events: interfaces can only have methods on them. Instead of having events, a class can expose one or more "source" interfaces which say to clients of the interface "if you implement this interface and then register it with me then I will call the methods on it in an event-like fashion." Wherever possible, the CLR will "collapse" the events from the class' source interfaces into the main interface (eg, the Application.WorkbookOpen event in Excel actually comes from the AppEvents_Event interface), but when they have the same name as a method it can't do that so you need to explicitly cast to the original interface.

Like I said, I missed a lot of detail there (and maybe even told some fibs) so if someone else wants to give a more detailed (and correct!) explanation, please do so.

But the gist of it all is this: Just as Luke Skywalker puts away the computer targeting system and is told to "use the force" in Star Wars Episode IV, so too should you put away IntelliSense and never use ApplicationClass when programming against Office.

Beware of AutoSave and DocumentBeforeSave

ptorr — Wed, 28 Jan 2004 12:41:00 GMT

One of the cool things about Word is that it auto-saves your work so that if the machine dies or the app crashes you can get most of it back again. One of the other cool things about Word is that you can customise the built-in dialogs -- such as the Save As dialog -- to save yourself some development time and keeping the UI familiar to users.

Unfortunately these things don't work too well together right now. I have code similar to this in dotWord:

/// Called when the document is opened.

protected void ThisDocument_Open()

{

ThisApplication.DocumentBeforeSave += new Word.ApplicationEvents4_DocumentBeforeSaveEventHandler(BeforeSaveHandler);

}

private void BeforeSaveHandler(Word.Document Doc, ref bool SaveAsUI, ref bool Cancel)

{

// Get the default Save-As dialog to show to the user

Word.Dialog fileSaveAsDlg;

fileSaveAsDlg = thisApplication.Dialogs[Word.WdWordDialog.wdDialogFileSaveAs];

// Show the dialog. This will actually save the document if the

// user clicks the "Save" button.

object missing = Type.Missing;

fileSaveAsDlg.Show(ref missing);

// Oooh, fun! It's like a choose-your-own-adventure

if (MessageBox.Show("Do you want to crash [Yes] or show a silly dialog [No]?",

"Save bug" , MessageBoxButtons.YesNo) == DialogResult.Yes)

Cancel = false;

else

Cancel = true;

}

In the real code I actually do some work to compute a decent file-name based on the blog title and today's date, but I omitted that from the sample since it uses reflection code that just gets in the way of the example.

Anyway, as the final MessgeBox suggests, you can either set the Cancel parameter to true (cancel the save operation) or false (continue the save operation). Neither is particularly cool in the case of an AutoSave, as indicated by the choice in the message box text. There doesn't seem to be a way to detect AutoSaves either. The good news is that AutoRecover can fully recover the file (since it really is saved before Word crashes), but still, Word shouldn't crash (and the silly dialog should go away, too!)

IMHO the AutoSave should not even trigger the DocumentBeforeSave event because it's not a real "save" -- the user hasn't committed to saving the file yet, so they may not want to give it a name or update a database or call a web service or perform other operations that you would typically do in this event handler. I think that either there should be a separate event for the AutoSave (or at a minimum a flag / parameter that you can query to figure out you're in an AutoSave) or else it should not trigger any events at all.

What do you think? Either way, this little gotcha is something to look out for that you're not likely to come across in normal testing of your application. The default AutoSave timeout for Word is 10 minutes, which probably means it never gets triggered when you're doing your testing. (It repros in VBA as well, so it's not just a managed code / VSTO thing).

Word Shortcuts

ptorr — Mon, 12 Jan 2004 05:44:00 GMT

Ever wanted to move some text around in a Word document, but didn't want to go through the hassles of copy and paste?

Just select the text, hit F2, move the cursor to where you want the text to be, and hit Enter.

Using Shift+F2 will copy the text instead of moving it.

If you want to move an entire paragraph up or down, use Alt+Shift+Up or Alt+Shift+Down.

And did you know Shift+F3 will toggle the case of the selection, from Proper Case to UPPER CASE to lower case?

You can use Excel templates with VSTO 1.0

ptorr — Mon, 05 Jan 2004 12:24:00 GMT

If you've tried out VSTO (and you should :-) ) then you may have noticed that Word has both Document and Template projects whilst Excel has only a Workbook project. If you were thinking that the reason was because Excel was naughty but Word was nice, you'd be half right. And if you were thinking it had to do with security, you'd be dead right. (I guess the "Security" category kind of gave that away though, right?)

As you may already know, the template models used by Word and Excel are very different. In a nutshell, Word documents inherit some information from their template and maintain a link back to it, whilst Excel workbooks are basically just copies of their templates.

Because Word documents maintain a link to their templates, we were always able to tell what security context to use for documents based off templates -- Word would tell us "Hey, the template being used for this document came from such-and-such a location," and we'd use that to set up the security context. You can see my old blog on trusting document locations for more info on the document's security context if you haven't read it already.

But poor old Excel didn't really have that information handy, and we couldn't in good faith let you build solutions off Excel templates without knowing how to run them securely. So for the majority of the VSTO product cycle (and probably for the beta, as well) Excel was deliberately designed not run VSTO code for new workbooks created from customised templates, and so we didn't have a project template (no pun intended) in Visual Studio for building such projects.

Then for some other obscure reasons (I think it was something to do with getting workbooks to load properly when hosted inside IE) Excel magically had the right information hanging around, and lo-and-behold we could now run Excel templates with the correct security context.

Unfortunately, this came too late in the cycle for us to add a new project to the VS toolset -- it was a fairly trivial code change to allow Excel to run customised templates, and it was a relatively minor addition for the runtime QA team to add Excel templates to their test cases, but any kind of changes in the project system (especially since they involved UI) were far too expensive and risky to make at such a late stage.

So the long and the short of it is that if you start an Excel Workbook project in VSTO and then use Excel to "Save As" the workbook as a template, you should be able to get it working. At one stage we even had sample code in the help files telling you how to simulate a template using some code in a workbook; I don't know if that still lives on or if we cut it out in time.

Yes, VSTO 2.0 should have designer support for Excel templates.

Thanks to Siew Moi for prodding me to write this blog :-)

Hacked up using dotWord

ptorr — Wed, 24 Dec 2003 05:46:00 GMT

I'm posting this with a hacked-up version of WordBlogX that has been trivially modified to talk to the .Text web service instead. Hopefully it will all go well, in which case I can continue using Word to write my posts, and sooner or later Rob will figure out how to do his custom action in setup and this baby will be ready to be re-released!

What does "Save As..." mean, anyway?

ptorr — Fri, 05 Dec 2003 10:47:00 GMT

I'm in the middle of writing another blog entry about saving (note to self: add a link later) which made me want to write briefly about something related that bothers me quite regularly. Professionally, that is, not personally ;-)

A lot of applications (like Microsoft Office Word, for example, which I'm using right now) have two basic options when it comes to saving files:

· File -> Save

· File -> Save As...

It's pretty obvious what Save does -- it saves the current state of the current document (whatever the application's idea of a "document" is) with its current name in its current location. If the document does not yet have a name and location, the user is prompted for this information.

But what was the user's intention when they hit the Save button? There are a bunch of possibilities:

· They are about to exit the application and want to save the state of the document for the next editing session

· They are concerned about program crashes, hardware problems, or power outages, and want to save their work-in-progress "just in case" something goes wrong

· They are going to copy the document from the current machine and work on it on another machine (eg, taking it home)

· They are going to copy the document from the current machine and give it to another person for collaborative purposes

· They are going to copy the document from the current machine and give it to another person as a finished document

· They are storing a final "authoritative" version of the document for historic / legal reasons

· Possibly some more...

These are semantically different actions, and ideally we might like our software to perform different tasks depending on what our intention was. For example, when I'm saving a work-in-progress as a mitigation against crashes or power loss, I might not want to think about a file name or location. It might be a "throw-away" file I'm never going to keep once I've finished with it, or maybe I won't be sure what it should be called until it's completed. In these cases, forcing me to think about where to save the file or what it should be called is a waste of my time. Obviously this is what things like AutoSave are designed to accomplish, although not all software supports AutoSave and even when it is available the user may want to explicitly save their current progress before a "risky" operation or after writing something particularly insightful.

As another example, when I'm working on a document with somebody else I probably want my application to store collaboration-related metadata such as revision marks, comments, and tracking IDs, but when I am ready to "publish" my file to the general public, I want all that information removed. Today programs like Microsoft Word are "optimised" for the former case (metadata is stored with the file) and you have to hunt around the Tools -> Options -> Security tab to turn it off. And then it's a global switch that applies to all future Save operations on all future documents; there's no one-time-only "Save and clean" operation that I can see in the product.

Save As... presents its own set of problems... again, it's pretty obvious what it does -- save the current state of the document with a new name and location -- but here there are at least three possible motivations:

· The user wants to make a new "version" of the current file

· The user wants to create a "checkpoint" of the current file

· The user wants to create an entirely new file, using the existing file as a "template" or base

The difference between the first two is basically whether you want to save the current state of the file and start working on the next "version" (eg, go from AnnualReport_Monday.doc to AnnualReport_Tuesday.doc and keep working on Tuesday's version) or whether you want to continue working on the main document but save a temporary copy with a different name (eg, working on SalesReport.xls you create a temporary SalesReport_BeforeReformat.xls but keep working on the main file). Applications like the Microsoft Office Suite tend to favour the first approach: Save As... creates a copy with the new name and makes that new copy the current version. If you want the second way of doing it, you have to first Save, then Save As... with the new name, then Open the original version again. Messy, huh?

The third motivation is when you really want to create a brand new file but rather than start from scratch you want to copy some of the formatting or the data from the current document. For example, using the data and formatting in BudgetForecast2002.xls to create BudgetForecast2003.xls -- it's not a new "version" of the 2002 spreadsheet; it really is a new spreadsheet.

Again, there are things that we might want the software to do differently depending on our intention. As mentioned above, getting the "checkpoint" semantics in Office requires some manual steps. In the "I'm really creating a new document" case, we might want to do a "deep" clone of content in the document (including OLE objects, linked images, etc.) so that they can be modified along with the new document and not affect the old document.

Unfortunately, having a cascading File -> Save sub-menu with (at least) nine different options under it would be horribly confusing for most users. They just want to "save" a file, for whatever their definition of "save" is, without wanting to worry about what all the other possible meanings of "save" might be. In actual fact, Save As... is conspicuous by its absence in our very own Visual Studio product -- you can't clone or checkpoint your solutions in VS. I guess they expect you to be a "real" developer and use source control instead...

So, sometimes things don't work out the way you want them... it's not easy designing software!

Cross-application coding and other questions

ptorr — Fri, 21 Nov 2003 07:51:00 GMT

Here's some questions for any of you that do (or perhaps would like to do) Office development. They are more targeted at non-professional developers using VBA (how many of them read my blog?) but even if you're a seasoned pro and you feel like answering, more power to you (and me! Ha!).

Your answers (and other comments) are much appreciated :-) and can be used by Microsoft in future products etc. etc. etc. Obviously this will be a very un-scientific poll (assuming there are any replies ;-) ) but it should still be interesting.

Anyway, here goes:

1. Do you have a need for building solutions that span multiple documents? For example, moving data between a weekly report and a monthly rollup report in Excel

2. Do you have a need for building solutions that span multiple applications? For example, integrating data from Excel into a PowerPoint presentation

3. If you are doing either of these things already, do you have the code in multiple places and rely on events or cross-customisation calls to do the integration, or is all the code in one place and it "drives" the other document / application externally? For example, if you had a solution with two Word documents, the code could be split between both documents and you relied on the Open event or explicit cross-document macro calls, or it could all be inside the main document

4. If you answered "Yes" to either question 1 or 2, but you're not doing this today, is it because you don't know how? Or you know how, but it's too hard? (What makes it hard?). Or for some other reason?

5. And finally, do you (or your users) call macros from one document in the context of another document? For example, in Excel you might have a UDF (User Defined Function) defined inside one workbook, but the user calls that function from another workbook

If you do answer, it would be really cool if you could also say whether you're a full-time coder or not, what tools you use today (VBA, VB6, VSTO, JScript <g>, etc) and how many people tend to use your solutions.

Thanks!