Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Choosing a C runtime library

Choosing a C runtime library

Rate This
  • Comments 28

Yesterday a developer in my group came by asking about a failure he saw when running the application verifier on his component.  The app verifier was reporting that he was using a HEAP_NO_SERIALIZE heap from a thread other than the one that created the heap.

I looked a bit deeper and realized that he was running with the single threaded statically linked C runtime library.  An honest mistake, given that it’s the default version of the C runtime library.

You see, there are 3 different versions of the C runtime library shipped (and 3 different versions of the ATL and MFC libraries too). 

The first is the statically linked single-threaded library.  This one can be used only on single threaded applications, and all the object code for the C runtime library functions used is included in the application binary.  You get this with the /ML compiler switch.

The second is the statically linked, multi-threaded library.  This one’s the same as the first, but you can use it in a multithreaded application.  You get this one with the /MT compiler switch.

The third is the dynamically linked library.  This one keeps all the C runtime library code in a separate DLL (MSVCRTxx.DLL).  Since the runtime library code’s in a DLL, it also handles multi-threaded issues.   The DLL library is enabled with the /MD switch.

But I’ve been wondering.  Why on earth would anyone ever choose any option OTHER than multi-threaded DLL version of the runtime library?

There are LOTS of reasons for always using the multithreaded DLL:

1)      Your application is smaller because it doesn’t have the C runtime library loaded into it.

2)      Because of #1, your application will load faster.  The C runtime library is almost certainly in memory, so the pages containing the library don’t have to be read from disk.

3)      Using the multithreaded DLL future-proofs your application.  If you ever add a second thread to your application (or call into an API that creates multiple threads), you don’t have to remember to change your C runtime library.  And unless you’re running the app verifier regularly, the only way you’ll find out about the problem is if you get a heap corruption (if you’re lucky).

4)      If your application has multiple DLL’s, then you need to be VERY careful about allocation – each DLL will have its own C runtime library heap, as will the application.  If you allocate a block in one DLL, you must free it in the same DLL.

5)      If a security bug is ever found in the C runtime library, you don’t have to release an update to your app.

The last one’s probably the most important IMHO.  Just to be clear - There haven’t been any security holes found in the C runtime library.  But it could happen.  And when it happens, it’s pretty ugly.  A really good example of this can be seen with the security vulnerability that was found in the zlib compression library. This library was shipped in dozens of products, and every single one of them had to be updated.  If you do a google search for “zlib library security vulnerability” you can see some of the chaos that resulted from this disclosure.  If your app used the DLL C runtime library, then you’d get the security fix for free from windows update when Microsoft posted the update.

The only arguments I’ve been able to come up with for using the static C runtime libraries are:

1)      I don’t have to distribute two binaries with my application – If I use the DLL, I need to redistribute the DLL.  This makes my application setup more complicated.

Yes, but not significantly (IMHO).  This page lists the redistribution info for the C runtime library and other components.

2)      If I statically link to the C runtime library, I avoid DLL hell.

This is a red herring IMHO.  Ever since VC6, the C runtime library has been tightly versioned, as long as your installer follows the rules for version checking of redistributable files (found here) you should be ok.

3)      My code is faster since the C runtime library doesn’t have to do all that heap synchronization stuff.

Is it really?  How much checking is involved in the multithreaded library?  Let’s see.  The multithreaded library puts some stuff that was kept in global variable in thread local storage.  So there’s an extra memory indirection involved on routines like strtok etc.  Also, the single threaded library creates it’s heap with HEAP_NO_SERIALIZE (that’s what led to this entire post J).  But that just wraps the heap access with an EnterCriticalSection/ExitCriticalSection.  Which is very very fast if there’s no contention.  And since this is a single threaded application, by definition there’s no contention for the critical section.

Using the multithreaded DLL C runtime library is especially important for systems programmers.  First off, if your system component is a DLL, it’s pretty safe to assume that you’ll be called from multiple threads, so at an absolute minimum, you’re going to want to use the multithreaded static C runtime library.  And if you’re using the multithreaded static C runtime library, why NOT use the DLL version?

If you’re not writing a DLL, then it’s highly likely that your app does (or will) use multiple threads.  Which brings me back to the previous comment – why NOT use the DLL version? 

You’re app will be smaller, more secure, future-proof, and no slower than if you don’t.

 

  • I use the multi-threaded static version of the CRT in every single one of my apps, and I can summarize why in one word: support.

    I've tried using the DLL version before and there are always the freaky systems out there with mismatched, missing, or corrupted DLLs and trying to diagnose a problem like that remotely is a nightmare. Files can get nuked/corrupted no matter how well-versioned they are, and there are always going to be people who run as admin all the time and turn off system restore or system file checker so there are no backups of the files around.

    Same thing goes for MFC, back when I was using it regularly I always used the static version.

    Before you say "the solution is to install copies of the DLL in your app's directory", that still doesn't prevent the DLLs from getting deleted/corrupted, and the end result is a larger download than if I'd used the static version.

    Fortunately for me, I write mostly stand-alone apps and I don't fall into the cases you mentioned where using the DLL version has great advantages.

    The only downside I see to this is one you mentioned about security updates. Given the lack of security holes in the CRT to date, it's a trade-off I'm willing to make to reduce support problems.
  • I guess I've been living inside the bubble for too long... Are there REALLY that many people that mess their systems up that badly?
    I'd think that if you blew away the MSVCRT40.DLL and it's friends that LOTS more things would blow up in peoples faces.

    Also, it sounds like most of your apps are standalone? If you've got more than a couple of DLL's in your project, then the fact that each of the DLL's contains a complete copy of the C runtime library can start interfering with the working set of the project (#2 above)
  • Maybe you've not been around long enough to remember the complete screw-up early in the life of VC6, where a VC runtime DLL was shipped which caused lots of other applications to crash.

    Although these applications were technically at fault, several of them were things which were shipped by MS, with the OS (Hyperterminal, for example)

    I found the experience of deploying MS code, in the MS approved way and having it break MS OS-included applications particularly appalling, and have only recently revisited dynamic CRT linking, 4-5 years later.

    Besides which, if you want to ship anything *small*, then it's much better to statically link it. You can easily ship a statically linked app in the 200K region which includes MFC and its own installer. Try doing that with a couple of dirty great DLLs hanging about.

    Not all of us have yet been completely desensitised to the size of a deployed image by the .NET framework redist, and not all of us want to have to pay Akamai to distribute our online applications for us...

  • Um. I've been around long enough. That's why I mentioned "ever since VC6" in my comment. We learned our lesson the hard way.
  • msvcrt.dll has come in many versions, and shipped with many products. The DLL Help Database (http://support.microsoft.com/default.aspx?scid=/servicedesks/fileversion/dllinfo.asp) lists 23 versions, and I know that's not complete. Couple that with Visual Basic 6's Package and Deployment Wizard's habit of redistributing whatever's in the Windows directory, and you have something of a nightmare. Thankfully on Windows 2000 and XP this is a protected file (protected with Windows File Protection).

    Security Bulletin MS01-060 (http://www.microsoft.com/technet/security/bulletin/ms01-060.mspx) included a patch for format string vulnerabilities in the C runtime.

    You're a bit safer with msvcr70.dll or msvcr71.dll. Version 7.0 has only been shipped in two versions, with FoxPro 7.0 and Visual Studio .NET 2002 (and .NET 1.0 SP1), while 7.1 also appears in two versions: with Business Contact Manager and with VS.NET 2003, Windows Server 2003 and .NET 1.1.

    These pre-release DLLs (FoxPro 7.0 and Business Contact Manager) are bad practice, IMO. Windows Installer is pretty good at DLL installation issues, anyway, and people should now be using the VC_User_CRT71_RTL_X86_---.msm merge module for shipping the runtime.
  • Sorry, I thought "ever since VC6" *included* VC6.

    If you knew about the VC6 DLL problem, you might have had an item 4 in your list of 'things against':

    4) We screwed this up completely in the past and you might not trust us not to do it again.

    If you say something like "Why on earth would anyone ever choose any option OTHER than multi-threaded DLL version of the runtime library?", you're implying that those of us who are doing things differently are doing them wrong. That's bound to grate a bit with people who have to earn a living in world which is slightly less rarified than Redmond.



  • Typically, once Microsoft learns its lessons, it tends to remember them. Check out Michael Grier's weblog (http://weblogs.asp.net/mgrier) for more info on some of the forthcoming technologies to help with versioning.

    As to the last thing: When you get right down to it, the nice thing about a 'blog is that I get to say what I feel in it. And I honestly believe that the reasons for not using the DLL version of the C runtime library aren't strong enough to justify not using it.

    The two Mikes (Dunn and Dimmick) brought up a couple of VERY valid points - sometimes customers machines are broken, and sometimes apps like foxpro toss trash into the system.

    But, as Mike Dimmick points out: If you're using VC6 and the static runtime library, then your app is vulnerable to the bug if it passes user input into the vulnerable function. If you've got the DLL version, then when the fix for MS01-060 is downloaded to your machine, you're likely to be safe.
  • "When you get right down to it, the nice thing about a 'blog is that I get to say what I feel in it. "

    Absolutely. And the nice thing about it having a comments field at the bottom, is that I can disagree with you in public.

    Actually, I don't really disagree with you that there are lots of reasons why dynamically linking the library is better, it's just that there are some good reasons why it can be worse.

    I'm afraid 'trust me, we don't repeat our mistakes' might not be enough for me. I like MS a lot (why else would I read your blog?), but some bits of it endlessly re-invent the wheel, often regressing horribly when they do. (Tried MFC programming in a recent version of VS?)

    It could be that the calm on the VCRT DLL was largely because it became owned by the OS rather than VS. I know that the VC team are very keen that this doesn't happen again (they've said so on the newsgroups) and it will be interesting to see if they can be trusted to look after it on their own again.

    As one of the Mike's said, the 7x.dll's are safer, but building version numbers into filenames hardly seems to be the solution to the world's problems. At one point there was a service pack promised for 7.0, though I don't suppose this is actually going to appear now - would there have been a new DLL name for that?


  • You're right Will.

    Btw, you really should check out Mike's blog. He's touched on some of the things that go into fusion in his most recent entry, and I'm sure that he'll be doing more stuff in the future.

    There actually are good solutions coming down the pike that should hopefully alleviate a lot of the versioning/dll hell problems.
  • >Are there REALLY that many people that mess their systems up that badly?

    Probably not a huge number. However, my products are not aimed at programmers or other techy people. People who not only aren't knowledgeable enough to fix problems, but not even aware of how/when the problem happened to begin with.

    If a DLL-related problem _does_ happen, I can't rely on the user's help in troubleshooting. Trying to track down a problem over email or a message board, when I have to explain concepts like right-clicking and version numbers, takes up a huge amount of time and (more importantly) puts a lot of stress on the user because they don't really know what's going on.

    When I statically link, I have zero problems related to the CRT version. However small the number of messed-up systems is, it's greater than zero.

    Another point that I just thought of is user experience. (MS is big on that these days, no?) ;)
    If a non-techy person installs my app and it bombs out with a "missing export 1234 in msvcrt.dll" error, they are going to assume my app is broken and try one of my competitors. They might never even ask me for help, so I wouldn't know the problem exists.
    Since I'm not (yet) making huge bank off sales, I can't afford to lose any customers. Especially to a problem that I can prevent.
  • I have to agree with a lot of the commenters here -- shipping dynamically linked to the CRT is undesirable from a support standpoint. You say that it has the advantage of automatically getting security fixes, but that also means the program inherits vulnerabilities if the DLL gets rolled back due to a broken installer. I cause enough bugs by myself; I don't need third party programs introducing more.

    There is one major advantage to MSVCRT that hasn't been noted here, though: it avoids allocating a thread local storage (TLS) selector per module. I've had cases where third parties wrote a bunch of plugins that were statically linked, and when my program tried to load them, all of the individual CRT instances consumed all the TLS entries on 98 and modules started failing to load. Global hooks and plugins should generally be dynamically linked for that reason.
  • I had totally forgotten about the TLS issues Phaeron. In your case, you'd be better off with the DLL CRT anyway (I really am a one-note song, aren't I), just for working set issues - which can especially be significant on W98.
  • There are several interesting reasons to use the shared msvcr*.dll:

    1. Page sharing across processes
    2. Fewer heaps in each process
    3. You need to do this is you pass malloc()d or new-d memory across DLL boundaries
    4. The TLS issue

    The perf team in MS is who pushes us to use the common C runtime mostly because of 2 - the actual code size issue tends to not be as big an issue as having a bunch of lurking heaps out there. (Also, if you use the process heap, there's hope that someday you'll be able to allocate memory in DLL_PROCESS_ATTACH but if you're using a private heap, you're forever subject to the deadlocks that happen today.)
  • This discussion starts with the assumption that a proper installer can help solve the DLL versioning problem. IMO, one of the primary flaws of the Windows architecture is the need for installers, and subsequently, uninstallers. The definition of an installer under Windows:

    "An installer is a program which takes something that is one nice neat place (the CD) and proceeds to strew it all over your hard drive".

    The fact that an installer is needed for basic applications is an indication that the system is too complex for users. And in the end it is too complex for developers as well. I want to ship my simple programs as nothing but an executable (remember how it was with Macintoshes?) and the user can uninstall my app by simply deleting it.

    I for one don't even trust software that comes with an installer because that gives the software an easy opportunity to screw up my system with bloatware and spyware.

Page 1 of 2 (28 items) 12