How to get CL.exe proxy filters (like STLFilt) working under VS2005

How to get CL.exe proxy filters (like STLFilt) working under VS2005

  • Comments 8

There's a class of tools that lots of people write that replace cl.exe with a 'wrapper' to do things link pre-process a command line [perhaps adding particular -D arguments], or filtering the output to deal with things like psychotically complicated error messages from template metaprogramming. These tools tend to be incredibly useful, and also tend to be incredibly straightforward to author. And with VS2005, they tend to be broken when running under the IDE. With 2005, the compiler toolset finally got serious about supporting Unicode. You can write code in Unicode (so you can have variable names like "a picture of a telephone" :-) ) You should also get your error messages in Unicode. The trouble arises when you realize the ethnocentric history of software. It's the stomping ground of the English-speaking white male. I'm not trying to be rude, racist, sexist, or generally offensive, I'm just speaking historically (and unfortunately, it continues to be a field dominated by men, though the few women I have been able to work with are truly brilliant). Whoops - get off that tangent and back to the point... The Win32 console file handles cannot handle Unicode. They're code-page based handles, so when you're running under the console, we just spit out code-page based errors & warnings. If those errors & warnings contain Unicode, you're just out of luck. Well, when cl.exe is running under the IDE, we figured we could improve the experience. Instead of spitting messages out stdout and stderr, we open up some named pipes, and spit out Unicode that the IDE can then display. Which is great, until you start thinking about the cl.exe proxy scenario. So, if you're writing a cl.exe proxy, prior to invoking the original cl.exe, undefine VS_UNICODE_OUTPUT. This successfully tricks cl.exe into thinking you're just compiling from the command line, and you're back in business (with non-Unicode messages, though :-( )

Update:  Leor Zolman (Mr. STLFilt himself!) mentioned he couldn't get this working.  Turns out there's one more little issue that you may not realize:  The response files being fed to cl.exe are probably unicode.  You have to uncheck that, or else handle unicode response files for everything to work properly.  If you turn off 'Use Unicode Response Files" in your project, go grab the source to STLFilt and change the doWin32Stuff function to have this at the beginning:

// from cl.cpp:
int doWin32Stuff(char *full_filt_cmd, char *cmdline)
{
    using namespace std;
    if (getenv("VS_UNICODE_OUTPUT"))
    {
        log("VS_UNICODE_OUTPUT is enabled - disabling!");
        // Clear the Unicode output flag to force the compiler to use the normal output pipes
        // I don't know perl at all, but this is how you do it from C:
        _putenv("VS_UNICODE_OUTPUT=");
    }

-Kev

Leave a Comment
  • Please add 1 and 4 and type the answer here:
  • Post
  • A related feature request is to be able to add a "custom link step" in the same way that you can add custom build onto any file.

    Not having this makes using Crinkler ( http://www.crinkler.net/ ) somewhat awkward... although if there ever was an extremely obscure niche "market", this is it ;)
  • Is this true for Visual Studio 2005 Express Edition?  When I try,

    size_t requiredSize;
    getenv_s(&requiredSize,0,0,"VS_UNICODE_OUTPUT");

    requiredSize comes back as 0.
  • I'm not having any success on modifying STLFilt.  I try to undefine the variable with

    _putenv_s( "VS_UNICODE_OUTPUT", "" )

    but it has no effect; the output is not filtered at all.

    When I try setting the variable to anything else, I get an abnormal termination:

    Project : error PRJ0002 : Error result 2 returned from 'C:\VC++Express\VC\bin\cl.exe'.

    So, I know that the changed environment is getting passed to cl, but not with the intended effect.  Can you suggest anything?
  • Honestly, I'm not sure how to undefine a variable.  I'll have to try this myself and see what I can do.  You may have to use the same CRT as cl (/MD, not /MT).  Give me a few days to try it out and report back with a new blog post.
  • Has anybody had any luck getting this to work?

    It's quite amusing that the article is all about how to get CL proxy filters to work by "undefining" a variable, yet nobody knows how to do that!  :)

    I want my STLfilt!
  • Hmm, well, in SP1 can we have a new environment variable?

    Perhaps: VS_ANSI_OUTPUT, which would of course, override the unicode one.

  • PingBack from http://blog.assarbad.net/20070103/memo-to-self-vs_unicode_output/

  • LOL, just happened upon this page while diddling with Google.

    And I STILL have no idea how to get this to work in STLFilt myself.

    It's infuriating.

    In case anyone's figured it out, please let me know! (Contact info at my web site...)

    Thanks!

    -leor

Page 1 of 1 (8 items)