July, 2007

  • The Old New Thing

    If the system says that an embedded string could not be converted from Unicode to ANSI, maybe it's trying to tell you something


    It's probably trying to tell you that an embedded string could not be converted from Unicode to ANSI.

    One of our programs is throwing the exception "Type could not be marshaled because an embedded string could not be converted from Unicode to ANSI." It happens only if we use the Chinese version of the program. Why are we getting this exception?

    I may be going out on a limb here, but I bet it's because an embedded string could not be converted from Unicode to ANSI.

    Unicode is big. Bigger than any ANSI code page. No matter what ANSI code page you pick, there will be Unicode characters that cannot be expressed in it. (And no, you can't set your ANSI code page to UTF-8. Michael Kaplan discussed it last October, and before that, last July, and before that, a week and a half previous (still July), and before that, two years ago February. I think Michael might need to change the subtitle of his blog to "Explaining why the ANSI code page can't be UTF-8 since 2005".)

    In particular, if you grab some Chinese characters and try to map them to the ANSI code page on an English system, they definitely won't map successfully.

    The real question, though, is why you're using ANSI in the first place. Get with the program, it's 2007 already. As Michael Kaplan noted, it's all Unicode all the time from now on. In that same article from Michael, he also does the necessary psychic debugging for the root cause: The DllImport declaration did not specify a CharSet, so you get the default, which is CharSet.Ansi!

    Change your character set to CharSet.Unicode and you'll be all set.

  • The Old New Thing

    The conversations backstage at computer Go tournaments


    Steve Rowe linked to an essay on why computers can't play Go well even though they've mastered other "difficult" games like chess. I was reminded of a description I received of what happens backstage at computer Go tournaments (i.e., tournaments that pit Go-playing computers against each other). ("Backstage" is a bit of a misnomer, of course; since the contestants are computers, you can talk all you want as loud as you want without worrying about distracting the players.)

    At computer Go tournaments, the programmers are like parents watching their children compete in a hockey game where they've only just barely learned how to skate. It's not so much a matter of who plays better as it is a matter of who sucks less. One programmer will lean over to the other and say something like "I hope my program realizes its left-hand-side is vulnerable before your program takes advantage of it."

  • The Old New Thing

    Image File Execution Options just inserts the debugger in front of the command line


    If you use the Image File Execution Options registry key to force a program to run under the debugger, all the kernel does is insert the debugger in front of the command line. In other words, the CreateProcess function figure out what program is about to be run and checks the Image File Execution Options. If it finds a debugger, then the debugger is prepended to the command line and then CreateProcess resumes as if that were the command line you had passed originally.

    In particular, it doesn't do anything with the other parameters to the CreateProcess function. If you passed special parameters via the STARTUPINFO structure, those parameters get passed to the debugger. And the PROCESS_INFO that is returned by the CreateProcess function describes the debugger, not the process being debugged.

    Specifically, if you specified the STARTF_USESHOWWINDOW flag and passed, say, SW_HIDE, as the wShowWindow, then the debugger will be hidden. This bites me every so often when I am called upon to debug a program that happens to be launched as hidden. I'll slap the debugger underneath it with Image File Execution Options, run through the scenario, and then... nothing.

    And then eventually I realize, "Oh, right, the debugger is hidden."

    To unstick myself, I fire up a program like Spy to get the window handle of the hidden debugger and fire up a scratch copy of Notepad so I can make it do my bidding and show the window.

    ntsd -Ggx notepad
    Break instruction exception - code 80000003 (first chance)
    eax=7ffdf000 ebx=00000001 ecx=00000002 edx=00000003 esi=00000004 edi=00000005
    eip=7c901230 esp=00a1ffcc ebp=00a1fff4 iopl=0         nv up ei pl zr na po nc
    cs=001b  ss=0023  ds=0023  es=0023  fs=0038  gs=0000             efl=00000246
    7c901230 cc               int     3
    0:001> r esp=esp-4
    0:001> ed esp 1
    0:001> r esp=esp-4
    0:001> ed esp 0x00010164
    0:001> r esp=esp-4
    0:001> ed esp eip
    0:001> r eip=user32!showwindow
    0:001> g
    0:001> q

    The first two commands push the value SW_SHOWNORMAL (numerical value 1) onto the stack. Then goes the window handle. And then the return address. Move the instruction pointer to user32!ShowWindow and we've simulated the function call ShowWindow(0x00010164, SW_SHOWNORMAL);. Once I let execution resume, *boom* the debugger window appears and I can continue my work.

Page 5 of 5 (43 items) 12345