Why does the CreateProcess function modify its input command line?

re: Why does the CreateProcess function modify its input command line?

jMarkP — Mon, 01 Jun 2009 17:11:05 GMT

My psychic Microsoft compatibility hack sense is tingling and I'm guessing the answer has something to do with a program which relies on the Unicode function modifying its parameter somehow.

Am I warm?

re: Why does the CreateProcess function modify its input command line?

Rob Manderson — Mon, 01 Jun 2009 17:20:09 GMT

I'd guess the ANSI version can do a string allocate for two reasons:

1 - It's a wrapper - the allocation is done 'before' calling into the real CreateProcess.

2 - The debugger will be launching the Unicode version directly.

re: Why does the CreateProcess function modify its input command line?

Jack Mathews — Mon, 01 Jun 2009 17:32:26 GMT

Since the ANSI version is essentially a shim utility library on top of the UNICODE kernel, a crash in ANSI functions will not cause the recursive crash you talk about.

re: Why does the CreateProcess function modify its input command line?

Mark — Mon, 01 Jun 2009 17:58:28 GMT

It looks like CreateProcessW does allocate from Peb->ProcessHeap to sort out the path. Not sure how long this has been the case, though.

re: Why does the CreateProcess function modify its input command line?

Alexandre Grigoriev — Mon, 01 Jun 2009 19:00:52 GMT

I'll chime in.

First, CreateProcessA doesn't need a mutable string, because it will first be converted to an UNICODE string and then passed to CreateProcessW.

Second, you could allocate temporary storage on stack (using _alloca), or use VirtualAlloc which doesn't depend on any usermode locks.

Third, the whole idea of running JIT launcher in a context of a failed process is flawed. At least, I hope it's done in context of a dedicated thread (in which case you've got a meg of stack reserved, too, and can use _alloca).

re: Why does the CreateProcess function modify its input command line?

Mon, 01 Jun 2009 20:57:02 GMT

Would a 64kB allocation on stack break something?

re: Why does the CreateProcess function modify its input command line?

Nick — Mon, 01 Jun 2009 22:39:01 GMT

Simple, really. Nobody uses Unicode so there's no reason to spend time fixing it!

I kid, I kid (well, kinda :)...

re: Why does the CreateProcess function modify its input command line?

waleri — Mon, 01 Jun 2009 22:57:31 GMT

I suspect the length of the argument of CreateProcessA may not exceed MAX_PATH.

re: Why does the CreateProcess function modify its input command line?

waleri — Mon, 01 Jun 2009 23:10:40 GMT

While we're at it, what is the reason *not* to fix the contract for CreateProcess? Even if argument is changed from LPTSTR to LPCTSTR this should be backward compatible. Unless of course CreateProcess *still* modifies that memory...

[Changing the function prototype breaks existing source code. Try it:
typedef BOOL (WINAPI *CREATEPROCESSFN)( LPCTSTR lpApplicationName, LPTSTR lpCommandLine, LPSECURITY_ATTRIBUTES lpProcessAttributes, LPSECURITY_ATTRIBUTES lpThreadAttributes, BOOL bInheritHandles, DWORD dwCreationFlags, LPVOID lpEnvironment, LPCTSTR lpCurrentDirectory, LPSTARTUPINFO lpStartupInfo, LPPROCESS_INFORMATION lpProcessInformation); CREATEPROCESSFN DoCreateProcess = MyCreateProcess; typedef BOOL WINAPI MyCreateProcess( LPCTSTR lpApplicationName, LPTSTR lpCommandLine, LPSECURITY_ATTRIBUTES lpProcessAttributes, LPSECURITY_ATTRIBUTES lpThreadAttributes, BOOL bInheritHandles, DWORD dwCreationFlags, LPVOID lpEnvironment, LPCTSTR lpCurrentDirectory, LPSTARTUPINFO lpStartupInfo, LPPROCESS_INFORMATION lpProcessInformation) { if (!RedirectCreateProcess) { // disable trappingin in future callers DoCreateProcess = CreateProcess; return CreateProcess(...); } ... }
I learned this lesson the hard way. -Raymond]

re: Why does the CreateProcess function modify its input command line?

Gabe — Mon, 01 Jun 2009 23:56:14 GMT

Is there any reason CreateProcessW couldn't trap the write-fail exception and only then allocate memory? That would solve the problem of attempting to call it with a string from a page marked PAGE_READONLY, so at least you don't get arbitrary failures.

re: Why does the CreateProcess function modify its input command line?

MadQ — Tue, 02 Jun 2009 03:19:52 GMT

Some programs also play around with the input command line. verclsid.exe seems to do this.

re: Why does the CreateProcess function modify its input command line?

NickPick — Tue, 02 Jun 2009 03:56:43 GMT

Exercise:

Explain why a design decision of CreateProcess function was probably not done by "somebody back in the 1980s"

re: Why does the CreateProcess function modify its input command line?

waleri — Tue, 02 Jun 2009 07:31:43 GMT

>>> Some programs also play around with the input command line. verclsid.exe seems to do this.

Isn't command line is a copy? After all, the child process should access it in its own address space.

re: Why does the CreateProcess function modify its input command line?

dave — Tue, 02 Jun 2009 16:22:22 GMT

>Is there any reason CreateProcessW couldn't

>trap the write-fail exception and only then

>allocate memory? That would solve the problem

>of attempting to call it with a string from

>page marked PAGE_READONLY, so at least you

>don't get arbitrary failures.

Well, I'd say that no-one gets arbitrary failures today, anyway. People who fail to read the documentation get exactly the failures that the documentation implied they'd get. 'Unexpected by programmer X' != 'arbitrary'.

And in any case, why futz around with an API that has managed to work adequately well for the past 16 years? Sure, it's a little odd the first time you bump into it, but aren't there better things to work on?

But if we are going to futz with CreateProcess, can we remove the mess that allows you to specify the image name EITHER through a separate parameter or as part of the command line?

(Joking... it's way too late for that).

re: Why does the CreateProcess function modify its input command line?

peterchen — Wed, 03 Jun 2009 00:17:50 GMT

@Gabe - that would be worse: a function that promises not to modify memory, but does. (And if it doesn't make that promise, there's no point in fixing it).

waleri - interesting thought! MSDN doesn't say anything about a limit, though.

Now, gotta chew through that sample...

re: Why does the CreateProcess function modify its input command line?

Brad — Wed, 03 Jun 2009 06:12:46 GMT

Since people seem to have danced around the answer, is it:

The JIT debugger is always launched with CreateProcessW? (so that case can never allocate from the heap).

re: Why does the CreateProcess function modify its input command line?

Alexey Borzenkov — Wed, 03 Jun 2009 09:27:09 GMT

> Why is it okay for the ANSI version of the CreateProcess to allocate a temporary string

I'm not 100% sure, but from what I seen during numerous debugging sessions I think that all path related ANSI functions use the same thread-local buffer, so it is either already preallocated, or is not allocated at all.

re: you don't want to allocate memory on the heap

Someone — Wed, 03 Jun 2009 11:24:14 GMT

"When a program crashes, the just in time debugger is launched with the CreateProcess function, and you don't want to allocate memory on the heap if the reason the program crashed is that the heap is corrupted."

I do not follow that argument. It would only make sense if that command line memory were the only allocation done by CreateProcess. If so, where would CreateProcess get the process stack or the memory for the debugger image from?

Or is that debugger running in some zombie state all the time, and was CreateProcess hacked to not really launch it, but revive it from the dead? If so, why would one add that code to CreateProcess, rather than creating a new function "StartJITDebugger"?

[Um, the debugger is a separate process. Heap corruption is process-local. -Raymond]

re: Why does the CreateProcess function modify its input command line?

Mike Dimmick — Wed, 03 Jun 2009 16:45:59 GMT

@Someone: when dealing with CreateProcess, you really have to think in terms of the calling, parent, process, and the child process it creates.

The article is talking about a buffer that CreateProcess is (not) creating in the context of the calling process. The process's initial thread stack is not directly created by CreateProcess but by the process startup code in the kernel, in the context of the new process. The virtual memory for the program code is a shared-memory section backed by the program image file, which is paged in on demand.

I'm surprised that Kernel32's default unhandled-exception code is creating the debugger process directly. I would have thought it would instead send a message to the Win32 subsystem to invoke a debugger on its behalf.

A debugger is just a standard Win32 program which calls DebugActiveProcess to debug another process, or creates a process with one of the DEBUG_PROCESS or DEBUG_ONLY_THIS_PROCESS flags. The debugger then calls WaitForDebugEvent to have Windows tell it about new threads, loaded or unloaded DLLs, debugging messages, thread exits, and exceptions. Breakpoints are set by modifying the code of the target process (with WriteProcessMemory) to place a breakpoint instruction at that location instead; that instruction causes an exception (code STATUS_BREAKPOINT, 0x80000003) to occur in the program, which is reported through WaitForDebugEvent.

To restart, the debugger replaces the breakpoint with the code that should have been at that location, sets the thread's processor context to single-step (execute one instruction), then restarts with ContinueDebugEvent. A STATUS_SINGLE_STEP exception is then raised in the program, the debugger rewrites the breakpoint, and again continues execution to continue at full speed.

re: Why does the CreateProcess function modify its input command line?

Jamie Anderson — Thu, 04 Jun 2009 02:08:10 GMT

I would hazard a guess that the ANSI wrapper functions in the kernel have their own private heap that they use to allocate memory. That would avoid the matter because any memory allocation will happen on a different heap to the one that's been corrupted.

[What if the private heap is corrupted? -Raymond]

Почему функция CreateProcess изменяет переданную на вход командную строку?

Блог Рэймонда Чена (перевод) — Thu, 04 Jun 2009 10:49:00 GMT

Одна из неприятных особенностей функции CreateProcess состоит в том, что параметр lpCommandLine должен

Why CreateProcess commandline parameter should not be read-only memory? « Sharing my thoughts…

Thu, 04 Jun 2009 22:09:26 GMT

PingBack from http://sarathc.wordpress.com/2009/06/05/why-createprocess-commandline-parameter-should-not-be-read-only-memory/

re: Why does the CreateProcess function modify its input command line?

Someone — Fri, 05 Jun 2009 00:42:58 GMT

@Mike Dimmick: thanks for the explanation. I knew most of what you said, except for what surprised you, too: that, apparently, the 'crashed' program launches the debugger (or, probably more precisely: the kernel calls CreateProcess to start the debugger while in the context of the crashed application). But even with that, I do not see why one would have to modify the string you pass as the lpCommandLine to figure out where the program name ends and the command line arguments begin.

The first thing I can think of is that it is not the figuring out, but the use of the program name as an argument to some other system call (FindFile?) that requires the modification. I guess the change would be to overwrite a space in the command line with a NULL. I have seen people do the reverse on Unix systems: creating a program's command line by replacing the NULLs at the end of all but the last of the argv entries by a space. I doubt that that is guaranteed to work, but it did on the one system where I saw somebody do that.

A second possibility could be that CreateProcess does some Unicode normalization in-place. That would also explain why the the ANSI version never modifies that argument.

re: Why does the CreateProcess function modify its input command line?

GregM — Fri, 05 Jun 2009 04:32:02 GMT

"I guess the change would be to overwrite a space in the command line with a NULL."

Yes, that's what the documentation for CreateProcess says.

re: Why does the CreateProcess function modify its input command line?

Random832 — Fri, 05 Jun 2009 22:04:44 GMT

Wait... what if there is no space between the program name and arguments - like "cmd/?" (or any /flags generally) - where does it put the null then?

re: Why does the CreateProcess function modify its input command line?

Falcon — Sat, 06 Jun 2009 13:01:47 GMT

@Random832: Try typing "cmd/?" into the Run dialog text box and see what happens!

(Tested on Windows XP, can't speak for Vista or any other versions...)

re: Why does the CreateProcess function modify its input command line?

teo — Mon, 08 Jun 2009 15:00:36 GMT

While CreateProcess is peculiar, the real fun is in ShellExecuteEx. Yesterday, I had to code a self-elevating program. Basicaly it boilds down to:

if(!IsUserAnAdmin())

ShellExecute("runas", my-own-command-line)

Well, doh! It turns out that Windows (whose mantra used to be "Easy stuff should be easy and hard stuff should be possible") considers this the hard stuff...

My first try:

ShellExecute(0, L"runas", argv[0], GetCommandLine() ... ), nope it won't budge

2nd try:

ShellExecute(0, L"runas", 0, GetCommandLine() ...)

And so on and so forth. At the end I had to write a helper function which goes through the command line of the current process, builds a specially escaped, massaged, tangerine-flavoured, painted-white-and-pink ... command line which after digested by the ShellExecute monstrosity will produce the one I started with. Btw, whoever designed the whole "runas" hack obviously decided that console applications are so out-of-fashion. What window handle a console app is supposed to feed it?

But that is just a minor hack, i fixed it in less than 15 mins. If you want to have a taste of Hell, go compile a program which should run on XP with Vista/2k8 Platform SDK and imports CreateVssBackupComponents() function. Now that is one fine mess. Where to start? The wrong documentation? The fact that on XP it's exported as extern "C++", which makes it exported with C++ mangled name, which obviously differs between 32 and 64 bit compilers and is not natively consumable by any other language in existence, being it another c++ compiler, delphi or what not? The most sickening part is the "fix" - in the form of an INLINE FUNCTION in the platform sdk headers; making it completely impossible to compile a program targeting XP using the official function name ... Microsoft, Microsoft, what have I done to bring thy wrath upon me?