Welcome to MSDN Blogs Sign in | Join | Help

Psychic debugging: Why doesn't my program show up when I call ShellExecute?

This question came in from a customer (paraphrased):

If I run my program from the command prompt, it works great, but if I run it from my launcher via ShellExecuteEx, it never appears.

See how good your psychic powers are at solving this problem before I give you the second question that gives away the answer.

Any luck?

Here's a second question from a different source (but which coincidentally came in the same day).

I'm trying to use ShellExecute to open a document. The function succeeds (returns a value greater than 32), but I don't get anything on the screen.

if (ShellExecute(Handle, NULL, FileName, NULL,
    NULL, NULL) <= (HINSTANCE)32) ...

The problem the second person is having lies in the last parameter to the ShellExecute function. It's nShowCmd, which is supposed to be an SW_* value, but which this person is passing as NULL. It so happens, that the value zero corresponds to SW_HIDE, which explains why the program doesn't appear: You told it to run hidden!

Now go back to the first problem. Do you see what the person most likely did wrong? The code probably went like this:

SHELLEXECUTEINFO sei = { sizeof(sei) };
sei.hwnd = hwnd;
sei.lpVerb = TEXT("open");
sei.lpFile = pszFile;
ShellExecuteEx(&sei);

Since the sei.nShow member was not explicitly set, the value was implicitly set to zero by the incomplete initializer. And as we noted above, zero means SW_HIDE.

It turns out my psychic debugging was correct. That was indeed the source of the first person's problem. Now you can use your psychic powers, too.

Published Monday, October 23, 2006 7:00 AM by oldnewthing
Filed under:

Comments

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 10:20 AM by Andrei

My favourite mistake I have tried to avoid!

This is the reason to avoid zeroes in enum's.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 11:24 AM by Doug

I must assume your customers don't fit the category of professional programmers.

Professional: "Hmm, it doesn't work.  Let's look at the code and check the functions we are calling."

Amateur: "Hmm, it doesn't work.  Must make a support call."

# Interesting Finds: October 23, 2006

Monday, October 23, 2006 12:42 PM by Jason Haley

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 1:49 PM by Tim Smith

Doug, you couldn't be further from the truth.

Professional: "Hmm, it doesn't work.  This operating system is so buggy.  Let me try this, I bet that will work."

Then again, I wouldn't exactly call those people professionals.  But I run into programmers who never assume it is their fault.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 1:55 PM by Mike Dunn

I often see this on message boards - folks who use NULL or zero for parameters they don't (yet) understand, and expect that the API will "do the right thing" and use good defaults. The trouble is, that works in _some_ cases (like the lpDirectory param to ShellExec) but not others (like nShowCmd).

A quick RTFMsdn would solve the problem, and hopefully the person learns this after getting their answer of "you need to set nShowCmd correctly, see the docs for the legal values".

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 4:29 PM by Problem solved

If SW_SHOW would be 0 and SW_HIDE would be 5 this problem wouldn't exist. NULL/0 should always be a safe default.

[I'm surprised. I would have expected that if SW_HIDE were 5 people would say "What a phenomenally stupid idea. The value for SW_HIDE should obviously be zero since zero means FALSE which means 'hide' and nonzero means TRUE which means 'show' (and the precise nonzero value indicates what type of show). Giving 'hide' the bizarro value of 5 just shows what a bunch of moronic developers they have at Microsoft." -Raymond]

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 4:53 PM by 640k

Both SW_FORCEMINIMIZE and SW_MAX are = 11, so I think the hope of not be treated as a moronic developer is already lost.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 5:10 PM by GregM

SW_MAX == largest value supported by ShowWindow, for easy range checking.  So yes, it is the same as SW_FORCEMINIMIZE, the largest value.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, October 23, 2006 7:08 PM by Igor

>If SW_SHOW would be 0 and SW_HIDE would be 5...

If you just read the MSDN you would know what value to put there instead of NULL and the problem wouldn't even exist so you won't have to solve it.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Tuesday, October 24, 2006 1:33 AM by Goran

Because of (possible) problems like the one with SHELLEXECUTEINFO I always wondered should I subclass any old structure and add a couple of constructors to it, only to avoid thinking about what to put in every member it might have every time I use it.

struct ShellExecuteInfo : public SHELLEXECUTE

{

 ShellExecuteInfo(LPCTSTR File, LPCTSTR Verb bool show);

 // ...

};

Yeah, it's not great, but may be a small improvement ( and a huge pollution of the namespace :-( ). Because I'm not sure about it, I seldom do it (i.e. when I get really fed up with RTFing MSDN).

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Tuesday, October 24, 2006 2:39 AM by peterchen

> NULL/0 should always be a safe default.

:wthf:

SW_* constants are used in many places, not everywhere the same value is the "sensible default".

It's your / our job to feed the correct parameters - all of them, not just those that we care about at the moment.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Tuesday, October 24, 2006 3:05 AM by Niklas

Running something in the background is a quite sensible default (you start an application in the background for printing a document or something).

It's easy to make a great handwaving argument about how well-designed it is in that he who asks for nothing gets nothing. I won't do that. I think you should know what to put and not to put in enumerations.

[I'm assuming you're joking. Enumerations were not in K&R and did not become part of the C language until 1989, six years after the ShowWindow function was written. Let me know when you've perfected that time machine. -Raymond]

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Tuesday, October 24, 2006 8:45 AM by required

Actually there is nothing in the MSDN documentation of ShellExecute which says NULL will hide the application, because the possible values of nShowCmd are not shown, only names.

It is just another example of an API tripping up a developer by using basic scalar types and assigning meaning to them beyond what can be gathered just by reading a description of the function. Compilers should check such things, based on the types of the arguments, not people.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Tuesday, October 24, 2006 8:51 AM by Neil

If you use image execution options to debug the app startup is the debugger clever enough a) to show even though you specified SW_HIDE b) to pass SW_HIDE on to the app so that when you get to ::ShowWindow(hWnd, nCmdShow); you get to realise that nCmdShow is SW_HIDE?

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Tuesday, October 24, 2006 5:24 PM by cmil

"Actually there is nothing in the MSDN documentation of ShellExecute which says NULL will hide the application, because the possible values of nShowCmd are not shown, only names."

No, the possible values _are_ shown. SW_HIDE and NULL might be equal but they are not equivalent. SW_HIDE is always a legal value while under most circumstances NULL is not. Just because the abstraction isn't enforced doesn't mean it can be abused.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Tuesday, October 24, 2006 9:17 PM by mikefried

It is too bad that the MSDN doesn't explicitly state the values of the "enumerations" (or maybe I just couldn't find them). I was recently poking around the WinMM.DLL methods for MIDI in/out (http://msdn.microsoft.com/library/en-us/multimed/htm/_win32_midi_functions.asp), and in order to get the values and types to put into my C# app (via P/Invoke) I had to open up VC++, type in the value I was looking for, and hit F12 to have VC++ search for the definition.

My folks sent me my old MIDI controller from high school (for me that was in the mid 90s), and I wanted to hook it up to my tablet via USB and make it use Windows to act as its synthesizer (and maybe to do a little more), so I needed a way to read messages from MIDI in on my USB to MIDI adapter and send the messages to MIDI out on the Windows side. I chose C# instead of my normal C/C++ for fun -- I wanted to work with P/Invoke and delegates.

Needing to use my C++ tools to find out how I needed to marshall my data/types/etc for C# was unfortunate.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Wednesday, October 25, 2006 4:06 AM by 640k

Some enums in windows.h (winbase.h) have a "max" defined which is 1 more than the actual largest value. Conclusion: You can't trust it.

# SQL?

Wednesday, October 25, 2006 11:28 AM by Roie

These people may have some background in SQL. In SQL, NULL is a valid value for any datatype, distinct from any other value including zero (for some reason, Oracle treats the empty string as NULL, but that's because Oracle's weird). It really does make sense, to your average programmer these days (i.e., one that didn't grow up on K&R and RAM measured in kilobytes).

In truth, I think the problem stems from the fact that ShellExecute is such an old routine, that hasn't been superceded (yet).

# re: SQL

Wednesday, October 25, 2006 4:59 PM by Good Point

Roie, I would make one small editorial change:

> It really does make sense, to your average [VB] programmer these days

If you want to use Win32 you should know what NULL is.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Wednesday, October 25, 2006 6:31 PM by Neil

ShowWindow was doomed once it went beyond the simple showing and hiding of windows (including one constant which goes as far as to change your z-order, activate another window according to an undocumented algorithm and trim your working set). Instead ShowWindow should have been left to do what it does well, and all the iconising and maximising should have been given to a separate function.

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Sunday, October 29, 2006 11:45 AM by The real problem

ShowWindow()'s name is to blame for this. To call a function names ShowWindow when you really wants to hide a window is plain wrong.

[A common thread I find is that when people say that X is a bad idea, they rarely say what would be better. -Raymond]

# re: Psychic debugging: Why doesn't my program show up when I call ShellExecute?

Monday, November 06, 2006 12:50 AM by Norman Diamond

> I would have expected that if SW_HIDE were 5

> people would say "What a phenomenally stupid

> idea.

Should've used negative numbers (teasing here).

Theoretically you could use all positive numbers and returned an error code if someone passed a zero.  Obviously it's too late but this kind of design is reasonable.

> Enumerations were not in K&R

... for some language versions of that book.

> and did not become part of the C language

> until 1989

Nothing became part of the C Standard until 1989, but a ton of stuff was added to the language by ATT and others before 1989.  I'm pretty sure ATT added enums while Microsoft was selling Xenix.

[I can't believe my eyes. You're actually saying that Microsoft should have violated the C language standard. -Raymond]
New Comments to this post are disabled
 
Page view tracker