The Old New Thing

But you've just run an article that told us how to use the least significant bits of a HANDLE! (I suppose you could call it "balance".)

Exactly; this seems like a mixed message to me.  A program that relies on kernel handles always being multiples of 4 is ok, but a program that relies on window handles always being multiples of 2 is bad.  I suppose someone will point out that the kernel handle implementation detail is documented whereas the window handle implementation detail is not documented.  I understand that, but doesn't Microsoft reserve the right to change implementation details at any time?  If Microsoft "fixed" window handles to always be multiples of 2, isn't it just as "safe" to rely on that as it is to rely on kernel handles being multiples of 4?  And if enough programs relied on kernel handles being multiples of something other than 4 would Microsoft "fix" that detail as well?

My ultimate goal in life is to create a poorly written program so popular that Microsoft changes their implementation details to suit it.

"I suppose someone will point out that the kernel handle implementation detail is documented whereas the window handle implementation detail is not documented.  I understand that, but doesn't Microsoft reserve the right to change implementation details at any time?"

There is a pretty hefty distinction between documentation stating "X is guaranteed to be Y" and documentation stating that "X is Y but this is an implementation detail and is subject to change".

If kernel handles are guaranteed to be multiple of four then it isn't an implementation detail, it's part of the specification.

Ken Hagan: The difference is that the definition for file handles includes the fact that the low bit is always clear (in fact there are Win32 APIs that use the low bit in a handle to signal changes in behavior).  The definition for window handles has no such guarantee.

Oh, John. I hope no one ever sets out to do that for the programs I code. And for MS, I'm sure it's happened before... :)

the phrase "implementation detail" pretty much means "something that happens just now, but isn't documented and might change" so I don't really understand people talking about "documented implementation details"

>A program that relies on kernel handles always being multiples of 4 is ok

He didn't say this... he just gave a technical example of why one could want to do this.

"But you've just run an article that told us how to use the least significant bits of a HANDLE!"

IMO, even if it's "documented" in the NT kernel line, it's still evil to use. It doesn't properly works with "special" handles (STD handles and current process handle).

In other words, it's not proper Win32 code. For NT native code, it's less evil; just bad programming practice.

Ulric wrote:

"He didn't say this... he just gave a technical example of why one could want to do this."

Yes, Raymond told us:

"(This series was written in response to Igor Levicki being unable to imagine "how this can save anything (in terms of performance)"

Raymond just gave his "imagination". This doesn't make it good practice.

Ah, it's another version of the programmer's 100 bottles of beer on the wall song:

100 implementation detail dependencies

fix one

release again

101 implementation detail dependencies

repeat until implementation detail dependencies is zero

This could take a while.  ;)

I'm still all for breaking software that uses "undocumented features" and putting the blame squarely where it belongs - on the developers who made up their own rules as they went along.

"and putting the blame squarely where it belongs - on the developers who made up their own rules as they went along."

How? If Microsoft just does that, the blame will be put on Microsoft!

Hey, the new version of Windows is broken, it isn't compatible with <big software of big company> and with <my favorite software>!

Microsoft engineers surely maliciously put incompatibilities to force migration to newer software! Evil Microsoft!

I think backwards compatibility is reason #2 of Windows success, even if on the long run, it makes a system full of hacks.

(The reason #1 is the excellent Windows 3.x which provided a powerful graphical and multitasking for 640K computers).

@Xepol: Ah, the luxury. Even though I work on a small company we still have to work around other companies inability to correctly integrate with our devices.

One such company has a bug in its communication with our devices and we get to take the blame for it. Why? Because the customer only sees that the communication doesn't work with our devices. It works with other devices. Never mind that our devices communicate using a different protocol. It's our fault, it's always our fault. The same goes for Microsoft, if a program doesn't work in Windows, it's always Microsoft's fault (no, I'm not being ironic, from a non-technical customers point of view it's true).

Darn straight of the Product X dev acknowledging a problem, and getting it fixed.

"Should only take five years, ten tops."

Oh the joys of application compatibility and legacy code!

"If Microsoft just does that, the blame will be put on Microsoft!"

Not if done properly. However, the effect will be the same anyway. "Product X doesn't work on Windows N+1... it's clearly the fault of Product X's developers, but I'm just going to stick with Windows N anyway."

People would not pay to upgrade to Windows N+1 and ALSO Product X N+1 just to get a version which works on the latest version of Windows.

"You can finally use those odd handles once the next version of Product X comes out and the current version falls out of support. Should only take five years, ten tops."

Yeah, because backporting a patch to fix this issue, along with the other bugs, in the next minor/bugfix release of the current version is ... what? Too hard? Oh no! The burden of a vendor actually fixing bugs in a product while it's still being supported!

WTF?!?

I hope no-one is actually paying this vendor for this "support", if the vendor isn't using that money to fix the bugs that the product has.

>> I hope no-one is actually paying this vendor for this "support", if the vendor isn't using that money to fix the bugs that the product has.

For what ? Supporting a fix noone really needs (it works) ?

You should be grateful they fixed it in future releases, few software vendors would have done so.

(beside, it could even be argued this is a bug at all. it's the use of an undocumented "feature". There are entire products based only on the use of undocumented features).

"For what ? Supporting a fix noone really needs (it works) ?"

Depends. There are other implementations of Win32 than just Microsoft's you know.

"You should be grateful they fixed it in future releases, few software vendors would have done so."

WTF? I should be grateful that vendors actually comply to relevant API specs?!? Should I also be grateful if they release software that doesn't randomly delete my hard drive? Rubbish. This is what vendors are *supposed* to do. This is what I'm paying them for when I buy their software, and when I pay for support. I should not be "grateful" to someone for upholding their end of a contract or bargain, I should *expect* it.

Seriously, the reason why most software is so damn awful is because of attitudes like this which *don't* demand that bugs, once discovered, are fixed; which assumes it's acceptable for software which you've paid money for to be broken.

Demand more from your vendors!

@Karellen:  backports can be a huge PITA when the fix crosses with other, more-recent patches or changes for new functionality.  Development & testing cost real money and if there's no market for a three-year-old version, where's the incentive for a business to spend on it?  By definition (for this thread) the software is working correctly for those that have it installed.

....and even if it *is* done, convincing customers to install a new version in production when the old one is proven & working correctly isn't simple, either.

Karellen:

I'm a long-time developer, currently working on Windows.  There is a rather large set of reasons why an application might not comply to relevant API specs:

They are wrong (e.g., the icon description was correct for about windows 2.0.  It's wrong now)

Various coder magazines (both MSFT and non-MSFT) have published useful shortcuts that turn out to be a bad idea.

The specs have changed (this has especially happened in the network space)

They are ambiguous to be point of uselessness.  (The sin function, for example, used to not specify whether it took radians or degrees)

The program had to work in a particular way, and the APIs or data structures had to be rather steamrollered to make everything work

I was one of the original developers for the first few versions of Microsoft Project for Windows (oops, I wasn’t supposed to let that out) and I was the one who implemented this hack.

It was a dark and stormy day, way back in early 1990, when test discovered that project was running out of memory unrepentantly on non-protect mode Windows 2.01 running on 640K RAM and with no expanded memory cards installed.  I traced the problem to the fact that Project 40 or so Windows controls to display its task from window and each of these controls required a few hundred bytes to be allocated from the user heap.  At some point, Windows would need to reallocate the user heap but couldn’t because of memory fragmentation, locked code segments, etc.

We were close to our ship date and the only solution I could come up with that wouldn’t impact the rest of the code base was to implement my own set of Windows controls (list boxes, edit controls, labels, buttons, combo boxes, etc) that supported the exact same Windows messages but minimized the memory required and made sure that memory was not allocated from the user or GDI heaps.  My only problem was that I needed an easy way for my message pump to be able to identify whether a window handle was a real Windows window or one of my windows, so I could forward the message to the right code.  I noticed that Window handles at that time were just byte offsets into the user heap and that these offsets were always even, so I hacked my code to use odd handles to identify my windows.  I knew that this was going to cause some grief for the Windows guys in the future, but this was the only way I could squeeze Project into 640K.

I left Project to form the original MSN team in 1992 and got some blowback from this.  Windows/NT apparently had to do some rework after discovering this issue during backwards compatibility testing.  That cost the Windows team some time and money, but Project has probably contributed better than $10B in profits to Microsoft over the years so it was probably a good trade off.

I’ve since run into two of the developers on the Windows side that had to deal with this, and we still chuckle about the good old days.  It’s amazing that this code has remained in Project for something like 18 years.  I guess, if it works, don’t mess with it.

--Jeff Lill

I think the right approach (long overdue) for Microsoft would be: not to work around third party bugs, but when an user tries to start some cr@pware, put a nice message box:

--This version of Cr@ptastic Entertainment Dung Throwing Turnament may not run correctly in this version of Windows because of bugs in it. Contact Cr@ptastic Entertainment for a fix. Click this button for more technical details. Click this button to run it anyway--

This would keep the ISV developers in the line. And Microsoft's legal department does have enough bodies to protect against Cr@ptastic Entertainment legal threats.

Alexandre:

Try reading this http://blogs.msdn.com/oldnewthing/archive/2003/12/24/45779.aspx and then you'll see why. I'd quote it, but it'd be the entire post.

If you upgrade to Windows X+1, and now that LOB app that you have to run *every day* pops up this stupid little box before it lets you run it, are you upset at the old version of the program, or are you upset at the new version of Windows? http://blogs.msdn.com/oldnewthing/archive/2003/12/23/45481.aspx#45510 And which one can you return and get your money back?

@Karellen: "Depends. There are other implementations of Win32 than just Microsoft's you know."

Ummm... No, there aren't. Win32 specifically refers to 32-bit versions of Microsoft Windows. Any other implementations might be compatible with Win32, but they aren't "implementations of Win32".

"Seriously, the reason why most software is so damn awful is because of attitudes like this which *don't* demand that bugs, once discovered, are fixed; which assumes it's acceptable for software which you've paid money for to be broken."

Sorry, but you're wrong. If the current version of my product is 5, and a bug is discovered in 3 by someone who purchased it then but hasn't bothered to upgrade since, they're out of luck. If they want bug fixes, they need to stay current; the bug may have been fixed in v4, but they didn't get it, and it's not a bug for the current v5 or the immediately prior v4. Anything before v4 is out of date and out of support.

I'm all for vendors supporting their customers, but it goes both ways; the customer has to continue to support me. Continuing to run an outdated version and expecting ongoing work isn't supporting me, but adding a burden, and I have enough of those.

"to implement my own set of Windows controls (list boxes, edit controls, labels, buttons, combo boxes, etc) that supported the exact same Windows messages but minimized the memory required and made sure that memory was not allocated from the user or GDI heaps."

Nowadays, this would be called windowless controls.

Adam V,

I understand that was the logic behind putting those numerous kludges into Windows. Unfortunately, it backfired. The users got used to the idea, that all problems of the third party software is caused by Windows.

At least, Vista could have had such dialogs of shame.

And actually, XP SP doesn't hesitate to say that the driver X is incompatible with the update.

Vista has such dialogs of shame.

And it doesn't contribute to its sales.

I need VS2002, it says me it's incompatible, therefore I need XP.

Whether documented or not, using the lower bits of a handle is fraught with risk. Imagine another piece of software that uses the same trick, but uses the bit for a different purpose. Can you prove that you never release the handles that you've changed "into the wild" where the other software might encounter them and misbehave? Can you prove that you never accept handles "from the wild" that might have originated from the other software. Bear in mind that the other software might be in-process. For example, it might be a library that was incorporated into a shell extension.

It seems to me that the effort required to prove it is safe exceeds the effort required to use an alternative mechanism (like a map that you maintain privately). A significant part of the risk is that not all of the code involved is code that you wrote. (Raymond will recognise this scenario. :) He is a clever guy, though, so he can probably *still* think of a counter-example.)

"Imagine another piece of software that uses the same trick, but uses the bit for a different purpose."

In fact, in this case, this is exactly what is happening:

http://blogs.msdn.com/oldnewthing/archive/2005/10/05/477802.aspx

PingBack from http://www.bwhacks.com/forums/hardware-software/35457-another-example-software-issues-resulting-bad-development-not-os.html#post634889