Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

End of Life Issues

End of Life Issues

  • Comments 23
Wow.  Yesterday I asked y'all when it was ok to end-of-life an API.

I'm astonished at the quality of the responses.  For real.

Almost everyone had great ideas about it - some were more practical than others but everyone had great ideas. 

Several people suggested a VM based solution, that's an interesting idea, but somewhat impractical with the current state of technology.  And it's beyond the scope of what I'm trying to discuss.  In particular, I was wondering what the criteria were for end-of-lifeing an API.  Also, Mike Dimmick pointed out one of the huge problems with VM solutions - it doesn't work for plug-ins.

A number of people talked about end-of-lifeing with the release of a new platform.  But the problem there is that the Windows appcompat contract is a BINARY contract, not a source level contract.  So we can't release a new version of Windows that doesn't support APIs, since that would break existing applications.

A couple of people suggested that open-source would be the solution to the problem.  But open sourcing old APIs that doesn't fix the problem, it just pushes it onto someone else.  And by releasing the source to a high level API, it locks down the interface in at a lower level, which is almost always a bad idea (because it removes your architectural flexibility - what happens if there are four different APIs that use a single low level API, you want to end-of-life one of them but keep the other three.  If you open source that one, then you freeze the low level API, which removes your ability to innovate the other three APIs (this is a grotesque simplification of the issues, I may write more on this one later).

Michael Kaplan wrote about a possible solution, but that doesn't really solve how you REMOVE APIs - it just describes how you can make changes without removing the APIs.

At least one person said that the software wasn't relevant that it was the data that mattered.  At some level, that's right, but it's a naive view of the world - the reality is that for most businesses, if their line-of-business application doesn't work, they don't care if their data hasn't been lost - their business is still just as shut down.  And many businesses DON'T have the source for their LoB applications.  Or they don't have the money to bring those applications up-to-date.  It can cost millions and millions of dollars to update a suite of LoB applications.  And those updates invariably involve UI changes, which means retraining costs.  Which most businesses aren't willing to take on.  So the existing apps have to continue to run.

Dana Epp was one of the first people to pick up on what I think is the most critical aspect of end-of-lifeing an API - you need to have a credible alternative to the API - one that offers comparable functionality with similar ease of use.  If you don't have an easy alternative for customers to adopt, then they won't adopt it.

But (IMHO) the most important thing that everyone missed (although Dana came close) has to do with the binary contract.  You must keep existing binaries working.

And that means that your ONLY opportunity for removing an API comes when you know that every application running on your system is going to be recompiled.  That happens when you switch hardware platforms.  So there have really only been three opportunities in the life of Win32 to do this - Alpha, PPC, X64.

Any other time has the possibility of breaking applications.

So to me, an API can be removed if:

  1. You can guarantee that there are no existing applications that call the API.
  2. You have a credible alternative API that is comparably easy to use.

Criteria 1 only occurs with the release of a new hardware platform, Criteria 2 is just a result of careful planning.

While I was doing the "How to play a CD" series, Drew noticed that my sample didn't work on his version of x64.  Why?

Well, because we thought we could end-of-life the MCI APIs for x64.  It turns out that virtually all of the MCI API set has been replaced with newer, more functional APIs and there were some architectural issues with the MCI APIs that made it attractive to remove them.  So we did.

And then we realized that we'd messed up.  Why?  Audio CD playback.

It turns out that we didn't have a reasonable alternative to the MCI APIs for CD playback - everything else had credible replacements but not CD playback.  And it turns out that a number of apps that were being ported to x64 relied on the MCI APIs for CD audio playback.

So we put all the MCI APIs back before we released, and addressed the architectural issues.  Because appcompat trumps architecture every time.

Having said that, a number of the people making comments were absolutely right - for Longhorn, there WILL be some scenarios that won't continue to work, as a result of some of the high level architectural changes we're making.  But we're being a lot more careful about ensuring that those scenarios truly ARE corner cases.

Having said all this stuff about when it's ok to end-of-life code, I MUST include Adrian's comment verbatim:

A lot of the comments here are looking at this from the point of view of somebody with existing applications upgrading to a newer version of the OS. It's an an important considertation, but it's not the one I am faced with day to day.

As developers, we rarely get to choose which platforms we target. The market determines that. I and many other Windows developers I know are building *new* applications, but the market demands that they run on older platforms as well as the "current" ones. A non-trivial number of people still run Windows 98 and NT 4, especially as you look at the international market.

The users of an obsolete API are not necessarily applications that are five or ten years old. They may be brand new applications that rely on an API that was been superceded long ago because it's available across all of the platforms that must be supported.

Sure, sometimes your application will dynamically use a newer, better API if it's available or fall back to the old one if it's not. But when the old API is sufficient and universal, it will often be used rather than the "current" one, even in sparkling new code.

Consider Avalon, a whole new presentation layer. How long will it take for it to completely replace GDI? If I'm writing a new general-purpose application today, I *have* to use GDI. Even if I were targeting a release date that coincides with Longhorn, I couldn't afford to ignore Windows XP, Windows 2000, Windows NT 4.0, Windows Me, and Windows 98. (In reality, we even ensure that the basic functionality of our app works even in Windows 95 and NT 3.1.) If I had unlimited resources, I *might* try to develop a parallel GDI/Avalon version. But when is the last time you were on a development project with unlimited resources?

I don't have a general answer to Larry's question, but using GDI as an example, I'd say you could retire it when Longhorn (or perhaps XP with Avalon extensions) is used about as much as Windows 98 is used today. That's probably four releases after Longhorn, or more than a decade away.

Adrian's spot-on - it's not enough that we ensure that the API has been superseded.  We also need to ensure that the replacement either is available on previous platforms or has been around for enough time that it it's a credible alternative.

 

 

  • Actually this brought to mind the MS layer for unicode. Could you pull something like this off? It seems like the very situation that Adrian mentioned with APIs (which seems blatently obvious once you read it. But alas..) is the one that was encountered with Unicode, since the Win9x platform wasn't natively unicode. If there are, in fact, viable alternatives, it seems that there could be a layer that would do this conversion. For exes already linked, you could have shimeng catch the imports and patch in the layer automagically. Then the exes work on the old, and the new, and the functionality of the old code is only in the new place - but just redirected from several, not duplicated.

    Thoughts?
  • This is what I get for doing many things at once.
    That wasn't clear at all.

    On the old machines, you have a layer to support conversion from new to old.

    On the new machines, you have a layer to support old to new.

    So on the old machines, new applications still run.

    And on the new machines, old applications still run.

    The question is whether all that work is actually worth it.
  • Which API are you talking about? The kernel one, or one of the libraries?

    If it's just a library, I don't see the problem. Just bump up the version major number. People using the new API make sure they are linking against say foo2.0 and all the old apps linked against foo1.0 still work. Eventually when enough time has passed you just don't install foo1.0 by default anymore,and if someone _really_ needs it they can download it as an option.

    This is more or less the way things work in Linux distributions.
  • I don't actually have my own x64 box (yet). There's a big lab full of machines if I need to borrow one, though.
    And I didn't actually even build for x64. I just checked out the header and the SDK entries for the APIs. Until I looked in mmsystem.h I had forgotten that some headers still have this in them:

    #ifdef _MAC
    #include <macwin32.h>
    #endif //_MAC

    That amuses me.
  • Vince, that doesn't end-of-life the API.

    You still have to support the v1 of the API.

    Just because you shipped v2 doesn't somehow magically make v1 go away. The question here is when can you nuke the v1 library.

    Linux doesn't solve this either, the platform stability guarantees that Linux provides aren't binary guarantees, they're source code guarantees (and even then, the're very weak).
  • As Window Snyder has told just perfectly:
    "Every time we rip a feature out because it is old and we think no one is using it, our customers scream that we are using it, and over the life time of Windows, that adds up to a significant code base."

    The main problem is that Windows really is
    a binary/executable OS. You cannot simply talk
    about compile-time compatibility (that can be
    solved well with 'obsolete' warnings or other
    decent compiler tricks), but you need to be
    run-time compatible.
    Sadly "current" applications are "DLL-hellish".
    That is a fact. As long as you have these
    kind of apps around you must create a runtime
    environment that supports DLL-hell (no matter
    how clean/well is your "new" OS is).
    It wouldn't be a problem if the Windows would
    have grown up in an Open-Source community,
    since that way you - most probably - would
    have access all "crucial" sources (talking
    about apps not the OS itself).
    If you have access to the source, you can still
    revert the run-time compatibility to a
    compile-time one.
    Sadly this is not the case here. So you have
    quite a few options:
    - hire the Wizard of Oz (I guess at Microsoft
    he is Raymond Chen) who does the run-time
    patching of the affected apps and creates
    the DLLhellish environment until the end
    of time, so that you can have satisfied
    customers
    - create a new application that does the same
    as the "old" non-working one, forcing
    customers buying a new tool instead of the
    old (still good) one that worked just fine
    with the old OS (well at least would drive
    me nuts if it turns out that my $500 tool
    stops working after the next OS release,
    but that would kill me if there would be
    no update to my tool (vendor went bancrupt)
    -> surely it is definetely not the
    responsitibility of Microsoft, but it is
    still a valid/real-life scenario
    - do not support run-time compatibility ->
    making customers mad, that's for sure

    So my opinion is: It is impossible to have
    a solution that satisfies everyone. You must
    value the pros and cons;
    - do I gain more from breaking the app.
    compatibility and most probably ending up with
    lots of annoyed customers
    - or is it simply just not the time when these
    changes are worth to be done (please do not
    tell me that the time is never right to do
    changes, there IS when the extra features
    (let it be stability or new stuff) simply
    satisfies the customer as well and making them
    forget the "pain" that loosing some apps would
    cause).

    But please do not forget the other (internal
    design) aspects of supporting backward
    compatibility. As your code-base grows, there
    will always be lot of security risks floating
    around coming from the "outdated" design
    coming from the old-code (so it is not just
    the customers' happyness that must be weighed,
    but the cost of internal maintenance as well).
  • When the DOS 1.x FCB functions stopped working on FAT32 drives in Win9x, was that a deliberate attempt to End-of-Life them or an oversight? The fact that they weren't removed from the NT-based OSes suggests the latter.
  • "So there have really only been three opportunities in the life of Win32 to do this - Alpha, PPC, X64."

    X64 is actually a Win64 system - its the change to a new ABI that matters as much as the hardware platform.

    Oh, and you forgot both of the IA64 users :-)
  • What is this "Windows appcompat contract"? I mean, I think everyone understands it in a way ... appcompat trumps architecture. But is there more to it? A legal thing? Is it spelled out somewhere?
  • There's no contract.

    AndyM: Your're right, I had forgotten ia64, since ia64 and x64 came at the same time, I tend to lump them together, although they are very different platforms.
  • Fascinating stuff, as always. More than ever it just underscores the need to be as sure as you possibly can that you've designed your API correctly before releasing it in a product. Once an API is in the wild it's out there essentially for good.

    I work on test & measurement equipment and we export a string based API over a physical interface known as GPIB. A roll of our hardware platform means nothing to our customers, they expect the same physical interface and API to be present on our new box, the same as the old box. This means that for an existing API we are *never* in the situation where we can guarantee that there are no apps that call our API.

    I'd be very interested to hear your thoughts on how the problem of designing and reviewing APIs should be tackled.
  • Would it be possible to release a version of Windows that specifically states that no backwards compatibility before X (where X may be Win95, 98,2000, or whatever is decided) is supported, for those users that prefer a cleaner install over the ability to use miscellanious archaic programs? This would not be ideal for everyone, but power users would probably appreciate the boost.
  • LarryOsterman Said:

    > You still have to support the v1 of the API.

    Why? Just freeze it as is, maybe with some extremely minimal support to fix security issues. If people want new features, they can port to the new version of the library.


    > Just because you shipped v2 doesn't somehow
    > magically make v1 go away. The question here is
    > when can you nuke the v1 library.

    After a period of time, say 5 years after announcing it is no longer supported, just stop installing it by default. People who still need it can enable it by hand.

    > Linux doesn't solve this either, the platform
    > stability guarantees that Linux provides aren't
    > binary guarantees, they're source code guarantees
    > (and even then, the're very weak).

    Again you are being abit confusing here.

    "Linux" refers only to the kernel.

    Linux the kernel is _very_ strict about supporting syscall-level compatibility, the whole way back to version 1.0 and before. I have run extremely old binaries ( amusingly, a CD-rom playing app like your series) that were in the a.out executable format, linked against libc4, and against extremeely old Motif library... and yet still managed to run fine.

    The sycall level binary compatibility is very important to Linux developers.

    The kernel-internal API is not though. Which is why you can't use binary-only device drivers without a lot of hassle. That's a different issue though.


    What the Linux DISTRIBUTIONS do is an entirely different story and varies from project to project and vendor to vendor. Though typically they work in the way I described. Older programs might be linked against the 1.0 version of the gtk+ library. THat library has been obsoleted and is not installed by default on newer distros, but you can get it and install it by hand if needed. Most users don't, and most programs do keep up to date with current library releases.

    Commercial programs for Linux do often link against extremely old versions of libraries, which is they primary reason the distros still try to maintain the older libraries.

    And before you claim Microsoft sainthood based on how well you keep compatibility, I find I can run old DOS games using the "dosbox" emulator on Linux and MacOS a lot smoother than on my girlfriend's machine running modern Windows.
  • Vince,
    The discussion here is "when can you REMOVE the API". You're asking "when can we IGNORE the API".

    As long as the code to implement the API exists, it's going to have to be supported. And apps are going to call it. And they're going to expect that the APIs will continue to work, regardless of what happens on your platform.

    Unless you're proposing reinventing DLL hell? There are two subtly different but similar versions of a DLL on the system - one offers one set of functionality, the other offers a somewhat different set of functionality, and apps have to choose which one they're going to use?
  • 1. Norton has broke on every new win release. Unfortunately I always have had the beta of the next Win when I've aquired NU. They don't feel any need to support their products. It's hard to whinge as I last paid cash for Ver 7 (and I still have ver 7 (with libraries) and 6 (standalone) files. But by sending a tip in to the SMH you get a free copy. I no longer bother as it won't work for very long.

    2. MS has broken EVERY SINGLE ONE of my applications. This "contract" excludes me.

    3. Like with the Win 95 aware header for Win 16 programs, break the API on new programs, wait a decade then kill the API.
Page 1 of 2 (23 items) 12