Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

It's the platform, Silly!

It's the platform, Silly!

  • Comments 69

I’ve been mulling writing this one for a while, and I ran into the comment below the other day which inspired me to go further, so here goes.

Back in May, Jim Gosling was interviewed by Asia Computer Weekly.  In the interview, he commented:

One of the biggest problems in the Linux world is there is no such thing as Linux. There are like 300 different releases of Linux out there. They are all close but they are not the same. In particular, they are not close enough that if you are a software developer, you can develop one that can run on the others.

He’s completely right, IMHO.  Just like the IBM PC’s documented architecture meant that people could create PC’s that were perfect hardware clones of IBM’s PCs (thus ensuring that the hardware was the same across PCs), Microsoft’s platform stability means that you could write for one platform and trust that it works on every machine running on that platform.

There are huge numbers of people who’ve forgotten what the early days of the computer industry were like.  When I started working, most software was custom, or was tied to a piece of hardware.  My mother worked as the executive director for the American Association of Physicists in Medicine.  When she started working there (in the early 1980’s), most of the word processing was done on old Wang word processors.  These were dedicated machines that did one thing – they ran a custom word processing application that Wang wrote to go with the machine.  If you wanted to computerize the records of your business, you had two choices: You could buy a minicomputer and pay a programmer several thousand dollars to come up with a solution that exactly met your business needs.  Or you could buy a pre-packaged solution for that minicomputer.  That solution would also cost several thousand dollars, but it wouldn’t necessarily meet your needs.

A large portion of the reason that these solutions were so expensive is that the hardware cost was so high.  The general purpose computers that were available cost tens or hundreds of thousands of dollars and required expensive facilities to manage.  So there weren’t many of them, which means that companies like Unilogic (makers of the Scribe word processing software, written by Brian Reid) charged hundreds of thousands of dollars for installations and tightly managed their code – you bought a license for the software that lasted only a year or so, after which you had to renew it (it was particularly ugly when Scribe’s license ran out (it happened at CMU once by accident) – the program would delete itself off the hard disk).

PC’s started coming out in the late 1970’s, but there weren’t that many commercial software packages available for them.  One problems developers encountered was that the machines had limited resources, but beyond that, software developers had to write for a specific platform – the hardware was different for all of these machines, as was the operating system and introducing a new platform linearly increases the amount of testing required.  If it takes two testers to test for one platform, it’ll take four testers to test two platforms, six testers to test three platforms, etc (this isn’t totally accurate, there are economies of scale, but in general the principal applies – the more platforms you support, the higher your test resources required).

There WERE successful business solutions for the early PCs, Visicalc first came out for the Apple ][, for example.  But they were few and far between, and were limited to a single hardware platform (again, because the test and development costs of writing to multiple platforms are prohibitive).

Then the IBM PC came out, with a documented hardware design (it wasn’t really open like “open source”, since only IBM contributed to the design process, but it was fully documented).  And with the IBM PC came a standard OS platform, MS-DOS (actually IBM offered three or four different operating systems, including CP/M and the UCSD P-system but MS-DOS was the one that took off).  In fact, Visicalc was one of the first applications ported to MS-DOS btw, it was ported to DOS 2.0. But it wasn’t until 1983ish, with the introduction of Lotus 1-2-3, that PC was seen as a business tool and people flocked to it. 

But the platform still wasn’t completely stable.  The problem was that while MS-DOS did a great job of virtualizing the system storage (with the FAT filesystem)  keyboard and memory, it did a lousy job of providing access to the screen and printers.  The only built-in support for the screen was a simple teletype-like console output mechanism.  The only way to get color output or the ability to position text on the screen was to load a replacement console driver, ANSI.SYS.

Obviously, most ISVs (like Lotus) weren’t willing to deal with this performance issue, so they started writing directly to the video hardware.  On the original IBM PC, that wasn’t that big a deal – there were two choices, CGA or MDA (Color Graphics Adapter and Monochrome Display Adapter).  Two choices, two code paths to test.  So the test cost was manageable for most ISVs.  Of course, the hardware world didn’t stay still.  Hercules came out with their graphics adapter for the IBM monochrome monitor.  Now we have three paths.  Then IBM came out with the EGA and VGA.  Now we have FIVE paths to test.  Most of these were compatible with the basic CGA/MDA, but not all, and they all had different ways of providing their enhancements.  Some had some “unique” hardware features, like the write-only hardware registers on the EGA.

At the same time as these display adapter improvements were coming, disks were also improving – first 5 ¼ inch floppies, then 10M hard disks, then 20M hard disks, then 30M.  And system memory increased from 16K to 32K to 64K to 256K to 640K.  Throughout all of it, the MS-DOS filesystem and memory interfaces continued to provide a consistent API to code to.  So developers continued to write to the MS-DOS filesystem APIs and grumbled about the costs of testing the various video combinations.

But even so, vendors flocked to MS-DOS.  The combination of a consistent hardware platform and a consistent software interface to that platform was an unbelievably attractive combination.  At the time, the major competition to MS-DOS was Unix and the various DR-DOS variants, but none of them provided the same level of consistency.  If you wanted to program to Unix, you had to chose between Solaris, 4.2BSD, AIX, IRIX, or any of the other variants.  Each of which was a totally different platform.  Solaris’ signals behaved subtly differently from AIX, etc.  Even though the platforms were ostensibly the same, they were enough subtle differences so that you either wrote for only one platform, or you took on the burden of running the complete test matrix on EVERY version of the platform you supported.  If you ever look at the source code to an application written for *nix, you can see this quite clearly – there are literally dozens of conditional compilation options for the various platforms.

On MS-DOS, on the other hand, if your app worked on an IBM PC, your app worked on a Compaq.  Because of the effort put forward to ensure upwards compatibility of applications, if your application ran on DOS 2.0, it ran on DOS 3.0 (modulo some minor issues related to FCB I/O).  Because the platforms were almost identical, your app would continue to run.   This commitment to platform stability has continued to this day – Visicalc from DOS 2.0 still runs on Windows XP.

This meant that you could target the entire ecosystem of IBM PC compatible hardware with a single test pass, which significantly reduced your costs.   You still had to deal with the video and printer issue however.

Now along came Windows 1.0.  It virtualized the video and printing interfaces providing, for the first time, a consistent view of ALL the hardware on the computer, not just disk and memory.  Now apps could write to one API interface and not worry about the underlying hardware.  Windows took care of all the nasty bits of dealing with the various vagaries of hardware.  This meant that you had an even more stable platform to test against than you had before.  Again, this is a huge improvement for ISV’s developing software – they no longer had to wonder about the video or printing subsystem’s inconsistencies.

Windows still wasn’t an attractive platform to build on, since it had the same memory constraints as DOS had.  Windows 3.0 fixed that, allowing for a consistent API that finally relieved the 640K memory barrier.

Fast forward to 1993 – NT 3.1 comes out providing the Win32 API set.  Once again, you have a consistent set of APIs that abstracts the hardware and provides a constant API set.  Win9x, when it came out continued the tradition.  Again, the API is consistent.  Apps written to Win32g (the subset of Win32 intended for Win 3.1) still run on Windows XP without modification.  One set of development costs, one set of test costs.  The platform is stable.  With the Unix derivatives, you still had to either target a single platform or bear the costs of testing against all the different variants.

In 1995, Sun announced its new Java technology would be introduced to the world.  Its biggest promise was that it would, like Windows, deliver platform independent stability.  In addition, it promised cross-operating system stability.  If you wrote to Java, you’d be guaranteed that your app would run on every JVM in the world.  In other words, it would finally provide application authors the same level of platform stability that Windows provided, and it would go Windows one better by providing the same level of stability across multiple hardware and operating system platforms.

In Jim Gosling post, he’s just expressing his frustration with fact that Linux isn’t a completely stable platform.  Since Java is supposed to provide a totally stable platform for application development, just like Windows needs to smooth out differences between the hardware on the PC, Java needs to smooth out the differences between operating systems.

The problem is that Linux platforms AREN’T totally stable.  The problem is that while the kernel might be the same on all distributions (and it’s not, since different distributions use different versions of the kernel), the other applications that make up the distribution might not.  Java needs to be able to smooth out ALL the differences in the platform, since its bread and butter is providing a stable platform.  If some Java facilities require things outside the basic kernel, then they’ve got to deal with all the vagaries of the different versions of the external components.  As Jim commented, “They are all close, but not the same.”  These differences aren’t that big a deal for someone writing an open source application, since the open source methodology fights against packaged software development.  Think about it: How many non open-source software products can you name that are written for open source operating systems?  What distributions do they support?  Does Oracle support other Linux distributions other than Red Hat Enterprise?  The reason that there are so few is that the cost of development for the various “Linux” derivatives is close to prohibitive for most shrink-wrapped software vendors; instead they pick a single distribution and use that (thus guaranteeing a stable platform).

For open source applications, the cost of testing and support is pushed from the developer of the package to the end-user.  It’s no longer the responsibility of the author of the software to guarantee that their software works on a given customer’s machine, since the customer has the source, they can fix the problem themselves.

In my honest opinion, platform stability is the single thing that Microsoft’s monoculture has brought to the PC industry.  Sure, there’s a monoculture, but that means that developers only have to write to a single API.  They only have to test on a single platform.  The code that works on a Dell works on a Compaq, works on a Sue’s Hardware Special.  If an application runs on Windows NT 3.1, it’ll continue to run on Windows XP.

And as a result of the total stability of the platform, a vendor like Lotus can write a shrink-wrapped application like Lotus 1-2-3 and sell it to hundreds of millions of users and be able to guarantee that their application will run the same on every single customer’s machine. 

What this does is to allow Lotus to reduce the price of their software product.  Instead of a software product costing tens of thousands of dollars, software products costs have fallen to the point where you can buy a fully featured word processor for under $130.  

Without this platform stability, the testing and development costs go through the roof, and software costs escalate enormously.

When I started working in the industry, there was no volume market for fully featured shrink wrapped software, which meant that it wasn’t possible to amortize the costs of development over millions of units sold. 

The existence of a stable platform has allowed the industry to grow and flourish.  Without a stable platform, development and test costs would rise and those costs would be passed onto the customer.

Having a software monoculture is NOT necessarily an evil. 

  • As an example of how different Linux distributions are I was trying to VPN into my workplace this weekend using SuSE 9.1 Professional. Using PPTP Client (http://pptpclient.sourceforge.net) and following their directions for SuSE 9.1 (my distribution, mind you) I was unable to connect to my VPN.

    They instruct you to disregard dependencies that are based on RedHat for the graphical configuration program. I did and tried to get the graphical program working. No go.

    All in all I spent nearly 6 hours playing with everything before I managed to connect. After I had connected successfully I discovered that data wouldn't send. No matter what I did, data would not traverse the newly created tunnel. My guess is that the negotiated key for data encryption is being lost somewhere along the way...

    Connecting with Windows XP takes about 45 seconds. And before anyone complains about "I bet you're connecting to other Microsoft products!" -- I'm not. I'm connecting to a WatchGuard FireBox which runs, ironically enough, an embedded version of Linux as its operating system.

    This is but one example. I've always toyed with Linux -- I have CDs for Red Hat Linux 2.1 from geez... 1995, 1996? As it has grown older it has become *more* difficult for me to use and configure because distributions change so much. Should this software install in /opt/bin or /usr/bin? Where are my configuration files? Every package becomes a nightmare to install and use -- there are no standards, and even when there are standards the distributions don't enforce them, so they become effectively meaningless.

    Of course most of this rant applies to X. On the command line, from a raw system, Linux can do what you need it to. Just don't install anything but the bare minimum and install from source. (ala Gentoo) Because you have a bare bones system you can decide where software goes, how it goes there, and why. I have seven programs to modify the volume of my sound card, but nothing to configure a VPN tunnel for my use.

    And don't get me wrong on the latest trend... naming icons on the menu after their purpose, rather than what they are. Yes, if a program has an icon of a notebook I expect it to be a word procesor of some sort. I don't need "Word Processor" spelled out in front of it. Tell me what the heck it is called: Open Office Writer. This gives me a unique description to latch onto and describe to folks, where Word Processor may be abiWord on another machine. *Not good.*

    Windows is more of an evolution of a specific vehicle. Windows NT, Windows 2K, Windows XP are all SUVs. They require a lot of gas but they have more power. Linux is more like different car models entirely. The engine is based on the same core components, and a lot of the interior is somewhat the same, but the exterior of the car, color scheme, radio type, automatic or manual... these are all decisions left to the distribution.

    It's a confusing mess. As a software developer I'm trying to hedge my bets by becoming familiar with the differences between .NET on Win32 and Mono on Linux, but so far it has been an expensive and painful battle.
  • My crystal ball tells me that you will be slashdotted in the next few hours with a headline saying "MS developer says MS monoculture is NOT evil" :-) And no, I'm not planning on submitting the story :-D
  • <em>In particular, they are not close enough that if you are a software developer, you can develop one that can run on the others.</em>

    Bzzt. Wrong.

    If you only care about <em>binary</em> compatibility, then there aren't too many cases where systems differ. You just need to know what you're doing - and many people quite evidently don't.

    If it's source-level compatibility you care about, things are generally much easier. Write your code in conformance with the published specs, rather than what you <em>think</em> the specs say, and don't mis-use feature-test macros, and you'll be fine. You still need to know what you're doing, of course, but it's a lot easier.

    Linux isn't a platform, however. The reason Richard Stallman bangs on about calling it 'GNU/Linux' isn't just ego-driven: Linux is just a kernel. It's sort of like the difference (though this isn't a fantastic analogy) between Win32 and NT - Win32 is the platform, NT is the OS.

    The biggest problem with binary-only distributions of software for Linux-based platforms is that vendors make assumptions. They assume certain configuration files will be in a specific place, when they might not necessarily be there. They assume certain libraries will both be present, and of a specific version, and that the distribution uses a particular naming scheme (rather than checking, or shipping their product with those libraries included). If they're unable to figure out a way to get around these assumptions, then they should talk to the developers of the software they're making assumptions about instead of blindly carrying on anyway, causing hell for users.
  • Mo, you've EXACTLY made my point in your comment. Because the configuration files are in different places on different distributions means that the vendor has to TEST all those different distributions.

    Testing adds to costs. Costs get passed onto users. If you're selling an application that costs $10,000 a copy, then your price supports the cost of testing all those configurations. When your app costs $129, you can't afford the test costs. The monoculture allows for the $129 word processing application, it couldn't exist without it.

    Amd senkwe: Maybe so. If /. picks this up, then so be it. I've said what I meant, and I meant what I said (An elephant's faithful 100%?)
  • It can be really frustrating to get one binary that works all the way back to Win 98 (which is what I support in my work - I can't be like MS and ignore the 9x users).

    I have to worry about different shell versions, IE verions, common control versions, bugs or different behavior among those, and on top of that know which APIs are only in 2000/XP.

    For ex, do I use SHGetSpecialFolderLocation, SHGetSpecialFolderPath, SHGetFolderPath, or...? While the existence and availability of the various APIs are well-documented, the UNavailability is not - tell me, which CSIDL_* values work on 9x? NT4? With shfolder.dll and without shfolder.dll? I have no way of knowing without writing a test app to call the SHGet* functions with all the CSIDL_* values.
  • I was enjoying your post until I got up to the bit about DRDOS not providing the consistency of MSDOS.

    Say what? Care to explain this? Apart of course from Microsoft's own deliberate code that would disable Windows from running on DR DOS.

    I ran DRDOS 6.0 and it was better than what Microsoft had at the time (MSDOS 4).
  • Mike: Hmm... You have a good point Mike, there CAN be a bewildering set of options to consider.

    With a smidge of digging (I searched for CSIDL and clicked on the 2nd link), I found this from MSDN:
    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/shellcc/platform/shell/reference/enums/csidl.asp

    The information there, combined with:
    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/shellcc/platform/shell/programmersguide/versions.asp

    appears to pretty thoroughly document which CSIDL_ versions work with which versions of the common controls, and what platforms those controls are used on.

    Is there more info that you need to know?
  • matthew, you're right, I unnecessarily tarred DR-DOS with the inconsistancy brush. DR-DOS was just as platform stable as MS-DOS.

    As long as you used DR-DOS as a MS-DOS replacement (and it was a good replacement, you are right), and stayed away from DR-DOS's multitasking extensions (which then tied you to the DR-DOS platform).
  • Mike does have a point about usability/discoverability. Why can't devs fire up MSDN, select a minimum/maximum OS for their apps to run on, and have it display only the information that they can use? It would be cool if VS could do the same with Intellisense. Imagine how much better life would be if VS suggested alternate APIs to use because of the platform(s) on which you told it you wanted to run your app.
  • Wow. Drew, that is an AWESOME idea. You can do that for C++/C#/VB in the CLR sections, it would be fascinating if the same could be done for the mainline MSDN documentation.

    Now the challenge is finding an MSDN person to suggest this to. I'll ask around.
  • “The problem is that Linux platforms AREN’T totally stable.”

    Anyone making that statement does not have a lot of experience with a Linux operating system. Our Linux servers uptime is rated in quarters / years while our Windows servers are rated in days / weeks. Check out netcraft for the the Internet's best uptime servers and you'll see that BSD/Linux servers own the top 100.

    “The problem is that while the kernel might be the same on all distributions (and it’s not, since different distributions use different versions of the kernel), the other applications that make up the distribution might not.”

    Last time I checked, Microsoft still sells a desktop version of Windows vs. a Professional vs. a Server grade operating system. Thus, different distributions of Linux are designed with different goals in mind.

    I don't think very many people will argue that the desktop market for Linux hasn't evolved yet, but the same can be argued for why Microsoft is still hasn't evolved into the server market as much as they'd like to have. Again, look for the Apache vs. IIS wars and why IIS gained strong ground in 2002 it has dropped back down to it's 2000 level.

    “Java needs to be able to smooth out ALL the differences in the platform, since its bread and butter is providing a stable platform. If some Java facilities require things outside the basic kernel, then they’ve got to deal with all the vagaries of the different versions of the external components.”

    As far as Java's bread and butter goes, Java was built to be portable not tied to a particular operating system. Java has great support in Linux/BSD operating systems and this doesn't seem to be a problem. The lava installation is a binary distribution and it's external dependencies are next to nothing. Even the desktop features are tied more into X then KDE or Gnome.

    “How many non open-source software products can you name that are written for open source operating systems? “

    Does does Sun's Java, Oracle, IBM's Lotus Notes, Word Perfect, RealPlayer, Adobe Acrobat, and VMWare not to mention at least a dozen games pop off the top of my head (Quake III even before the source code release). Not bad for an operating system that focuses on the server market more the desktop.

    “What distributions do they support? Does Oracle support other Linux distributions other than Red Hat Enterprise? The reason that there are so few is that the cost of development for the various “Linux” derivatives is close to prohibitive for most shrink-wrapped software vendors; instead they pick a single distribution and use that (thus guaranteeing a stable platform).”

    Red Hat has focused on the Enterprise market and have built a solid reputation so it's only natural that Oracle would choose that Vendor because of their business. However, while I'm not 100% certain on RedHat as being the only Linux distribution, Oracle runs on FreeBSD. I think the key here is which Operating does Oracle run better on? Linux/BSD or Windows (Search on google for that answer or ask your local friendly Oracle Rep.).
  • Jeff, platform stability doesn't refer to how long a particular computer stays up. It has to do with how much a platform changes from machine to machine, from version to version.

    Win32 applications written for XP Home run on XP server, and they run on XP pro. In general, this is true for all platforms (Mike Dunn did correctly point out that there ARE platform differences, but they're relatively limited). He's right, it CAN be a challenge writing an application using todays SDK that will run on Win95, unless you are careful to stay within the well documented boundaries of Windows as it existed in 1995. But the applications written for Win95 still run on Windows today (unless they were written with dependencies outside the Win32 API set).

    The Windows operating system IS differentiated into different products. But the Windows PLATFORM is the same regardless. If your app runs Windows XP home, your app is almost certainly going to run on XP Pro. And it's almost guaranteed to run on W2K3 server as well (it might not because W2K3 server enterprise edition doesn't have audio enabled by default, and if your app depends on audio, it might fail, that's one of the platform differences).

    You cannot make the same statement about differing Linux distributions. That's what Jim Gosling was complaining about - Java VM's have to be tested on every possible Linux distribution, because Java is a shrink-wrap solution. And that testing is expensive.


    Going through the rest of your list of non open source Linux applications...

    The Real Networks player is described as being a "user supported" player. From the web site, it looks like it's an open source distribution, so it doesn't count - it's an open source product, not closed source.

    Adobe Reader does appear to be a shrinkwrapped copy, so that one is very real. They also don't charge for it, so it's clearly a loss-leader (like the free reader is for Windows), so they're clearly recovering the costs of development of the free player somewhere else. They appear to only support Linux 2.2 on x86 computers, fwiw.

    I looked at the Corel web site and couldn't find any indication where I could get a copy of Word Perfect for Linux. I found press releases announcing that they were doing it here (http://www.corel.com/servlet/Satellite?pagename=Corel2/Products/Home&pid=1047023021963), but no software. It appears that they stopped development at the 0.9 version.

    VMWare appears to be an operating system, although it does support Linux management console machines. This IS a good example of a closed source Linux product, since the management console for Linux costs only $199, and claims to support most versions of the 2.2 or 2.4 Linux kernel.

    Lotus Notes supports ONLY the following Linux versions:Red Hat Enterprise Linux AS 2.1 (Uni- processor only), and UnitedLinux 1.0. They don't support "Linux", they support two Linux distributions, RHEL, and UnitedLinux 1.0.

    The only Oracle platform I could find supported was RHEL, but there may be others, I just couldn't find them - it may be on the Oracle web site, I just couldn't find ANY indication of the platforms on which Oracle is supported - all the other vendors above listed their supported versions without too much effort, I could not make the same statement about Oracle.

    And on which platform Oracle runs better is irrelevent to my article. My thesis is simply that in the absence of a stable platform, the cost of software development is passed onto the consumer. Oracle charges tens of thousands of dollars a copy for their software, the cost of cross platform development and testing is built into the cost of their product.

    This article isn't about whose operating system is better. It's about the cost of software development and who pays for it. If you have a single stable (unchanging) platform, then the software vendor can reduce their development costs and produce software for far less than if your have to support multiple platforms. When software vendors costs go down, then those cost savings get passed on to the consumer.

    If you don't have the single stable platform then you need to test all the varients of the platform. That adds to the cost of writing software. If the software vendors costs are higher, they will pass their costs onto the consumer, and software cost goes up.

    So I come back to my original conclusion: The Microsoft Monoculture has enabled cheap software. If the monoculture didn't exist, software wouldn't be as cheap.
  • Jeff, sorry, but I have to call BS on that.
    Yes, I can make linux run for a long time on a particular kernel, but I have to be extremely careful about what patches and hardware I use. Even then, I ran into bugs. The Linux VM does have certain race conditions.

    Larry is talking about the platform of Linux in general, and he is right. If you look at a lot of apps for Linux, they have a ton of dependencies on specific versions. If you run an older redhat, you can't even do a source level build with some of these systems.

    Note, I have used just about every form of Unix, VMS, Windows, etc.... he's right.

  • Larry, AFAIK, win32g was the (very early)precursor to DirectX (and also seems to be a virus payload), while win32s was the 32-bit subset for Windows 3.1 and 3.11.

    http://www.webopedia.com/TERM/W/Win32s.html

  • There is more stability in the Windows world than in the Linux world, but it's nowhere near as bad as you make out.

    90% of applications will use the same (or mostly the same) kernel calls, and a version of libc. The kernel has compatibility back to very very very old versions, glibc has compatibility back to very very very old versions of glibc.

    If you use hidden features of glibc, yes, you will be screwed. But you know what, your Linux distributor can do the equivalent of compatibility shims by forcing you to load a specific library before you run.

    If instead of just assuming that a file is in /usr/bin, you have an entry in a config file "this application is located here", then you can run on pretty much any Linux platform. (Note, you should probably make it possible to load your config file from an arbitary location)

    The only key exception I can think of for most vendors is a) changing APIs for UIs (which can still run, you just need to install the (older) libraries), and b) kernel modules, which rely on the internals of the kernel to use.
Page 1 of 5 (69 items) 12345