Raising the level of software development abstraction
Have you ever wondered how it is possible that we as an industry perform so poorly? According to The Standish Group, the majority of all IT projects fail to complete within budget and on time. This of course comes as no surprise to anyone who has been involved with commercial software development projects, it is a known "fact" that we have a constant struggle with the quality of the software we produce. This state of current situation cannot continue forever, we have to grow up as an industry and come up with better ways to produce software. Historically, one answer to this has been raising the level of abstraction. A good example from the previous years is the introduction of virtual memory - instead of dealing with hardware memory pages the applications see a virtualized view of the memory. The virtual memory can be extended into using hard disk space if physical memory is low. Similar types of abstractions have advanced developers' lives ever since, such as object oriented languages and virtual execution environments.
We all know that the range of developer productivity is huge. Sure, there are some amount of brilliant developers who produce the majority of deliverables in a development project. Similarly, there is a set of developers at the low end of the productivity scale - there are even developers who have negative productivity (e.g. somebody is capable of producing more bugs than the other developers have time to fix!). However, most developers fall to the category of "average" productivity. If we are going to raise the productivity of software development projects as a whole, we clearly need to make better use of the brains of brilliant developers. This leads into the idea of frameworks and reusability. It is far more effective to have the highly skilled and experienced developers to write sets of reusable components that the other developers use as their tool set. This is of course nothing new as an idea, but if you take a look at average custom software projects it is amazing how many of them are built as one-off projects.
With one-off projects I mean development projects where by far the majority of the code is developed as a unique solution essentially from scratch. Sure, there probably are a set of reusable components involved (typically at least user interface controls), but usually there is no concept of a reusable application architecture. This idea of reusable architecture can also be viewed as software product line -thinking, where we already at the design phase are taking into consideration the possibility of making multiple "copies" of the same implementation. With multiple copies I mean producing multiple instances of the same solution for roughly the same problem. Of course it will not be copies of exactly the same bits, but the overall type of solution is the same. The differences that can be found between different solutions need to be identified and implemented into the product line as configurable settings or pluggable implementations.
This is more or less what we (Microsoft) are targeting with software factories and domain specific languages (DSLs) in Visual Studio 2005 and beyond. Jack Greenfield and Keith Short have written an excellent book called "Software Factories" (link to amazon), take a look at it! We are also implementing a set of DSL -based modeling tools as part of Visual Studio 2005, they are already part of the current Visual Studio 2005 Beta versions, which you can get from here. Also, the DSL implementation in Visual Studio is going to be extensible (of course!), we already have a public preview of the DSL tools here that works with Visual Studio beta 1. There are also some good blog entries about Visual Studio modeling features, such as this one.
The idea in DSLs is that we have languages for each of the different problem domains. We recognize the fact that it is useful to be able to describe the exact behaviour of a certain module instead of having just an overview scetch of the functionality. I fact, the DSLs and models that are based on those need to be exact enough so that we can compile them directly into working application. Programming language does not always necessarily need to be a textual one, the models are examples of graphical programming languages. And of course, the models are stored as XML in files on disk so that they can be easily versioned with current version control systems as well as transformed into different types of diagrams should the need arise.
Hopefully software factories is the start of a new era of better developer productivity! Time will tell whether that happens, but they have very much potential in doing that IMHO!
One thing to add: as a Finn, I have to mention also that there is a long-time DSL veteran company based in Jyväskylä, Finland called MetaCase.
