Time for my annual blog post. :)   :(

> Hi,

Hi!

DSLs are notations that are specifically tuned to expressing ideas in a particular world or for a particular purpose. So when you cook up an XML schema, you're creating a DSL. DSLs can take the form of text or graphics - it depends what is more appropriate for the application. For example, graphical DSLs include the notations used to design user interfaces in .NET or Java.

People use DSLs for a variety of purposes, including: expressing ideas in a particular field, for people to communicate with each other or with computers; transmitting information between programs; and generating program code (http://www.bing.com/search?q=code+generation).

Well-known examples of DSLs include:

  * Special purpose programming: Regular expressions; SQL. For manipulating text strings and databases, these languages are a lot more expressive than general purpose programming languages like Java or C#.

  * Specific businesses: The languages used to transmit money between banks and to book airline tickets. These languages are used to communicate between computer systems, but it's important that humans can also read them.

So why would people want to create their own DSLs? For the same reasons: transmitting or storing specialized information; and writing specialized parts of computer programs. Transmitting information has fairly obvious advantages, so let's think about using DSLs to write programs.

Some years ago, I worked with a company that wrote software for creating telephone bills. Their clients were phone operating companies. The software was written in Fortran - they had been running for many years!  I was told that when they first started, the job their software had to do was very simple: for each customer, it just added up the cost of each call and printed a bill for the total. But when mobile phones arrived, things got more complicated. The competition between the phone companies meant that they evolved complex discount schemes, to try to make their service look cheap, while still making money. This made things difficult for my colleagues writing the billing programs - not only did they have to understand these complex schemes, but they also had to install a different scheme for each of their clients. And to make it worse, they needed to be able to change the scheme every few months: the phone operators would regularly invent new methods of calculating their bills.

The company's analysts, whose job was to understand the requirements, had evolved a way of using diagrams to express the charging schemes. They showed a kind of flowchart, where each phone call went through different filters and was added to different totals. The diagrams made it easy to discuss the scheme with their clients, the phone operating companies. At first, the analysts would pass these diagrams on paper to the programmers, who would then modify the Fortran. But it was very unreliable to update the Fortran every few months, and to maintain a different version for each client.

Then someone had the bright idea of using the diagrams directly to generate the program code. That enabled them to have just one application, instead of many, and it meant that the software was easy to update without risk of programming errors. It wasn't just easy, though: they had to refactor their software quite a lot to separate the fixed parts from the variable parts, which would be generated from the diagrams. But it was good to do that anyway, because it improved the quality of the programs,  and they fixed some long-standing bugs along the way.

One question that the software developers discussed a lot was whether they should use the diagrams to generate Fortran, which would then be compiled; or should they write a Fortran application that would interpret the diagrams at run-time. The first alternative had the advantage that it was easy to develop from their existing source code: they created a tool that took the diagrams and generated the variable parts of the source code. The second alternative took more effort to develop, and tended to run slower; but it had the advantage that it was possible to update the bill calculation scheme on the client's site. It was even possible for the same software to run different schemes at the same time - which was good for one of their clients that operated more than one brand, with different charging schemes.

This story illustrates one of the nice things about DSLs (whether you generate code from them or interpret them): they narrow the gap between the customer's requirements and the software, making the development team much more agile in its response to changing requirements. Table driven software has been around for a long time of course, so the idea of generating or interpreting from a specialized notation isn't particularly new. But a good DSL is one that the software's users can understand. Effectively, you're giving the users the means to update the software for themselves. From that point of view, DSLs in graphical form are particularly useful, because they're easier non-programmers to understand, and easier to discuss on the whiteboard.

A drawback of DSLs is that you have to develop them! Designing a language can be quite expensive, even with something like the Microsoft DSL toolkit (http://code.msdn.microsoft.com/vsvmsdk). But if you know that you will have many variants of the software you are developing, a DSL can provide a good return on the investment.

For many applications, you can reduce the investment by adapting an existing notation, such as one or more of the UML diagrams. This has the advantage that you can use existing UML tools (http://msdn.microsoft.com/en-us/library/57b85fsc(VS.100).aspx), and your development team are likely to be familiar with the notation already. For example, the phone bill company might have used a UML activity diagram - perhaps using stereotypes to distinguish different kinds of operations. Then you only need to write the software that generates the variable parts of the source code from the UML. There's some material about this here: http://msdn.microsoft.com/en-us/library/ee329484(VS.100).aspx

Some projects go through a number of stages: they start by writing a specific application. Then they make some of that application variable, generating parts of its source code from some UML diagrams. They might add stereotypes to customize the UML for their purposes. They will customize the UML tools, adding commands that generate common patterns, and adding validation rules to make sure you only draw diagrams that make sense for this particular application. In a complex application, they might use more than one type of diagram in an integrated way. They might create their own DSL for aspects that aren't suited to the UML notation.  In most of the cases I have seen, DSLs are used to  generate parts of the application, rather than the whole thing.

Each of these steps requires more investment, but brings benefits. Generated code is usually reliable (after you have tested the generation process!), and it is easy to update the DSL. So at each step, there is a cost assessment to be made. If you are only writing a single small application with a short life, it is probably easier to code it directly. If you are creating a product line or a large product with a long life, DSLs are a good option.

Hope this is useful. Good luck with the master's.