DSL = Metalinguistic Abstraction
July 27th, 2007 at 2:23 pmThe idea of Domain Specific Languages (DSL) has been lately brought into attention by the freshly popular programming language Ruby. In particular, the famed web framework Ruby On Rails, which is built as a DSL on top of Ruby has been a prime example.
What most people are not aware of, however, is that DSLs is just a new incarnation of the very old concept born in the world of Lisp called Metalinguistic Abstraction. It is a recurrent theme in the seminal MIT textbook, Structure and Interpretation of Computer Programs (SICP) that was written in 1985 to present general computer programming concepts using the Scheme language (member of the Lisp family).
Metalinguistic Abstraction is the process of solving programming problems by creating a new language that is better suited to tackle these problems. Unlike the traditional software engineering process of dividing problems to exactly defined subproblems and solving those subproblems, metalinguistic abstraction instructs the programmer to imagine a new programming language in which the solution of these problems can be expressed most naturally, and then implement this language.
While this sounds a bit detached, Rails actually provides a good, modern example of how such a concept can work. The designer of Rails asked himself “what language do I need to write web applications more naturally”. He then went forward to implement this language on top of Ruby. Naturally, he could have created a completely new programming language. However, building it on top of an existing system provides several powerful benefits:
- You don’t need to craft a parser for the new language.
- You don’t have to make up new syntax, just use the existing syntax of the language you’re building on top of.
- Most importantly - since you are just building a layer on top of an existing programming language, all the facilities and power of this language are available to you, for free. This greatly enhances the immediate usefulness of the new DSL.
Ruby is a good language to build DSLs in, and Rails is a prime example of this. Ruby is better in this task than the other popular languages - Perl, Python, C++, Java, C#, etc. However, it is much inferior to Lisp. This is in no way to diminish Ruby’s success or start a language war, I’m simply stating a fact. Lisp has some unique concepts that make metalinguistic abstraction natural, namely uniform syntax and macros.
Since metalinguistic abstraction is such a powerful programming technique, it is highly recommended for any programmer to learn Lisp (whether it is Common Lisp or Scheme doesn’t really matter). Even if you’ll never use it for any official project, just the act of learning it and understanding the programming techniques possible in it will surely expand your mind and make you a better programmer.
Related posts:
July 27th, 2007 at 7:48 pm
The “embedded DSL” approach you describe is also very popular in Haskell (www.haskell.org). The use of monads to describe the way in which side effects are propogated through a computation is particularly powerful, although people often find it hard to get their heads around at first.
Paul.
July 27th, 2007 at 8:06 pm
You reuse the syntax of the host language, you reuse the semantics of the host language, and so your choices are the names of the functions, methods, and classes.
How in the world is that creating a new language? How in the world is that not exactly what Perl, Python, C++, Java, PHP, and yes Ruby programmers do all day long every day — that is, writing and using APIs?
July 27th, 2007 at 11:02 pm
> How in the world is that creating a new language?
It isn’t. But it does make LISP programmers feel better about themselves.
“We don’t use your dirty APIs. We have advaced Metalinguistic Abstraction techniques that you cannot possibly comprehend.”
Consider this passage:
“Unlike the traditional software engineering process of dividing problems to exactly defined subproblems and solving those subproblems, metalinguistic abstraction instructs the programmer to imagine a new programming language in which the solution of these problems can be expressed most naturally, and then implement this language.”
And then this one:
“Most importantly - since you are just building a layer on top of an existing programming language, all the facilities and power of this language are available to you, for free. This greatly enhances the immediate usefulness of the new DSL.”
The author is claiming that DSLs are different than traditional layering and abstraction techniques, and then turning around and saying how great their DSL is because it’s layered on top of a base langauge.
July 28th, 2007 at 2:25 am
Good article Eli. I especially like the way you summarized this…
Ruby is a good language to build DSLs in, and Rails is a prime example of this. Ruby is better in this task than the other popular languages - Perl, Python, C++, Java, C#, etc. However, it is much inferior to Lisp. This is in no way to diminish Ruby’s success or start a language war, I’m simply stating a fact. Lisp has some unique concepts that make metalinguistic abstraction natural, namely uniform syntax and macros.
More about Lisp vs. Ruby/Python focusing on one of Lisp’s unique features, namely recursion.
July 28th, 2007 at 6:00 am
Specifically, I thought that languages that are good for DSLs are capable of delayed processing semantics… ruby blocks, functional programming, anonymous functions, macros, or futures and promises. While chromatic refers to making sensible function names, good DSLs tend to have a unique order of processing, which can’t be inferred from code reading. For instance, Rake (Ruby’s equivalent of Makefile) introduces dependency semantics. This makes it most unlike an imperative language any more.
July 28th, 2007 at 6:09 am
> For instance, Rake (Ruby’s equivalent of Makefile) introduces dependency semantics. This makes it most unlike an imperative language any more.
Any language with higher-order functions has delayed processing semantics. Any language that can walk a directed graph can have dependency semantics. Rake exploits zero language features that Ruby does not have on its own; Rake uses those same language features as would any program which implements the same algorithm.
Rake is unlike an imperative language because Ruby is unlike a purely imperative language, not because Rake is some mythical DSL! Rake shares the syntax and semantics of all other Ruby programs. If you understand Ruby but can’t divine what a Rake program does at first glance, that’s because you don’t understand the language of its domain, not because it’s a DSL.
That’s a hugely important distinction.
July 28th, 2007 at 7:09 am
chromatic:
It isn’t *different* from the way the other languages do it. It just fits the concept better. I’ve used an written DSLs in Perl, Ruby and Lisp. In Lisp it is much simpler and much more natural, especially due to macros that are unavailable in other language. And what makes macros possible is the uniform syntax of Lisp.
Jonathan Allen:
I think you’re confusing metalinguistic layering with procedural / object oriented layering. This is the distinction between top down and bottom up design. A good article on this topic is Paul Graham’s “Programming Bottom-Up”:
http://www.paulgraham.com/progbot.html