Can Smalltalk blocks be better than Lisp Macros?
April 2, 2005, 7:22:10 pm

Ever curious about languages and language design - and always wanting to make my own, I've been investigating what makes "Lisp so darned good" with a friend. The opinions expressed here are my own - I don't know if my friend will agree with them or not. But I'm going to compare Smalltalk's use of BlockClosure's to Lisp's use of Macros and explain why in the end Smalltalk's approach is superior.

First, I need to explain why they are the same. So I'll do that with a nice practical example. Say you want to open a file, do something with it, then have it close once you're done. The regular code to do this would be something like:

file open: 'blah.txt'. file doSomething. file close

This has some disadvantages. If the doSomething fails, will the close ever happen? So, the code should be rewritten as:

file open: 'blah.txt'. [file doSomething] ensure: [file close]

Hey, this doesn't even handle the case where the file may not exist! So again, we have to rewrite the code to do this:

[file open: 'blah.txt'] on: FileNotFound do: [:sig | ^self]. [file doSomething] ensure: [file close]

Now, most programmers would take that pattern and copy+paste it around thinking they have finally got the answer. What if they don't? What if you need to change it again universally for all files. What if your input starts becoming a URL instead of a local machine file name. Too bad! You've Copy+Paste'd the code every where. In-steps where Macros and Blocks meet:

openFile: filename ifNotFound: notFoundBlock do: aBlock

| file |

[file open: filename] on: FileNotFound: [:sig | ^notFoundBlock value].

[aBlock value: file] ensure: [file close]

And to use it, we write:

self openFile: 'blah.'txt ifNotFound: ["do nothing"] do: [:file | file doSomething]

In Lisp, you'd also write such a macro. In Lisp it'd be called with-filename or something along those lines and it'd be written as such:

(defmacro with-openfile (filename, &body body) `let (file (open-file (filename))) body (file) close-file (file))

I'm no lisp expert and I don't know if the code above works, but the intent should be clear enough :)

So! What's different between using Macros and Blocks. First off, Lisp can do exactly what Smalltalkers do - except to use their function they have to write 'lambda' in front of the argument every time. Smalltalkers enjoy a syntax sugar for lambda which is [].

The macro does -not- take a lambda block as its argument. Instead, it rewrites and inserts code in to the calling location. This has two side effects:

  1. The resulting code will naturally run faster
  2. The resulting code will be much harder to debug

Instead of debugging you call to open a file and doSomething, you're suddenly debugging all the stuff you wanted to hide. In the Smalltalk block solution, the debugger lets you skip past all the code in openFile:notFound:do: and go straight to the block of code that is relevant to your program.

This is a huge win situation - it makes the land you live in - the code - easier to explore and work with. But, there is a big draw back. It runs slower.

Normally, code that runs slower shouldn't matter. Computers are fast and 'optimise' is the last step any programmer should take when developing software. So am I suggesting already that the only reason lisp macros win over Smalltalk blocks is speed?

Not quite. I think Smalltalk can win there too!

In a self-optimising environment, the block code should be 'unwound' or 'inlined' back in to the calling code. So should the code that calls the block. This results in a flattening of the code structure - exactly like what the macro does. Naturally, this requires a sophisticated dynamic inlining optimiser - but such a thing isn't out of reach.

There is one more disadvantage I'd like to sprook with the Lisp Macros. It's the "thought" expense. As a developer - why do I have to decide if a "piece of code" is a macro or a method? In the Smalltalk way of doing things, you don't. There are no macros and with the right optimisations important bits of code end up running like a macro any way.

In lisp though, the developer is required to -choose- at that point whether they are making a macro or a function. This is akin to having to decide if your number should be an Integer or a Float or a Fraction or a Double or.. something else. You shouldn't have to make these -optimisations- ahead of when they are required.

By Vincent Foley on April 2, 2005, 10:29:53 pm

I thought about macros and blocks when I had nothing to do about a month ago. In the end, I prefer blocks over macros.

By Michael Lucas-Smith on April 2, 2005, 10:43:46 pm

Thanks, that's a good read too. I didn't realise that there would be so much 'noise' in the lisp macro usage. Keyword selectors are generally never going to be as concise - but they are more maintainable and readable. That's worth more to me than perl-isation.

By Hmm on April 2, 2005, 11:14:00 pm

April joke?

By Michael Lucas-Smith on April 2, 2005, 11:50:46 pm

Nope, no joke.

By Vassili on April 3, 2005, 1:55:16 am

I started adding a comment here, but ended up writing enough for a separate post.

By jarober@gosmalltalk.com on April 3, 2005, 10:54:28 am

Trackback from: Smalltalk Tidbits, Industry Rants

Link: Discussing macros and closures by jarober@gosmalltalk.com

By Peter Seibel on April 3, 2005, 11:55:43 am

While with-open-file is often held up as a fine example of a Lisp macro (and it is) it is far from the end of what all you can do with macros. More complex macros let you essentially embed domain specific languages into your Lisp programs. For some examples of the latter kind of macro you can check out Practical Common Lisp in particular chapters 24, 30, and 31. As others have noted, Lisp macros and Smalltalk blocks are quite different language features--it just happens that there is some overlap between what can be implemented with macros and blocks.

By Alex on April 3, 2005, 12:44:36 pm

For probably one of the best explanations of why Lisp Macros are different (from closures) see Paul Graham's On Lisp.

In a nutshell, it appears that most of what you have described is why writing [...] is better than (lambda () ...)? Macros do remove the need to type lambda and a whole bunch of other things).

So why is a Lisp Macro different? It is (roughly) a function that generates code that will be expanded into the current context at compile time (accept in Smalltalk terms?).

Understanding that you can do (just about) anything in any language (after all, it is all done in assembler anyway) macros are written to save typing (and often make code clearer) (and there are good and bad macro writers!).

Consider the Lisp cond (compact, clear, flexible, minimal typing...) and consider what you must do in Smalltalk (lots of nested ifTrue?). If cond did not exist in Lisp, I can easily write it. How do you that in Smalltalk? That is just one example of what macros do.

You point out that you do not like having to decide whether to use functions or macros. The answer will almost always be - use a function unless nothing else will do. In other words you use a macro when there is no other way to do it.

By James Robertson on April 3, 2005, 1:40:22 pm

Comment by James Robertson

Alex,

While I get your point about cond in Lisp, the particular issue is o ne I do run across in Smalltalk, and I don't need to resort to nested ifTrue:ifFalse: statements. How? If I find that I am nesting such boolean comparisons, I simply add a common message to all the objects in question, and then send it along to them. I think all this means is that Smalltalk skins the same cat differently - it's neither better nor worse. I'd recommend a look at Vassili's post on the subject for more.

By Alex on April 3, 2005, 2:49:01 pm

James,

Thank you, I did read Vassili's comments first.

As for polymorphic behavior, that is a totally different discussion. Lisp can, through generic functions, specialize on multiple arguments. So if the issue was about coding such a situation, a good Lisp programmer would have suitable, and more flexible than Smalltalk, options. However, I do not want to get into a Lisp vs. Smalltalk war here - lots of battles, no winners!

The point of my post was that if you want to do something like cond, you can do so easily in Lisp - that is, add a new linguistic element. The resulting code is clean, concise and performs well. It was merely an example of a use of Macros, not meant as a discussion of uses of conditionals.

By Michael Lucas-Smith on April 3, 2005, 5:34:30 pm

Comment by Michael Lucas-Smith

This is one case where Smalltalk's keyword selector syntax actually gets in the way. It's very hard to invent new syntaxes in Smalltalk unless they 'look' like a keyword selector. Interesting.

By Brad Might on April 4, 2005, 3:40:18 pm

Common Lisp...the industrial strength version you would want to compare Smalltalk to, already has a with-open-file macro:

http://www.lispworks.com/documentation/HyperSpec/Body/m_w_open.htm

(with-open-file (stream filespec options*) (do-something-with-file stream))
First off, you are confusing anonymous functions/closures with macros.

yes, lisp anonymous functions (smalltalk blocks) are introduced via lambda

(lambda (x y) (perform-operations x y)
I will agree that that is more verbose than :x :y | x doSomething: y. Point made...apples to apples.

with-open-file would be implemented as 

(defmacro with-open-file ((stream filespec &optional options) &body body)
  (let ((stream (open filespec options)))
    (unwind-protect   ;;; <---- your 'ensure' option
      ,@body
      (close stream))))
Note: no need for lambda in here.
By Niall Ross on April 15, 2005, 8:03:30 am

Re adding new linguistic elements to Smalltalk, avoiding nested ifTrue:'s, etc., is this discussion of how easy it is to add a case statement to Smalltalk relevant?

By Josh Staiger on May 1, 2005, 3:54:14 pm

Responding to Brad above,

Wouldn't the macro for with-open-file have a &rest parameter for options, instead of an &optional parameter? 

(defmacro with-open-file ((stream filespec &rest options) &body body)
  (let ((stream (open filespec options)))
    (unwind-protect   ;;; <---- your 'ensure' option
      ,@body
      (close stream))))

In case you're wondering, I was looking for the source for this macro definition and happened across this entry.