I'm so special (11 Feb 2006)

The large application which I help to develop has an embedded Lisp interpreter and compiler, and over time I also left my marks in that subsystem. It was only after a considerable amount of tinkering with the innards of the interpreter that my insights into Lisp finally reached critical mass. I guess I understand now why Lispniks are so devoted to their language and why they regard all those other languages as mere Lisp wannabees.

While learning Lisp, bindings and closures were particularly strange to me. It took me way too long until I finally grokked lexical and dynamic binding in Lisp. Or at least I think I get it now.

Let us consider the following C code:

  int fortytwo = 42;

  int shatter_illusions(void)
  {
    return fortytwo;
  }

  void quelle_surprise(void)
  {
    int fortytwo = 4711;
    printf("shatter_illusions returns %d\n", shatter_illusions());
  }

A seasoned C or C++ programmer will parse this code with his eyes shut and tell you immediately that quelle_surprise will print "42" because shatter_illusions() refers to the global definition of fortytwo.

Meanwhile, back in the parentheses jungle:

  (defvar fortytwo 42)

  (defun shatter-illusions()
    fortytwo)

  (defun quelle-surprise()
    (let ((fortytwo 4711))
      (format t "shatter-illusions returns ~A~%" (shatter-illusions))))

To a C++ programmer, this looks like a verbatim transformation of the code above into Lisp syntax, and he will therefore assume that the code will still answer "42". But it doesn't: quelle-surprise thinks the right answer is "4711"!

Subtleties aside, the value of Lisp variables with lexical binding is determined by the lexical structure of the code, i.e. how forms are nested in each other. Most of the time, let is used to establish a lexical binding for a variable.

Variables which are dynamically bound lead a more interesting life: Their value is also determined by how forms call each other at runtime. The defvar statement above both binds fortytwo to a value of 42 and declares the variable as dynamic or special, i.e. as a variable with dynamic binding. Even if code is executed which usually would bind the variable lexically, such as a let form, the variable will in fact retain its dynamic binding.

"Huh? What did you say?"

1. defvar declares fortytwo as dynamic and binds it to a value of 42.
2. The let statement in quelle-surprise binds fortytwo to a value of 4711, but does not change the type of binding! Hence, fortytwo still has dynamic binding which was previously established by defvar. This is true even though let usually always creates a lexical binding.
3. shatter-illusions, when called, inherits the dynamic bindings of the calling code; hence, fortytwo will still have a value of 4711!

Kyoto Common Lisp defines defvar as follows:

(defmacro defvar (var &optional (form nil form-sp) doc-string)
  `(progn (si:make-special ',var)
          ,(if (and doc-string *include-documentation*)
               `(si:putprop ',var ,doc-string 'variable-documentation))
          ,(if form-sp
               `(or (boundp ',var)
                    (setq ,var ,form)))
          ',var))

In the highlighted form, the variable name is declared as special, which is equivalent with dynamic binding in Lisp.

This effect is quite surprising for a C++ programmer. I work with both Lisp and C++, switching back and forth several times a day, so I try to minimize the number of surprises a much as I can. Hence, I usually stay away from special/dynamic Lisp variables, i.e. I tend to avoid defvar and friends and only use them where they are really required.

Unfortunately, defvar and defparameter are often recommended in Lisp tutorials to declare global variables. Even in these enlightened times, there's still an occasional need for a global variable, and if you follow the usual examples out there, you'll be tempted to quickly add a defvar to get the job done. Except that now you've got a dynamically bound variable without even really knowing it, and if you expected this variable to behave like a global variable in C++, you're in for a surprise:

  > (print fortytwo)
  42
  42
  > (quelle-surprise)
  shatter-illusions returns 4711
  NIL
  > (shatter-illusions)
  42
  > (print fortytwo)
  42
  42

So you call shatter-illusions once through quelle-surprise, and it tells you that the value of the variable fortytwo, which is supposedly global, is 4711. And then you call the same function again, only directly, and it will tell you that this time fortytwo is 42.

The above code violates a very useful convention in Lisp programming which suggests to mark global variables with asterisks (*fortytwo*). This, along with the guideline that global variables should only be modified using setq and setf rather than let, will avoid most puzzling situations like the above. Still, I have been confused by the dynamic "side-effect" of global variables declared by defvar often enough now that I made it a habit to question any defvar declarations I see in Lisp code.

More on avoiding global dynamic variables next time.

• Practical Common Lisp: Functions
• The Idiot's Guide to Special Variables and Lexical Closures

to top