elisp - Emacs lisp: why does this sexp cause an invalid-function error?

Question

The sexp in question is

(((lambda (b)
  (lambda (a)
    (+ b a))) 3) 5)

which, to me, looks like it should evaluate to 8, and in other lisps (e.g. Racket) it does, but in elisp it instead throws this error:

Debugger entered--Lisp error: (invalid-function ((lambda (b) (lambda (a) (+ b a))) 3))

It appears to be telling me that

((lambda (b)
  (lambda (a)
    (+ b a))) 3)

Is not a valid function. This seems wrong, because when I evaluate that expression I get

(lambda (a) (+ b a))

which looks like a valid function to me. Does anyone have any idea why this happens? Does it have something to do with dynamic scoping?

Answer 1

There are two issues here. The first is a matter of syntax, as other answers have pointed out. The second, which is mentioned in the question, is a matter of scoping issues.

The Syntactic Issue

In Emacs Lisp (and other Lisps in the Lisp-2 family), a function call looks like (f args...) where f is a either a symbol that has a function value, or a lambda expression. For instance,

(list 1 2 3)  
=> (1 2 3)

because list has a function binding. Also,

((lambda (x y) (list x x y y)) 1 2)
=> (1 1 2 2)

because (lambda (x y) (list x x y y)) is a lambda expression. What you can't do, though, is use something the value of which of is a function.

(let ((id (lambda (x) x)))
  (id 3))

signals a Lisp error: (void-function id). But we can call function values using funcall:

(let ((id (lambda (x) x)))
  (funcall id 3))
=> 3

Note: That's a reasonably good way of looking at it, but in fact things are a little more complicated. See 9.2 Kinds of Forms in the manual for the details and esoteric bits such as function indirection.

So, now we can address the syntax issue. The original code, reformatted a bit to indicate which functions are getting which arguments, is:

(((lambda (b)
    (lambda (a)
      (+ b a)))
  3)
 5)

As I understand it, the intent is to first call (lambda (b) ...) with the argument 3 to get back an anonymous function, (lambda (a) ...). In Emacs Lisp that would be:

((lambda (b)
   (lambda (a)
     (+ b a)))
 3)
=> (lambda (a) (+ b a))

Now, you also want to call the returned anonymous function with 5. We use funcall to do that:

(funcall ((lambda (b)
            (lambda (a)
              (+ b a)))
          3)
         5)

The Scoping Issue

Disappointingly, this code produces a Lisp error: (void-variable b). This is where we finally run into the issue of dynamic versus lexical scoping. Because the variable b was bound dynamically, its value is not preserved in the anonymous function (lambda (a) (+ b a)). We can check to see that this is what's happening by surrounding the whole form in something that binds b and seeing what happens:

(let ((b 100))
  (funcall ((lambda (b)
              (lambda (a)
                (+ b a)))
            3)
           5))
=> 105

I'm not much of an Emacs Lisp hacker, so I'm not sure of the best way to get lexical closures in Emacs. I read that Emacs 24 has it, but I'm still on 23 here. Based on this answer, though, we can use lexical-let to get the results we need:

(funcall ((lambda (b)
            (lexical-let ((b b))
              (lambda (a)
                (+ b a))))
          3)
         5)
=> 8

lexical-let establishes the lexical binding that we need, so that the anonymous function (lambda (a) ...) does have that 3 stuck into it. More specifically, we introduced a lexical binding of b, and it is that lexical binding that (lambda (a) …) references. In fact, if we look at the returned anonymous function now, it's not simply (lambda (a) (+ b a)), but is printed in a more complex (and less useful) way:

((lambda (b)
   (lexical-let ((b b))
     (lambda (a)
       (+ b a))))
 3)
=> (lambda (&rest --cl-rest--) (apply (lambda (G27322 a) (+ ... a)) (quote --b--) --cl-rest--))

As an aside, it doesn't matter that the lexically-bound variable b has the same name as the dynamically bound b; we could have used (lexical-let ((c b)) ... (+ c a) ...), instead.