CS 3540 Reading: Local Recursive Functions

`letrec` as Syntactic Sugar

A Wrinkle in Code

the front cover of the book A Wrinkle in Time, by Madelein L'Engle. — A Wrinkle in Time
Source: Wikipedia

In your reading assignment for today, we learned that we can use a let expression to create a local function in Racket. Function names are like any other variable bindings in Racket, so we don't need any special mechanism; let does the trick.

In Session 18, we asked whether this works for recursive functions, too.

Consider the case of list-index. It is an interface procedure that calls a recursive helper function, list-index-with-count, to do its work. No other function will ever need to call list-index-with-count; it exists only to do the recursive work of list-index.

That sounds like it is a good candidate to be a local function:

(define list-index
  (lambda (target los)
    (let ((list-index-with-count
            (lambda (target los base)
              (if (null? los)
                  -1
                  (if (eq? target (first los))
                      base
                      (list-index-with-count
                            target (rest los) (+ base 1)) )) )))
      (list-index-with-count target los 0)) ))

But trouble ensues... Dr. Racket doesn't even give us a chance to execute the function. Its type checker displays an error even before we can load the file successfully:

list-index-with-count: unbound identifier in module in: list-index-with-count

What went wrong?

When I ask you a question, you usually know enough to figure out the answer. This time, the answer is the same one we found in Session 17's opening exercise, when y and z could not be initialized as we expected:

The region of the name list-index-with-count is the body of the let expression.

The first call to list-index-with-count, within the body of the let expression, is fine. But list-index-with-count is recursive and calls itself from within its own body. The body of the let does not include the definition of list-index-with-count itself! The recursive call within its body uses an undefined name.

This is perfectly clear if we translate the let expression into a semantically-equivalent application of a nameless lambda:

(define list-index
  (lambda (target los)
    ( (lambda (list-index-counted)
        (list-index-counted target los 0))
      (lambda (target los count)
        (cond ((null? los) -1)
              ((eq? target (first los)) count)
              (else (list-index-counted target (rest los)
                                                (+ count 1))))) )))

This code declares list-index-with-count as formal parameter on a function. We pass the body of the function as an argument using a nameless lambda. But this function calls what used to be named list-index-with-count! When we call the nameless function that is created, list-index-with-count is a free variable.

Can we use a nested let expression to solve this problem? Something like:

(let ((list-index-with-count ...))
  (let ((list-index-with-count ...))
      ...))

After Session 17's quick exercise, you know that this won't help us. The new local variable shadows the outer one.

We are stymied. let can't support the idea of a recursive function. Why? Because it is merely a syntactic abstraction of a lambda application. The arguments passed to the lambda are evaluated before they are passed to the function and only then bound to their names.

To iron out this wrinkle, we need something more powerful than let.

Ironing Out the Wrinkle

In another style of programming, this might not be a big deal. We could try to work around the limitation. But in functional programming, we create functions all the time. We also recurse over tree and list structures. We want to be able to so so as flexibly as possibly. For this reason, Racket provides another special form, named letrec, that supports local recursive definitions.

The syntax of letrec is identical to that of let:

<letrec-expression> ::= (letrec <binding-list> <body>)

     <binding-list> ::= ()
                      | ( <binding> . <binding-list> )

          <binding> ::= (<var> <exp>)

However, the semantics of the letrec expression differ from the semantics of the let expression in an important way. In a letrec, the region associated with each <var> is the remainder of the letrec expression, including the binding expressions that follow.

With letrec, we can define list-index using a local function:

(define list-index
  (lambda (target los)
    (letrec ((list-index-with-count
                (lambda (target los base)
                  (cond ((null? los) -1)
                        ((eq? target (first los)) count)
                        (else (list-index-counted target
                                                  (rest los)
                                                  (+ count 1)) )))))
      (list-index-with-count target los 0)) ))

... and satisfaction fills the room:

> (list-index 'd '(a b c d e f d))
3

Actually, we can now simplify list-index-with-count a bit. Because the value of target remains the same throughout the body of list-index and the body of list-index-with-count, we don't really need to pass target to list-index-with-count:

(define list-index
  (lambda (target los)              ;; once target is passed in ...
    (letrec ((list-index-counted
              (lambda (los base)
                (cond ((null? los) -1)                 ;; ... its value
                      ((eq? target (first los)) base)  ;; never changes
                      (else (list-index-counted (rest los)
                                                (+ base 1)) )))))
      (list-index-counted los 0))))

This makes for a more cohesive function, at the small expense of tracking target up to the declaration in list-index.

Quick Exercise: What happens if we remove los from list-index-helper's parameter list, the way we did target?

Local recursive function definitions are quite useful in cases that require an interface procedure. The helper function is the real function, while the interface procedure exists only to send the initial value for some argument.

Local Recursive Functions

As you are learning Racket, this type of construction may be hard to read for a while. Simple local variables tend be defined with simple values, but local recursive functions tend to be a bit longer and more complex. On the other hand, they are a clean, compact way to implement many solutions that require interface procedures to kick off a computation.

Perhaps you can help yourself understand the ideas in Racket by referring to another language you know:

Nearly every language you know allows local variables.
Many, such as Ada, allow local procedures or functions. In Java and C++, we cannot create functions that are local to other functions, but we can create private member functions that are local to a single class to serve as helpers to public member functions.
Java allows inner classes: "local" classes that are defined within the definition of another class.

Local function bindings offer some significant advantages over helpers declared at the top level:

We reduce the potential for naming conflicts, because the region of the name is local to a single lambda. (We saw this earlier with non-recursive functions.)
Readers can find local definitions more easily, because they occur near where they are used.
We can pass fewer arguments, because variable references can be made to bindings in the enclosing scope.
We can limit the scope of programming changes, because we limit the scope of the bindings. This is a great value in writing and maintaining large programs.

`letrec` as Syntactic Abstraction

It is not often the case that Racket provides a new keyword or a new piece of syntax to solve a problem. letrec is an exception, but a well-motivated one. Recursive programming is a fundamental technique in functional programming, so it is important that Racket make writing recursive functions as easy and straightforward as possible.

But we don't need a new piece of syntax. Like the let expression, letrec is a syntactic abstraction. We can implement the equivalent of a letrec expression in "vanilla" Racket, using (1) a feature of the language we have not studied yet, but which you know well, and (2) a bit of a hack. Can you imagine how?

Code

You can download the code for this reading as a zip file. It contains the original version of list-index, the failed attempts to use let to implement the helper, and the successful version that uses letrec.

Practice, Practice, Practice

Try to implement some earlier code with helper functions using letrec. positions-of from Homework 4 is another case with an interface procedure and a helper needed only by the interface. The helper functions for most mutually-recursive problems are also good candidates, too!

letrec as Syntactic Sugar