Session 7

Thinking Functionally:
Programming with Higher-Order Functions

CS 3540
Programming Languages and Paradigms

Opening Exercise

Suppose that we had the data for the total error problem in a Python list:

     >>> pairs = [ [2, -7], [-4, -20], [ 7, 8], [-13, 2],\
                   [6, -5], [-10, -1], [-2, 4], [  7, 2] ]

Write Python code to solve the problem:

Write a function named total-error that takes one argument, a list of this form. The function returns the total of all the differences in the list.
Python has an abs function, too. But this is Python, so you can use assignment statements and a loop!

A Solution

Unshackled from the chains of Racket and functional programming, we might produce code that looks like this:

    total_error = 0
    for p in pairs:
        difference = abs(p[0]-p[1])
        total_error += difference

Why is it so much easier to write this code than a functional solution in Racket? When you encounter a problem to solve, you start thinking about the problem -- breaking it down into parts, solving the parts, putting the parts back together -- in a particular way. These are habits you have learned and practiced for a couple of semesters.

Changing your mindset to a functional approach requires you to establish new habits and to break old ones. Creating new habits is a challenge, even when we want to change.

I know that many of you are not asking to change habits, or develop a new programming style. But it will make you a better programmer, and it will prepare you for something that is happening in industry right now. Give it a try, and you will be surprised.

One thing we can do when we are trying to break old habits and create new ones is to watch for the triggers that cause us to fall back into an old habit and have a plan for what to do instead. Let's see if we can identify triggers for some common procedural habits and match them up with alternative courses of action in functional programming.

Growing a Solution

Let's look closer at our loop:

    total_error = 0
    for p in pairs:
        difference = abs(p[0]-p[1])    # 1. do something with p
        total_error += difference      # 2. do something with result from #1

This loop does two kinds of thing: process the elements of the list and process those results. In FP, we try to tease apart different tasks into different code...

Trigger: loop does different kinds of action.
Action: decompose the loop into separate loops, each with a single responsibility.

So, let's focus on the "do something" part of the loop. Instead of processing the results immediately, we will save them to be processed later.

    results = []
    for p in pairs:
        difference = abs(p[0]-p[1])    # 1. do something with p
        results.append(difference)     # 2. record result from #1

Now, let's factor the action out into a function:

    def error_for(two_list):
        return abs(two_list[0]-two_list[1])
... and then use our new function:
    results = []
    for p in pairs:
        difference = error_for(p)      # 1. do something with p -- in a fxn
        results.append(difference)     # 2. record result from #1

This is a trigger: a loop that does something with every item in a collection. The alternative action: map the function over the list.

map implements the entire loop -- as long as you give it the error_for() function to apply to each item:

    results = map(error_for, pairs)

Yes, that is Python. The Python map function produces a "map object" that we can loop over, not a list, but the idea is similar.

We have made progress toward our solution:

    pairs   = [ [2, -7], [-4, -20], [ 7, 8], [-13, 2],\
                [6, -5], [-10, -1], [-2, 4], [  7, 2] ]

    results = [9, 16, 1, 15, 11, 9, 6, 5]

Now, let's do the second part of our original loop: add up the results:

    total_error = 0
    for r in results:
        total_error += r               # 1. accumulate sum from item r

This is another trigger: a loop that combines the value for every item into a single answer. The alternative action: use a "reducing" function.

Python doesn't have a single, simple way of reducing a list. We could replace our loop with

    total_error = sum(list(results))

In Racket, we have been using apply to reduce lists. apply implements the entire lopp -- as long as you give it +.

Growing a Solution in Racket

Now that we know the triggers, we can think about implementing our solution functionally in Racket. First, let's port our data back to a Racket list...

    (define pairs
      '((2 -7) (-4 -20) (7 8) (-13 2) (6 -5) (-10 -1) (-2 4) (7 2)))
... and our error_for() function to Racket...
    (define error_for
      (lambda (two_list)
        (abs (- (first two_list)
                (second two_list)))))

Now we can map error_for() over the list...

    (define results (map error_for pairs))
... use apply to total up our results...
    (define total_error (apply + results))

We can do this without the temporary variable results if we replace the symbol with the expression that creates its value:

    (apply + (map error_for pairs))

And that is the body of the function we need to write:

    (define total-error
      (lambda (list_of_pairs)
        (apply + (map error_for list_of_pairs))))

The apply can be a simple reducer. Functions like + and average are operators that apply can use to combine values.

[ take stock ].

Check-In Exercise

Suppose that we have lists of strings of this sort:

    (define names
      '("Johnny" "christine" "FRANK" "juliette" "Joanna" "eugene"))

Your task:

Write a Racket function average-length that returns the average length of the strings in a list of strings.

For example:

    > (average-length '("hi" "lois"))

    > (average-length names)
    6 2/3

Consider a possible solution.

Another Pattern: Filtering a List

Introduce a new problem: find the games in which the home team was picked to win.

     >>> pairs = [ [2, -7], [-4, -20], [ 7, 8], [-13, 2],\
                   [6, -5], [-10, -1], [-2, 4], [  7, 2] ]

Work through Python evolution: loop-and-if trigger.

An initial imperative solution:

    results = []
    for p in pairs:
        if p[0] > 0:                   # 1. if p meets a condition
           results.append(p)           # 2. record it in our result

As before, move the operation into a function:

    def home_team_expected(two_list):
        return two_list[0] > 0

And use the function:

    results = []
    for p in pairs:
        if home_team_expected(p):      # 1. if p meets a condition, in a fxn
           results.append(p)           # 2. record it in our result

This is a trigger: a loop with a bare if, selecting pairs that meet a condition. The alternative action: use a filter.

filter implements the entire loop. All we have to do is give it home_team_expected():

    results = filter(home_team_expected, pairs)

As with map, Python's filter produces a "filter object" that we can loop over.

We can do all of this directly in Racket:

    (define home_team_expected
      (lambda (two-list)
        (positive? (first two-list))))

    (filter home_team_expected predictions)

Racket's filter gives a list.

[ take stock ].

Putting It All Together

map, filter, and apply are useful separately, but their real power comes when we use them together.

Recall our list of strings:

    (define names
      '("Johnny" "christine" "FRANK" "juliette" "Joanna" "eugene"))

Your task:

Write a Racket function total-starting-with, which returns the total number of characters in the names that start with a given letter.

For example:

    > (total-starting-with "j" names)
    > (total-starting-with "e" names)
    > (total-starting-with "a" names)

Convert all the strings to a canonical form (lowercase) before processing.

If you need a primitive function, ask!

Evolving a Solution

    ; (map string-downcase names)
    ; (filter (lambda (s)
    ;           (string-prefix? s "j"))
    ;         (map string-downcase names))
    ; (map string-length
    ;      (filter (lambda (s)
    ;                (string-prefix? s "j"))
    ;              (map string-downcase names)))
    ; (apply + (map string-length
    ;               (filter (lambda (s)
    ;                         (string-prefix? s "j"))
    ;                       (map string-downcase names))))

    (define length-of-names-starting-with
      (lambda (char list-of-strings)
        (apply + (map string-length
                      (filter (lambda (s)
                                (string-prefix? s char))
                              (map string-downcase list-of-strings))))))

Thinking Functionally

The patterns of data in our solutions look something like this:

    MAP      from  ((a b ...) (c d ...) (e f ...) ...)
               to  (   d1        d2        d3     ...)

    FILTER   from  (d1 d2 d3 d4 d5 d6 ...)
               to  (d1    d3       d6 ...)

    APPLY    from  (d1 d2 d3  ...)
               to  n

You can create new habits, with attention and practice. Take baby steps. Use the REPL to help you build code you trust.

Wrap Up


Eugene Wallingford ..... ..... January 30, 2018