Session 11

Implementing Algorithms

CS 3530
Design and Analysis of Algorithms

Let's Do a Puzzle!

This is a puzzle in board reconstruction. We are given as input:

Our task is to produce an nxn board that satisfies the input.

For example, on this input:

    1 1
    1 1

We might produce this board as output:

    X .
    . X

Or this board:

    . X
    X .

As you can see, there may be more than solution for a given input. You just need to find one, if it exists.


    3              . X .
    1 1 2          . X .
    1 2 1          X . X

    3              X . .
    1 3 1          X X X
    2 2 1          . X .

    4              . X . .
    1 2 3 2        X X . .
    2 3 2 1        X X X .
                   . . X X

    4              . . X .
    1 2 3 2        X X . .
    2 3 3 0        X X X .
                   . X X .

Do you notice any invariants in the inputs and their corresponding solutions? How about the partial solutions?

Let's Design an Algorithm

Do you remember when we discussed graph coloring a few sessions back? This board coloring task is common in scheduling and resource allocation. For example, we may need to schedule n people to n tasks, where each person may work on multiple tasks and each task may require multiple persons.

Design an algorithm to construct an nxn matrix that satisfies a board description given as input.

We are beyond the stage of staring at a blank page saying I have no idea what to do. Use the ideas you've seen (top-down, bottom-up, zoom-in, brute force, ...) to get an idea for the 'shape' of an algorithm.

We are also beyond hand waving. Someone should be able to apply or implement your algorithm, as written. So be prepared to have another student read it.

Don't worry if the problem seems hard, or if an answer does not come to you right away. You will learn a lot just by trying to turn your ideas into words, and then into a clear algorithm. Practice will make you better! Persistence will allow you to learn.

Your algorithm must guarantee finding an answer where possible. It does not have to be efficient.

Make sure that your algorithm works for the examples we've seen so far. Try to construct counterexamples that break your algorithm. (I will!)

If you get done before I call time, determine the complexity of your algorithm.

If you still have time, try to design a more efficient algorithm.

We should be able to solve this puzzle with an algorithm that is O(n²), or close.

Trying Brute Force and Greed

I borrowed this problem and some of the discussion that follows here from David Ginat's "Colorful Challenges" column in the June 2004 issue of inroads magazine.

What kinds of solutions did you create?

How might we use brute force to solve this problem? We could do an exhaustive search of the possibilities until we find a 'winner'.

    for all combinations of occupied/empty squares
        if board satisfies the input
           return true
    return false

How many combinations are there? CHOOSE(n² k), where k is the sum of the values in all rows. For our first example above, that is CHOOSE(4 2). For the last example above, that is CHOOSE(16 8). If we assume that the board is roughly half full, then we have CHOOSE(n² n/2). Ack!

We could, instead, try a greedy solution. Construct the board row-by-row, coloring the leftmost slot(s) available in each row:

    for row = 1 to n do
        for col = 1 to n do
            if row_input[row] > 0 and col_input[col] > 0
               put token at board[row][col]
               row_input[row] -= 1
               col_input[col] -= 1
    return (all values in both input lists are 0s)

Unfortunately, this approach fails for some simple cases. Can you construct an example where this algorithm fails to construct a board when one exists?

How about this:

    3              X . .
    1 1 2          . X .
    1 1 2          . . ?

Ginat suggests a slight variation: Construct the board diagonal-by-diagonal, beginning with the main SW-NE diagonal and then alternating above and below. This takes care of the failure case that foiled our original algorithm:

    3              . . X
    1 1 2          . X .
    1 1 2          X . X

Unfortunately, this fails for other simple cases:

    3              . . X
    1 1 3          . X .
    1 1 3          X . ?

Looking at diagonals points us in a direction that may lead to a greedy approach that works, if designed carefully: Color the squares at the intersections of the highest row/column pairs. But how do we break ties? Making that decision requires a bit more design.

One way to salvage a greedy approach is to do something that the brute force algorithm above does naturally: when we fail, move on to another board. Applied to our greedy algorithms, we would keep track of the last decision point, and if we ever reach a dead end, we can back up and choose differently there. This technique is called backtracking. Unfortunately, this approach leaves us in a predicament that is similar to the brute force algorithm, computationally: exponential time in the worst case.

Can we do better?

Zoom, Zoom, Zoom

Is there an invariant can we use to zoom in on a more efficient algorithm? Let's assume that we will color the rows in order...

At each point in the process,

One way to implement the idea is to always select the columns with the highest values remaining to be colored next. This ensures the maximum number of different columns available for subsequent rows. We call this a least commitment strategy.

Try this idea on the troublesome case above:

    1 1 3
    1 1 3

      1 1 3

      1 1 2
    0 . . x     (first)

      1 1 1
    0 . . x
    0 . . x     (next)

      0 0 0
    0 . . x
    0 . . x
    0 x x x     (last)

Even better, as soon as the invariant fails, we can stop, because no legal coloring is possible. This algorithm never hits a dead end.

How complex is this algorithm?

This results in O(n²) performance overall. That is the lower bound on coloring an nxn board anyway.

How easy would it be to implement this algorithm?

Selection Sort as Brute Force

[ These notes are probably incomplete. ]

Bubble sort and selection sort are often among the first sorts we learn as programmers. Bubble sort doesn't seem intuitive at all, but selection sort is almost prototypical brute force:

repeatedly choose the smallest value not used yet

Quick Exercise: What is the invariant of selection sort?

After k iterations, we have selected the smallest k items from the original set. They reside in slots [0..k-1] of our answer.

Here is a Java implementation of the selection sort, applied to the problem of sorting the characters in a string.

What is the complexity of selection sort?

This is an important point, because:

Wrap Up

Eugene Wallingford ..... ..... February 18, 2014