Today is an exam day, so let's not do anything too stressful. Instead, let's do some magic that will awaken your algorithm-design muscles.
We are at the Breslin Center ticket office in East Lansing, Michigan, buying tickets for the Big Ten basketball tournament. The line is full of fans from three schools: Michigan State (yea!), Michigan (boo!), and Iowa (ho-hum). The folks are all mixed up, and things are starting to get ugly.
What's the problem? The Michigan State fans don't like the Michigan fans, and the Michigan fans don't like the MSU fans. The trash talk is getting out of control. (I blame the Michigan fans, myself.)
The ticket manager decides on a solution that both averts any chance of a riot and ensures that Michigan State fans get first dibs (and Michigan fans last) on the available tickets: He will rearrange the line so that all MSU fans are at the front, all Michigan fans are at the back, and the calm Iowa fans are a buffer in the middle.
Work with someone else, if you like, to...
Yes, that says linear.
For simplicity, you can write an algorithm that manipulates an array of containing the characters S, I, and M.
This is a sorting problem. We could use a bubble sort, a selection sort, or an insertion sort to achieve O(n²) time.
(How easy are these algorithms to implement with people in a line?)
We could use quicksort, mergesort, or heapsort to achieve O(n log n) time.
(How easy are these algorithms to implement with people in a line?)
Oftentimes, algorithms of a higher complexity class are more intuitive, more straightforward, and easier to implement ad hoc than algorithms in a lower complexity class. Efficiency has its price!
Can we do better than O(n log n)? Yes, because we know there are only three distinct values.
We can use insertion sort as an inspiration -- and quicksort, too! (And, to help you develop the idea, you may consider partitioning a crowd of just MSU and Iowa fans...)
"In the business", this problem is known as the Dutch Flag Problem. It was created by Edsger Dijkstra. In the canonical problem, we are sorting the colors in the flag of Dijkstra's homeland: red, white, and blue.
We always say that we can't sort in better than O(n log n) time, but that is only for the general case, in which when we don't know anything about our input data. When we know something about our input data, we can often design an algorithm that takes advantages of its characteristics. As we saw back on the day of our first quiz, knowledge matters.
What do we know in the case of the nearly-rioting fans? There are only three distinct values. So we can sort them by partitioning them, a lá quicksort.
Consider this two-value example:
SIISIISSS
We can make one pass through the array, swapping Ss from the back of the array with Is at the front. To solve the three-way problem we can make two passes, the first to partition the Ss from non-Ss, and the second to partition Is from Ms in what remains.
Because there are only three distinct values, we can generalize this idea without too much complexity. On each iteration, the algorithm maintains four partitions: Ss at the front, Is next, items not yet considered next, and Ms last.
Each of these sections may be empty. As soon as the third section becomes empty, we are done. So:
s ← 0, i ← 0, m ← n-1
while i ≤ m
if A[i] = S
swap A[s], A[i]
s++ and i++
else if A[i] = I
i++
else /* A[i] = M */
swap A[i], A[m]
m--
Try this algorithm out on this input: MISMIS.
The idea of partitioning works precisely because we know that the data consists of only d distinct values. We can generalize the idea for any particular value of d, though the complexity of a one-pass algorithm quickly becomes hard to manage.
But: there's more! There is another way to sort those pesky fans in linear time. Conceptually, it's even simpler than the partitioning approach:
s ← 0, i ← 0, m ← 0
for k from 0 to n-1 do
if A[k] = S
s++
else if A[k] = I
i++
else /* A[k] = M */
m--
for k from 0 to s-1 do A[k] ← S
for k from s to s+i-1 do A[k] ← I
for k from s+i to n-1 do A[k] ← M
In some cases, we aren't allowed to overwrite or rewrite the values. Perhaps each fan is a person with his or her own identity. Then we could make a second pass over A, moving values from A into a new array A_sorted. Knowing the counts would let us put the values in the right spots, by partition.
To paraphrase Bacon and quote Hobbes, Knowledge is power.
The first technique shown above uses the partitioning pattern we've seen several times this semester. We've generalized the binary partition of binary search and quicksort to handle (in this case) three distinct values.
The second technique is known as sorting by counting. You can sort by counting even if you have n distinct values in your array, by counting how many items in the array are smaller than each item. At the end, your counts give the positions of each value in the sorted array!
This idea is sometimes called comparison counting sort. As you found on Homework 1, this algorithm is neither stable nor in-place. Unfortunately, in addition to using O(n) space for counters, it is also O(n²) in time. So, as the number of distinct values goes up, the counting approach becomes much less attractive.
Sorting-by-counting can be done efficiently in-place using a variation called distribution counting.
Other kinds of knowledge can be useful, too, and give rise to even simpler algorithms. For example...
Quick Exercise: You are given an array A[0..n-1] of values whose keys are the integers 1..n. You are given an empty array S[0..n-1] to hold the sorted values. Give a one-line sorting algorithm.Quick Answer: Well, A[i] will belong in slot S[ A[i]-1 ], so:
for i ← 0 to n-1 do S[ A[i]-1 ] ← A[i]
I love it when a plan comes together, as "Hannibal" Smith used to say on The A-Team).