Handling "Hard" Problems

For many optimization problems (TSP, knapsack, job-scheduling), the best known algorithms have run-time's that grow exponentially.

You could wait centuries for the solution of all but the smallest problems!

Ways to handle these "hard" problems:

• Find the best (or a good) solution "quickly" to avoid considering the vast majority of worse solutions

  e.g, Backtracking and Branch-and-Bound

• See if a restricted version of the problem meets your needed that might have a tractable solution.

  e.g., TSP problem satisfying the triangle inequality

• Use an approximation algorithm to find a good, but not necessarily optimal solution

Backtracking

Idea:

• Search the "state-space tree" using depth-first search

• Use the best solution found so far to prune partial solutions that are not "promising,", i.e., cannot lead to a better solution than one already found.

In the worst case, the algorithm is still exponential.

Backtracking State-Space Tree

State-Space Tree for 16-cents coins {1, 5, 10, 12, 25, 50}

What would the tree look like if we considered each subproblem from smallest coin to largest?

Recursive Backtracking Template

Backtrack( treeNode n ) {

treeNode c;

for each child c of n do

if promising(c) then

if c is a solution that's better than best then

best = c

else

Backtrack(c)

end if

end if

end for

} // end Backtrack

C++ Coin-change Backtracking Solution void Expand(int numberOfCoins /* level of search tree too */, int changeToBeReturned) {

int i, j;

for (i = 0; i < numberOfCoinTypes; i++) {

partialSolution[i] = partialSolution[i] + 1;

changeToBeReturned = changeToBeReturned - coinValues[i];

numberOfCoins++;

if (Promising(numberOfCoins, changeToBeReturned)) {

if ((changeToBeReturned) == 0) {

if (!solutionFound || numberOfCoins < bestNumberOfCoins) {

// Found a new best solution, so save it

bestNumberOfCoins = numberOfCoins;

solutionFound = TRUE;

for (j = 0; j < numberOfCoinTypes; j++) {

bestSolutionSoFar[j] = partialSolution[j];

} // end for (j...

} // end if(!solution...

} else {

Expand(numberOfCoins,changeToBeReturned);

} // end if (changeToBeReturned...

} // end if (promising...

partialSolution[i] = partialSolution[i] - 1;

changeToBeReturned = changeToBeReturned + coinValues[i];

numberOfCoins--;

} // end for (i...

} // end Expand

int Promising(int numberOfCoins, int changeToBeReturned) {

if (changeToBeReturned < 0) {

return FALSE;

} else if (solutionFound && changeToBeReturned > 0

&& numberOfCoins+1 >= bestNumberOfCoins) {

return FALSE;

} // end if

return TRUE;

} // end Promising

Branch-and-Bound (Best-first search)

Idea:

• Search the "state-space tree" using no specific search pattern

• Instead, expand the tree node that appears to lead to the best solution - node with the best bound

• Use the bound values to prune partial solutions that are not "promising,", i.e., cannot lead to a better solution than one already found.

In the worst case, the algorithm is still exponential.

Bound for Coin-change Problem

Bound - should be a true limit on the best solution possible for any node derived from it

For 41 cents with the set of coins {1, 5, 10, 12, 25, 50}

All the bounds are the same, so coin-change problem is NOT a good branch-and-bound problem!

Template for Branch-and-Bound

Branch-and-Bound(node & best) {

priority_queue_of_nodes PQ;

node c, n;

n = root of state-space tree

best = value(n)

PQ.enqueue(n)

while not PQ.empty( ) do

PQ.remove(n)

if bound(n) is better than value(best) then

for each child c of n do

if value(c) is better than value(best) then

best = c

end if

if bound(c) is better than value(best) then

PQ.enqueue(c)

end if

end for

end if

end while

} // end Branch-and-Bound