Today's notes look a little different from those in a typical day. Today was much more dynamic than lends itself to PP slides. Thus, this is just an approximation of what we did today.
During PA01, I asked you to consider how you might test that your BoundedInteger class works properly. Many of you commented on the fact that this was difficult. You weren't really sure what to do. Today we tried to scratch the surface of this issue.
While we won't spend a huge amount of time on this (really, only today FORMALLY) I hope you don't come away from this thinking that I think that this isn't important. Just the opposite. I suspect that testing is probably one of the most underrated skills and abilities you can develop as a programmer. Just to give you a clue as to how serious a subject this is, our department offers an ENTIRE SEMESTER course on testing (810:175).
When we talk about testing, we are really talking about one of two kinds of error detection.
The first type of error detection is one that most of you are already getting pretty good at. That is, checking for syntax errors. These are easy to catch (although not always easy to fix) because you hit the compile button on your IDE or type the compile command at the console, and the computer tells you if there are problems.
The second type of error detection is the one that I am really interested in today. This is the detection of logic errors. These are errors where the computer "works." But it doesn't work correctly.
In the rest of the discussion today we will talk about how to test for and try to fix this second kind of error.
Positive testing is the testing of cases that are expected to succeed and confirming that you get the answer you expected.
Examples of positive tests from yesterday's lab include typing
java Analyze -min 3 5 7 1 9 8
And getting the answer of "1"
Another example from PA01 includes setting up a "minute" BoundedInteger with a starting value of 50 and then adding 5. We EXPECT the code to work and produce 55.
Negative testing is the testing of cases where you expect your code to 'fail' the situation. In many cases these are the conditions that produce exceptions or error messages. You want to confirm that your code will produce these exceptions or error messages and handle them gracefully. For example, in yesterday's lab a user might type
java Analyze -min
This should generate an error message, and we should confirm that it does so.
There is another vocabulary word we should talk about with all of this. That is the concept of boundary testing. Boundary testing is not a third type of testing. Instead, it is a special case of each of the previous two tests. Boundary testing is the idea of testing the "first" positive test case and the "first" negative test case when there are boundaries between the two. For example, if we wanted to completely test the toUpperCase() method used above, we would want to test not only 'a' and 'z', but '`' and }' (the characters right before a and after z in an ascii table).
For example, a minute timer set to 55 and told to add(4) should produce 59. If instead we told it to add(5) it should have produced 0. We normally want to test both of these issues.
We also should talk about blackbox versus whitebox testing.