Black-box and Glass-box Testing
Applied class for 2026-04-16-solid-principles-ii (Week 7). Applies the black-box/white-box testing concepts from java-testing (Week 5, 2026-03-26-testing) to a worked example. Most of these tasks ask for a set of inputs achieving some coverage goal — many different answers are equally correct, so answers here are reveal-only worked examples rather than checkable exercises.
1. Black box testing
The following class tracks how many passengers are on a bus. Only the specification is given — no bodies — because black-box tests are written against the specification, not the implementation.
public class Bus {
/**
* Construct a new instance of the bus with a specified capacity.
*
* @param capacity The maximum passengers at any one time.
*/
public Bus(int capacity) {...}
/**
* Calculate how many passengers are currently on the bus.
*
* @returns The number of passengers on the bus.
*/
public int getCurrent() {...}
/**
* Calculate the average number of passengers on the bus between stops.
*
* @returns The average passengers on the bus.
*/
public int getAverageCount() {...}
/**
* Exchange passengers at a bus stop.
* Passengers get off before new passengers get on.
*
* @param on The number of passengers boarding the bus.
* @param off The number of passengers disembarking the bus.
*/
public void stop(int on, int off) {...}
}The class above spells the tag @returns, not the standard @return. The course is inconsistent about specification-tag spellings (see csse2002-diagram-notation).
Task 0 — write a smoke test
There is a huge range of smoke tests you could come up with. Your smoke test should utilize the Bus class under normal/expected operation:
Bus bus = new Bus(10);
bus.stop(5, 0);
bus.stop(3, 1);
assertEquals(7, bus.getCurrent());
assertEquals(6, bus.getAverageCount());The Javadoc says passengers get off before new passengers get on, so stop(3, 1) on a bus holding 5 goes \(5 - 1 = 4\), then \(4 + 3 = 7\); and the average between the two stops is \((5 + 7) / 2 = 6\).
See java-junit for assertEquals and week8-lab-junit-testing for writing these as real tests.
Task 1 — develop boundary scenarios (with justification)
Develop some boundary scenarios/inputs to use for testing. For each scenario you should justify why it is a good boundary case. You are not expected to know the expected output for each scenario.
There are many different situations to try; the following are just some examples.
| Scenario | Justification |
|---|---|
| Construct a bus with capacity 0 | A bus that can have no passengers is an uncommon scenario we may expect bugs from. |
| Construct a bus with a negative capacity | It is not clear how a bus with a negative capacity should behave; we should aim to clarify its functionality and test it. |
| Ask for the average number of passengers after no stops | An average often involves division by the number of times — we want to ensure that asking for the average after zero stops doesn’t create an exception. |
Negative values for on and off |
Behaviour for negative values is unclear — will a negative number of on passengers result in passengers getting off? |
| More passengers get off at the stop than are currently on the bus | Results in undefined behaviour. |
| More passengers on the bus than capacity | Behaviour is not specified, so we wish to see how Bus behaves. |
Task 2 — how can we make Bus’s behaviour fully specified?
Many of the boundary scenarios/inputs above are a result of unspecified or unclear behaviour of the bus class. How can we fix Bus to make all behaviour specified and clear?
If we specify that getCurrent() <= capacity is an invariant of the Bus class (see java-specification), then we must protect the invariant throughout by either throwing exceptions or adding preconditions:
- Add a
throws IllegalArgumentExceptionto the constructor ofBuswhencapacity < 0. - Add a precondition to
stopthat bothonandoffare greater than or equal to zero. - Add a
throws IllegalStateExceptionto thestopmethod ifgetCurrent() + on - offis less than zero or greater than capacity.
The two exception types do different jobs (see java-exceptions): the argument itself is nonsense in case 1, whereas in case 3 the arguments are individually fine but would drive the object into a state the invariant forbids.
2. Glass box testing
Glass-box testing helps us cover edge cases of program control flow. This is possible as we are allowed to use the internal structure (control-flow) of a method to construct our test cases.
There are different levels of code coverage; in this course we focus on three:
- Statement coverage
- Branch coverage
- Path coverage
Consider what inputs would achieve each level of code coverage for countOccurrences:
public static int countOccurrences(int[] array, int target) {
int count = 0;
for (int num : array) {
if (num == target) {
count++;
}
}
return count;
}- Statement coverage — every statement must run at least once, and the only one that isn’t run unconditionally is
count++. So the array must contain the target at least once:[2]with target2. - Branch coverage — the
ifmust additionally be false once, and theformust additionally terminate (it always does).[2, 3]with target2is enough on its own; so is[2, 3, 2]. - Path coverage — approximated as 0, 1 and 2 iterations crossed with the
ifoutcome each time, exactly the seven-row shape used forascending. Because there is no earlyreturn, all seven are reachable.
3. Ascending
/**
* Returns true if and only if numbers is an array of ascending
* integers.
*
* @require numbers != null
* @ensure \result is true iff for all indices j such that
* 0 <= j < numbers.length - 1, numbers[j] <= numbers[j + 1]
*/
public boolean ascending(int[] numbers) {
boolean result = true;
for (int i = 0; i < numbers.length - 1; i++) {
// invariant: for all j, 0 <= j < i, numbers[j] <= numbers[j + 1]
if (numbers[i] > numbers[i + 1]) {
result = false;
}
}
return result;
}The // invariant: line is the loop invariant — see java-loop-invariants. It plays no part in the coverage answers below, which turn on the loop’s shape: the body has no early exit, so every combination of iteration count and if outcome is reachable.
The three coverage levels are questions about the method’s control-flow graph, so it helps to have it in front of you:
Figure 1: Control-flow graph of ascending. Every statement lies on some path; the two labelled edges out of numbers[i] > numbers[i + 1] are the only branch in the loop body, and the back edge is what makes exhaustive path coverage infeasible and forces the 0/1/2-iteration approximation. Statements and decisions are both drawn as plain rectangles; the teal arrows are the flow of control.
Task 3 — statement coverage
Provide a minimal set of inputs that achieve statement coverage.
[2, 1]
One iteration, with the if true, is enough to touch result = false — the only statement not on the unconditional path.
Task 4 — branch coverage
Provide a minimal set of inputs that achieve branch coverage.
[2, 1, 2] or the pair [2, 1] [1, 2]
Either way the if has to come out true once and false once.
Task 5 — path coverage
Provide a minimal set of inputs that achieve path coverage.
| Input | Justification |
|---|---|
[] or [3] |
0 times through loop |
[3, 4] |
1 time through loop, if false |
[4, 3] |
1 time through loop, if true |
[3, 4, 5] |
2 times through loop, if false, false |
[3, 4, 2] |
2 times through loop, if false, true |
[4, 3, 5] |
2 times through loop, if true, false |
[5, 4, 3] |
2 times through loop, if true, true |
Writing the justification for each row helps ensure no cases are missed.
Both [] and [3] give zero iterations, because the loop bound is numbers.length - 1. The 0/1/2-iteration approximation is the one from java-testing — exhaustive path coverage of a loop is infeasible.
4. Maximum — bonus
Returns the maximum number within an array of integers, if one exists, else returns Integer.MIN_VALUE.
/**
* Returns the maximum number within an array of integer,
* if one exists, else returns Integer.MIN_VALUE.
*
* @require numbers != null
* @ensure numbers.length > 0 ==> (
* (\forall int i; 0 <= i < numbers.length ==> \result >= numbers[i]) &&
* (\exists int i; 0 <= i < numbers.length && \result == numbers[i])
* )
* @ensure numbers.length == 0 ==> \result = Integer.MIN_VALUE
*/
public int maximum(int[] numbers) {
int currentMaximum = Integer.MIN_VALUE;
for (int number : numbers) {
if (number > currentMaximum) {
currentMaximum = number;
}
}
return currentMaximum;
}Task 6 — statement coverage
[1]
Task 7 — branch coverage
[2, 1]
The first element beats Integer.MIN_VALUE (if true), the second does not beat 2 (if false).
Task 8 — path coverage
| Input | Justification |
|---|---|
[] |
0 times through loop |
[Integer.MIN_VALUE] |
1 time through loop, if false |
[1] |
1 time through loop, if true |
[Integer.MIN_VALUE, Integer.MIN_VALUE] |
2 times through loop, if false, false |
[Integer.MIN_VALUE, 1] |
2 times through loop, if false, true |
[2, 1] |
2 times through loop, if true, false |
[1, 2] |
2 times through loop, if true, true |
Integer.MIN_VALUE is doing real work here: it is the only value for which number > currentMaximum can be false on the first iteration, because the accumulator starts there and the comparison is strict.
5. Index Of — bonus
/**
* Returns the index of the given number within the array of numbers,
* or -1 if number is not in numbers.
*
* @require numbers != null
* @ensure numbers[\result] = number ||
* (\result == -1 &&
* \forall int j; 0 <= j < numbers.length ==> numbers[j] != number)
*/
public int indexOf(int[] numbers, int number) {
for (int i = 0; i < numbers.length; i++) {
// invariant: \forall int j; 0 <= j < i ==> numbers[j] != number
if (numbers[i] == number) {
return i;
}
}
return -1;
}This one differs from the other two in exactly one structural way: the if body returns, so the true branch leaves the method instead of continuing round the loop.
Figure 2: Control-flow graph of indexOf. Unlike ascending, the true branch of the if is an exit, not a step — so no path can take the if true and then iterate again, which is why two of the seven path-coverage rows in Task 11 read Cannot be tested. The exit is labelled in magenta.
Task 9 — statement coverage
[] 1 and [1] 1
Two calls are needed: return i and return -1 are both statements, and no single input reaches both.
Task 10 — branch coverage
[] 1 and [2, 1] 1
[2, 1] with target 1 takes the if false then true; [] takes the loop condition false immediately.
Task 11 — path coverage
| Input | Justification |
|---|---|
[] 42 |
0 times through loop |
[1] 2 |
1 time through loop, if false |
[1] 1 |
1 time through loop, if true |
[1, 2] 3 |
2 times through loop, if false, false |
[1, 2] 2 |
2 times through loop, if false, true |
| Cannot be tested. | 2 times through loop, if true, false |
| Cannot be tested. | 2 times through loop, if true, true |
The last two rows are unreachable for the reason drawn in Figure 2: taking the if true on the first iteration returns immediately, so there is no second iteration to have an outcome at all.