EP & BVA Practice Assignment: explanation

A couple of years ago, I created a simple website designed to assist QA engineers in practicing some of the most commonly used test design techniques: Equivalence Partitioning and Boundary Value Analysis. The site provides a specification of a feature along with multiple implementations; one is correct, and the others are not. The objective was to show that the correct test input can effectively detect all errors. However, my initial intent was not clear to the end users because I did not provide correct answers or any explanations! Let me do that in this post.

Code example

Software developers use predicates to alter the execution path of a program. In my assignment, the specification states:

Any integer that is greater or equal to -10,000 and less or equal to 10,000 is accepted by the program.

When the trainer receives an input, it is supposed to check whether the input falls within the allowed range. If it does, the value is accepted; otherwise, it is rejected. In the correctly working Implementation 1, this requirement is enforced as follows:

if (userInput >= -10000 && userInput <= 10000) {
  // accept
}

These two predicates control the behavior for all integers from -10,000 to 10,000. Hence, a single mistake can introduce a bug:

if (userInput > -10000 && userInput <= 10000) {
  // accept
}

Here, I changed the first predicate from >= -10000 to > -10000, and now the value -10,000 will be rejected by the program. Bugs occur near to the border of the partitions with higher probability because of mistakes like this one, which is why Boundary Value Analysis is so important.

Hopefully, these code snippets also illustrate why Equivalence Partitioning and BVA are often combined. One would have to try very hard to make a mistake in this scenario:

if (userInput >= -10000 && userInput <= 10000) {
  // accept
}

That would allow all values from -10,000 to 10,000 to be entered, except for anomalies like 8247 or some subset of integers!

Cheat sheet

Let's see how various ON, IN, OFF, and OUT values could help us in finding the issue from the example above.

This table shows the output for 8 different predicates based on the input values I've chosen. The first implementation i >= -10000 is correct:

• It returns true for -10,0000 because it is equal to -10,1000
• It returns false for -10,001 because it is less than -10,000
• It returns false for -15,000 because it is less than -10,000
• It returns true for -5000 because it is greater than -10,000

However, the remaining implementations do not meet the requirement. Discrepancies with the expected output are highlighted in red. I really like this table because it clearly shows how these values can detect errors.

Note:

• ON value can reveal two incorrect implementations: 2 and 7, and no other value can do that
• IN value can reveal one incorrect implementation: 5, and no other value can do that
• OFF value can reveal one incorrect implementation: 6, and no other value can do that

Therefore, we must include ON, IN, and OFF values in our tests. The OUT value can find errors in the implementations 3, 4, and 8, but other values can also do that. We might skip it.

This won't be true for all predicates though. Try to create your own tables where the correct predicates (for implementation 1) are:

• i > 10000
• i == 10000
• i != 10000

Like in my table, add incorrect implementations (2-8). Then identify ON, OFF, OUT, and IN values. After that, list the output for each implementation and highlight discrepancies. You'll see how the values necessary for your tests change depending on the requirement. You can always refer to these tables in your future work.

Recommendation

Practical Test Design is a fantastic book on the subject. It presents a wide set of test design techniques along with a thorough analysis of why they work. The authors reference various papers, research studies, and other books, providing a deep insight into this area. If you earn your living by creating test cases, this book is definitely worth reading.