Here are some examples where Testing differs from Coverage testing. I am assuming that the term "Coverage" means Code Coverage branch/loop coverage both. In summary testing is of the following types viz. Functional(Feature/Requirement) Testing, Code Coverage Testing.
- Overflows/Underflows of variables, for example the following function, coverage is just passing any values of a and b, not caring about how overflows for example if a*b > 32-bit, the return value is correct.
int foo(int a, int b) {
return a * b ;
}
- Handling Exceptions from lower layers: For example say we have the following code, where the innerfoo function can throw exceptions. A 100% coverage which mocks the innerfoo function will still miss the cases which are not handled i.e. coverage will test whatever is written - but won't test if whatever is written is enough from the functionality pov.
int foo() {
try {
innerfoo();
} catch( ExceptionName e1 ) {
// catch block
} catch( ExceptionName e2 ) {
// catch block
} catch( ExceptionName eN ) {
// catch block
}
}
- Coverage testing does not cover Range Testing, for example,
if (b < 100) {
//do something
}
in which case, while coverage will be 100% if we pass a value of a that returns a b
such that it goes in the if block. However, coverage testing will miss noting that the code for the
else is missing and if b >= 100 then it is not clear that the function foo will do the right thing
In summary thus, Coverage does not check functional/requirements.
- Testing Features vs. Testing code that is written/ Functional Testing/Feature Coverage/Requirement Coverage: Code coverage will cover all branches but it won't be able to find out how much of the code written actually matches what is desired from the feature i.e. if the feature is supposed to do X, Y, Z then the code does that is found out by feature testing - code coverage will only test if the testing is covering the written code. This is also called Functional Testing.
- When people refer to the word "Unit Testing" it effectively means doing feature/requirement testing for the function.
- When people refer to integration testing, it refers to unit-testing the unit-tests i.e. for example imagine there are 2 gears that are individually unit-tested but integration testing is unit testing the behavior of 2 gears locked-together. For example verifying that turning left gear anticlockwise will turn the right gear clockwise. In terms of code, coverage will verify the unit-testing of individual components but again as mentioned previously may not cover overflows/exceptions i.e. will verify whatever is written is fine in terms of code-logic and execution, but wont verify whatever is written is what we want!
