Being familiar only with tests in an Object Oriented environment, it was quite tough to figure out how to properly conduct unit tests.
For testing a function, we did what we usually do: mock every external function call.
So, our code effectively looked like this:
function X
{
if Y($1) is "true" then echo "Yes"
else echo "No"
}
X.test
mock function Y { return "true" }
assertEquals "X in good case" "Yes" X(1)
mock function Y { return "false" }
assertEquals "X in bad case" "No" X(2)
Y.test
assertEquals "Y with good result" "true" Y(1)
assertEquals "Y with bad result" "false" Y(2)
Extra credit to those who already sit back laughing, "You fools, this obviously had to go wrong!" ...
Guessed what happened?
We had done some refactoring to Y in the meantime, and in the end, the unit tests for Y looked like this:
Y.test
assertEquals "Y with good result" "Yes" Y(1)
assertEquals "Y with bad result" "No" Y(2)
Of course, the refactoring and TDD made sure that Y was doing what it should be, and we simply assumed that regression tests would catch the error on X - guess what: they didn't!
Because we had always mocked the behaviour of Y in X, there was no such test "Does X do what it's supposed to do in real circumstances?"
Lesson Learned:
If the function works in context, it does what it's supposed to do - but if the function works in isolation, there is no guarantee that it works in context!
We changed the way of writing unit tests as follows: "Rather than use the most isolated scope to test a function, prefer to use the most global scope possible without relying on external resources".
No comments:
Post a Comment