And perhaps it goes without saying that code review is good at catching particular *types* of bugs, static analysis catches another type…testing, another.

Clearly, code review is no silver bullet. Just because you had a ton of eyes swarming over you code, and you got a “ship it” from your reviewers, doesn’t mean you didn’t introduce a bunch of new show-stopper bugs.

So I guess my take home message is that “ship it” is not necessarily equal to “everything is dandy”.

Just a friendly warning to fellow beginners who are just starting the code review habit.

Another way to approach this problem is to ask “Why didn’t our automated testing catch this issue?” Granted, testing i18n’ed apps is problematic due to the sheer volume of validations you need to make, but technically, it’s not hard to automatically test that the correct string is being used at runtime.

Suppose the person writing the code hadn’t mis-spelled ‘without’. Do you think that code review should have caught the issue then? I would much rather rely on automated tests to do that sort of mind-numbing validation

*A* bug remained, but the bug that was detected by code review *was* fixed.

To me, there were *two* bugs, not one. The string in the locale file was:

no_student_without_a_group_message

So there are two bugs in the code:

1. Using “students” instead of “student”

2. Using “wihtou” instead of “without”

The code review caught the second bug, but not the first. Like I said, I’m kind of splitting hairs here.

The more interesting point you raise is: “isn’t it plausible that while our diffs look good, we’re not preventing ourselves from adding new bugs into the code base?”

Yes, that is completely plausible. The key is this: what does it really mean for a diff to “look good”? In this case, if the review does not include the locale file, then it is easy for the diff to look good.

If you have a static analysis tool that can do the grunt work of verifying the locale files, then great! But if not, that’s a problem….

*Some* bugs can be caught by using static analysis tools, others cannot. For those that cannot be caught by a static analysis tool, you’re better off doing code review than not because at least you have an opportunity for someone to spot the problems. Assuming, of course, that all the relevant files are included in the review.

Reviews catch bugs – reviews also miss bugs. At the end of the day, the code review is only as thorough as the reviewers choose for it to be.