How to Find Bugs

Your code has bugs in it. Here are some ways you can find them.

Test It Yourself

I do two things to test the code I write before anyone else sees it. First, I tend to write automated unit tests as I go along, in a kind of test-driven development style. These don’t so much find bugs as prevent them from occurring in the first place, as they force me to think about the requirements the code must fulfil before I write it. They are probably at their most useful though when you discover a bug through other types of testing. Then you can use a new unit test to reproduce the bug, fix it, and be confident that it will stay fixed through future versions of the code.

Second, I write a simple test script to carry out manually. This is essential for software that isn’t easily unit-testable, such as GUI code. It also helps to cover wider scenarios than unit testing covers, because unit tests tend to be focused on a single, simple scenario, while the test script might use the software to complete a larger task and so test more of the system at once.

Get Someone Else to Test It

The biggest problem with testing your own code is that you’re the one who wrote it. Any assumptions you made during the development will probably also be made during the testing, without you realising it. “Assumptions are things you don’t know you’re making” as someone once said to me. So you’re likely to have the same blind spots when testing that you did when developing, and you won’t find the bugs that are in there.

Another problem is that you don’t really want to find any bugs. It’s your code, you’re proud of it, and the deadline is tomorrow. Actually finding a bug would be a pain. So it’s very likely that you’ll subconsciously sabotage your own testing, not really testing it thorougly, in order to get it done without finding any bugs.

This is why it’s really important to have someone else test your code. If possible, this should be a dedicated tester; someone who is an expert at finding the types of mistakes developers frequently make and who understands what the software should do. A good tester makes developers nervous about handing over their code for testing, and that’s a good thing. But failing that, another developer will do. The important thing is that they’re not you, and they want to find bugs in your code.

If you’re doing the testing for another developer, try to be as thorough and as devious as possible. Think about the things you’d hate for someone to try with your code; all those tricky edge cases they may not have thought about. Blank inputs, negative numbers, invalid data. There are definitely bugs in there, you just have to find them.

Get Someone to Code Review It

Code Reviews are used in a lot of organisations. But I’ve often found that they focus primarily on making sure new code adheres to company coding standards, and not so much on finding bugs. It’s quite easy to check against coding standards with a cursory glance, but finding bugs takes considerably more time and effort, involving a careful line by line examination of the code, making sure that you understand at each step what the code is doing and what the developer intended.

But code reviewing with the intention of finding bugs can be very effective. Subtle bugs that may not have been discovered in testing can be found in this way. I take particular satisfaction in finding a bug when I review someone else’s code.

Demo It

You’ve tested your code to destruction and it’s ready for the big demo in front of all the VIPs. Halfway through the demo though, a big embarrassing error message pops up. “That’s not supposed to happen” you mumble. “It’ll be fixed before we release.”

Demoing the software in front of people can help you find bugs in several ways. It puts you in a different frame of mind, so those habitual paths you take through the software when you’re sat at your desk might change. If you always use the + button to add a user, you might find you use the menu option during the demo, without really thinking about it, and find that it doesn’t work.

Also, the questions people ask during the demo might lead you to bugs. “Can we search for sales leads who haven’t been contacted in the last month?” someone asks. “Of course” you say, only to find that it falls over when you try.

Usability Test It

Usability testing is a great way to make your software easier to use and understand. But it can also find help you find bugs. The way I like to do this is to write out a quick activity I want the user to try. If you’ve developed a new watersports specialist shopping website, it might be “Using the site, buy a large man’s wetsuit”. Then I give the task to a random person and let them loose.

Watching the user try and do this will give you many insights into how you can make the software easier to use. But it will often also throw up bugs as the user does things in ways you didn’t expect. And these should be easy to reproduce because you can watch and write down the exact steps they followed.

Document It

If you have to produce any kind of documentation as part of the software you’re developing, this can also help to find bugs. Writing user documentation helps put you in the mind of the user, and this can lead you to think of ways of using the software that you might not have thought of before. So you try it out, and find that it doesn’t work.

Writing documentation for other developers can also find bugs. Often when you’re describing exactly how something works from a technical point of view, you’ll realise there’s a case in which it won’t work. Or you won’t be sure and you’ll try it out. “Given an empty import file, the software will display a message informing the user. Hmmm, I don’t think I’ve ever tried that. Nope, it doesn’t work!”

Release It

If you have a pool of dedicated and enthusiastic users who will test your software before its official release, then you could release a preview version and run a beta test. Or you could just release it and start collecting feedback and bug reports from real users. Either way, you are very dependent on the diligence of your users in reporting the bugs that they find while using the software. You may also find that you get bug reports, but that these are really vague and difficult to reproduce.

There are a few things you can do to help find bugs after release. You can add reporting for crashes and other undesirable events, then investigate these when they occur. It’s really easy to release some software, and just assume it’s working fine when it’s not. Most users won’t report bugs and crashes. They’ll just stop using your software instead.

Adding extensive logging to your application can help you diagnose those vague bug and crash reports. In a previous job, the software we produced logged almost every event that occured within the code. If we knew the rough time a problem had occurred, many problems could be diagnosed simply by looking through the log files.

Use It Yourself

Finally, if you actually use the software you write, for your own work, then you’ll find bugs and annoyances in it. This isn’t going to work in every case, and there are going to be some blind spots in the way you use the software versus the way others use it. But this helps to avoid the scenario where you release it and assume it’s working when there are showstopping bugs in there.

This concludes my brief summary of some ways to find bugs in your code. I guess the conclusion is to test from as many angles and in as many ways as possible. Happy bug hunting!