Bug Fixing

by breeve 20. March 2011 12:12

Pilots pay attention to lots of things but nothing focuses their attention more than an airplane fresh out of maintenance. A Boeing 747, for example, has 6 million parts including linkages, fuel lines, bolts, wires and countless gages. Even the most safety conscientious mechanic can forget to tighten a bolt or secure a fuel line. At no time during the lifetime of an aircraft is it more vulnerable than in the safety of a hanger under the comforting sounds of hammers, wrenches, and the occasional cussword.

Take an Airbus A330 departing Vancouver International Airport on November 6, 2003. Shortly after takeoff, the control tower informed the pilots that smoke and vapor were pouring out of the #2 engine. Although the pilots made a safe landing after shutting down the engine, further investigation revealed why the airplane looked more like Superman streaking across the sky while taking an in-flight bathroom break than a normal Airbus flight. Mechanics had unnecessarily removed the low pressure fuel line while fixing another issue and failed to reconnect it back properly.

If the most well intentioned mechanic can cripple an airplane, bug fixing programmers can cripple a program. And just like mechanics who make mistakes on big complex airplanes and little airplanes alike, all but the smallest trivial programs are targets for mistakes. But unlike airplane maintenance, software maintenance has a little dirty secret: fixing one software bug somehow creates two more.

And if fixing one bug creates two bugs, it is easy to see that no piece of software is bug free. Most complex software has outstanding bug counts measured not in hundreds but thousands. Despite escalading bug counts, software gets away with it because unlike airplanes—where everything must be functioning perfectly or disaster will strike—all but the most mission critical software can function with bugs because most are either never hit or infrequently hit.

With large bug counts, deciding what to fix is non trivial. To mange the bugs, it helps to rank each one according to two variables. The first—perhaps the easiest—is how many customers are affected? If usage stats are gathered this becomes easy; if not more subjective. The second is how bad is it. Does it crash the program every time or is it a misspelled word at the bottom of the page in 2 point font that no one reads?

The graph below shows an example. Here we have plotted the bugs with the percentage of customers affected on the horizontal axis and the severity on the vertical axis. As we move to the right, bugs become more visible while towards the top more severe.

Looking at the graph above, we might expect to fix all bugs above and to the right of a imaginary line running from the top left to the bottom right. But in reality, the bug fixing boundary follows more of a y = 1/x curve than a linear one as shown in the graph below. This is because the less severe and less visible bugs congregate towards the left and bottom edges of the graph. All other bugs are either severe enough or visible enough to be fixed first.

At first glance of the graphs above, it might appear that only excluding bugs that are close to the left and bottom edges will result in most bugs being fixed. But like middle school students at their first dance, a surprising number of bugs cling to the edges of the graph. This can be attributed to internal opinions or small one off requests from individual customers that produces bugs that turn out to be either not important or visible enough.

But it’s a third parameter not the other two which is often overlooked: the risk of making a fix. How complicated is it and will it break other things resulting in more than two new bugs? Imagine this parameter being the z index of the graphs above springing out of the page towards you.

With this system, you don’t have to be a wizard to determine what to fix and what not to.

Tags:

Essays

Powered by BlogEngine.NET 1.5.0.7
Theme by Mads Kristensen

About Me

I am a Principal Engineer with 13 years experience developing and releasing software products. I started developing in C/C++ then moved into .NET and C# and have tech lead multiple projects. I have developed products in Windows Forms, ASP.NET/MVC, Silverlight, and WPF. I currently reside in Austin, Texas.

Own Projects

Pickaxe - An easy to use web page scraper. If you know a little SQL and how basic CSS selectors work, no web scraping product will be easier to use.

Download Page

Source Code

Created ASP.NET MVC forum originally targeting home owner associations but now in use by an investor group.

http://vtss.brockreeve.com

A language for querying PGATour golf strokes.

http://pga.brockreeve.com/

Real time bidder for car ads demo

http://cargawk.com/

Simple front end tic tac toe

http://tictac.brockreeve.com/

Currently Reading