What is a bug?
In this discussion paper, I examine the layman and technical definitions of the word "bug"
and related terms. I later posit more precise operational definitions, and present the scenarios
where bugs appear (or not).
Created: November 30, 2002
Updated: December 25, 2002
We, humans, are notoriously imprecise. Do you always agree with what someone else calls an anomaly,
a bug, a defect, a deficiency, an error, an exception, a fault, a problem, or an issue? When was the
last time you matter-of-factly said:
- That's not a bug.
- That's a "feature".
- That'll be an enhancement (or change request).
- We need to reset the customer's expectations.
That a reported problem may meet with any (or all) of these responses, leads us to consider the need
for more concise operational definitions.
The literature is rife with inconsistent usage of terminology. At one extreme, McConnell [69-72]
makes no distinction between errors and defects in the examples he cites. At the polar opposite,
Humphrey [311-313] elaborately states a bug is a defect but not all defects are bugs, and all defects
result from errors but not all errors produce defects. Even the software measurements collected by
pseudo-authoritative organizations reflect a lack of consensus: NASA and DoD refer to
"defects", the Software Engineering Laboratory refers to "errors", and the
Army refers to "faults" and "anomalies". [Christensen and Thayer, 478-483]
Turning to standards and more authoritative sources, let us examine the prevailing definitions:
(Source: "Randy Rice's Software Testing Glossary", except where noted otherwise):
- (IEEE) anything observed in the documentation or operation of software
that deviates from expectations based on previously verified software products or reference
- (1) things the software does that it is not supposed to do, [or] something
the software doesn't do that it is supposed to. [Telles and Hsieh, 5];
- (2) a fault in a program which causes the program to perform in an unintended or unanticipated
- (IEEE) the sudden and complete failure of a computer system or component.
- (1) (IEEE) a product anomaly;
- (2) synonymous with fault, both imply a quality problem discovered after the software has
been released to end-users (or to another activity in the software process); [Pressman, 203]
- (3) fault; [SWEBOK, Ch.5, p.6]
- (4) error, fault, or failure; [SWEBOK, Ch.11, p.9]
- (5) noncomformance to requirements.
- (1) (ISO) a discrepancy between a computed, observed, or measured value or
condition and the true, specified, or theoretically correct value or condition;
- (2) a quality problem discovered by software engineers (or others) before the software
is released to the end-user (or to another activity in the software process). [Pressman, 203]
- (IEEE) an event that causes suspension of normal program execution; types
include addressing exception, data exception, operation exception, overflow exception, protection
exception, underflow exception.
- (IEEE) the inability of a system or component to perform its required
functions within specified performance requirements.
- an incorrect step, process, or data definition in a computer program which
causes the program to perform in an unintended or unanticipated manner.
- (DOD) an unplanned event or series of events resulting in death, injury,
occupational illness, or damage to or loss of data and equipment or property, or damage to the
- (IEEE) a human action that produces an incorrect result, and the subject
of the quality improvement process.
- (1) (IEEE) a condition or capability needed by a user to solve a
problem or achieve an objective;
- (2) a condition or capability that must be met or possessed
by a system or system component to satisfy a contract, standard. specification, or other formally
- (3) a documented representation of a condition or capability as in (1)
- (IEEE) a measure of the probability and severity of undesired effects; often
taken as the simple product of probability and consequence.
- side effect
- an unintended alteration of a program's behavior caused by a change
in one part of the program, without taking into account the effect the change has on another part
of the program.
- (IEEE) a document that specifies, in a complete, precise, verifiable
manner, the requirements, design, behavior, or other characteristics of a system or component, and
often, the procedures for determining whether these provisions have been satisfied.
First, we state the following indisputable facts: (see tangent discussion in
- There is the user/customer's intention or expectation.
- There is a specification (or documented requirement).
- There is a software implementation, provided by the developer/vendor.
The general, imprecise definition of "bug" captures this relationship:
- any deviation between the customer's expectation, the specification, and/or
Next, we examine the typical use of "error" vs. "defect" in these common
messages (with these Google-based rankings):
- An error has occurred (~75,000 pages, vs. 68 for defect)
- Unknown error (~57,700 pages, vs. 478 for defect)
- Unrecoverable error (~26,900 pages, vs. 3 for defect)
Clearly, based on dominant usage, "error" and its ilk (i.e., mishap, crash, failure,
exception, and anomaly) can be considered observed events, i.e., manifestastions that occur at some
time, often with observable symptoms or effects. We'll choose to differentiate "error"
from "defect" in this manner, as the latter is rarely used in the context of an event
(e.g., "a defect has occurred").
Thus, we define "error" as follows:
- any observed event (incident) or detected deviation between the
specification and the implementation.
Pressman's use of temporal constraints (i.e., before vs. after) to differentiate between
"error" and "defect" is obscure, but not without merit or justification.
Constrained by a qualifier, the "Zero-Defects" concept (as advocated by Crosby) may seem
less daunting to the organization considering a quality improvement program. Also, in some circles
(or cultures), "defect" carries harsher overtones than "error".
And so, we state our definition of "defect" as follows:
- a deviation between the specification and the implementation, detected
after release to the customer (or the next activity in the software process).
The operational definitions established herein are neither earth-shattering nor revolutionary.
We simply refine and tighten the definitions, hopefully resulting in more consistent usage.
Considering expectation, specification, and implementation, we arrive at
these five scenarios:
- Expectation = Specification = Implementation. No bugs. Many happy campers.
- Expectation = Specification != Implementation. Error or defect to be corrected by
- Expectation = Implementation != Specification. Customer is happy with code-to-intent.
Vendor has a requirements bug, to be addressed in the "as-built" product specification.
- Expectation != Specification = Implementation. Customer is unhappy with code-to-specifications.
Customer has a requirements bug, to be addressed by a change request or reset expectations.
- Expectation != Specification != Implementation. Likely the result of ambiguous or undocumented
requirement(s). General disagreement, unhappiness, and finger pointing.
Some closing comments:
- Scenario 2 implies employing software development and QA practices for error preventation and
detection, such as prebugging, inspections, and testing.
- Scenario 4 may explain the increasing interest in methodologies relating to "Requirements
Engineering" (or "Requirements Management") and "Domain Analysis".
Unfortunately, this interest would be customer-driven (given the aforementioned scenario which
favors the vendor/developer) and in itself, may present little quantitative (risk or cost
reduction) incentive for the vendor/developer to add process overhead.
- Scenario 5 is somewhat rare (i.e., breakdown in communications and understanding).
- IEEE Standard 610.12-1990. IEEE Standard Glossary for Software Engineering Terminology.
- IEEE Trial Draft SWEBOK Version 1.00. Software Engineering Body of Knowledge.
- Christensen, Mark J., and Richard H. Thayer. The Project Manager's Guide to Software Engineering's Best Practices. Los Alamitos: IEEE, 2001.
- Crosby, Philip B. Quality Is Free. New York: Penguin, 1980.
- Humphrey, Watts S. Managing the Software Process. Reading: Addison-Wesley, 1989.
- McConnell, Steve. Rapid Development. Redmond: Microsoft, 1996.
- Pressman, Roger S. Software Engineering: A Practitioner's Approach. 5th ed. New York: McGraw-Hill, 2001.
- Telles, Matt and Yuan Hsieh. The Science of Debugging. Scottsdale: Coriolis, 2001.
Copyright © 2002 Anthon Pang.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU
Free Documentation License".