Abstract

This paper presents an overview of inspections, walkthroughs, and reviews. What are the similarities and differences? Who participates in these activities? Why reviews are beneficial? And what kinds of errors do reviews find?

OPMT 1115 - Software Quality Assurance
First presented: December 2, 2002
Updated: December 25, 2002

Summary

Technical reviews...

• vary in degree of formality and number of participants,
• supplement testing,
• find defects earlier and quicker,
• find more defects than simply testing alone,
• facilitate knowledge transfer through discussion forums, and
• increase visibility into progress and readiness.

Overview

For some, inspections, walkthroughs, and reviews are all synonymous with "technical reviews". What they all have in common is that they are used to detect errors (or defects) early in the software development life cycle. Reviews can be used anytime. The earliest is after the requirements are written up. The latest is after the software has been delivered to the customer ... which may be too late (in terms of prevention).

For specific contrast:

Walkthroughs	Inspections
Informal review	Formal technical review
Involves two or more peer developers.	Group involves multiple participants of varying roles and responsibilities: reviewers trained in inspection, moderator (or inspection leader), reviewers (trained inspectors), developers (authors, producers), recorder (or scribe), and possibly a separate reader.
Possible errors identified, typically in code.	Can be used to detect errors in requirements, design, code, test cases, and/or other project artifacts.
Changes are only suggested. Further investigation and correction are outside the scope of this review.	Inspections specify action items, e.g., for rework and re-inspection.

Other types of reviews include:

• Code reading - a code walkthrough of two or more reviewers who read the code and report errors back to the developer.
• Management reviews - evaluate the project-level plan and the project's progress per the plan. The results of the review would include action items for corrective action, and changes in resources, scope, and/or schedule.
• Audits - independent evaluations of a process and/or product.

Note 1: You may also hear software inspections referred to as a form of static analysis or static testing, i.e., testing that doesn't involve running the program. Static testing also includes tools that analyze code to generate test cases or identify problems (e.g., lint).

Note 2: There's no limit on what can be inspected, so inspections should be limited to those items where the benefit is likely to be worth the cost. Consider the context (rigor vs. scope vs. resources vs. time vs. costs) and be practical. A less formal walkthrough process may be adequate. (Humphrey, 172)

Structure

Inspection Pre-Requisites:

• Structured review process - integrated in the software development life cycle.
• System of check lists - to govern each step of the review process, including objectives.
• Defined roles and responsibilities.
• Template forms and reports.

Pre-Inspection Inputs:

• Producer certifies code compiles, has been spell-checked, and all specifications have either been met or explicitly waived.
• Source listing and documentation (e.g., requirements and design).

Post-Inspection Outputs:

• Inspection error log, report, summary, etc.
• Rework to fix identified problems.
• Re-inspection.

Inspection Objectives: (Humphrey, 463-486)

• Identify all the design errors in the product.
• Identify any cases in which the code does not implement the design as intended.
• Identify any improper use of interfaces.
• Inspect for adherence to usability considerations.
• Inspect for appropriate style guidelines and to any required standards or constraints, such as:

• Naming
• Commenting
• Flagging
• Compiler restrictions
• Size constraints
• Performance limitations

• Verify that previously identified issues have been resolved.

Benefits

Why conduct reviews?

• Find errors early.
• Improve quality.
• Increase productivity.
• Shorten schedule.
• Mitigate risk.
• Supplement testing.
• Avoid preventable defects.

Studies have found reviews to be very effective in error detection. Examples (from secondary sources) include:

• Walkthroughs found between 30 and 70 percent of errors in a program. (McConnell, 73)
• Code reading found twice as many defects per hour of effort as testing. (McConnell, 74)
• Inspections found 60-90% of defects in a program. (McConnell, 75)
• Inspections produced a net schedule savings of 10-30%. (McConnell, 75)
• An hour of inspection avoids 33 hours of maintenance; inspections are 20 times more efficient than testing. (McConnell, 75)
• The introduction of inspections in software maintenance reduced production crashes by 77%. (Humphrey, 186)
• Inspections increased productivity 14%-25%. (Humphrey, 186-187)
• Inspections assist programmers to recognize and fix their own errors early in the process. (Humphrey, 171) Finding problems which could not be caught in testing. (Humphrey, 186)
• A study of 2019 user-found problems that resulted in code changes, found 57.7% of the problems could have been detected by design inspections, 62.7% with code inspections. (Humphrey, 187)
• When programmers know their work will be critically examined, inspections motivate better work -- that is, developers take more pride in quality and avoid sloppy mistakes. (Humphrey, 171)
• Reviews provide visibility into the state of the project. It provides an opportunity for discussion, intermediate milestones, and an assessment of the adequacy of some aspect of the project. (Christensen and Thayer, 291)

Errors

Over time, inspections have revealed a number of common problems: (O'Neill, 404-405)

• Source code not traceable to requirements. Verification procedures are imprecise and changes cannot be managed.
• Lack of rigor and discipline in software engineering practices (in design and programming). Result: high defect rates.
• Ad hoc recording of software product designs and source code. Result: software product understanding, adaptability, and maintainability are directly impacted.
• Poorly stated/understood/applied rules for construction in the application domain. Result: common patterns and templates are not exploited for later reuse.
• In web development, the code and upload development paradigm, parallels the criticisms of code-and-fix.

For example, in a code inspection, reviewers may uncover these common implementation errors: (Telles and Hsieh, 125-161)

References

Standards:

• IEEE Standard 1028-1997. IEEE Standard for Software Reviews.

Books:

• Christensen, Mark J., and Richard H. Thayer. The Project Manager's Guide to Software Engineering's Best Practices. Los Alamitos: IEEE, 2001.
• Humphrey, Watts S. Managing the Software Process. Reading: Addison-Wesley, 1989.
• McConnell, Steve. Rapid Development. Redmond: Microsoft, 1996.
• O'Neill, Don. "Software Inspections: Positive Results to Date." Software Management. 6th ed. Los Alamitos: IEEE, 2002.
• Pressman, Roger S. Software Engineering: A Practitioner's Approach. 5th ed. New York: McGraw-Hill, 2001.
• Telles, Matt and Yuan Hsieh. The Science of Debugging. Scottsdale: Coriolis, 2001.

Copyright

Copyright © 2002 Anthon Pang.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".