The following is a guest post from Herb Krasner, an Advisory Board Member for the Consortium for IT Software Quality (CISQ) and industry consultant for 5 decades.

Demands of the competitive global economy have placed a strong emphasis on quality across the IT industry, and it shows no signs of going away. Meeting the customer’s expectations at a high degree of conformance is no longer expected to come at a premium – it is just expected.

In a previous post, we looked at the magnitude and impact of the soaring cost of poor software quality in the US and where those hidden costs are typically found. We now turn our attention to what you, as a leader in your organization, can do about it. Calculating the cost of software quality is an important first step in identifying areas of opportunity to add value from IT while reducing costs, accelerating deliveries and remaining efficient/competitive.

Basically, the costs of software quality (COSQ) are those costs incurred through both meeting and not meeting the customer’s quality expectations. In other words, there are costs associated with defects, but producing a defect-free product or service has a cost as well. Calculating these costs serves the purpose of identifying just how much the organization spends to meet the customer’s expectations, and how much it spends (or loses) when it does not.

Knowing these values allows management and team members across the company to take action in ensuring high quality at a lower cost. While analyzing the COSQ at an organization may lead to the revelation of uncomfortable truths about the state of quality management at the company, the process is important for eliminating waste associated with poor quality. This often requires a mindset and culture shift from viewing software quality defects as individual failures to seeing them as opportunities to improve as a collective team.

In this post, we focus on the various costs of software quality and how those can be measured.  In the future, we will examine more closely the discussion of achieving disciplined and mature software development and how it affects a software asset’s total cost of ownership.

As highlighted in the figure above, we show that investing in software engineering discipline and in the Cost of Good Software Quality (CGSQ), will dramatically reduce the Cost of Poor Software Quality (CPSQ).

The American Society of Quality (ASQ) uses the following formula to calculate the Cost of Quality (COQ):

Cost of Quality (COQ) = Cost of Poor Quality (COPQ) + Cost of Good Quality (COGQ)

We use that same formula for the Cost of Software Quality.

Below is a summary of how to properly identify and track both CPSQ and CGSQ.

Cost of Good Software Quality

A discussion of what is meant by good quality software can be found here. The cost of good software quality is as variable as the organizations I have encountered. Some groups invest a lot in proactive quality management and planning, while others make do with patchwork systems and reactive programs aimed at solving problems after they occur.  

COGQ is divided into different categories. These are the costs associated with providing good-quality work products, systems or services.

There are three categories: prevention costs (investments made to prevent or avoid quality problems), appraisal costs (costs incurred to determine the degree of conformance to requirements and quality standards) and management control costs (costs to prevent or reduce the likelihood of failures particular to its management functions: contract reviews, planning, goal establishment, and progress review and control of each software project).   

Below are some examples of typical costs within each category:

Prevention

  • Error Proofing (Defect Prevention Programs)
  • Capability Assessments
  • Training
  • Improvement Initiatives

Appraisals

  • Quality Assurance
  • Inspection/Testing
  • Setting up effective test environments
  • Supplier and included component (e.g. OSS) assessments
  • Design and Code Reviews

Management Control Costs

  • Costs of carrying out contract reviews
  • Establishing quality goals, objectives, gating/release criteria and quality standards
  • Costs of preparing project plans, including quality management plans
  • Costs of periodic updating of project and quality plans
  • Costs of performing regular progress review and control
  • Costs of performing regular progress control of external participants’ contributions to projects

Cost of Poor Quality

COPQ, like its counterpart COGQ, is also divided into different categories. These are the costs associated with providing poor-quality work products, systems or services.

There are four categories: internal failure costs (e.g. costs associated with defects found before the customer receives the product or service), external failure costs (e.g. costs associated with defects found after the customer receives the product or service), technical debt (the cost of fixing the structural quality problems in an application that, if left unfixed, put the business at future serious risk) and management failures (costs incurred by executives and below dealing with the ramifications of poor-quality software).  

Below are some examples of typical costs within each category:

Internal Failure Costs

  • Rework
  • Re-Testing
  • Re-Designing/coding
  • Scrap
  • Downtime
  • Failure Analysis
  • Work Stoppages
  • Crisis Management and Overtime

External Failure Costs

  • Grievances/Complaint Handling
  • Recalls
  • Sales Reductions
  • Returns
  • Patches, Repairs & Servicing
  • Warranty Claims
  • Company Reputation Damage
  • Company Devaluation
  • Lawsuits

Technical Debt

  • Structural Problems
  • Increased Complexity Due to Shortcuts
  • Future Refactoring
  • Debt Service and Interest

Management Failures

  • Unplanned costs for professional and other resources, resulting from underestimation of the resources in the planning stage.
  • Damages paid to customers as compensation for late project completion.
  • Damages to other projects planned to be performed by the same teams involved in the delayed projects. The domino effect may induce considerable hidden failure costs.
  • Excessive management crisis mode behaviors, like lots of meetings to solve urgent problems.

Some Strategies for COSQ Measurement and Improvements

Most of the measurement of software quality and its related costs are readily tracked with today’s tools if you are willing to insert additional data to capture the effort involved with COSQ.  The major component of which is staff effort data, which can easily be converted to $$ when needed. The tools for tracking internal and external problems and defects already exist. What is needed is the recording of the total team effort involved in investigating and resolving those problems and defects.  That will give you the major component cost of CPSQ.

Once these basic mechanisms are in place, quality improvement programs can be meaningfully baselined and tracked over time.

Every company is at a different point in the evolution of its understanding of its key metrics/performance indicators and COSQ.  Once the basics are in place, management can consider leveraging the following strategies to reduce their company’s CPSQ and positively impact quality and bottom line performance.

1. Create an action plan for software quality and process improvements in your IT shop

  • Establish baselines and benchmarks
  • Form a cross-functional action planning team
  • Select improvement targets/pilot projects
  • Apply software quality modeling and measurement standards

2. Improve supplier relationships for both product and process improvements

  • Collaborate during development process, engage suppliers in the corrective action process (from incoming, or customer-reported problems), develop supplier scorecards, audit suppliers based on their product/process risk levels

3. Focus product development on Prevention

  • Define Critical to Quality attributes, pull in lessons learned from defect information from similar products’ risk files and quality system information

4. Make quality and achievement information and metrics visible across the organization

  • Collect real time quality data, defects/dispositions, inspection/QA/testing results, and rework, just to list a few, to trend problems and see systemic issues
  • Evaluate quality and compliance risk from audit results (internally and externally), complaints, reportable events
  • Use statistical analysis to monitor real-time quality data

5. Leverage technology

  • Evaluate tools that help analyze and track software quality attributes
  • Deploy a quality management platform to support operational efficiency which can increase accountability, productivity and reliability

Each of the above initiatives have costs to implement, but also savings when achieved. The costs may increase CGSQ (either through appraisal or prevention categories), and the cost savings can impact both the CPSQ and the CGSQ.

An Example of What Can Be Accomplished

As seen in the below graph for one of the companies that I have worked with, significant improvement in COSQ was achieved over a 5-year period.  This led to a 200% improvement in organizational software development productivity over that period.

Conclusion

Understanding Cost of Poor Software Quality in your organization is the first step toward gaining executive buy-in for quality-led operations. This is fundamental to achieving the potential benefits of agile, DevOps and Proactive/Predictive Quality Management. With a CPSQ number in hand, you have the basis for a business case to invest smartly in quality. Determining CPSQ may sound daunting, but in fact, it’s very achievable and simply requires some tried-and-true methods along with a cross-functional team to get the brainstorming on paper.  

Enhancing your organization’s approach to calculating COSQ is as much a culture change initiative challenge as it is a process improvement program, where planning and cross-functional team buy-in is required to ensure long-term COSQ calculation repeatability and success.

You can calculate the contributed cost of errors to your company’s bottom line with our Cost of an Error calculator: calculator.overops.com/cost-of-an-error

We hope this post aids you and your team’s efforts in outlining the many factors that impact the COSQ, the associated benefits of uncovering the specific COSQ issues at a deeper level, and the value of the many lasting benefits of pursuing superior software quality in your IT systems.

Herb Krasner is a retired Professor of Software Engineering at the U. of Texas at Austin now serving as an Advisory Board Member for the Consortium for IT Software Quality (CISQ). He has been an industry consultant for 5 decades, helping organizations baseline and improve their software development capabilities.