Contents
		Introduction  Who's paying?  Hard to measure  Case study  Executive summary

Given its relatively low profile, there is no requirement yet for Australian companies to pursue CMM ideals to win corporate or government contracts. This could change in the long term, with some state governments said to be contemplating phasing in requirements that suppliers be quality certified.

For now, however, CMM remains a distant ideal for the majority of Australian companies. Perhaps it shouldn't be, however: a recent case series from Carnegie Mellon University's Software Engineering Institute, which developed and maintains the CMM body of standards, found that large adopters of CMM processes had delivered significant benefits.

At Boeing Australia, for example, a project to map development processes transferred CMMI Product Suite guidelines to the ISO 15504 (SPICE, or Software Process Improvement and Capability dEtermination) framework, an alternative method for assessing development processes. Over 18 months, Boeing reported a 33 percent decrease in the average cost to fix a defect, with time to turn around new releases cut in half and a 60 percent reduction in post-audit work.

Overseas, other providers were equally bullish about CMM. At General Motors, CMMI increased the percentage of milestones met during software development from around 50 percent to 95 percent; the average number of days late reduced from 50 to fewer than 10. US-based Lockheed Martin Management & Data Systems enjoyed a 20 percent reduction in unit software costs, and saw award fees--given for standout contract performance--increase by 55 percent between its achievement of CMM Level 2 in 1993 and full CMMI Level 5 certification in 2002.

IBM Global Services Australia has publicly said that integration of CMMI Level 5 ideals into its processes has saved more than US$230 million for its customers over five years and increased development productivity by 76 percent. Of 1000 IBM projects undertaken during 2002, 98.5 percent were on time and 99.2 percent within budget; the number of production problems has decreased by nearly two-thirds, and the level of "high impact" problems by over 90 percent.

Other firms reported similar benefits. Accenture reported a 5:1 ROI for quality activities, while defence contractor Northrop Grummann Information Technology rated ROI at 13:1 (calculated as defects avoided per hour spent in training and defect prevention) after following CMM Level 5 processes during the construction of an inventory tracking system.

CMM also had an impact outside of the companies surveyed, with software development customers enjoying delivery of better code, faster. Aerospace firm Thales Training & Simulation, which achieved CMMI Level 2 compliance in 2001, noted that schedule variances decreased as its process maturity increased. Cousin organization Thales Research & Technology, saw a 60 percent reduction in the cost of customer acceptance after progressing to maturity Level 3 of the Software CMM (SW-CMM), a predecessor to the current CMMI process.

These improvements don't come overnight, however: the Standard CMMI Appraisal Method for Process Improvement (SCAMPI) audit process is rigorous and intense. Accenture, for one, reported spending 8045 hours--six months' solid work for a team of eight--implementing the CMMI Product Suite. And each successive level of competence involves additional effort: the median time to move from CMM Level 1 to Level 2 is 24 months, according to Gartner. IBM's CMMI Level 5 certification required a review of 1100 developers in four cities, testing compliance with 425 distinct CMMI practices; its SCAMPI audit required five weeks' work from a team of nine auditors.

Clearly, compliance with CMM requirements translates directly into both cost savings (from shorter development times and fewer resources spent fixing problems) and smoother delivery for customers. It also offers intangible benefits, such as the creation of a quality development mindset that improves overall product delivery and innovation. Furthermore, having full CMM certification is still rare enough in Australia that it means something to potential customers.

While they may take CMM's ideals under advisement when planning quality activities, however, most Australian companies still appear to prefer ISO certification. Yet when every second company is ISO certified, presuming that the certification assumes a certain level of expertise, or ability to deliver, can be very problematic. Don't assume that ISO-certified suppliers are necessarily more effective than non-certified organizations; many smaller firms, particularly, lack the resources to commit to formal certifications but may well have developed a body of best practices upon which to call during their projects.

It is these practices, ultimately, that deliver the most value for customers of quality-certified organizations. Prudent assessment of tender bids, including a concerted effort to differentiate suppliers based on the experiences of their past customers, can be far more revealing than simply assuming that a particular certification assumes a particular level of competency. The key is to understand not how many certificates the potential supplier has on its wall--but how many of the vendor's employees regularly read them.