Metrics for judging the success of application-development projects are too complex for many organizations to understand, according to a survey commissioned by software firm Borland and carried out by Forrester Consulting.

The survey was conducted among the heads of application-development projects at 20 companies, all of which had revenues in excess of US$1 billion per year.

Borland's director of solutions marketing, Andy Seager, said: "Everyone is agreed we need metrics to understand projects because a good many IT [departments] simply don't know what to deliver."

According to the research, two factors "conspire to deter application-development organizations from attempting to improve their metrics programs". The first factor is the cost and complexity involved in collecting data for the metrics; and the second factor is over-reliance on what Forrester calls "superficial metrics".

"The lack of in-flight project metrics that really describe the work being performed on a project is a major fault of most application-development metrics programs," said the report.

To add to the problem, it is a fault that "most [IT departments] aren't even aware of", the report continued. "Without metrics about business value, application-development organizations are unable to communicate with their customers about their contributions to the bottom line or even prioritize the work in a way that makes real business sense."

Organizations sometimes do not get the right metrics in the first place because of the level of work involved in getting hold of them, according to the report. "The number-one obstacle to gathering meaningful metrics is the manual effort involved. Nearly half of the companies Forrester interviewed cited this as a challenge, and several of the companies reported that they spend nearly a third of their time on metrics collection," the report said.

In addition to these issues, the report added: "Development organizations struggle with the technical complexities involved in the trending and aggregation of metrics--where the bulk of the value of measurement is found."