Metrics Matter Even in Product Development

What do metrics mean to business? Since the time that W. Edwards Deming introduced statistical process control to the Japanese in the late ‘40s/early ‘50s, manufacturing metrics have led the world down a quality revolution that, while significantly matured in the last half of the 20th century, remains fundamentally incomplete. However, as programs such as Lean Manufacturing and Six Sigma succeed on the shop floor, similar improvement methods and philosophies have yielded comparably little progress in the product development systems that feed the production schedules of manufacturing. Many have made gains in cycle time, product cost and other elements, but few feel they have achieved the desirable level of process control.

So what is it that is keeping engineering, marketing and affiliated product development-related functions from achieving the quantum improvements enjoyed by their manufacturing brethren? One has only to look at the metrics.

The critical component of successful quality in manufacturing is that its events and phenomena consist of controllable, finite variables. The output yield of capital equipment, such as machine tools, can be optimized to a predictable level with extreme accuracy. But the output of a product development team, by comparison, seems intangible, unpredictable, and uncontrollable. After all, product development activities are largely non-physical, unrepeatable, and exponentially more variable. Most say this is because product development is a “creative process,” while manufacturing is its “physical execution.”

Think about it. There is a giant difference between “How long will it take to make 100 widgets?” and “How long until you finish the drawing for the control interface?” In the first instance, the parameters are relatively finite, in the second, nearly infinite. The widget producer knows his machine makes between 48-52 widgets every hour, plus he knows that this is dependent on limited variables such as inventory, machine setup time and a few other predictable and controllable things. By contrast, the engineer may provide an estimate for completion time, but his accuracy is much worse, plus there are many more variables that can affect him. He need only be distracted by one phone call from the person who took over his last project or to discover that a part he specified is no longer available to be delayed for an unpredictable period of time, potentially costing the project significant time and money that nobody will notice until it is too late, if at all.

If you take an honest look at the way companies are structured, there is also a very critical cultural component to this problem. Compare the demographics of shop floor staff with engineering staff. You will quickly notice a socio-economic gap between the two. Engineers are some of the most highly educated employees of any company, and often in highly specialized disciplines from prestigious universities. Such people are notoriously skeptical by training, and typically have little affection for attempts to have their work “controlled” by things such as metrics.

On the other hand, manufacturing personnel live in a daily environment of control, with defined reward systems for compliance. Good shop floors maintain tight work discipline for cleanliness, preparation, and safety. These things are nearly non-existent in engineering and marketing departments, making this side of the house much less willing to accept changes that incorporate increased accountability. Metrics are often used to expose problems that many don’t want credit for. These flaws often result in a staff that becomes proficient at “gaming” performance measurement systems “Dilbert style.” To ignore these critical cultural issues is an immediate death sentence for any improvement effort.

So why then pursue such a challenge? The downstream effects of this lack of control in product development mirrors what happens with poor production methods: excessive rework, abundant waste and scrap, and embarrassing and costly late deliveries to customers. Once companies initiate effective quality programs on the shop floor and get a taste of the good life of statistically controlled, predictable and accurate manufacturing, it is not at all a surprise that their efforts would then turn to the rest of their business. What is most desired is a process that makes your entire business predictable—a “stretch goal” that is perhaps unachievable.

Why shouldn’t companies simply be satisfied with efficient manufacturing alone? It’s simply the economics. It is widely argued that 80% of a company’s profits can be directly attributed to 20% of the organization—the product development function—which includes engineering, R&D, marketing and all of the pieces that compose this cross-functional group. In theory, to press on this leverage point and streamline development would complete the picture and enable one to take full advantage of previous gains in manufacturing efficiency. The result would undoubtedly be a money-making machine.

So what’s the reality? Many people treat metrics like any other “best practice”: they search for whatever a majority of similar businesses and industries are using and gravitate towards anything “plug-and-play”. Still others are able to look deep internally and compile the set of specific measurements that indicate the success, failure and progress of critical and core functions. Some succeed with comprehensive tools like the Balanced Scorecard, while others reject systems that have a complex and data-intensive focus. While there is no “one-size-fits-all” metrics system, there is an elastic waistband of measures if you take the time to find what’s right for you.

Product developers can and will buy into metrics programs if they are designed so as to cause action and improvement rather than merely assign blame and foster distrust. Like any other improvement process, it pays dividends to ensure affected people fully understand the connection between what is being measured and how that specifically affects decision making. While metrics can create a minefield that has disaster potential, businesses will always seek them out to satisfy their hunger and greed for control, so what’s important to remember is to be mindful of side effects and unintended consequences. Remember the old saying, “If you can’t measure it, you can’t improve it.” And the new saying, “If you can’t measure it, you can’t blame anybody.”