Bad idea: You can’t measure productivity

The key points from Martin Fowler’s 2003 article “Cannot Measure Productivity”:

• The reason why we can’t measure productivity is because we can’t measure output.
• Using lines of code (LOC) as a measure of output is bad because bad designs tend to require more lines of code.
• Using function points, as a measure of output is bad because it’s not stable across estimators.
• Any true measure of software development productivity must be based on delivered business value.
• Business value is not just hard to measure, it may have a time lag too.

I agree with every point except the first one.

Productivity should be based on impact, not output. Because impact may be lagging, it can be useful to measure intermediate outcomes as a shorter-term productivity indicator.

Productivity should be based on impact, not output.

What about frequency and lead time for impact?

When a (competent) executive talks about productivity, they’re not referring to how many lines of codes are being produced, nor even how many features are being produced; they’re referring to how much economic value is being produced. The kind of productivity (competent) businesses care about is impact over investment.

This is what I see as the actual key point from Fowler’s article. Instead of “Cannot Measure Productivity”, it should have been titled “Productivity Should Be Based On Delivered Value”.

There’s a useful mental model by Kent Beck and Gergely Orosz in Measuring developer productivity? A response to McKinsey:

Effort → Output → Outcome → Impact

We engage in activities (effort) to build features (output) that cause changes in customer behaviours (outcomes) that eventually lead to value to the organization (impact).

Productivity is the ratio of the value of Impact to the cost of Effort.

Productivity = Value of Impact ($) / Cost of Effort ($)

Because impact may be lagging, it can be useful to measure intermediate outcomes as a shorter-term productivity indicator.

As Fowler mentioned, impact can often be lagging with product development. However, this does not mean that you can’t measure impact nor that impact isn’t worth measuring. This does mean that it can be more useful to use outcomes (i.e., customer behaviours) instead of impact for shorter-term tracking. (See Outcome over Output)

For example, my theory of product could be that people who add more items to shopping lists will eventually buy more things. So, I could assess how productive different approaches, different teams, etc. are in getting customers to add more items to shopping lists.

Example theory of product

Eventually you do want to assess impact. This is essentially checking that your theory of product, that is “these outcomes lead to this impact”, is correct. For example, did people who added more items to shopping lists actually buy more things. This allows us to answer the top-level productivity question around “How much did we get given how much we invested?”