Recently I delivered a presentation about being outcome-focused when introducing tooling and technology, over merely measuring output. Output tends to target lines of code, task completion and busy work. Whereas a focus on outcomes tends to measure proof-of-value, business challenges being met and sustained user satisfaction.

Many organisations have spent time and treasure seeking success with AI tooling, yet struggle to attain valuable user and business impact. So measuring and targeting success with AI is an increasing concern. To put it bluntly, for some, their claims and reputations are on the line.
I cautioned against an over-focus on output by citing Goodhart’s Law, which states “When a measure becomes a target, it ceases to be a good measure”. This has been recently demonstrated by the phenomenon of tokenmaxxing.
An attendee asked, how can you promote tool adoption that is not merely targeted by measuring tool sign-ins? Doing so could lead to users just signing in to show apparent usage, but not actually getting value from the tool. It could also be a result of teams being coerced to use a tool, despite their experience that the tool isn’t helping them.
The attendee’s question deserves careful consideration. My recommendation is not to think about metrics initially. Rather to focus on the user problem to be solved, and how that can ladder-up to business impact. Take a product-centric approach which incrementally validates whether the tool is desirable, viable and feasible.
Desirability leads to user behaviour changes. These are the leading indicators that can be measured by repeat usage, user recommendations and user satisfaction. Viability is the consequential business impact. These are the lagging indicators that can be measured by cost reduction, revenue growth and market share. Feasibility ensures a tool can be built, financed and operationalised.
While the DVF framework provides the ‘what,’ alone it’ll fail to address the ‘why’ organisations struggle—the hidden, cultural inhibitors to success. So, for many organisations the near-insurmountable challenge isn’t the relationship between types of metrics. It’s the competing pressures, ill-fitting culture and organisational psychology.
Sadly, many organisations have become so pressure-bound for ROI from AI, or certain leaders are so narrowly focused on resource utilisation, that genuinely addressing user needs and creating business impact is an afterthought.
For many organisations, teams don’t feel safe to push back. Teams silently suffer the consequences of the sunk cost fallacy and commitment bias.
So, organisations shouldn’t only institute disciplines that judge success with the right balance of measures. They need to re-imagine their culture and operations for safe-to-learn innovation that selects for tools that genuinely address user and business needs. Using models such as Westrum’s cultural topologies, culture itself can be examined and influenced.
Give me a call if you’d like to know more.

