Dean Latchana – latchana.co.uk

Why Your AI Tooling Isn’t Working: It’s Not the Tech, It’s the Culture

Recently I delivered a presentation about being outcome-focused when introducing tooling and technology, over merely measuring output. Output tends to target lines of code, task completion and busy work. Whereas a focus on outcomes tends to measure proof-of-value, business challenges being met and sustained user satisfaction.

Many organisations have spent time and treasure seeking success with AI tooling, yet struggle to attain valuable user and business impact. So measuring and targeting success with AI is an increasing concern. To put it bluntly, for some, their claims and reputations are on the line.

I cautioned against an over-focus on output by citing Goodhart’s Law, which states “When a measure becomes a target, it ceases to be a good measure”. This has been recently demonstrated by the phenomenon of tokenmaxxing.

An attendee asked, how can you promote tool adoption that is not merely targeted by measuring tool sign-ins? Doing so could lead to users just signing in to show apparent usage, but not actually getting value from the tool. It could also be a result of teams being coerced to use a tool, despite their experience that the tool isn’t helping them.

The attendee’s question deserves careful consideration. My recommendation is not to think about metrics initially. Rather to focus on the user problem to be solved, and how that can ladder-up to business impact. Take a product-centric approach which incrementally validates whether the tool is desirable, viable and feasible.

Desirability leads to user behaviour changes. These are the leading indicators that can be measured by repeat usage, user recommendations and user satisfaction. Viability is the consequential business impact. These are the lagging indicators that can be measured by cost reduction, revenue growth and market share. Feasibility ensures a tool can be built, financed and operationalised.

While the DVF framework provides the ‘what,’ alone it’ll fail to address the ‘why’ organisations struggle—the hidden, cultural inhibitors to success. So, for many organisations the near-insurmountable challenge isn’t the relationship between types of metrics. It’s the competing pressures, ill-fitting culture and organisational psychology.

Sadly, many organisations have become so pressure-bound for ROI from AI, or certain leaders are so narrowly focused on resource utilisation, that genuinely addressing user needs and creating business impact is an afterthought.

For many organisations, teams don’t feel safe to push back. Teams silently suffer the consequences of the sunk cost fallacy and commitment bias.

So, organisations shouldn’t only institute disciplines that judge success with the right balance of measures. They need to re-imagine their culture and operations for safe-to-learn innovation that selects for tools that genuinely address user and business needs. Using models such as Westrum’s cultural topologies, culture itself can be examined and influenced.

Give me a call if you’d like to know more.

Ron Westrum’s model of Organisation Culture

When AI Becomes a Target: The Perverse Outcomes of Compelled Usage

We’ve seen this movie before. Organisations are so keen for return on their AI investments, they compel their workforce to utilise AI tools, which leads to perverse outcomes.

A recent example is Amazon setting a target for staff to use internal AI tools. To meet the target, and to consume AI tokens, some individuals have apparently automated unnecessary activities. This phenomenon of tokenmaxxing improves the rank on an internal leaderboard.

Obviously, this spurious use of AI tools wasn’t the intent. Such unintended consequences are an example of Goodhart’s Law:

“When a measure becomes a target, it ceases to be a good measure”

Rather than set targets, organisations ought to consider the underlying needs and purpose of their workforce and jointly discover solutions that will alleviate their problems.

The most impactful problem-solution fit might not even be technological. Often it is related to some misaligned combination of purpose, priorities, coordination, and process.

So rather than succumbing to pressure and becoming fixated on rolling out ill-fitting AI tools, genuinely partner with your workforce to develop sustainable solutions to their challenges. The set of solutions may or may not include AI tools.

The work I do helps managers and teams identify the mismatch between purpose and tools. It leads to a deeper appreciation and a set of interventions which genuinely enable greater individual and team performance.

The rule of thumb is to focus on purpose & people, process and tools, in that order of importance. It’s not to repeat the bad movie plot of compelling AI usage by any means.

Do less Proof-of-Concepts

The challenge with proof-of-concepts

To understand how AI could be beneficial, many organisations are undertaking a programme of AI proof-of-concepts (PoCs). They’re seeking to demonstrate how such genuinely phenomenal tools could benefit them. I fear this approach is mistaken.

On their own a portfolio of PoCs does not validate whether they’d sufficiently address a prioritised set of business needs. Neither do PoCs validate whether the capability can be integrated and scaled in the field, without duplication or being cost prohibitive.

At best such PoCs are little more than technical demonstrations of a capability. At worst they create distraction, sunk costs and delay.

Introduce discipline

I believe we first need to put technology out of our mind. Instead we need to focus on the problems of customers, colleagues and the organisation. Identify the need before potentially overspending on an AI solution that may go nowhere.

Once there’s common alignment and prioritisation of the problem to be solved, we then need to canvas for possible solutions. Such solutions may not be technological; a solution could be related to purpose, people or process. I often ask whether a problem can be solved without touching one line of code.

Whatever the potential solution, discipline is needed to ensure its given the oxygen (e.g. funding) only if it shows demonstratable promise to sustainably solve the prioritised problem. This represents proof-of-value.

Truthfully, something like only 1 in 10 will graduate from proof-of-concept to become proof-of-value. This is the nature of discovering the needle in the haystack of organisational complexity.

What kind of value?

When assessing a solution, there are a number of frameworks for value. Here’s a couple.

Consider David Kelley’s design thinking framework Desirability, Feasibility, Viability. It checks whether users actually want or need it (desirable), whether we can actually build it (feasible), and whether our business should do it (viable).

Another example is Technology Readiness Levels, which help determine which solutions are little more than concepts, and which actually show value in the field and are not financially exorbitant.

Conclusion

Many organisations are keen to see value from AI and automation. However, they often take a backward approach of expending talent, capital and time to see what proof-of-concepts stick to the wall. This will likely lead to PoC fatigue, frustration and deepen operational messiness.

Rather than take this solution-first approach, organisations should take a targeted outcome-focused approach. This starts with agreement on the problem to be addressed. Then organisations should institute a discipline of evaluating which solutions are proven to show sustainable value.

Underpin the outcome-focused approach with techniques like hypothesis-driven development and Changeban.

So it’s not so much AI adoption. It’s more outcome-first adoption, which may be achieved with some AI solution.

Wardley Map for Stock Markets

During a Saturday afternoon run, the thought occurred to me that a Wardley Map could illustrate the different make-up of various stock markets.

The FTSE 100 index is weighted toward mining, energy, and banking. These are established commodities and utilities where businesses aren’t usually driven by disruptive innovation. The FTSE 100 is a value index where investors generally benefit from dividends, but lack growth potential. So, I wondered if FTSE stocks tend to be in the Commodity stage of the Wardley Map.

The S&P 500 index has higher risk and potential for greater capital growth. Companies in this index have more tendency to create new products and services. So, I thought they would fall into Custom and Product stages of the Wardley Map.

With this brainwave, I spoke a gabbled note into my phone as I ran home. That evening while the house was quiet I threw my consideration into my favourite LLM. I was excited to learn my premise was sufficiently sound so I created this Wardley Map.

The map illustrates clearly the contrast between stocks in the two indices. Safe & stable FTSE 100 stocks are on the right. More risky and high-potential S&P 500 stocks are on the left.

I was helped to realise the value chain (vertical axis) is from the perspective of an investor who needs to balance growth potential with stability. These ‘user needs’ are represented by the blue circles.

A prudent investor would likely balance their investment in both indices represented by the red circles. This would then expose them to the individual stocks in both indices illustrated by the green circles.

I then dabbled with the idea of showing some relationships between some companies. For example, in order for NVIDIA to provision data centre hardware there’s an indirect demand dependency on data centre build out. This is represented by the gold dashed line linking NVIDIA to VERTIV. VERTIV provides the data centre infrastructure and is innovating in power and cooling.

I could have gone further by adding another gold dashed line linking data centre companies to those providing energy (right side).

At the risk of complicating the map, I illustrated one more consideration. That is the movement of Apple. They’re predominantly a ‘cash cow’ with aspects of lower radical innovation, such as with their iPhone. Therefore, I illustrated it with a black arrow showing them evolving to the right.

I went to bed somewhat satisfied. What do you think?

AI and Organisational Fragility

Recent research in The Lancet reveals a systemic warning: after using AI assistance, experienced doctors’ independent diagnostic skills dropped by 20%.

This is the Automation Paradox — where “efficiency” atrophies the very intuition we rely on.

If we outsource the “hard-won lessons” to algorithms, we will likely lose the expertise for independent thought.

Are we augmenting our people or hollowing them out, leading to organisational fragility?

We must protect human judgment by designing for productive friction, ensuring AI remains a tool, not a crutch.

Ping me if you’d like to explore this.