No time for discovery? Breaking News: It's your fault. How to Explain its Financial Value to CEOs, CFOs, etc.
There is a lack of understanding of the value of Discovery. Let's see concrete tools to earn trust with a language that any stakeholder would understand.
Hey! Nacho here. Welcome to Product Direction’s newsletter. If you’re not a subscriber, here’s what you missed:
Subscribe to get access to all posts.
Imagine walking down the street and spotting someone with $100 intending to blow it all on 10 lottery tickets. There are 100.000 possible numbers1, so they would have a 0.01% chance of winning the prize.
You approach them and propose a deal: for only $2, you'll reveal the range of 10,000 numbers where the winning combination will lie. The person, captivated by the allure of a potential $150,000 prize, agrees, knowing they can only afford 9 tickets now. However, their chances of winning have just jumped from 0.01% to a respectable 0.09%.
As they prepare to purchase, you tantalize them with an even better offer: for an additional $18, you'll narrow down the potential winning range to 5,000 numbers. While their ticket count decreases to 8, their chances surge to 0.16%.
You're not done yet. You unveil your final, most tempting offer: for $50, you'll pinpoint the winning range to a mere 100 numbers. They're down to just 3 tickets, but their chances of winning have skyrocketed to 3% – a staggering improvement from the initial 0.01%.
The Value of Discovery (in terms that your CFO will understand)
While the lottery story may sound ridiculous, it is an exaggerated version of how discovery value works.
Companies have development teams that will work on features that, if decided merely with opinions, fail to deliver the desired outcomes more than 70% of the time.
So, on average, if a team delivers 10 features in a year, only 2 or 3 will move the needle.
But this is the most expensive way to learn. The company cost of a typical product team can be more than $500.000, so we are talking about a +$350.000 waste (just for 1 team!).
With Discovery, we spend a portion of our team's efforts and time collecting evidence to reduce the chance of building something that fails and discarding ideas that prove not worthy of our efforts.
So, if you invest 20% of your time in Discovery, you can “only” build 8 features. But what if you reduce the +70% failure rate to 50%? Instead of 2 or 3 needle moving features, you will have 4. Even when you account for this 20% of discovery as waste (which, of course, isn’t), total unprofitable work is reduced to €300.000.
What if you invest a considerable 40% in discovery? If your continuous experimentation leads you to a +80% success rate, while you can “only” build 6 features, 5 will move the needle, further reducing “waste” to €250.000.
Product Discovery isn't a luxury. It's a necessity.
Important note: you will still fail 70% of the time! With this 40% discovery time, you cheaply test many more ideas. The 6 you are building are the finalists of a process where you discarded many more options with research and experimentation.
Additional benefits
While I focused on the financial value -one that is tangible and easy to prove- there is an incredible number of benefits:
You don’t create a bloated product that is hard and costly to maintain.
Since we progressively invest in each idea we are analyzing, starting with very cheap experiments, we actually get to test many more ideas, maximizing our chances of finding winners.
Instead of shipping things your users don’t care about most of the time, you constantly ship things they value, increasing satisfaction and LTV.
Instead of “build and pray,” you continuously learn, increasing your ability to build successful product increments over time.
You not only collect facts to develop better solutions, but you also gain evidence to create a more solid strategy.
… and the list goes on.
You don’t have a time issue. You have a trust issue.
Like in our lottery story, no rational agent who understands this value will dismiss it (and most CEOs, CFOs, etc., are very smart rational agents).
So, not having time or resources for discovery is a problem of trust.
They don’t believe that 70% of initiatives fail.
They don’t trust that there are tools to collect facts to validate these ideas cheaply.
Let’s see it in more detail.
1. They don’t believe that 70% of initiatives fail
Sadly, most companies don’t properly track their track record and the impact of initiatives.
Considering many (if not most) companies operate as feature factories, the focus is on measuring feature delivery, not results.
Often, a very specific feature metric is reported without understanding how it truly impacted the business. “Feature adoption increased 5%.” Did it increase engagement? Did it reduce churn? Did it increase LTV?
And even when results are measured, hardly anyone keeps track of features over time.
No one likes this truth. So, we sugarcoat the results in many (if not most) companies. And not just the product team; there are many forces at play. Imagine a feature factory where sales requested a feature: it is also in their best interest to show how valuable it was, even when its direct correlation to metrics is hard to prove.
Eventually, not even CEOs or CFOs gain (or ask for) a clear understanding of the team’s track record, and a +70% failure rate becomes hard to believe.
2. They don’t trust that there are tools to collect facts to validate these ideas cheaply.
An even harder problem is agreeing on how we determine our confidence level and what is required to improve it.
Budget allocation and prioritization work most of the time by creating some sort of business case (or assessment or impact estimation). The “value” the initiative creates is pitched, and the case's credibility is argued primarily based on opinions. So it’s a game of “in which opportunity we believe.”
Unfortunately, confidence is not part of the business case equation most of the time, and there is hardly any way to put a scale to it or compare initiatives for which we have different types of evidence.
So, when you bring user interviews, experiments, and other tactics to the table, we still debate how valuable that information is and how it affects (or primarily does not affect) the decisions already made.
How to Showcase Discovery’s Value
That was a long intro to the most critical part: a systematic and logical way to show the value of Discovery.
There are 3 things entirely under your control that you can start doing to give Discovery the place it deserves:
Show the track record
Look back at your last year and truthfully review the impact of your initiatives. What was your hit rate? How many moved the needle?
Bonus points if you find highly visible ones for the storytelling. That pet project some executives pushed for. Some strong business cases from sales promising millions that never came.
Show early wins “silently introducing” some discovery
We all have some flexibility to introduce some small discovery tactics. Be strategic, and select a highly visible initiative for which you can collect evidence and show the value of the learning. A typical example is running a fake door test that challenges the value stated in the business case.
Keep a track record of decisions affected by your discovery efforts.
Agree on a confidence score
You have set the stage to put confidence higher on the map. You can leverage a tool like Itamar Gilad’s confidence meter as a starting point, but be flexible to account for what your stakeholders consider valuable evidence or not (my favorite debate is “competitors have it” -how valuable is that fact?)
Score all the business cases with a confidence score. This doesn’t dismiss the business case but signals which ones you must prioritize for discovery.
Start systematically updating and sharing the confidence scores with your discovery results.
This last item is the one I see most struggle with and the one I help product teams implement.
In any case, I would love to hear your experience. Does it resonate? Are you having other problems? Do you want to discuss some of these points further? Let’s chat.
Become a subscriber, and don’t miss any posts!
I’m basing the example in Spain’s National Lottery :)