How to unlock business value with AI by focusing on user experience and enterprise value levers
Extracting business value is hard. I mean really hard… And when it comes to advanced analytics that process is even more difficult. After struggling to convince executive teams and boards of the value of these solutions, I have realized there are 5 critical components of extracting business value from AI projects.
In this post I hope to illuminate these principles as best I can and hope that it will spur further conversation. I don’t consider these principles exhaustive by any means, but I do consider them to be the most common source of frustration across most companies embarking on their journey into AI
1. Can a front line employee understand each prediction?
To and end-user, most predictions without explanation of what is making up the prediction is useless. Imagine you are a sales rep being told that this opportunity only has a 50% chance of closing. This does two things. 1) Immediately annoys you. “How dare a computer model tell me my deal stinks”, 2) Removes all trust in the model in the future.
On the other hand, if you tell the user
Your opportunity has a 50% chance of closing because the close date has moved twice in the last quarter — we suggest you re-qualify the importance of the deal with the customer
there is an intuitive understanding of why the prediction is what it is. That is explainable and more importantly actionable
2. Does your business process match your features?
Further to principal 1, make sure the features you use to explain your predictions can be modified in a business process.
I know everyone likes to throw a tone of features at a model to get the most accurate results, but if your end-user can’t change the value of a feature, then it is at best useless information and at worst demoralizing to them to present that feature as a reason code.
As a simple example, say your model predicts that the industry in which a business is in is a large determining factor of success. Can your salesperson change the industry? No, so telling them they are “32% less likely to close the deal because the prospect is in manufacturing” will just annoy them, or worse make them abandon the deal in a sort of wicked self-fulfilling prophecy. I have experienced this behaviour many times in the sales departments I have consulted for.
3. Can the C-Suite use the models to make decisions?
So you can accurately predict an outcome for a forecast in the current month. Great, what happens when you are predicting terrible attainment of, say, 62%? Does your model let the executive team know what they should do to fix that problem? My guess is no. An accurate prediction of a number if fine, but a better prediction is how to change that number, and how much time is needed to change it. If you tell the CRO and CMO they will miss forecast by 62% next month, you had better also tell the CMO what they need to do to make sure that doesn’t happen. For example, “you will miss by 30% next quarter unless you increase add spend by 12% this week based on current conversation stats”
On the individual feature level, let’s go back to the industry column. If you do have a bunch of predictions that are resulting in “industry = manufacturing” is a lot less likely to close, that is a tangible decision making element for the C-Suite. They could review the performance of the model against this explanation and simply chose to not expend any more resources on selling into the manufacturing sector.
4. Is there a strong business case for AI vs. simpler methods?
There is still a lot of hype around AI and this is driving everyone to look at an AI solution for every problem. But ask yourself, from a business value perspective does using AI tools have a positive net present value (NPV)? What if we use a custom vs commercial solution (more on how to choose that later post)?
I have personally fallen into this trap. In one example, we developed an AI tool that would allow us to deduce NPS from audio transcriptions of customer phone calls. This was an attempt to avoid overwhelming the customer with NPS surveys and get a better completion rate. All said and done, the 3-month trial cost us about 3x the amount of revenue we saved. When we retroactively performed a detailed NPV calculation including all compute costs, labour costs to build and train the models and keep them up to date, and some intelligent assumptions about the increase in retention rates, we determined that the NPV was negative over 5 years. So we abandoned this approach for a simple NPS survey.
5. Do your models support cross-business unit decision making?
A tool that supports decision processes in a single business unit is fine, but true value unlock comes when you can automate decision processes across business units. This is the key factor in choosing to build or buy a tool.
Let’s take working capital for example. As a refresher, working capital is made up of three elements: Days Payables outstanding, Days Sales Outstanding and Days Inventory outstanding.
A simple prediction of days it will take to collect payments from a customer will add obvious business value in accelerating cash collection, but a model that can optimize all three of these elements will add materially to your balance sheet and ultimately affect your valuation if you are private or your stock price if you are public. That is something that any C-level will get behind.
