Trusting machine-powered financial forecasting

Human-centered design focuses on taking a holistic approach to trusting machine learning powered forecasting by incorporating individual requirements of the key user groups and the interactions across them. In this approach, user groups, also known as personas, are not defined by their function or business unit. They’re defined by their business objectives.

To promote successful adoption of the solution, it’s important to ask: 1) Did we account for the right user groups or personas when designing the capability, or were the personas defined too broadly? 2) Did we successfully connect the dots between each user group to create a comprehensive solution?

If adoption challenges are observed in a set of users, it could signal a missed or misaligned persona during the initial design. It’s critical to define meaningful personas through comprehensive discovery and research. The personas become the basis of the solution design, creating a unique experience for each persona while addressing any cross-persona interactions.

Understanding the impacted user groups and accounting for any personas you might have missed along the way can be a powerful mechanism to improve the user experience and shift focus to building a comprehensive forecast using the algorithmic platform.

Machine-learning financial forecasting capabilities can unlock benefits such as improved productivity—enabling a more detailed and granular forecast than before and generating more accurate outcomes, without driving additional effort to produce it. While these solutions serve as a central component of the forecasting process, they are not intended to replace the process entirely. In fact, deploying a forecasting solution presents an opportunity to rethink current processes and take advantage of the automation it enables. However, it is critical to ensure the solution fits into your organization’s forecast approach. A top-down forecast is unlikely to require the same granularity and complexity as a bottoms-up forecast, so they would require different considerations. With either approach, a thoughtful strategy is essential to work through the data management, process and talent needs to support the desired capabilities. Adoption challenges can often be traced back to a disconnect between the algorithmic solution and the broader forecast process. Introducing complex models and large datasets without the mechanisms to understand them can create a “black box” perception whereby the machine outputs seem unexplainable.

Making sense of the algorithmic forecast may require narrowing your focus to allow the machine to do the legwork to get the most meaningful components right. Focusing on the elements that drive most of the activity or volume for the business may be best suited for the algorithmic forecast in order to maximize its impact while reducing the noise created by superfluous data. When thinking about how you leverage the forecasting solution, consider the rationale behind your data collection: Are you including immaterial line items or products in your forecast? The solution isn’t intended to consider every line item, but rather to forecast the line items that matter most to how you plan your business.

Equally as important as the immersion of the algorithmic forecasting solution is the interaction between those who use it. No matter how your organization is structured, there are specific skills and interactions that will allow people and machines to work smarter together. Removing silos that have formed, whether by design or organically, can enhance knowledge sharing across domains. The key knowledge domains at play sit in data science, FP&A, and the functions of the impacted business units. The way they interact with both the machine and with one another is a critical indicator of the operating model effectiveness.

Data scientists are typically the ones doing the “heavy lifting” directly in the machine, especially at inception when the capability is being established, trained, and drivers are being added and evaluated. Finance business partners are responsible for driving and explaining the forecast to gain acceptance from their supported businesses and leadership. This requires the agility to make and explain changes. Often, this can be a cause of communication breakdowns and poor model adoption.

Classically trained finance business partners may struggle with forecast outputs that at first glance either do not appear explainable, or the explanation does not match a traditional forecast approach. Data scientists in turn tend to focus less on the broader business implications, so they may struggle to explain the cause and effect of the forecast inputs and outputs. Missing from this equation is the translation of analytics to insights, and the traceability of insights back to the analytics.

The role of a translator is to serve as the interpreter at the intersection of data science and finance terminology. The role requires the breadth but not necessarily the depth of knowledge needed for the data science and finance roles alone. In order to bridge the gap, translators need to understand both the basics of data science and how data science outputs are consumed by finance and the business units.

A well-designed solution, process, and operating model alone will not drive decision-making. All are important considerations for adoption but in order to understand the real-time decision triggers at play, we need to take a closer look at the underlying behavioral components. This can be considered the finance adaptation of the “last mile” problem, where adoption can be considered the final and most challenging leg of the algorithmic forecasting supply chain. Since this “last mile” is often embedded as a step or stage in a larger finance transformation journey, it can further add to organizational reluctance as the new operating model and the capabilities and tools to support it may all be evolving at the same time.

In this case, making sense of algorithmic forecasting is less about inherent model logic and more about understanding the adoption barriers or noise clouding users’ choices. Once the source of the noise is uncovered, we can influence behavior through “nudges”¹ that remove barriers and reinforce the machine capabilities. In order to do this effectively, the lens should broaden beyond design and process. It should consider the prevailing culture and incentives. The most accurate forecast may not always be the “right” one if there are competing or misaligned motivators.

Inserting steps to monitor and analyze performance can be a powerful tool for continuously improving performance while reinforcing its use. This can be done by comparing the machine outputs to status quo forecast and tracking for accuracy. Asking “Where were my assumptions more accurate than the algorithmic forecasts?” can help inform machine improvement opportunities. Asking “Where was the model more accurate than my assumptions?” can demonstrate where users can partner with the machine to create a more accurate forecast.