The intelligent enterprise has been saved
More than a decade after the concept of big data became part of the lexicon, only a minority of companies have become insights-driven organizations—and culture may be the culprit.
The amount of data available to organizations every day continues to proliferate at a staggering volume. But technologies such as analytics and artificial intelligence (AI) have the potential to help businesses make better use of these massive volumes of data. In the Age of With™1, they represent the next step in human collaboration—humans and machines working together in a designed system.
But not every organization is optimizing the opportunities available in the Age of With. Some do little or nothing with data to aid their decision-making. Others carry out analytics projects in pockets of the business. Far fewer consistently embed analysis, data, and evidence-based reasoning into their decision-making process.
Most large companies fall into the last two categories, as do all the companies surveyed recently by Deloitte in our Becoming an Insight-Driven Organization survey (see sidebar, “Survey methodology: Becoming an insight-driven organization”).
It’s understandable why many companies feel they are far down the path of becoming an insight-driven organization. Many have invested in creating the requisite data initiatives, analytics, or data science groups. Many have created chief data officer (CDO) or chief analytics officer (CAO) organizations. The vast majority have invested in tactical solutions.
These evolutions seem natural. It has been more than a decade since the term “big data” became part of the lexicon. Many legacy issues that traditionally posed barriers have now been eliminated or reduced. These include the high cost of data storage, expensive proprietary software, and the need to devote capital to expensive data centers.
But what is the reality? How many companies have truly evolved into insight-driven organizations?
To find out, in April 2019, Deloitte posed questions to more than 1,000 executives working at large companies (500+ employees) who interact with, create, or use analytics as part of their job. The goal of the Becoming an Insight-Driven Organization survey was to see how many identified their company as being in the top two categories of Deloitte’s IDO Maturity Scale (see figure 1)—analytical companies and analytical competitors2—as well as discovering how fully they leverage data and tools to make decisions, and what role culture and talent play in IDO maturity. Among our findings:
To obtain a cross-industry view of how organizations approach business analytics and where they fall along an analytics continuum, Deloitte conducted an online survey of 1,048 executives (senior managers or higher) who interact with, create, or use analytics as part of their job and work for US-based companies with 501+ employees. Thirty-six percent are senior vice-presidents and above, and 13 percent are from the C-level. We eliminated 928 potential respondents representing approximately 35 percent of the potential survey base because they do not have any interaction with analytics in their companies. The survey was fielded in April 2019 with a margin of error of ± 3.03 percent at the 95 percent confidence level.
Three-quarters of respondents to Deloitte’s Becoming an Insights-Driven Organization survey report that their organization’s analytical maturity has increased over the past year, and nearly as many—70 percent—expect business analytics to be more important in the next three years than it is now.
Accompanying these indicators of increased organizational awareness is the finding that over the next few years, business analytics as an organizational priority is expected to be on par with such critical drivers of business value as risk management, reputation management, product/service innovation, and managing growth expectations (see figure 2). In other words, analytics is becoming an established fact of business life and no longer the sole domain of the IT or finance department.
AI, the more technology-intensive relative of business analytics, is not yet used as commonly as some other business and management tools—though other Deloitte surveys suggest that its use is growing even more rapidly than business analytics.3
As figure 3 shows, analytics is being used to support a number of strategic areas representing a broad spread of fairly mainstream use cases with no low frequencies. The top use—identifying business process improvements—aligns with one of the top organizational priorities among respondents shown in figure 1—overall cost efficiency. Using analytics to improve processes, such as optimizing pricing or inventory levels, represents a classic use of analytics.
Slightly less common is the use of analytics to guide product and service development or monitor competitors. This is not surprising. Typically, there is less structured numeric data available in these functions. Also, as is discussed later in this report, companies are far less likely to rely on unstructured data such as product images or customer comments gathered from interacting with a chatbot while shopping online.
Being an insight-driven organization is not the result of any single factor; it is multidimensional. For organizations to fully leverage the insights they derive and embed them into decisions and actions, a combination of three drivers is required: data and tools, talent, and culture.
Although it’s only one component of success, investment in and adoption of data and tools is perhaps the most visible and easily measured manifestation of where a company is on its journey to becoming an insight-driven organization.
The traditional workhorses of the data analytics universe—spreadsheets such as Microsoft Excel and business intelligence tools such as Microsoft Power BI or IBM Cognos—are the most commonly used tools. Yet 67 percent of the companies surveyed also use at least one advanced tool such as SAS, an open source tool like R, a programming language such as Python, or an AI tool (figure 4).
This range of tool sophistication is encouraging. However, most organizations in the survey are at risk of hindering the success of their analytics by limiting the types of data they employ and limiting the chance to adopt a cross-enterprise approach.
According to our Becoming an Insights-Driven Organization survey, most organizations (64 percent) today report relying on structured data from internal systems/resources. Far fewer (18 percent) have taken advantage of unstructured data (such as product images or customer audio files) or comments from social media. These data types can be difficult to put in the row-and-column relational format characteristic of traditional data storage, but over the last decade several new technologies have emerged to address that, including Hadoop and other open source projects, cloud-based architectures, approaches to managing streaming data, and new storage hardware environments.4
These forms of data are often more challenging to interpret but can deliver a more comprehensive and holistic understanding of the bigger picture—particularly of the world outside an organization. A look at companies that exceeded their business goals shows a connection with this appreciation for unstructured data: Executives who say unstructured data is one of the most valuable sources of insights are 24 percent more likely to have exceeded their business goals, according to our Becoming an Insights-Driven Organization survey.
The majority of companies today also adopt a fragmented, siloed approach to analytics tools and data, which correlates with diminished business success. Sixty percent of companies in the survey that use different tools and systems across different teams or business units exceeded their goals last year. But among a much smaller group—the 26 percent that use a single, common set of tools and methods across the enterprise for accessing and analyzing data—an impressive 80 percent exceeded their business goals last year. The absence of one enterprise approach to finding insights is a common barrier to effectiveness.
Analytics: The systematic quantitative analysis of raw data to draw conclusions that help drive business strategy and performance.5
Artificial intelligence (AI): The development of computer systems that can perform tasks previously performed only by humans.
Big data: Big data is structured and unstructured data generated from diverse sources in real time, in volumes too large for traditional technologies to capture, manage, and process in a timely manner.6
Machine learning (ML): A method of data analysis that automates the building of analytical models. These algorithm-based models are primarily built from statistical techniques and theoretical computer science and leverage large datasets to continuously learn and improve.7
Procter & Gamble (P&G) has long maintained a common, enterprisewide approach to data management and analytics as part of its Global Business Services organization. It established global standards for data type and quality; with these standards, aggregating and comparing data across product lines and regions became significantly less complicated with the data being stored in standard formats in the central data warehouse. Over time, managers found that the single, companywide database played a much more strategic role in aiding decision-making by serving as the “one truth” for the entire corporation. The company’s IT organization also automated the generation of reports that were used across multiple business units. In addition to simplifying the task of retrieving data and performing some basic analyses, the reports served to standardize the way data was visualized across the company.8 With consistent visuals, analysts and managers from one unit could step into a role, or even a meeting, with a different product or region and quickly understand the situation. The IT organization also developed scaled and standard solutions for accessing analytics, including the Decision Cockpit, a web-based customizable dashboard that tracked the most relevant data and news for each individual employee. For management teams, P&G developed a patented physical environment for information-based executive decision-making called the Business Sphere. Leadership teams would typically gather in over 50 Business Spheres around the world at regular intervals to review product and market performance for the previous periods and make decisions about how to proceed going forward.9
Talent can be spread broadly across an organization or concentrated among a select few. D.J. Patil, appointed as the US government’s first-ever chief data scientist in 2015, liked to say that data science is a team sport.
But if data science comprises teams within organizations, the teams are generally small and homogeneous. The survey data shows that two-thirds of organizations rely on a select group of employees who have been trained on analytics or data science, versus 27 percent who say that all employees are trained (quite rare in our experience) or the small minority that trains no employees.
Organizations need to embrace a diversity of roles and skills. Instead of relying on siloed teams of highly technical quantitative experts, companies would do well to cultivate a wide variety of people throughout the organization who are curious, numerate, and capable of translating between analytics/data science methods and business requirements. This might be called the “democratization” of data science.”10
Data from the survey confirms the merit of this approach. In companies where executives report that all employees have been trained on analytics, 88 percent exceeded business goals, compared to just 61 percent of companies in which only select employees have been trained on analytics.
Concomitant with the need to involve all employees in the use of analytics for decision-making is the need for user-centered design and stakeholder analysis. Decision-makers, data scientists, and business analysts, all must care about the business outcomes and be consulted frequently on analytics projects. Effective users of analytics are those who understand what matters most, prioritizing the right questions and zeroing in on the right decisions to improve. They need to understand where to find the insights and quickly know what to discard, to avoid being overwhelmed by the massive volumes of data that businesses now routinely generate or receive.
The issue of improving companywide access and use of analytics is not restricted to employees at lower levels in the organization. The survey shows that 67 percent of those surveyed (who are senior managers or higher and interact with data) say they are not comfortable accessing or using data from their tools and resources (see figure 5). Surprisingly, comfort level rises with title level, which could show access to resources also rises with level in the organization. In any case, the fact that two-thirds of executives at large organizations are not comfortable navigating what is now the lifeblood of every business is a gap that could yield huge dividends if addressed.
Principal Financial Group, based in Des Moines, Iowa, offers insurance, retirement, and asset management products. Recruiting data scientists and AI specialists is difficult anywhere, and especially away from hotspots such as the Bay Area, Boston, and New York; so Joseph Byrum, the company’s chief data scientist, has been forced to be creative about building a talent ecosystem. In addition to hiring data scientists, he’s explored crowdsourcing and close relationships with university analytics programs. Over the course of his career, Byrum has worked with thousands of crowdsourcing projects, and he strongly believes that they generally yield more innovative solutions than working with internal domain experts. He works with several different crowdsourcing firms. In addition to crowdsourcing, Byrum and Principal also rely heavily on an even less common talent source—student capstone projects in analytically oriented programs at universities. There are now hundreds of such programs in the United States alone,11 and many of them have “capstone” programs in which students are expected to work on real business problems with corporate sponsors. Byrum has set up over 60 capstone projects at a variety of universities—many involving important problems for Principal to solve. Of course, Principal does need to hire some internal people as well to help develop AI and analytics solutions, and Byrum has thus far built a team of about 20. He hires often through the capstone projects, which Byrum finds much more useful than interviews. He says that the staff that he does hire are a little different than most data science types in other companies—they have quant skills, but also design thinking and systems engineering skills. He also puts a strong emphasis on their ability to communicate and collaborate and uses psychological profiles to ensure that members of his team complement each other.12
Among the key drivers that help companies scale from carrying out analytics projects in pockets to becoming an IDO, a data-driven culture is the most difficult to establish. It also appears to be the one factor holding back many organizations.
A data-driven culture is one in which important decisions are made based on data and analytics (assuming that data is available). It is important to note that data, technology, and solid statistical and machine learning capabilities are facilitators but not necessarily drivers of insights-driven decision-making. There has to be a willingness to act on analytically derived insights—to make decisions, change processes, and adapt behaviors based on insights rather than intuition.
The survey highlights this powerfully by revealing a strong correlation between culture and business performance: Organizations that reported having the strongest cultural orientation to data-driven insights and decision-making are twice as likely to have reported exceeding business goals in the past 12 months. Forty-eight percent of these businesses say they outperformed their target versus just 22 percent of those with a more diluted analytics culture (see figure 6).
A twofold difference in not just meeting but exceeding business goals is a significant difference, and it contributes to the dividing lines between the insight haves and have-nots we describe in the introduction.
Creating any form of desired culture is a challenge for most organizations, and analytics is no exception. Only 39 percent in our survey say their company has a strong cultural orientation to data-driven insights and decision-making, and a similarly low percentage (37 percent) feel that employees in their company are aware of the importance of data analytics.
But, as with other areas in this survey, organizations that have a strong analytics culture and employee awareness of analytics are more likely to have exceeded business goals, illustrating the bottom-line benefits of investing in creating the right culture and elevating employee awareness of the importance of analytics.
How companies assign responsibility for analytics is a critical factor in moving up the maturity scale. Here the results are encouraging: Analytics is deployed companywide at the majority of organizations (57 percent), while one in three deploy across specific company groups and only one in 10 deploy them in an ad-hoc manner.
As with talent, the survey data indicates that diffusing responsibility across organizational lines is more effective than localized responsibility (see figure 7). However, the survey shows that most rely on a pool of identified experts rather than charging all employees with responsibility for developing insights or learnings from data analytics. In other words, analytics is not yet a team sport.
The concept of necessary-but-not-sufficient is a critical one for insights-driven maturity. It is easy to mistake the necessary for good enough; but building tools and acquiring data are not sufficient to reach the top levels of IDO maturity. As figure 7 illustrates, spreading responsibility for analytics across the enterprise and making analytics-gathering and decision-making a team sport are crucial to success. Acquiring tools without engaging teams will not lead to better business outcomes.
As stated earlier, when companies in the survey were asked to rank themselves on Deloitte’s IDO Maturity Scale (see figure 1), 63 percent placed themselves in the bottom three categories—analytics aware, localized analytics, and analytics aspirations, meaning they are not insights-driven organizations (see figure 8).
The methodology used in the Becoming an Insights-Driven Organization survey eliminated approximately 35 percent of the potential survey base because they did not have any interaction with analytics in their companies. This means that the insights-driven organizations in our survey make up 37 percent of a group that is, at a minimum, analytics aware. Given that the overall population of companies includes those that are not even analytics aware, the number of insights-driven organizations in real life is very likely even smaller than this survey indicates.
In addition to measuring where companies placed themselves on the maturity continuum, we also wanted to examine what patterns emerge when we looked at how they approach analytics culture, data and tools, and talent. To analyze this, Deloitte cross-referenced respondent’s self-ranking on the continuum with other factors.
In general, development in key input (tools, talent, culture) and outcome (business success) metrics rises across the maturity continuum.
An interesting exception is the group that is lowest on the continuum, those that are analytics aware. They rated themselves higher (close to the top performers’ rankings) in all the key factors making up analytics maturity. Given that they represent only 6 percent of respondents, this anomaly represents a very small percentage of the data. It is possible that their atypical responses may be a layman’s appraisal of the factors driving analytics maturity. In other words, they may be an example of the adage, “You don’t know what you don’t know.”
The otherwise steady progression of best practices on the continuum below illustrates the importance of leadership and culture in becoming an insights-driven organization. These two factors are most likely to be the drivers of this segmentation, in our experience. For example, committed leaders can provide data and tools, and hire the necessary people.
In our experience—reinforced by this survey—the vast majority of companies do not have initiatives in place to address data-driven culture issues. Initiatives around data and technology, and even hiring specialized talent, are not enough to bring about the cultural changes needed to help companies evolve to being insight-driven.
Here are some recommended steps to bring about the cultural changes needed to become an IDO:
Our 2019 Becoming an Insights-Driven Organization survey results clearly show that insights-driven organizations represent a minority of businesses today, despite the number of years technologies such as big data and analytics have been readily available. The most common culprit is culture. Buying and using analytics tools is not hard—changing behaviors is. By emphasizing education, enlisting executive sponsors, and modeling and rewarding the right insights-driven behaviors, organizations doing business in the Age of With can benefit from human collaboration made greater with AI and analytics.
Achieving your business outcomes, whether a small-scale program or an enterprisewide initiative, demands ever-smarter insights—delivered faster than ever before. Doing that in today's complex, connected world requires the ability to combine a high-performance blend of humans with machines, automation with intelligence, and business analytics with data science. Welcome to the Age of With, in which Deloitte translates the science of analytics—through our services, solutions, and capabilities—into reality for your business.
Learn more