The datafication of HR has been saved
HR departments capture enormous amounts of data, but these typically stagnate in various systems and are rarely used for strategic purposes. It’s not that companies haven’t tried, with HR data warehouses and “HR analytics” teams that run reports. But the returns have been frustrating—until recently.
Capturing and analyzing data about people is a hot topic. Articles about big data and the “Moneyball effect” appear in nearly every Sunday New York Times. We know that Facebook, Google, LinkedIn, and Twitter monitor and store much of our daily web activity and have just found out that the National Security Agency and telecommunications companies monitor call details, including location and history.
And data scientists are now learning how to perform sentiment analysis on our emails,1 deliver pinpointed advertisements to our mobile phones, and even assess and find job candidates through their social activities. It’s not a surprise that data science, a relatively new profession, is now considered “The sexiest job in the 21st century.”2
All this noise has created tremendous changes in Human Resources. LinkedIn, for example, has become a large provider of recruiting solutions, largely driven by its database of global professionals. There are now dozens of start-up companies building tools to use social data, HR data, and testing data to assess and source people better. A new company called Identified is collecting much of the data available on LinkedIn, Facebook, Yahoo, and Twitter to create a next-generation candidate search engine that offers recruiters a pool of more than 500 million workers. A company called Smarterer is offering skills-based assessments for contract workers. A company called Evolve, which analyzes the performance of call center employees to understand turnover, learned, among other things, that ex-convicts are often more effective call center workers than high-end college graduates.
Many suggest that HR professionals don’t understand data. Yet there is a rich history of data science related to the HR profession. In the late 1800s Fredrick Taylor,3 a mechanical engineer, analyzed the job performance of steel workers and laid the foundation for an industry of industrial psychologists who measure what we do on the job. He found, for example, that a worker who lifted 50-pound “pigs” was far more productive than one who lifted the then-typical 75-pound “pigs” because he had more time to rest. This time and motion study caused a whole reengineering of the steel manufacturing process.
Human Resource departments capture enormous amounts of data about people: turnover, engagement, hours of training, compensation, job mobility, performance ratings, as well as where we went to school, our college degrees, and nearly 200 other items.4 But such data typically sit around stagnant in various HR systems and are rarely used for strategic purposes.
It’s not that companies haven’t tried. Companies have been building HR data warehouses for 25 years, and many HR departments have an analytics team that runs reports. But until recently, these investments don’t seem to have paid off much.
Our recently completed two-year study of this topic indicates a major shift is taking place.5 While organizations invest more than $14 billion in HR software,6 today fewer than 4 percent can perform predictive analysis on their people and only 14 percent perform statistical analysis at all. HR managers tell us that they want to be more “data driven,”7 but most have not yet been able to build a solid set of talent analytics systems and capabilities.
There is evidence that proficiency in talent analytics is associated with strong performers. We looked at the financial performance of companies in the top 14 percent in our research (levels 3 and 4, as defined in the talent analytics maturity model shown in figure 1—those actually able to correlate HR data and business data to predict and improve outcomes), and we found them to be the highest-performing companies in terms of shareholder value. Organizations that operate at these levels outperformed the S&P 500 by 30 percent over the last three years and reported a fourfold better ability to make data-driven decisions about people, including whom to hire, promote, and let go of, and how much to pay.8 A similar study by the Massachusetts Institute of Technology (MIT) and University of Pennsylvania found that companies with mature analytics functions in general produce 5–6 percent higher financial returns.9
Our research suggest that more than 60 percent of all companies are now engaged in this area,10 yet there is a chasm between the “haves” and the “have nots.” The 14 percent at the top of our maturity curve have invested in people-related analytics, moved up the learning curve, and are now engaged in some impressive problem solving.
A large financial services company saw dramatic variations in sales performance and retention among its hundreds of sales representatives. The team hypothesized that there were hidden factors that might be causing these patterns, so they built a model to try to predict sales representative performance and retention. Traditionally this company sourced its sales team from top universities and recruited candidates with excellent grade point averages.
The analytics team pulled together demographic, job experience, recruiting, and environmental data on the entire sales organization and compared the high performers against the average. After significant statistical analysis, the team found that the company’s assumptions were wrong: The high performers were not those from the top schools, nor did they have the highest grades.
As figure 2 shows, the high performers in this particular company could be identified by far less academic criteria, making high-performance recruiting much easier.
This example illustrates an important point: One of the greatest benefits of talent analytics is the debunking of typical management myths. In this case, by shifting to a new way of assessing sales candidates, the company generated more than $4 million of new revenue in the first six months.
A large Canadian bank suffering from theft and embezzlement in its branches spent many years investing in training and monitoring tools to reduce fraud. Despite these ongoing programs, theft continued—and seemed particularly high in smaller branches.
The operations team, partnering with HR, embarked on a talent analytics project to correlate patterns of loss against such factors as employee tenure, age, experience, training, educational background, management demographics, and geography. After many months of effort, the company found that the factor most correlated to theft was the number of miles from the branch office to the district manager. People in this particular role who felt unsupervised were more likely to act unethically.
After years of largely ineffective investment in training and compliance programs, the bank reorganized its district managers to bring them closer to the branches, and the loss rate dropped dramatically.
A US-based food service company found highly inconsistent renewal rates among its large clients. Some remained loyal clients for many years, and others seemed to drop their relationships abruptly.
The company hypothesized that there were team characteristics that must be creating these losses, so they assembled an analytics team (operations staff in partnership with HR) to look at data. After analyzing more than two years’ worth of account, team, management, and organizational data, the team uncovered a surprising, but in hindsight obvious, factor. The biggest factor that correlated to client loss was a pattern of Occupational Safety and Health Administration safety regulation violations in the team’s food service operations.
It turns out that safety, which was considered a “compliance program” for many years, had a very large impact on client perception and service satisfaction. These safety-related accidents reflected poorly on the company and gave customers an incentive to rapidly change suppliers.
While safety had long been a priority, the company had always considered safety training a compliance program, implemented to reduce insurance costs. After this analysis, the company reprioritized safety and safety training as top priority in its service delivery operations strategy.
This finding helped the company improve client retention as well as institutionalize new operational practices for sales management. The company now includes safety training and compliance at the top of every monthly account review to make sure that all field employees take safety and safety education seriously.
A global technology company suffered from the loss of many high performers to competition. To understand the causes of this turnover the HR organization hired a team of statisticians to look at a large number of people-related variables (more than 100) to understand what may be causing the problem.
After months of analysis the statisticians found a direct relationship between high performer turnover and compensation. The problem was not simply paying people more, but paying them differently.
The team found that midlevel performers, who greatly enjoyed working at the company, would not leave even if their raises were as low as 90 percent of industry average. But high performers (those in the top 10 percent) were far more sensitive, and would very likely quit if their annual raises were not at least 115 percent of the industry average.
What management had been doing was “keeping everyone happy” by trying to distribute raises through a normal distribution curve. What this research showed, which we have seen in other situations, is that high performers in service and intellectual property–related companies are not just somewhat better than midlevel performers; they are often orders of magnitude better. This warrants a much more unequal distribution of compensation, which makes some managers very uneasy.
After months of socialization and training, the company found this change was far more difficult than they realized—so they had to formally rebalance salary budgets and develop new tools for compensation review that reward hyperperformers with compensation increases much higher than average. (We recently talked with another software engineering firm that told us they found the same effect and now pays high performers as much as three times the salary of midlevel performers.)
Companies that implement talent analytics often question their long-held beliefs and revise their management decision process. Rather than rely on gut feel, they let the data inform people decisions.
A vice president of HR for a financial firm told us: “The biggest challenge we had with our analytics findings was convincing our top executives that their gut feel was wrong. It took us many months, but over time they realized that data could make them even smarter in their decisions about who to hire and promote.”
Each of these companies invested in an analytics strategy over a number of years, and, while the specifics of each implementation varied, they all went through the same four stages shown in figure 1.
The companies we studied started by focusing on building scale, quality, and experience in all areas of operational reporting. They then patiently, steadily, and continuously moved up the maturity curve to build predictive models. In many cases, it took these companies five to seven years to reach level 3 or 4.
Without a strong reporting infrastructure, clean and credible data, and a detailed understanding of what data you have and where it came from, analytics projects simply do not scale.
A pioneer in this market told us that the worst thing his company ever did with regard to analytics was to analyze retail store turnover without multiple years of validated data. As soon as they presented their work to management, one of the executives asked: “Have you looked at the seasonality of the data?” Oops. They went back to the drawing board and built a multiyear analysis. This company now has an advanced analytics team with more than 12 people who study multiyear talent trends, and the team has become critical to business and organizational planning at a senior level. But it took several years of maturity to reach this point.
Unfortunately, building a clean and integrated set of HR data is not easy. We estimate that 75 percent or more of the effort in talent analytics is invested in reaching levels 1 and 2. This is where companies have to find all their HR data sources, rationalize the definitions of various data elements, find ways to clean the data, and aggregate it into some usable system. This work takes several years of cleanup, a partnership with IT, and the skills to implement a scalable reporting infrastructure.
Why are companies stuck here? Usually it’s because the CHRO has not been willing to make the investment needed to build a true analytics function and thus has not taken the time to build the business case for an integrated analytics team.
Despite the huge consolidations in HR software providers, people-related data are nearly always present in many systems. One of the critical steps in putting HR on a more analytical path is to bring together the disparate data sources needed to build a data dictionary.
The challenge is more complex than it seems. First, HR has often not built an adequate business case, so it lacks support from IT. Second, there are few and perhaps no broadly accepted standards for HR-related data,11 so information in different systems must be rationalized and defined in a consistent way, which can be time-intensive and controversial. Third, HR data are often seasonal and regional, so what one business unit calls “turnover” is not the same as another, and the analytics team has to standardize all these measures. In one example from our research, a company did an extensive turnover analysis of its workforce and then went to the head of the Brazilian operation to present the dismal results. The Brazilian business leader dismissed the analysis as invalid: “We run our operation with a high percentage of contractors. So of course the turnover is higher.”
There is a significant amount of effort required in the rationalization of data—and while this isn’t the sexiest part of HR, it may be one of the most important.
Much has been written recently about the lack of data scientists in business, so one might ask where HR will get the skills. Often the problem is not finding statisticians and math talent but rather building an interdisciplinary team.
In the early stages of analytics, the team requires technical staff who work with IT to bring data together, build the data dictionary, and put a reporting process in place.
But as the process advances, the team can become increasingly multidisciplinary and work closely with other analytics teams in the company, as well as with external data providers and the executive team. It will often expand to include business analysts, a math or statistics specialist, and people who know how to visualize and present data in an understandable form. As described by Tom Davenport, the resulting communication barrier can undermine the impact of otherwise very well-constituted teams.12
Despite the generally high quality of the HR analytics tools purchased from their enterprise resource planner or other vendor, many companies still found themselves stuck in reporting molasses. While good tools are important, the biggest hurdles companies face at levels 1 and 2 are often insufficient patience, process, skills, and organization:13
Despite the huge consolidations in HR software providers, people-related data are nearly always in many systems. One of the critical steps in putting HR on a more analytical path is to bring together the disparate data sources needed to build a data dictionary.
One of our clients is a well-known health care provider with deep experience in health industry analytics. This company has a history of measuring patient outcomes and drives much of its business success by constantly looking at the organizational factors that result in healthier patients. The HR analytics team in this company (which is operating at level 4) had spent more than six years building its internal data warehouse and is now examining the impact of “source of hire” on nursing quality. The leader of this team is a senior IT professional who has worked in the company for more than 15 years and has extensive experience doing performance consulting with many departments throughout the company. It is his deep understanding of the data and the business, coupled with support from leadership and IT, that has made them a big success.
If you talk with the companies at levels 1 and 2, you often hear comments like “we aren’t investing enough in analytics” or “we don’t have an integrated team yet.”
In order to succeed in this fast-growing area, companies need to make an investment. The talent analytics team should be considered a new center of excellence within HR, as important as recruiting, compensation, and training.
And the business case should focus on more than reporting. Many analytics projects start by developing reports that look at the efficiency and effectiveness of all HR programs: What is our “time to hire” and “cost to hire?” Which candidate sources are producing the highest-performing staff? How much training are we delivering, and how satisfied are our learners? What are our compa ratios, and how well are performance scores being distributed?
While these are interesting and important questions to answer, they will likely not drive enough value to justify a major new analytics team.
A solid business case focuses on business-critical operational problems. For example, if turnover is high in a manufacturing plant, it would be useful to know the level of workers’ tenure, training, compensation, engagement, background, education, and other factors that may be causing this turnover. Health care providers suffer tremendous financial expense from nurse turnover and high levels of infection in various parts of the service chain. A talent analytics plan focused on solving these problems would be the foundation of a strong business case.14
In the early 1900s we electrified the world—literally. Businesses realized they could rely on a steady stream of electricity to run manufacturing plants at night, invest in motors and robots to improve productivity, and eventually invest in computers and other tools to change the way they did business.
As Einstein noted, “Science is the refinement of everyday thinking.”15 Just as electricity changed the way we manufacture and deliver products, so can data change the way we think about managing people. Far too many important people decisions are still made on the basis of gut feel or anecdotal experience.
While only 14 percent of companies in the study have reached high levels of maturity in talent analytics, we see tremendous growth in this market over the next few years. This type of analysis may well become table stakes to attract and retain talent and will likely also expand into new areas as well. Consider the following three major trends: more data, more tools, and more techniques.
There are many sources of people-related data we have yet to collect: employee location, social activity, professional networks and network connections, psychographic data, project results, and more. Once companies have an infrastructure for analytics in place, we can bring more and more information together for analysis.
One of the more interesting areas of analysis is learning how to measure performance by looking at how people collaborate, the projects they’ve worked on, their peers and associates (their internal corporate “social data”), and their innovations and successes. One of our clients mentioned, for example, that their analysis shows that engineers who have friendships and regular meetings with other top engineers are far more productive than those who work in more isolated groups. Another client found that manufacturing teams that work in diverse environments outperform those who work in less diverse groups.
Think about all the possibilities to unlock the secrets of high performance.
The tools to collect, analyze, and visualize data are rapidly evolving. New tools like Tableau and Qlikview make visualization much easier, and a whole industry of start-ups is bringing managed analytics services to market.
There are three major categories of new tools: visualization and analysis tools (desktop tools that let users analyze data and create reports); middleware and database tools (tools that collect, clean, store, and query data—including tools that use Hadoop16); and managed analytics services (companies that provide integrated services to collect and make sense of your data). With this market growing so rapidly, it’s important for the analytics team to keep abreast of advances in the tools market.
While it still takes time and experience to make sense of data, the tools market is burgeoning, and we expect to see more and more third-party analytics suppliers do the analysis for their clients. Just as managed analytics companies analyze credit card fraud, market basket information, and customer relationship management data, there soon may be managed analytics firms that help us understand our people data.
Finally, data scientists and statisticians are inventing new ways of analyzing data every day. The University of California at Berkeley recently introduced a master’s program in data science, and many other educational institutions and for-profit training companies are teaching new statistical techniques. A new technique called structural equation modeling, which is based on social science, provides new techniques for finding cause from correlation. And new companies such as Kaggle (an international community of data scientists) bring math and statistics specialists together to let them bid on interesting projects, share ideas, and test and improve their analytic skills.17
I recently read a PhD thesis from an MIT student who called himself a “social data scientist.” He studied the statistical inferences and human characteristics of Twitter streams and found, for example, that young people use the smiley face differently from older people: Younger people use : ), while older people use :- ). What is the relevance of this? Probably not much right now, but what it tells us is that the techniques and tools for big data are expanding, and one day we will be able to measure people performance in real time (similar to a call center) and immediately point out when a management error or performance problem is likely to occur.
This “datafication of HR” is part of a broader trend affecting nearly every business function. Data about our people are often the most powerful data we have. Companies that learn how to harness the data hidden in their HR systems will likely find tremendous opportunities to drive improved performance, customer service, and business growth.