Posted: 30 Nov. 2022 9 min. read

Call the Data Science Team!

Addressing workforce challenges with HR advanced analytics

Authored by Gary Parilis and Eric Lesser

The last two years have seen a seismic shift in understanding employee activity, expectations, and behaviors and the associated impacts on operations, engagement, productivity, and business performance. The breadth and volume of data collected and created by today’s organizations represent an unprecedented opportunity to use data science to inform and enable more effective decision-making and action planning. However, a question we are often asked is, “When is the right time to bring in resources with the technical skills and experiences to apply more advanced analytic methods”?


While the need for applying data science to work, workforce and workplace opportunities and challenges continues to increase, the availability of individuals who have the necessary experience is often limited. In some companies, HR organizations may only have a few professionals with advanced analytics experience; in others, these resources might exist only at a larger corporate level and already have a large backlog of projects. In any case, there is a severe and growing shortage of data scientists in the talent market. The Bureau of Labor Statistics projects a growth rate of nearly 28% in demand for individuals with these skills.1

It is important for senior HR executives to know what scenarios require the time and attention of more technically oriented data scientists versus others with more readily available analysis or dashboarding skills. Bringing in data scientists to work on less technically demanding projects can result in lost opportunities for the organization, as well as frustration among those asked to perform tasks that do not require advanced methods and tools.


To enable HR executives to make more effective decisions about how, and when, to engage data scientists, we have identified several project archetypes that typically require more advanced analytics solutions:

Predictive Modeling

Perhaps the most well-known of the advanced analytics applications, predictive modeling uses machine learning algorithms and historical data to predict future outcomes and determine what factors influence these predictions. A typical example is the use of predictive modeling to anticipate and reduce employee attrition. It allows leaders to model future attrition, as well as develop interventions based on individual and aggregated employee risk level. Other applications for predictive modeling include assessing leadership potential for succession planning, and forecasting hiring needs, labor supply and demand by role, or skills availability in local markets.

Scenario Testing and Projecting Return on Investment

A powerful benefit of a predictive model is that it can be used to test the anticipated results of potential interventions suggested by the model. Modeling these changes involves looking at different combinations of interventions and gauging their impact. For example, if an attrition model suggests that base salary and hours of overtime are key drivers, simulation tools enable the testing of various combinations of incremental adjustments, providing projections of the number of exits prevented, and the corresponding savings in turnover-related costs.


Simply put, optimization is the search for the best-fit solution to a problem. An optimization algorithm cycles through possible combinations of variables, and projects the outcome in each scenario to find the most appropriate solution. For example, optimization can be used to develop a scheduling approach that assigns shifts to those individuals with the appropriate skills at the right time, in the right locations. Optimization techniques are most important when there are tradeoffs, such as a model that estimates the ideal compensation by role and location, balancing wage and benefit costs against productivity, and the cost of turnover.

Pattern and Exception Detection

Given the hundreds of thousands of workforce-related transactions and behaviors that occur within organizations every month, it is often difficult to detect when there are meaningful changes within small subpopulations. Data mining can detect leading signals that may indicate a looming issue that needs to be addressed, as well as automate the delivery of early warning alerts to managers and leaders so that issues can be addressed promptly. For example, if there is increased absence or tardiness in a particular group, it might be a sign that there are larger issues about workload or managerial guidance that require attention. This kind of automated data exploration can also expose equity and fairness concerns affecting certain groups, which might otherwise go undetected.

Analysis of Unstructured Data

Much of the valuable insight about the workforce is trapped in unstructured data sources, such as social networking platforms (e.g., Glassdoor, Twitter), employee survey comments, resumes, project descriptions, meeting notes, group chats, and exit interviews, as well as voice, images, and video. Advances in natural language processing have improved the accessibility of patterns, trends, and sentiment from these kinds of data. For example, this technology can be used to analyze text from performance reviews or employee recognition, or to infer the skills each employee has, based on resumes, course descriptions, and project information. Beyond simple reporting, unstructured data can be converted to more conventional structured data for inclusion in other analyses. For example, open-ended survey comments, performance text, and resume content can be automatically categorized and then included as inputs into a predictive attrition model. Another common application of this technology is the use of chatbots to answer employee questions regarding learning opportunities, employee benefits and service transactions.

Clustering and Grouping

In many situations, it is useful to group individuals based on similarities and differences, which are often hidden in the data. Unlike simple classification based on a small number of variables like demographics, tenure, or role, this method uses machine learning to seek patterns across multiple factors, discovering groups whose members are similar in many ways. Imagine that, to establish flexible return-to-office policies, you want to identify a half dozen employee “archetypes,” based on patterns of how and where they work and collaborate, what is important to them, their degree of satisfaction with their team, the types of work they perform, and demographics, such as their tenure in the organization. Based on different clustering algorithms, individuals are assigned to groups, making it possible to establish initiatives tailored to each segment, and then track the behaviors, engagement, productivity, and retention resulting from those actions.


There are many types of analytics tasks, and it is important to carefully consider the complexity of the need to determine whether they require data scientists with specialized machine learning and AI skills or can be addressed by business analysts using dashboards or Excel. To make the most of limited people analytic resources, organizations should consider the following questions to determine when to bring in individuals with more advanced data science experience:

  • Is there a need to make more accurate predictions about the future?
  • Is it important to base decisions on the costs and anticipated results of different combinations of interventions?
  • Is finding the best combination of interventions (those with the best possible outcome) too complex a task without an automated tool?
  • Do we need to unlock information that exists in unstructured data, such as text, which cannot be analyzed with tools designed to analyze more structured data?
  • Will our analytics objectives be helped by identifying holistic clusters of employees based on combinations of factors?

Considering these questions can help guide decisions about whether to engage data science resources or address issues with more traditional analytics personnel.



The Data Science Talent Gap: Why It Exists and What Businesses Can Do About It

Join the conversation