Deloitte’s Early Warning System Identifies Likely COVID-19 Outbreaks Three Weeks out | Deloitte US has been saved
By Greg Szwartz, life sciences data science practice lead, and Lavan Srinarayanadas, M.D., manager, Deloitte Consulting, LLP
What is the risk in having kids return to their classrooms? When should employees go back to the office? How can airlines reduce infection and cancellation risk? Should sales reps meet with clients in person? We are eight months into the COVID-19 pandemic and these questions are still difficult to answer—and the answers typically vary by location and population. While knowing when COVID-19 cases have spiked can be useful, historical data doesn’t indicate where and when future outbreaks are likely to occur. Other factors—including the susceptibility of a population, economic and social resiliency, and whether local hospitals have the capacity to handle a spike in cases—should also be considered.
In the early days of the pandemic, we began building a model that could help spot future COVID-19 outbreaks and help leaders make informed decisions about closing schools or canceling sporting events. After lockdowns went into effect in late March, we adopted a new goal: re-opening safely. Our early models didn’t always work as expected. For example, we didn’t anticipate the sometimes significant contrast in infection rates between some large and small counties (the math used to forecast transmission rates is fundamentally different between rural and urban settings). We also encountered data irregularity issues, which were compounded by uneven region-by-region reporting. After five months of daily re-calibrations, we have a model that works like a Sim City of the entire US. We are able to predict transmission rates with enough accuracy to forecast the magnitude and direction of COVID-19 infections down to the county level.
D.SMaRT (Deloitte Simulation Modeling and Resourcing Tool) is a customizable early warning system that can forecast infection rates three weeks out with more than 80% accuracy. The model mirrors the physical dynamics of infection-spread by simulating groups of individuals who interact with each other. It can predict likely transmission rates based on factors such as social distancing, mask mandates, and testing trends. This digital twin is inhabited by a simulated population based on density, demographics, and social and economic information.
Mobility is one of the leading indicators of infection spread. If residents are moving around outside of their homes (e.g., going to stores and restaurants), transmission rates are likely to increase. Additionally, we looked at the 300 largest counties in the US and found that increases in reported flu-like symptoms—based on data from the US Centers for Medicare and Medicaid Services (CMS)—can be predictive of more COVID-19 cases the following week.
In other analyses, we used causal inference statistical models to determine likely infection spread in the absence of a mask mandate. We identified counties that were statistically similar based on their demographic, socioeconomic, and health characteristics. We then measured the effect of mandatory mask policies between August 12 and September 2. Counties that had a mandatory state-level mask policy were 6% less likely to experience an outbreak. For optional mask policies, we did not find a statistically significant effect.
Beyond informing policy, we are using D.SMaRT as an early warning system for COVID-19 outbreaks, which can be useful in making a variety of decisions:
It is important to consider a wide range of inputs when trying to identify potential infection risks when opening schools, offices, or sports arena. A physics-based simulation is an important input, but it is one of many puzzle pieces that are needed to create a complete picture. As we continue to understand how to contain this virus, we will likely be better positioned to accurately assess risks. The ability to predict potential outbreaks could help businesses determine when it is safe for employees to return to work. As we get back to school, to work, and even to the ballpark, we are taking on risk.
But this is a level of risk we can quantify and manage if we make the effort to understand the probabilities, costs, and benefits of the decisions we’re making. This is a Moneyball problem. We won’t bat 1000 by playing the odds, but we will likely win more often when we make data-driven decisions.
Acknowledgement: Kevin Coltin
Gregory Szwartz is the leader of the Life Sciences Data Science and the Precision Engagement practices at Deloitte. Szwartz focuses on applying quantitative analysis to strategic and operational decisions in life sciences and health care, and actively works on the creation and dissemination of customer insight for business planning and product strategy. He has over 20 years of consulting experience in life sciences, with the majority focused on commercial analytics, market access, and patient safety.