Big data meets small data

Perspectives

“Old school” meets “new school” analytics

Short takes...on analytics

A blog by John Lucker, principal, Deloitte Consulting LLP and David Steier, director, Deloitte Consulting LLP

New technologies and processes are emerging to augment or replace what was cutting edge just a few years ago. Machine learning and the enterprise data warehouse (EDW) are two areas where questions of “how much” new-school analytics is necessary to replace old-school data management and business intelligence approaches.

Old faithful? The stable EDW

Rumblings of the death of data warehousing are nothing new. Some say data warehouses are past their prime, restricting the speed and flexibility analytics depends upon to deliver relevant business insights. Old-school EDWs are being challenged by newer, more cost-efficient ways to handle big data as data warehousing vendors put their brands on Hadoop clusters and related technologies. However, many enterprises continue to count on the security and reliability of their established EDWs to manage and extract intelligence from their data.

Some organizations are augmenting the production EDW with smaller data marts on data appliances for less-intensive, shorter-term applications. This approach helps enterprises to realize the efficiency gains of newer technologies while retaining the EDW’s advantages of security, reliability, and concurrent user support.

Machine learning – What about the human factor?

Like the data warehouse, traditional approaches to statistical modeling and analysis are being challenged by newer approaches. Machine learning uses semi-automated models that get better over time to perform certain tasks. Offering a cost-effective alternative to manual programming and analysis, machine learning is effective in instances where “the human touch” is difficult or time-consuming. The efficiency gains machine learning offers are significant -- automation can deliver thousands of models in the time it takes a human to create a hundred or so.

But to be effective, these models have to be informed and validated by human overseers. Statisticians and data scientists are often required to specify variables, adjust model parameters, and interpret the content of models. Models without translation into business terminology are simply models, not insights.

Old school meets new school

Rather than asking, "Do we need this technology or not?" instead ask, "How do we create the most effective information environment for our business?" The mix of old- and new-school approaches often answers that very need. While machine learning’s automation can power productivity, we shouldn’t underestimate the role of smart human analysts. And while technologies like Hadoop  improve our ability to handle large data sets, don’t rule out the reliability and stability of the EDW.

What do you see happening as old school and new school technologies collide?  For more on big data and analytics trends, explore Deloitte’s interactive infographic, Analytics Trends 2014.

Meet the Leader

John Lucker

John Lucker

Principal | Human Capital

John is a principal in Deloitte Consulting LLP, Gl...More

Contact us

Contact us via our online form

Submit RFP

Submit via our online form

Find an office

Find a local office

Services

Deloitte Analytics

Our business analytics specialists look inside data to help create and refine strategies for delivering insights that yield informed business decisions.

Services

Deloitte Analytics labs

The Deloitte Analytics Labs help business leaders discover what the science, technology and application of business analytics can deliver in practice. 

People

John Lucker

Principal | Human Capital