Text analytics – ride the wave or stay ashore?

Short takes on analytics

Blog written by Cindi Thompson, specialist leader, Deloitte Consulting LLP.

Text analytics

As you assess the potential of big data, don’t ignore the unstructured data sitting on your desk, web and social media sites, and even sensor networks. Many companies have begun to combine these disparate sources of internal and external data to achieve new efficiencies. When integrated with other data sources, unstructured data can bring businesses new insights. Examples abound of the strategic and tactical improvements enabled by the information provided including; detecting and preventing fraud, targeting and removing product deficiencies, and complying with emerging regulations.

One way to tap into the potential of unstructured data is through text analytics. Text analytics is the practice of semi-automatically aggregating and exploring textual data to obtain new insights by combining technology, industry knowledge, and practices that drive business outcomes.  

While some disregard text analytics as another fad, others believe it can be an effective complement to existing analytics programs. Combined with the analysis of structured data, text analytics can help businesses in their efforts to uncover signals and patterns. For example, to understand the psychology of markets, a leading news agency used behavioral psychology to guide high speed text analysis on news and social media. They developed indices related to individual emotions – like gloom, fear, anger, and trust – that potentially provide an early read on market sentiments. By mining this multi-structured data, the company developed a product that supports market technical analysis and helps make trading decisions. The product was sold to buy side firms and hedge funds, representing a new revenue stream.

Sounds great, right? So why isn’t everyone riding the wave? There are several barriers to adoption for text analytics:

  • Early adopters first:  Practical text analytics is still in its infancy, and those companies accustomed to taking bold technology moves and market leaders invested in specific business outcomes will likely be the first to explore its possibilities, while others may wait.
  • New tools, new challenges:  The learning curve to navigate the new text analytics tool sets can seem daunting. These tools usually depend on unfamiliar techniques, including document categorization, entity resolution, information extraction, and sentiment analysis.
  • Document management a prerequisite:  Many organizations are just beginning to tackle their fundamental document and data management strategies. Having effective data collection and metadata management policies can improve the efficacy of text analytics.
  • Talent in short supply: Relevant skill sets are also behind the curve in many businesses, and some tools require targeted knowledge to use effectively -- for example, adding the industry knowledge needed for accurate results, programming in unstructured data query languages, and effectively interpreting text analytics output.

Which leaves us with the question -- should you jump in? With 20 percent market growth in 2012 on top of a $1 billion base from the previous year, text analytics isn’t going away. This means that if you don’t take advantage of unstructured data, your competitors likely will.

Has your organization started leveraging unstructured data? What challenges have you encountered?

Did you find this useful?