Borrowing from the sciences: Scientific methodology solves real-world business problems

Scientific methodology solves real-world business problems

In 1772, James Cook set sail to explore the South Pacific, equipped with a precision scientific device developed by John Harrison. This marine chronometer allowed Cook to determine longitude within one-half of a degree, guiding his journey and allowing him to map his precise location with a degree of accuracy that was previously impossible.¹ This device helped make Cook a household name and subsequently revolutionized commercial marine navigation.²

Throughout history, business has been quick to embrace useful scientific advances and to apply them to realize the financial impact. Today, companies are looking for ways to find value in large amounts of accessible data. Advances in machine learning and analytics make it possible to apply sophisticated algorithms to this data. By successfully leveraging scientific developments, often made without an applied business context, businesses are deriving value and gaining a competitive edge.

Applying the right equation

Scientific methodology is being applied today to solve real world problems across many industries, including health care, where a tremendous amount of data is readily available and the potential business value is high. Other industries, from oil and gas to retail, are also recognizing the value of applying advanced algorithms to collected data.

In this blog, we discuss two examples of how today’s organizations are leveraging methodologies developed in the scientific domain.

Markov chains

A Markov chain is based on the idea that a present state can be used to predict a future state, regardless of what might have happened in the past. For example, if we were to model the weather as a (first-order) Markov chain, then if it’s raining today, there’s a greater chance that it will rain tomorrow, irrespective of if it were raining yesterday. At first glance this would seem to be a reasonably general assumption, and Markov chains have helped to solve problems in chemistry, medicine, physics, speech recognition, and artificial intelligence.

In business, Google’s popular search engine was originally derived using Markov chains. The PageRank algorithm, developed as a way to measure the importance or relevance of a web page, sets the standard for search engine optimization. Using a Markov chain to model the shared links among web pages, the algorithm identifies the most relevant sites, which are those with the greatest probability of access. These probabilities can be obtained using a straightforward procedure from linear algebra, specifically an eigen-decomposition.

This same idea can be applied to recognize the most important entities whenever data exists about how those entities are linked. For example, the most important players on a soccer team can be inferred from the data on who is kicking to whom during a match. Pharmaceutical companies are using this same approach to identify physicians who have the most influence within a particular provider network. Using patient data obtained from prescription records, the pharmaceutical company can identify the most influential physicians by analyzing patient connections within the data set.

The Markov chain is also the basis of the hidden Markov model (HMM), a particularly powerful statistical model that has been applied in problems as diverse as gene sequencing to conflict modeling. One of the HMM’s primary applications is in speech recognition, the technology that enables speech recognition in mobile devices.

In the oil and gas industry, one company is applying the HMM to listen to its pipeline. Low-cost acoustics sensors that are externally attached to pipelines, coupled with advanced HMM-based analytics, are used to measure flow rates, determining flow compositions and listening for tell-tale signals that indicate a potential maintenance event. This technology promises to disrupt current flow measurement approaches that rely on very expensive and invasive sensors coupled with relatively simple analytics.

Matrix completion algorithms

In 2006 Netflix sponsored a competition to predict how its users would rate movies. Somewhat surprisingly, it turns out that this problem is mathematically identical to the problem of filling a large and very sparse matrix. This problem arises in a wide variety of applications and many scientists have studied sparse matrix completion algorithms, including Terry Tao, winner of the Fields Medal, the highest scientific award for mathematicians.³ Netflix successfully leveraged some of the underpinning science arising from its competition, and currently 75 percent of viewer selections come via a Netflix recommendation,⁴ a feature that helped propel the growth of the company’s video-streaming concept into a business serving more than 81 million subscribers worldwide.

Sparse matrix completion algorithms have found applications in areas far removed from how users rate movies, particularly in the detection of medical billing anomalies. After receiving medical services, a claim consisting of medical procedures and diagnosis codes is generated and ultimately used to bill the insurer. As any particular claim consists of only a very few (usually fewer than 10) codes from the many tens of thousands of available medical codes, claims data can be mathematically represented as a very sparse, high-dimensional vector. Insurers are currently leveraging sparse matrix completion algorithms on the matrices representing many claims to detect provider billing errors, missing codes, and other anomalies.

Getting down to business

When it was first introduced, Harrison’s marine chronometer was one of a kind and represented a significant portion of the voyage’s expense. As the technology matured, marine chronometers became more widely available and more affordable. Soon, they were standard equipment on ships. Similarly, the science underpinning business analytics was formerly known only within universities and large research labs, but now is being leveraged widely in the commercial world.

What tools could your organization borrow from the sciences to gain a competitive edge and realize value? Learn more about how the marriage between business and science is delivering deeper insights to organizations in Analytics Trends 2016: The Next Evolution.

References

¹ “Solving the Problem of Longitude.” Captain Cook Society.

² John Harrison, Encyclopedia Britannica. Encyclopædia Britannica Online, s. v. "John Harrison", accessed May 17, 2016 ;Encyclopædia Britannica Online, s. v. "James Cook", accessed May 17, 2016.

³ “Fields Medal.” Wolfram MathWorld. 

⁴ Vanderbilt, Tom. “The Science Behind the Netflix Algorithms that Decide What You’ll Watch Next.” Wired.com. August 7, 2013.