In an on-demand world, businesses thrive on real-time data

But this is starting to change as more businesses invest in streaming data pipelines. Streaming data enables a host of use cases, including real-time analytics and rapid communication with Internet of Things (IoT) devices, reflecting people’s expectations for up-to-date information.

“The world has changed,” says Mindy Ferguson, vice president, messaging and streaming, Amazon Web Services. “Think about how we used to sit down to watch the nightly news once a day. Now we have access to news around the clock. We expect to know when things happen in real time.”¹

Streaming data enables enterprises to take faster data-driven actions. For example, credit card companies can monitor transactions in real time to identify fraud before payments process. Gaming companies can analyze customer behavior and make offers to players based on how they’re playing the game. Property managers can turn their buildings into smart offices that automatically adjust environment settings such as lights and temperature based on where people are at any given moment.

IoT is another major use case for streaming data. As enterprises add more connected devices to their networks, the volume of data generated increases substantially. Particularly, when devices are deployed to the field, that data is most valuable nearest to the time it was generated.

Streaming data pipelines help move data from connected devices across multiple sources to centralized repositories where it can be better leveraged, ensuring that users are working with the freshest information possible. Data lakes are one such repository of raw data. They utilize streaming data to support use cases such as business intelligence, analytics, and reporting, enabling applications such as predictive maintenance, environmental monitoring, and smart city management.

“Streaming and messaging services bring life to those real-time use cases,” Ferguson says. “It’s how people bring data into their applications. The advantage is being able to bring it in once, use it across your organization, and put it to use before the value of that data diminishes.”

Ferguson expects streaming data—and the real-time analytics use cases it supports—to become a strategic imperative for most businesses in the coming years, and eventually become a core technology. Data volumes and the velocity at which data is created are only increasing, and other tech initiatives are going to increase the demand for fresh data. This may put pressure on executives to develop a roadmap for making streaming data a bigger part of their operations. It may also impact the way they hire and train people, as well as how they decide which projects to fund.

For example, AI requires huge volumes of training data, and the more recent the data is, the more accurate a model will be. Streaming data could play a role in enterprises’ AI goals. As organizations look to expand their use of generative AI, this capability could be game-changing. By combining streaming data with large language models, organizations can extract meaningful patterns and trends from large volumes of data as it arrives. This powerful combination can be used for sentiment analysis, anomaly detection, topic classification, intelligent chatbots, and real-time translations, among other use cases.

“Anything outside of using real-time data becomes very frustrating for the end consumer and feels unnatural now,” Ferguson says. “Having real-time data always available is becoming an expectation for customers. It’s the world we’re living in.”