My take: How to develop transformative generative AI applications

By Naveen Rao, vice president of artificial intelligence at Databricks

It’s been a few years since generative AI took the world by storm with image and text generators that demonstrated the profound capabilities of the technology. However, many businesses are still trying to figure out what to do with it.

For many, it’s a problem of defining what success looks like. Too few organizations are developing end-to-end applications that clearly define metrics around usefulness, accuracy, user experience, and other types of business impact. Generative AI is great at returning answers to questions and creating content, but that alone isn’t necessarily enough to create an application that people will find useful.

Part of generative AI’s maturation process is understanding its limitations. Early on, everyone wanted some sort of sci-fi fantasy, a magic computer that could understand everything and spit out the perfect answer. The reality is that what defines a perfect answer is quite subjective.

One such way to address this is to build AI agent systems that break the problem apart rather than asking one model to do it all. The idea is to go from a large language model (LLM) that knows a few things to a system built from multiple components that creates an end-to-end application. LLMs are good at parsing language, but this is only part of what useful applications need to do. Systems that can do this kind of parsing and then call different functions and assemble an output that is usable to workers will create a much greater impact than an LLM-based tool on its own. We need a multipronged system with interconnected functions that can be independently engineered, verified, and monitored. 

The fact is, it’s harder to control a monolithic tool. When an LLM alone acts as your data store, computation layer, and interface, engineers have a hard time opening it up and tweaking specific components. That’s why Databricks is working on solutions to modularize these end-to-end systems. Every technology starts this way. You have to figure out how to break it apart, test each component individually, and build reliable, interdependent systems.

As more enterprises start developing AI applications in this manner, we expect them to see the business value of use cases improve. Right now, most generative AI applications involve developing bots to answer simple questions. Imagine a chatbot that can help employees navigate human resources systems without requiring help from human HR staff. These types of use cases can deliver some operational efficiency, which is important, but they won’t deliver the kind of transformation functionality that businesses are really looking for.

The higher-value use case will involve examples like coding assistants that know your organization’s application programming interface landscape and can connect data and tools across the organization, or inventory management systems that will allow users to search through extensive product catalogs to more quickly find a highly specialized part. These types of use cases are starting to come along now, and we expect them to deliver significant business value soon.

In the future, even more transformative use cases await. These could include electronic health record companies that allow patients to ask questions based on their medical records, or virtual financial advisors that counsel clients. These types of use cases are hard to do now using only LLMs, because in highly regulated environments, any error could prove disastrous. To get to a future state where these kinds of transformative use cases are possible, organizations need to start working on data governance and access management now.

Over the last 20 years people have been building infrastructure to gather, organize, and govern data, but they’ve found limited uses for that data. Primarily, data is viewed in business intelligence dashboards. That’s useful, but not transformative. Now we can use data in completely new ways that build new technologies. It’s not just about making a better business decision; it’s about building transformative applications for your business, and impacting how customers interact with you, or the economics of that interaction. Transformative AI applications will be the payoff for all the work technology teams put toward building data infrastructure. The value of data is going up.