Four prominent trends in the API and integration market – Deloitte On Cloud Blog | Deloitte US has been saved
A blog post by Kalpesh Potghante, senior manager, Cloud Engineering, Deloitte Consulting LLP; and Raj Yerapotini, managing director, Cloud Engineering, Deloitte Consulting LLP.
As a leading systems integrator for top-tier clients, we are in a unique position to observe trends that are shaping the application programming interface (API) and integration market today and influencing its future. In this blog, we present the top trends valued by clients as they move toward realizing their API and integration strategy and evolve their enterprise API and integration platform capabilities to scale to the demands of the future.
As clients across industries are aggressively adopting cloud, they have realized that a well-defined API and integration strategy sits at the core of this approach. A well-thought-out strategy will enable them to build a flexible, scalable, profitable, and long-lasting ecosystem of connected systems and applications to serve their partners, customers, and employees. Clients are increasingly seeking API-rich applications and platforms that seamlessly integrate in an ecosystem and ease the way of doing business in the digital age.
API and integration product vendors are also investing heavily in advancing features and capabilities of their platforms at a decent clip. With disruptive and aspirational use cases such as robotic process automation (RPA), artificial intelligence (AI), machine learning (ML), the Internet of Things (IoT), and blockchain gaining traction, vendors are working harder than ever to ensure that their platforms and/or products are equipped to meet these demands of the near future.
Transition from CoE to the C4E operating model
Despite APIs being a game-changing way to deliver and manage IT, getting started and anchoring a mindset shift toward “product versus project” is difficult in terms of people and process. Traditionally, organizations have adopted the center of excellence (CoE) model to scale and transform an enterprise capability. However, the IT delivery capacity in this model has quickly become a bottleneck that has stifled the pace of innovation and hindered the exponential growth demands that IT teams face today.
To overcome challenges posed by the CoE model, a good number of progressive IT organizations have embarked on the journey to establish a center for API enablement (C4E). The C4E concept is predicated on a federated model of operation versus a centralized one. The cross-functional C4E team is collectively responsible for productizing, publishing, and harvesting reusable assets and driving standards and best practices across the enterprise. They continually promote consumption and collaboration, which drives up “self-service” capabilities for the developer community, thereby improving the ability to scale and lowering the cost of quality.
Deloitte has witnessed a significant uptick in its clients, asking to help them establish a C4E as part of enabling their API and integration strategy. Deloitte has successfully helped these clients in shifting from a production-based to a production- and consumption-based delivery model. Clients have started to realize that a well-established and mature C4E for APIs paves the path for faster time to market and reusability of assets and scales seamlessly across an ecosystem of diverse partners.
API-first mindset and API-led connectivity
As IT leaders evaluate newer cloud technology options at their disposal, they are also on the lookout to maximize their IT investments with legacy technologies such as the traditional ERP and CRM monolith solutions. With flexible, agile, and evolving enterprise architectures composed of various cloud and on-premises technology solutions, building APIs on top of existing systems (investments) to unlock data assets is proving to be the go-to modus operandi for clients to maximize their legacy investments. This approach also goes hand in hand with the microservices-based approach to enterprise architectures and paves the path for a future replacement and/or upgrade of these monolith solutions with newer best-of-breed technology options.
Architects and developers alike have also significantly shifted their thinking and are approaching technology strategy and implementation projects with an API-first mindset. This mindset puts APIs at the center of architectural decision-making, with the idea that APIs will be consumed by mobile devices, websites, enterprise systems and applications, partner portals, and vendor ecosystems.
A key element of operationalizing an effective C4E and evangelizing the API-first mindset is to promote the creation of reusable and purpose-built APIs. API-led connectivity provides an approach for connecting and exposing data and functionality as a service. With this approach, rather than connecting applications and systems point to point, every asset becomes a managed API that is discoverable through self-service.
The diagram above shows the three categories under which APIs within an API-led approach fall: system APIs, process APIs, and experience APIs.
System APIs provide a means of accessing underlying systems of records (e.g., proprietary databases, one’s ERP, etc.) and exposing that data to the user while providing downstream insulation from any interface. These system APIs will also change more infrequently and will be governed by central IT, given the importance of the underlying systems.
Process APIs are the underlying business processes that interact and shape the data and should be strictly encapsulated, independent of both the source systems from which that data originates and the target channels through which that data is delivered. These APIs perform specific functions and provide access to noncentral data and may be built by either central IT or line-of-business IT.
Experience APIs are mediums through which data can be reconfigured such that it is most easily consumed by its intended audience from a common data source instead of separate point-to-point integrations for each channel (e.g., the capability to develop tailored APIs for apps). Services are easily consumable by various mobile web apps, providing an excellent digital experience.
For several of Deloitte’s clients, API-led connectivity is central to their API and integration strategies because the technologies they use to engage with their customers, employees, and partners continue to evolve. The convergence of enterprise technologies like IoT, SaaS, big data, social, mobile, and APIs is providing powerful new tools to allow businesses to do more, unlock new revenue streams, understand their customers better, and innovate faster than ever before. But to do so, they need to integrate these new technologies with APIs. Traditionally, these integrations have been done via point-to-point connections in an ad hoc way whenever a project requires. This leads to complicated and brittle systems that are prone to failure and require a great deal of IT time and resources to maintain.
Hybrid integration platforms
Organizations are still grappling whether to go “all in” with a single platform or vendor that offers end-to-end API management and integration capabilities or to stay on multiple, fragmented-but-cohesive sets of API and integration platforms at the enterprise level. Fund and budget allocation to transformation efforts, siloed (project- versus enterprise-level) decision-making, operating models (CoE versus C4E), evolving vendor landscapes and capabilities, merger and acquisition activity, and existing IT investments are some of the leading factors driving enterprises to a future with a hybrid API and integration platform.
Such a hybrid platform provides the enterprise all the technology capabilities to integrate data and applications across any on-premises or multicloud environment for internal or external consumption. Key capabilities of the hybrid platform are end-to-end API life cycle management, application integration, data integration, and high-speed and high-volume data transfer. For example, an organization might choose to use APIGEE for its strong API management capabilities and leverage Boomi for data and application integration. It might even have exceptions where it prefers to use MuleSoft for projects integrating with Salesforce and Informatica for large-volume, ETL-style data integration patterns into enterprise data lakes.
Clients are increasingly engaging with Deloitte for advice on and implementation of API and integration services on a hybrid integration platform. For instance, when enabling a C4E, clients prefer one at the enterprise level that spans various API and integration platforms versus one C4E at a single-platform level.
Event-driven architectures
Another trend catching traction, especially with enterprise architects, is around event-driven architectures. Advancements in technologies, especially in the ERP and CRM technology space, have accelerated the ability of these platforms to generate and track events as data changes within their platform. This has enabled integration architects to move away from the traditional ETL style of data integration pattern to an event-based pattern using APIs that monitor these events in applications and transport changes incrementally rather than in bulk. APIs are thus moving beyond the two-way synchronous communication mechanism between a client and a server to an event-driven mechanism.
To break it down further, event-driven architectures have three key components: producers, routers, and consumers. A producer publishes an event to the router, which filters and pushes the events to consumers. A producer is typically the ERP or CRM application configured to track transactions and log event data in objects that it manages centrally. The router is typically a process API that monitors events and pulls out updated transactional data to be sent to a queue, for example. The consumer is typically a process API that can either track events or poll the queue for new transactions and transmit to the final downstream system, such as a data lake. Producer and consumer services are thus decoupled, which allows them to be scaled, updated, and deployed independently.
By decoupling services in this fashion, they are only aware of the event router, not each other. This means that the services are interoperable; if one service has a failure, the rest will keep running. The event router thus acts as an elastic buffer that will accommodate surges in workloads, improving scalability and reliability of the overall architecture. Development is also accelerated by this architectural pattern, as new producers and consumers can be onboarded with ease without the need to write custom, tightly-coupled code. Auditing data also becomes a lot easier, as one can now track producers and consumers independently and enforce security policies in a controlled, end-to-end manner.
With all the advantages listed above, it is clear why this pattern appeals to IT buyers and architects. In addition, this pattern also appeals to business owners, especially ones that rely heavily on data analytics and visualization to operate, as data is now fed to them in near-real time as opposed to once a day through data lakes.
Our approach
Preaching and practicing the API-first approach and API product mindset, Deloitte has helped several clients understand, envision, and define their API strategies and product road maps for internal, partner, and external consumption. Deloitte has helped clients with use cases like creating digital customer experiences with a cloud-native development approach, application modernization with a data-as-a-service layer for legacy back-end applications, frictionless enterprise experience within an API ecosystem, and API modernization by migrating APIs from other platforms.
Get in touch
David Linthicum
Managing Director | Chief Cloud Strategy Officer
As the chief cloud strategy officer for Deloitte Consulting LLP, David is responsible for building innovative technologies that help clients operate more efficiently while delivering strategies that enable them to disrupt their markets. David is widely respected as a visionary in cloud computing—he was recently named the number one cloud influencer in a report by Apollo Research. For more than 20 years, he has inspired corporations and start-ups to innovate and use resources more productively. As the author of more than 13 books and 5,000 articles, David’s thought leadership has appeared in InfoWorld, Wall Street Journal, Forbes, NPR, Gigaom, and Lynda.com. Prior to joining Deloitte, David served as senior vice president at Cloud Technology Partners, where he grew the practice into a major force in the cloud computing market. Previously, he led Blue Mountain Labs, helping organizations find value in cloud and other emerging technologies. He is a graduate of George Mason University.
