Cloud analytics, a hybrid strategy | Deloitte UK has been saved
Limited functionality available
More and more organisations are adopting cloud. Gartner predicts that global public cloud spend will reach $354 billion in 2022, up from $227 billion in 2019. These are some big numbers! What are the implications of this rapid growth in cloud usage, and what should organisations be thinking about when adopting cloud?
I want to explore the benefits of cloud adoption, specifically focusing on the area of data science and analytics. I also want to discuss the challenges that organisations need to overcome as they move over to the cloud, and finally how analytics teams should adapt their processes to match the speed and agility that cloud has to offer.
So, let’s start with the benefits of cloud adoption, looking through a data and analytics lens:
1. Cloud enables organisations to take advantage of big data
Today organisations are collecting a lot more data than ever before. Digital propositions, ubiquitous information capture technologies and devices, they are all generating significant information and in many cases in real time. Cloud’s scale and elasticity is a game changer here. It’s because of cloud that organisations today can consume and utilise this scale of data and can use data science and analytics to solve complex challenges. This would not have been possible for the typical organisation in the pre-cloud world. The infrastructure cost, maintenance and operations would have been too much. Just imagine having to forecast how much data your organisation will consume in the next quarter, projecting requirements for new servers, and then ordering them in bulk even before you have generated any of that data!
2. Speed to market
The IAAS, PAAS, and SAAS offerings* on cloud provide organisations with the ability to expedite the process of data science - from setting up infrastructure, ingestion, experimentation, and training to the operationalisation of analytics insights at pace. In the pre-cloud days, there was an experimental operational disconnect i.e., organisations were slow to leverage new data, models needed to be built from scratch, and even when ready deployment process took several months. This impacted the speed to market and benefits realisation from analytics.
Let’s look at some examples of what speed to market looks like:
* Provision of on demand, instant virtual services such as compute infrastructure, software, and platforms to develop, manage and run applications.
Cloud enables organisations to develop truly personalised machine learning models that are unique to their customers. Developing and utilising models is an extremely resource-intensive job, as organisations may have millions of customers. These models not only consume individual customer data but they also reflect the customers’ unique sensitivities to signals - it’s like having to deploy and scale millions of models. It wouldn’t have been possible without cloud and is why most organisations initially developed models that generalised segment level behaviours. This limited not only the predictive accuracy and the value of analytical insights but also the customer experience.
4. Enables continuous learning
Most importantly, the cloud enables organisations to continuously learn (which is the Holy Grail when trying to harness the true potential of analytics). In pre-cloud days, analytical models had a lag when picking up new kinds of behaviours in data or they would experience a spike in their error rate. Models needed manual intervention and recalibration, which was resource and time-intensive (it could take weeks) - it also increased the time needed to realise the value of the data.
Today cloud providers offer tools and services built for the Continuous Integration, Continuous Deployment and Continuous Training (CI CD CT) paradigm, which allow data scientists to build automated end to end data and machine learning pipelines, which can be dynamically orchestrated and managed through automated trigger monitoring, e.g. triggers could include data drifts or increasing error rates. This enables real-time recalibration, redeployment, and serving of models. It gives organisations the ability to respond at a pace to changing customer behaviour, market trends, and evolve into being a continuously learning organisation. In summary, through all of the above, cloud enables organisations to compete in the real-time digital economy.
5. Finally…futureproofing the enterprise
Lastly, cloud future proofs an organisation by giving it access to next generation technologies as soon as they are available to the market. Access to Quantum, for example, next generation AI, and other exponential technologies such as blockchain. Obviously there will be differences in how the above benefits can be realised by industries and organisations.
Despite its many advantages, there are some challenges organisations need to think about when they move to the cloud. Let’s talk about that next.
Challenges of cloud adoption
1. Business requirements and costs
Even though the cloud is built for pay as you use, costs can escalate quickly. Generally, in the context of data science and analytics, organisations need to consider how model training and running costs will pan out. However, this is not as simple as calculating compute (IAAS) requirements as data scientists may use a multitude of other services that cloud offers for building applications. These costs need to be carefully thought through as even getting data into the cloud can be costly, let alone training and running algorithms! In addition, when going ‘cloud native’ and moving legacy analytical applications to the cloud, organisations need to consider what to transition and what not to. Experience has shown that it is difficult to predict how applications will escalate costs during their transition. In some cases, it may not be feasible or advisable to move/build all applications to the cloud due to customer, market, risk or regulatory requirements.
2. Security and privacy
Analytics professionals must have a solid understanding of security and privacy issues on the cloud. Until now, analytics professionals have worked on company owned, managed, and protected infrastructure - but cloud necessitates a different security and privacy mindset. This is despite most cloud providers having a higher grade security infrastructure than most organisations.
3. Vendor lock-ins
Although there has been an increase in the number of cloud providers in the market, organisations tend to use a single provider. The nature of the cloud marketplace is suited to suppliers - cloud providers – and organisations that rely on a single provider may experience challenges from being locked in, which includes the loss of organisational agility.
4. Lack of organisational readiness
In pre-cloud days, analytics professionals would develop an insight or a model on their laptops and send their codes over to IT teams for implementation. Sticking with the same process defeats the purpose of moving onto the cloud. Cloud is built for a self-service model - it empowers analytics professionals to have end to end control over their analytics application. However, most analytics professionals, including data scientists, are not well versed in using the cloud. Also, legacy technology and IT and risk processes in organisations are not built to maximise the value of running analytics on the cloud, and can often be bottlenecks.
The challenges above (and others) have started to push organisations in the direction of a multi/hybrid cloud strategy. While this may be a good strategy, it does come with its own set of problems i.e., how we deliver such environments so they are secure, connect, and run applications. This brings us to the final part of the blog, where we explore what organisations need to do to leverage cloud for analytics effectively.
Leveraging cloud analytics
1. Develop capability, best practices and processes
2. Have a clear roadmap for the transformation of legacy analytics models and applications as enterprise cloud/hybrid cloud becomes a reality in your organisation
3. Build an understanding of tools to measure and track cloud economics and costs
Costs can escalate rapidly and there are tools out there to help you assess and measure the costs of moving applications over to the cloud. Make sure you leverage these tools!
Cloud is here to stay, that’s for certain. Even though the move to cloud comes with challenges, it can enable organisations to truly unlock the value of data and analytics. It can significantly reduce the time taken to realise the value of data, and data science and analytics teams in organisations must develop the mindset and skillsets to unlock this potential.
Sulabh is Chief AI Officer at Deloitte Consulting & Innovator with 20 years product innovation, data science and technology experience. He leads the Deloitte AI Institute in the UK, bringing together the best of global AI thinking and helping advance human-machine collaboration in the Age of With. Sulabh advises Global C-level executives on AI and exponential technology led transformations, particularly in the financial services industry. He works with a fantastic team of highly skilled experts spanning Fintech, AI theory, applied data science, ML engineering and innovation/design thinking, and has led significant greenfield and brownfield transformations. Some of the projects Sulabh has led include - AI driven UW in commercial insurance, AI powered Digital Banking solutions, satellite and imagery based systems for risk mitigation, intelligent fleet solutions based on telematics and auto quotation in motor insurance.