Blog 3: Scaling in the cloud has been saved
Article
Blog 3: Scaling in the cloud
Blog 3/4 from the series about the Deloitte Impact Foundation initiative ‘Cognitive Deforestation Prevention’ that aims to prevent illegal deforestation by building an artificial intelligence solution that predicts where illegal deforestation will happen
Sebastian Panman de Wit is also part of the Artificial Intelligence team at Deloitte. Through this Deloitte Impact Foundation initiative, he experienced what it’s like to work for an organization with social responsibility as its core value. In his blog he talks about his experience and explains the complexity and purpose of the AWS Cloud technology used for the Early Warning System.
Introduction
Sebastian: “In the past years I have been part of the Artificial Intelligence team of the Analytics & Cognitive department of Deloitte. This involved doing a lot of cool AI-related projects, of which most are at big corporations. Working on a project with the WWF - an organisation with social responsibility as its core value – is quite a different experience. The project is a lot of fun and the technology and expertise required is not any less complex. Within the project I have mainly focused on leveraging the Cloud in the right way. Additionally, I worked on the Data Science aspect, which included optimizing the data preprocessing part of the solution.
Labeled satellite images as an input
One of the most important Data Science activities is the collection of the right data. That is why we are working together with a university spin-off called SarVision that provides us with labeled satellite images. To these images we add contextual data such as elevation, population density, and land types. To take the elevation data as an example: you can imagine that areas at high altitudes might be harder for loggers to reach and thus to deforest. Together with academic and domain experts we try to figure out what information might be valuable for the prediction of illegal deforestation.
From pixel values to valuable data
Most data that we use is meaningless to the cognitive model in its raw form. Therefore we use multiple so-called processors that transform this raw data into information-rich data. For example: raw data containing locations of roads are transformed into data containing information on the nearest roads to a certain area. Areas close to a road may provide easy logging transportation and may, therefore, be of higher risk of illegal deforestation. In total, we use over 10 different data sources with almost 20 data processors to create information-rich data that allow our cognitive model to effectively predict illegal deforestation.
Technology stack Early Warning System
Using the Amazon Cloud to save the rainforest
The amount of data we are working with is huge. Around 100GB of raw data is transformed into more than 3TB of information-rich data. Therefore is it important that we use the right technologies for the right tasks. To create a scalable and maintainable solution we work together with Amazon Web Services (AWS) to leverage the Cloud throughout our data pipeline. We developed a microservice-like infrastructure where we can use the scalability of the Cloud to ensure fast data transformations. In simple words: data transformations take about the same processing times for small amounts of data as for large amounts of data. This allowed us to reduce the total preprocessing time from 12 hours to 4 hours and keep this consistent with larger datasets. Additionally we use other Cloud solutions such as AWS Sagemaker, Lambda Functions and Step Functions to accelerate our Data Science activities. These technologies will make scaling the solution to large landscapes a lot easier!
Grateful to be part of this initiative
With all the different technologies and programming code lines, I sometimes forget the ultimate goal of the solution; that is to prevent illegal deforestation. However receiving photos from the users that went in the field and found indicators of illegal deforestation, reminds me that all the underlying technology we are developing, is making this possible. I’m very happy and grateful to be part of this great Deloitte Impact Foundation initiative.”
Contact
Recommendations
Cognitive Deforestation Prevention
This WWF program uses artificial intelligence to prevent illegal deforestation and is supported by the Deloitte Impact Foundation
Blog 4: Enabling action
Blog 4/4 from the series about the Deloitte Impact Foundation initiative ‘Cognitive Deforestation Prevention’ that aims to prevent illegal deforestation by building an artificial intelligence solution that predicts where illegal deforestation will happen
