Beyond the hype: delivering value with computer vision #2

A focus on animal welfare

To monitor animal welfare in slaughterhouses, more and more countries are making camera surveillance (CCTV) mandatory.¹ Even though these cameras increase the possibility to detect welfare deviations, the thousands of hours of resulting video footage pose significant challenges. Only a small, random fraction of the video footage can be reviewed and therefore most video footage remains unseen.²

Therefore we developed the AI4Animals solution to make existing camera systems more effective. The solution uses computer vision algorithms to detect animals, people, and objects and how they interact. Their movements are analyzed to determine potential issues. For example, a pig staying behind could mean the pig is lame and hurt. A group of pigs staying in one spot for a longer period of time indicates a bottleneck, where the pigs can get stressed. The AI4Animals solution automatically detects and selects these fragments from hours of video, so welfare issues can be identified in an early stage.

This innovative solution is currently delivering value on a daily basis. The solution is:

• Implemented at multiple Dutch slaughterhouses, with more locations coming in the near future
• Analyzing more than 100 hours of video data every day
• Detecting different types of potential deviations in multiple process steps
• Monitoring processes for both pigs and cows
• Providing Animal Welfare Officers with a dashboard to review fragments and assign preventive actions, helping to continuously improve animal welfare

As discussed in our previous blog, a number of key challenges need to be tackled when implementing a computer vision solution. In the following sections we will first provide a general overview of the solution and touch upon different aspects needed for a successful implementation. After that, we will dive deeper into the challenges of bringing computer vision models into a real-word setting.

A wider lens: The many aspects of a successful implementation

Like in most real-life applications, the computer vision models in AI4Animals are part of a cascade of steps that also involve other analytical and technical solutions. The visualization below shows that it is not only important to properly recognize objects, but also to analyse movement patterns to conclude if, for example, a pig is staying behind. In addition, a (web-based) dashboard is created for the end-user to trigger preventive actions, and to incorporate user feedback to continuously improve AI4Animals.

This means that many challenges which are not related to computer vision had to be overcome in order to move towards a successful implementation:

• Out-of-the-box computer vision models may not offer the right insights. Creating a solution that is integrated into current processes and co-created with domain experts ensures that computer vision models deliver the right insights at the right time.
• Without users that accept the solution and incorporate it into their daily work routine, a computer vision solution cannot deliver value. User adoption is encouraged with a user-friendly tool and user trainings, and a solution that is recognized by leadership.
• A computer vision solution that works on your machine is not scalable. A valuable solution requires scalable (cloud) infrastructure to store and analyze the massive amounts of video footage generated on a daily basis, constructed in a way to balance between costs and accuracy.
• Computer vision solutions often involve videos and/or images that contain personal data. Ensure privacy regulations are followed by involving privacy experts from the start and incorporating privacy measures such as setting a retention period for all sensitive data
• Video data that is analyzed is often confidential and needs to remain so. The security of the system can be enforced using pen tests performed by Cyber Security experts.

Zooming in: the computer vision technology within AI4Animals

Bringing computer vision models beyond a proof of concept poses a whole new set of considerations and challenges in selecting, training and deploying them.

In model selection, the scalability and efficiency become decisive factors as the data grows to hundreds of hours of video footage per day. Additionally, different processes require different models and modelling techniques. For example, with a large number of animals, an efficient object detection (for example EfficientDet³ ) is most appropriate. For other processes, with fewer animals and where more detailed analysis is required, it can be more appropriate to use markerless pose estimation (such as DeepLabCut).⁴ This is a computer vision technique that can track body parts and with this, estimate the body posture of animals – all without using physical markers.

When training the model, it is important to keep in mind that a model that performs well on the examples it was trained on does not necessarily perform well on situations it was not trained on. For example, a model trained to recognize a metal stunning device may struggle when new, previously unseen metal objects (such as wheelbarrows or bicycles) appear. We therefore incorporate active learning: taking predictions of which our initial model is uncertain and adding it to our training set.

Deploying computer vision models at scale calls for the adoption of MLOps and DataOps best practices⁵. This includes versioning which data was used for which model, monitoring the data that is processed, and continuously monitoring the quality and performance of models in production.

Concluding

Delivering value with computer vision takes more than just a good (prediction) model. Implementing a solution in a real-life application means different model considerations and a wide range of other aspects such as cyber security, privacy, a scalable infrastructure, and user adoption.

What’s next?

In our next blog, we will dive into another use case that successfully implemented computer vision: Field Hockey Insights. Field hockey is known to be an innovative sport where new technologies are quickly adopted. With loads of video data available we are developing a player detection tool for the Dutch National Hockey team that augments already available event data, which opens up new possibilities to support tactical decision-making, engage fans and increase sponsorship value.

Want to read the first blog in this series ‘Real world applications and challenges of computer vision’? Click here.

¹CCTV cameras to be compulsory in Scottish abattoirs; Video surveillance cameras in Spanish slaughterhouses become mandatory by the end of 2021; France to install CCTV in abattoirs after animal slaughter scandals; CCTV to be mandatory in all Queensland slaughterhouses

²Opinion on CCTV in slaughterhouses by the Farm Animal Welfare Committee (FAWC)

³EfficientDet: Scalable and Efficient Object Detection

⁴DeepLabCut: a software package for animal pose estimation

⁵If data is the new oil, where is your industrial refinery?