Algorithm Assurance has been saved
Services
Algorithm Assurance
The growing use of algorithms
With an increasing number of use cases, there is also a growing need to understand risks arising from the use of algorithms and how management and governing bodies can obtain assurance that these risks are controlled and maintained. Our experts are able to provide this assurance over algorithm controls and to address these issues, leveraging their diverse range of skills in assurance services and algorithm technology.
Algorithms: the need for a robust control framework
While the growth of algorithms is welcomed, they also come with a growing need for assurance over algorithm controls. The use of algorithms is increasingly coming into the public eye, causing regulators and senior management to consider how well the associated risks are being controlled and managed. A robust algorithm control framework is fundamental to algorithm risk management and should cover key areas including governance and oversight, algorithm pre- and post- go-live testing, specific algorithm controls around key risk, monitoring, surveillance and appropriate levels of documentation.
In each of these areas, minimum standards are now being adopted across many markets, which internal and external stakeholders are increasingly looking to attain and benchmark against. The sort of questions companies now need to ask include:
- Are we sure our algorithms are treating our customers fairly, including under stressed conditions?
- Are we comfortable our algorithms are not deliberating distorting markets, including under stressed conditions?
- Are we comfortable our algorithms meet their design objectives, including under stressed conditions?
- Do we have sufficiently designed controls to manage our algorithm-related risks?
Because this area is so new, regulations and guidelines are still evolving, creating a challenging algorithmic accountability, transparency and compliance landscape. Our team is well positioned to help clients understand ‘what ‘good’ looks like’ in their algorithm control framework.
What are the requirements for trustworthy algorithms?
Providing stakeholders with confidence in the use of algorithms is the cornerstone for the effective use of these technologies. Boards therefore need to consider the following six elements during the implementation and ongoing use of algorithms:
Fairness
Robustness
and reliability
Respect
of data privacy
Safe and secure
Responsibility
and accountability
Explainability
and transparency
Fairness
Fairness is a key concern for many operating with self-learning algorithms. In a world where bias is “human”, the risk consists in the algorithms being trained on biased data sets. They will then identify the “trends” and apply them even more forcefully to new datasets resulting in enormous reputational risks. Consider the recent story of the exam algorithm introduced in Britain during the pandemic: With schools in lockdown, the algorithm was used to determine A-level grades based on pupils’ performance earlier in the year. The result was that based on past data, pupils of historically high-performing (mostly) private schools were disproportionately advantaged compared to those from lower-performing (public) schools, causing an uproar in the media and eventually the removal of the algorithm.2
Source: ‘Trustworthy AI’ is a framework to help manage unique risk | Deloitte US
How Deloitte can help?
Deloitte can assist you when addressing these challenges. Our services cover the topic of “Algorithm Assurance” in a comprehensive way, focusing on the following elements:
- Governance and internal controls: Readiness assistance and assurance of the governance framework and internal controls relating to the implementation and operation of algorithms
- Algorithm review, modelling and data: Assessment of models used in the development, training, testing and deployment of algorithms. Review of documentation to ensure the explainability and transparency of models and decisions taken by the algorithm.
- Monitoring: Review of monitoring processes to ensure an early identification of biased or incorrect decisions.
With our extensive experience reviewing processes, technology and data models of our clients we are best positioned to provide you and your stakeholders with assurance on the robustness and reliability of the controls around your algorithms. Moreover we can assess your compliance with the criteria for “Trustworthy AI” as developed by Deloitte US or any other available governance framework1 for the ethical use of algorithms.
Algorithm Assurance provides you and your stakeholders with transparency and instills trust in your systems and algorithms.
1 ALTAI - The Assessment List on Trustworthy Artificial Intelligence | Futurium (europa.eu)
2 A-levels and GCSEs: How did the exam algorithm work? - BBC News
Contacts
