Meaningful human intervention in algorithmic decision-making

Organisations are increasingly using algorithms and artificial intelligence (AI) for algorithmic decision-making. Consider assessing a credit application or online job applications. If organisations want to use algorithmic decision-making, they must comply with certain rules. People have the right to human intervention in algorithmic decisions about them that affect them. The GDPR contains rules regarding this.

Meaningful intervention

Human intervention ensures that a decision is made carefully and aims to prevent, for example, people from being excluded or discriminated against, unintentionally or otherwise, by the outcome of an algorithm. This human intervention should not merely have a symbolic function; it should contribute meaningfully to decision-making.

The design is essential

The design of meaningful human intervention is essential. If an employee is hampered by time pressure or an ambiguous system, for example, this could impact the outcome of the decision.

Organisations often have questions about how to set this up. That is why the AP, the Dutch data protection authority, has developed tools for meaningful human intervention in algorithmic decision-making. These tools can help organisations and companies that intend to use algorithmic decision-making to guide the design of human intervention. For this reason, the document has been written to be as practical as possible to ensure it best addresses questions organisations have about the design.

The AP provides examples and an overview of questions that can help organisations guide the design of human intervention in a meaningful manner. These questions and examples address the relevant factors of humans, technology and design, process, and governance.

Result of consultation

Responses to the consultation have been collected and included without reference to the organisation or individual. Government agencies, independent foundations, companies, sector organisations and academics have provided input. Responses to the consultation included suggestions for particular questions. See the summary of responses to the consultation (in Dutch).

Document status

With this document we aim to help organisations design meaningful human intervention in the most practical way possible. The document provides practical tools and is intended to guide organisations in designing meaningful human intervention in algorithmic decision-making.

Scope

This document is about meaningful human intervention, which ensures that the decision-making is not solely automated as referred to in Article 22 (1) GDPR and Article 11 (1) LED. These articles relate to ‘decisions based solely on automated processing’. By this we mean a decision that is entirely based on automated processing and has legal consequences for data subjects or otherwise affects them to a significant extent. If this involves meaningful human intervention, it is not regarded as a decision based solely on automated processing. In that case, the decision is not covered by Article 22 of the GDPR.

Besides the design of human intervention, there are other key concepts from the article that may raise questions. For instance: when is the outcome of the process considered a ‘decision’? When does a decision affect someone ‘to a significant extent’? The document provides no answer to these questions.

Follow-up

To help organisations on their way, the AP will get started with the practical implementation of other key concepts regarding automated decision-making in the GDPR this year, such as what it means to be ‘affected to a significant extent’, and what information a data subject must be provided with about the automated decision.