Algorithmic risks in the workplace

 Challenges and risks

The use of algorithms in the workplace can benefit workers, for instance  when the system supports them  in working more efficiently or safely and when a system ensures a fairer distribution of tasks amongst employees. It can, however, also have an adverse impact on workload, autonomy, and a contestable employer-employee relationship. Here are a few examples of challenges and risks identified by the fall 2023 edition of the Dutch Data Protection Authority’s AI & Algorithmic Risks Report:

• An algorithm monitoring workers may cause them to feel  permanently surveilled and analysed. Monitoring of workers may also shift the employment relationship from human-to-human to algorithm-to-human.  The previous causes a risk of work being viewed as a product rather than  as something human. 
• Getting a grip on the risks associated with a worker management algorithm requires a tailored and context specific approach. A system that manages a group of warehouse workers functions different when compared to a system deployed in a fast-food restaurant.  The risks involved may therefore also be different from one domain to the next. Assessing the risks that these systems introduce is therefore always contextual. 
• Algorithms in the workplace may increase workload. After all, a system will be less able to take account of personal circumstances and external factors.  Human managers are better equipped to take into account factors which extend beyond productivity. 
• Algorithms in the workplace may have far-reaching impact on people’s autonomy and behaviour. Some AI systems are proven to  analyse the emotion in a call centre agent’s voice and challenge them on a lack of audible empathy in order to increase productivity.
• Algorithms in the workplace may even be a hindrance to innovation and quality improvement rather than being an enabling factor. It is harder for workers to shape their own work practices and improve their work processes while being directed by an algorithm. This deprives the organisation of the opportunity to harness each worker’s individual experience and knowledge and the human ability to adapt and be creative to solve problems. 

What is needed? 

Several aspects of deploying AI systems in the workplace will soon be regulated by the EU AI Act. The  AI Act will apply to the deployment of AI systems for the recruitment and selection of new employees and  for several decision making processes regarding them. EU legislation governing algorithms that manage platform workers is currently in the works in the Platform workers directive. Furthermore, the Dutch Dutch Data Protection Authority sees opportunities to further discussion on the topic of algorithms in the workplace and increase the transparancy of their use.  

To find out more about the risks of algorithms and AI in the workplace, read Chapter 3 of the fall 2023 edition of the ARR.