When designing human-in-the-loop automations and AI workflows, you have to figure out how much human should be in the loop. It looks like Workday is finding this out the hard way.
Workday has a set of powerful human-in-the-loop workflows that accept, screen, evaluate and accept or reject candidates–they save a ton of time in an otherwise labor-intensive process.
The product design of this human-in-the-loop feature needs to be tuned so that humans can influence the process and then participate in the process effectively. For instance, have the AI match some criteria, but have a human make the final call.
It’s a balance. The objectives are well-meaning, but the development of these products and the tactics shown in the features need to be handled carefully. If the process leans too heavily into AI to make decisions without proper attenuation, and doesn’t include human judgement in the workflow, then it could expose the company (or customer company) to risks like, say, discrimination lawsuits.
It looks like we’re going to find out if that may have happened. Workday is facing a class action lawsuit charging that their algorithmic and AI-based applicant evaluation tools unfairly discriminated against applicants over 40.
In order to compile the dataset needed to determine if the charges are true, the judge in the case has ordered Workday to supply a list of ALL the companies that used these features, even those that used the features AFTER the lawsuit was filed.
At its core, this will probably be like a ‘product failure’ case (I’m not a lawyer). Did it cause harm? Did Workday know there were these kinds of risks? And did they make decisions to move forward anyway? If yes, then guilty.
It’s one of the ways that product design, and the product design process, has real risk management impact for companies. What kind of impact? Well, while the case works through the courts, investors aren’t waiting. Since the case has moved forward on May 22, $WDAY started dropping and hasn’t recovered. The company has lost 17.5% or $12.2B of its value. The AI lawsuit has been widely cited as a key reason for investors to stop backing Workday.