In 2018, Reuters reported that Amazon had scrapped an experimental recruiting tool after finding it marked down CVs that included the word “women’s”. The model had learned from ten years of mostly male hires.
The lesson
An algorithm trained on past decisions repeats past patterns, including unfair ones. Removing gender from the data is not enough, because other details act as stand-ins.
The checks
- Test outcomes by group before and after launch.
- Keep a human decision-maker and a way for candidates to ask for review.
- Know the rules: New York City requires bias audits of automated hiring tools, and the EU AI Act treats hiring AI as high-risk.
Comments