“It’s the most fair thing to do, but it doesn’t make any sense”: Perceptions of mathematical fairness notions by hiring professionals

Hiring is a high-stakes DEI-compatible field, where negative decisions by Machine Learning (ML) systems can deny an individual the opportunity for financial stability. The increasing adoption of ML systems to automatically assess job applicants is raising concerns about the (un)fairness of algorithmic decision-making. Despite their usage in hiring, there is no understanding of the alignment of organizations on the fairness of algorithmic decisions, which prevents them from justifying its use and creating DEI-compatible policies regarding technologically enabled hiring. Towards that, we interviewed 17 professionals from executive functions, talent acquisition, HR, I/O psychology, and DEI operations. We take a direct approach to address fair hiring, where previous work in fairness in ML is limited to lay people’s perception of fairness. We also depart from existing interviewee sampling methods by explicitly searching and including professionals from different genders and ethnic backgrounds to obtain diverse insights. For effective communication with professionals who are not necessarily mathematically aligned, we designed user-friendly illustrations and explanations of six mathematical fairness notions in the context of early candidate selection in hiring. The iterative designs developed with the participation of independent researchers aided the professionals in expressing a rich set of themes surrounding understandability, perception of fairness, perception of diversity, and applicability of existing fairness notions. Qualitative analysis of the responses reveals several findings; fairness notions miss many hiring-related contexts, diversity and fairness considerations may conflict, and there is discomfort in weighing merit and diversity in selections. Moreover, professionals who identified as a minority had a different stance on fairness. We conclude that a qualitative approach in collaboration between designers, practitioners, and policymakers is the key to refining and contextualizing future technologically enabled fair hiring policies. Our participants’ intrinsic motivation to engage with the topic of fairness strengthens our case.