International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 12 Issue: 08 | Aug 2025
p-ISSN: 2395-0072
www.irjet.net
Algorithmic Bias Detection in AI-Powered Recruitment Tools: An Empirical Analysis Using Resume Matching Scores Pratham Mehta¹ 1Research Scholar, Oklahoma School of Science and Mathematics, Oklahoma, United States of America
---------------------------------------------------------------------***--------------------------------------------------------------------1.1 Research Objectives Abstract - This research presents a comprehensive analysis of algorithmic bias in AI-powered recruitment tools using a dataset of 9,544 resume-job matching scores from 2025. We evaluate major AI recruitment platforms including Workday, Greenhouse, Lever, and others that utilize automated resume scoring and matching algorithms. Employing three primary bias detection metrics—disparate impact ratio, demographic parity, and equalized odds—we identify significant variations in scoring patterns across different job categories and candidate profiles. Our findings reveal that matched scores exhibit a mean of 0.661 with substantial variance (σ=0.167), indicating potential systematic biases. The study demonstrates disparate impact ratios below the 80% threshold for certain demographic groups inferred from educational institutions and skill distributions. This research contributes to the growing body of evidence regarding AI bias in hiring and proposes a framework for continuous bias monitoring in recruitment AI systems.
The primary objectives of this research are: 1. To quantify algorithmic bias in AI recruitment tools using standardized fairness metrics 2. To identify patterns of discrimination across different job categories and candidate profiles 3. To develop a reproducible methodology for bias testing that organizations can implement 4. To provide recommendations for bias mitigation in AIpowered recruitment systems
1.2 Scope and Limitations This study focuses on resume-to-job matching scores as the primary indicator of potential bias. While we cannot directly access demographic information due to privacy constraints, we employ proxy indicators including educational institutions, geographic locations inferred from institutions, and skill distributions to identify potential bias patterns.
Key Words: AI Bias, Recruitment Algorithms, Disparate Impact, Resume Screening, Algorithmic Fairness, Machine Learning Ethics, HR Technology
1.INTRODUCTION
2. LITERATURE REVIEW
The proliferation of artificial intelligence in recruitment has fundamentally transformed hiring practices, with over 75% of large organizations now utilizing AI-powered tools for resume screening and candidate evaluation [1]. However, recent research demonstrates that these systems exhibit systematic biases across gender, racial, and other protected characteristics, with documented discrimination rates reaching up to 85% against certain groups [2].
Recent academic research provides substantial evidence of algorithmic bias in AI recruitment systems. The University of Washington's 2024 study analyzed over 3 million resume-job description combinations, finding that whiteassociated names were preferred 85.1% of the time compared to Black-associated names at only 8.6% [3]. This represents one of the most comprehensive empirical demonstrations of racial bias in modern AI hiring tools.
This study addresses the critical need for empirical bias testing of commercial AI recruitment tools using realworld data. We analyze a comprehensive dataset of 9,544 resume-job matching pairs from 2025, focusing on platforms that perform automated scoring and matching including Applicant Tracking Systems (ATS) with AI capabilities, resume screening tools, and talent intelligence platforms.
© 2025, IRJET
|
Impact Factor value: 8.315
Amazon's discontinued AI recruiting tool serves as a landmark case, having been trained on predominantly male resumes over a 10-year period, resulting in systematic discrimination against women candidates [4]. The system penalized resumes containing the word "women's" and downgraded graduates from all-women's colleges, demonstrating how historical biases become encoded in AI systems.
|
ISO 9001:2008 Certified Journal
|
Page 398