Josep Curto
Academic Director of the Master's Degree in Business Intelligence and Big Data at the Open University of Catalonia (UOC) and Adjunct Professor at IE Business School
Is it of good quality? Are the conclusions supported by sound data?
“As regards its quality, we can consider it to be of high scientific quality. It is no surprise that it has passed the rigorous peer-review process of PNAS (Proceedings of the National Academy of Sciences) and features a pre-registered experimental design (AEARCTR-0013538) inspired by classic correspondence studies in labour economics, now adapted to language models (LLMs). The results are supported by a very robust and well-structured dataset: 35,280 CVs generated across 98 real-world occupations (US SOC classification) and evaluated on six models, both proprietary and open-source (GPT 5.4, Claude Opus 4.6, DeepSeek 3.2, Mistral Large 3, Grok 4.1 and Llama 3.3 70B). From a data science perspective, it is particularly commendable that the authors do not use another LLM as a judge (thus avoiding circular biases), but rather expert-selected dictionaries, penalised logistic regression (Ridge) and cosine similarity against real job postings from Glassdoor, as well as publishing all the code and data on Zenodo to ensure replicability.”
How does this work fit in with existing evidence?
“This work provides a more holistic view. Until now, the scientific literature on generative AI had documented gender and racial biases in isolated tasks (for example, writing letters of recommendation or evaluating CVs in isolation). The focus here is on systemic risks.
This study goes a step further by demonstrating the cumulative or compound effect (adapting the theoretical framework of systemic discrimination proposed by Bohren et al.). In the real world, both candidates and HR departments use the same models: some to draft their CVs and others to screen them. The paper demonstrates that these two processes are not independent, but rather feed into one another.”
What impact does AI assessment bias have, according to this research?
“There are several points to note:
- The study refutes the idea that AI penalises women uniformly. In reality, it acts as a structural mirror of the labour market: in male-dominated professions, female candidates receive lower employability scores (reducing their likelihood of passing the initial screening), whilst in female-dominated sectors they receive an equivalent bonus. An aggregated average analysis would conceal this problem as both extremes cancel each other out. This highlights the need to carry out an intersectional equity analysis.
- In salary recommendations, AI assigns women lower salaries. This is mainly due to evaluation bias (rating the same profile more poorly solely on the basis of gender), exacerbated by generation bias (the model drafts CVs for women with more traits associated with non-promotable or support roles, and fewer traits geared towards high-visibility positions).
- When the same AI ecosystem generates the CV and then evaluates it, the initial distortions are magnified rather than corrected. In other words, we observe one of the consequences of the phenomenon known as model collapse (where patterns are reinforced and the quality of the model deteriorates).”
Are there any significant limitations that need to be taken into account?
“Although the article is of high quality, five key limitations should be noted (the authors themselves acknowledge some of these):
- It is a controlled audit experiment, not a field experiment. It abstracts from real human interaction: in a company, a recruiter may oversee the tool and a candidate usually edits the draft provided by the AI.
- The study is conducted exclusively in English and uses occupational data from the United States. In languages with marked gender inflection, such as Spanish, the propagation of bias from generation to evaluation might behave differently (and even more intensely).
- Salary outcomes are sensitive to how gender is indicated (explicit vs. proper name) and to the specific model (for example, GPT 5.4 exhibited atypical behaviour compared to the other five models when using proper names). Furthermore, the most recent versions of the models do not show a reduction in bias; in some cases, it is even greater.
- In the baseline design, the models generate CVs without any real prior experience or education. Although the authors demonstrate in sensitivity tests that evaluation bias persists when real credentials from LiveCareer are included, generation bias decreases when the user provides detailed information from the outset.
- It provides a snapshot in time, given the speed at which new models emerge. That does not mean it is irrelevant, since new models cannot generally be understood as an update to the previous one, but rather as a new artefact due to their internal structural changes. In this sense, they do not always lead to improvements, and it is often the case that they introduce new problems.”
What are the implications?
“From my perspective, the conclusions of this study send clear messages to organisations, developers and regulators:
- Automating recruitment processes with generative AI does not guarantee objectivity; without specific audits, companies run the risk of automating and scaling up existing inequality under the guise of technological efficiency.
- Instructing the model to ‘be gender-neutral’ is practically ineffective. Conversely, explicitly instructing it to favour the recruitment of women leads to massive overcorrection across all sectors, without distinguishing where there is actual under-representation.
- Given that models are constantly updated and global averages are misleading, companies must implement granular audits by role and sector, make statistical adjustments during the screening phases, and incorporate various governance mechanisms, aligning with regulatory frameworks such as the European AI Act, which classifies recruitment systems as high-risk.
- It is necessary to implement genuine governance safeguards to avoid token compliance.”