Bias Audits Explained

Last reviewed: July 2026

What Is a Bias Audit?

A bias audit is an independent, statistical evaluation of an AEDT that measures whether the tool produces disparate impact across protected groups (sex, race/ethnicity, and intersectional categories). Under NYC LL144, an audit must be conducted by a qualified independent auditor within the 12 months before the AEDT is used.

Who Qualifies as an Independent Auditor?

DCWP rules require that the auditor be independent of both the employer and the AEDT vendor. Specifically, the auditor must:

• Not be involved in the development or training of the AEDT
• Not be affiliated with the employer or the AEDT vendor
• Have no financial interest in the outcome of the audit
• Have the technical expertise to conduct statistical bias testing

Source: NYC Admin Code § 20-871(a); DCWP Final Rules § 5-301

What the Audit Measures

The audit evaluates the AEDT's impact on candidates by:

• Selection rates — The percentage of candidates selected/recommended, broken down by sex, race/ethnicity, and intersectional categories (e.g., Black women, Hispanic men)
• Impact ratios — The selection rate of each group divided by the selection rate of the highest-selected group
• Score distributions — How scores are distributed across protected groups

The Four-Fifths Rule (0.80 Threshold)

The EEOC's Uniform Guidelines (29 C.F.R. § 1607) establish the four-fifths rule: a selection rate for any protected group that is less than 80% (0.80) of the rate for the highest-selected group indicates potential adverse impact.

Cost & Timeline

How to Prepare

1. Gather 12 months of hiring data — Every candidate who was evaluated by the AEDT, including demographics (sex, race/ethnicity), selection outcomes, and scores
2. Document your AEDT — What it does, how it works, what inputs it uses, what outputs it produces
3. Identify your auditor — Choose an independent auditor with statistical expertise (e.g., DCI Consulting, Holistic AI, BLDS Consulting)
4. Provide data access — Give the auditor access to your hiring data in a secure, compliant manner
5. Review the audit report — Check impact ratios, flagged categories, and remediation recommendations
6. Publish the summary — Post the required summary on your public website
7. Set annual renewal — Calendar reminder 60 days before expiration

What Happens If the Audit Finds Bias?

If impact ratios fall below 0.80 for any group, the audit will flag it. You are not automatically penalized — but you must:

• Document the finding in the published audit summary
• Take corrective action (adjust the tool, change inputs, add human review)
• Re-audit after remediation to verify improvement
• Be prepared to explain your remediation efforts if DCWP investigates

Having an audit that finds bias is better than having no audit at all. The law requires you to conduct the audit and publish the results — it does not require the audit to show no bias.