Not Every Candidate Has the Same AI

TL;DR: Letting candidates use their own AI means a candidate on a paid frontier model submits stronger work than an equally capable candidate on a free tier, and both followed your rules. Banning AI isn't enforceable, and allowing anything makes submissions hard to compare. Providing the same environment to everyone is the option that gives you scores you can defend.

At Gradient, we talk to a lot of teams about how they're handling AI in take-homes. One pattern comes up often and gets discussed rarely: the brief permits AI, nobody asks which tool a candidate used, and two submissions produced under very different conditions get scored against the same rubric.

Why candidate AI access is a fairness problem

Picture two candidates with the same brief and the same three days.

The first has a paid $200 plan on a frontier model, a second subscription for research, and a machine that runs both. The second is on a free tier with a usage cap and hits it on day two. Both followed the rules and submitted honest work, and a good chunk of the difference between the submissions traces back to the subscription.

At the entry level it gets sharper. Some universities hand every student a paid account and some don't, so a take-home that permits AI without supplying it is partly measuring which school a candidate attended and the tools they provide.

Validity and adverse impact, in plain terms

In the assessment world, you have to consider a few things:

Validity asks whether an assessment measures what it claims to measure. A take-home meant to measure analytical judgment, which partly measures tool access, has a validity problem: the score conflates two variables.

Adverse impact asks whether a selection process produces meaningfully different pass rates across groups of candidates. Tool access tracks income, employer, and school, so a process that rewards better tools can produce uneven outcomes without anyone intending it.

Three ways teams handle AI access

Ban AI in the brief

Often, firms address this by banning AI use entirely. Unfortunately, this is very hard to verify. Detection tools are unreliable and misflag non-native English speakers. You've also designed an assessment around a way of working that doesn't match what you expect on the job.

Allow candidates to use anything

Most teams are here, usually by default rather than by decision. This option is easy, but reduces validity meaningfully: since submissions arrive from different toolsets, your rubrics are comparing work produced under different conditions.

Provide the AI environment yourself

Every candidate gets the same model, the same tools, and the same time limit. The inputs match, so the submissions are comparable and the rubric means what it says. However, this takes real infrastructure to run.

Where we landed

We built Gradient around the third option: our AI skills assessments give candidates a neutral, third-party environment, with time upfront to explore it. We standardize the model and harness per assessment, and block access to web search, so each candidate gets the exact same environment and experience. This ensures consistency and fairness over time.

Even if you don't use Gradient, we encourage teams who are designing take-homes to be thoughtful about fairness: if you expect people to use AI, make sure they have access to quality tools. If you expect them not to use AI, consider if that's realistic, and how it might skew results if some do use AI and just don't disclose it.

If you're working out which one fits your process, we'd be glad to help you think it through. And if you're rethinking the assignment itself, we wrote about that in What to do when AI can solve your take-home.