How to run a take-home coding assignment

How to run a take-home coding assignment well, now that any current AI agent can produce a plausible-looking answer to a generic prompt in minutes, comes down to three decisions: scope the task so a copied answer doesn't actually finish it, cap the time window and say so upfront, and grade the reasoning behind the submission instead of just whether it runs. Get those three right and the format still works. Get them wrong and you're grading whoever has the best prompt.

How long should a take-home coding assignment be?

Two to four hours of focused work, stated explicitly in the brief — not "a few hours, take your time," which candidates correctly read as "however long it takes to look as strong as possible." An unbounded window doesn't improve signal; it selects for who has a free weekend rather than who's the stronger engineer, and it costs you candidates who have other offers moving faster. If the role genuinely needs a longer evaluation, split it: a short take-home as a filter, then a longer paid or live session for finalists.

What should you actually ask candidates to build?

Something with enough ambiguity that there's no single reference solution to paste in. A generic "build a REST API for a todo list" prompt has been solved a thousand times in every training set that matters — an agent (or a candidate copying a five-year-old GitHub repo) produces a correct-looking answer with almost no signal about who they are. Better prompts:

• Extend an existing, slightly awkward codebase rather than starting from a blank file — it tests whether they read before they write.
• Include one underspecified requirement on purpose, and note in your rubric whether the candidate asked about it, made and stated an assumption, or silently picked one and moved on.
• Ask for a short written rationale alongside the code — the tradeoffs they considered and why they picked this approach. This is the part a plausible-looking AI submission is weakest at, because it has to reflect a specific decision the candidate actually made.

Should candidates use AI on a take-home coding assignment?

Pick a policy and design around it — the mistake is leaving it unstated. If your engineers use AI tools daily, banning AI on the assignment tests a version of the job that doesn't exist, and you'll select against candidates honest enough to ask before quietly using it anyway. The more useful policy is to allow it openly, ask for a one-line note on how it was used, and grade what a tool can't supply on its own: the choice of approach, the tradeoffs named in the rationale, and whether the candidate can defend every line in the debrief. We cover the broader version of this — interviewing candidates who work agent-first rather than just permitting AI on a take-home — in how to interview engineers who use AI coding agents, and the prevalence of undisclosed AI use specifically in AI cheating in technical interviews, explained.

Take-home assignment vs. live coding interview: which one do you need?

They test different things, so the honest answer is usually both, not either. A take-home shows what a candidate produces with time, references, and (usually) AI available — closer to real working conditions than anything timed and watched. A live session shows how they think and communicate under observation, and it's where you verify that the take-home reflects their own judgment rather than an agent's. Use a take-home alone and you can't tell those two things apart. Use a live session alone and you lose the realistic, unobserved-conditions signal a take-home is good for.

How to evaluate a take-home coding assignment fairly

1. Write the rubric before you send the first assignment, not after reading the first submission — otherwise the bar quietly shifts to fit whoever you liked.
2. Score the decisions, not just the output. Did they handle the underspecified requirement well? Is the rationale specific to this codebase, or something that could have been pasted onto any similar prompt?
3. Weight the debrief into the score, not just the diff. A candidate who can't explain a choice they supposedly made, or who stalls when you ask them to change a requirement live, is telling you something the submission alone didn't.
4. Calibrate across reviewers.Have two people score the same submission independently before you rely on a single reviewer's judgment across a whole pipeline.

Running the debrief

Keep it short — thirty to sixty minutes — and make it specific to their submission, not a generic set of questions you'd ask anyone. Pick two or three concrete moments from their code or rationale and ask why they made that call. Then change one requirement on the spot and ask how they'd adapt the solution. Someone who did the thinking themselves can follow that thread immediately; someone reciting an explanation they don't fully own generally can't, and the gap shows up fast.

Where DeftBench fits

An unsupervised take-home tells you a result was produced; it doesn't tell you who produced which part, or how. DeftBench takes a different approach to the same problem: candidates work in a live browser IDE with a real AI agent wired in, and the session is captured as telemetry — prompts, tool calls, and edits attributed to human versus agent — so the evaluation is evidence-linked instead of a guess about what happened off-screen between submission and debrief. If you're weighing that against an async take-home tool, see the DeftBench vs CoderPad comparison, or talk to us about a pilot.