I’ve been on both sides of the table enough times to have formed an opinion that won’t endear me to people who design interview loops: most technical interviews measure the wrong thing, and they get more wrong the more senior the role.
I’ve hired and interviewed engineers as an engineering manager, and I’ve recently been back on the candidate side of a few processes myself. The thing that keeps standing out is the gap between what the interview tests and what the job turns out to need. They overlap far less than the format quietly assumes.
What the Standard Interview Measures
The default technical interview tests a fairly specific skill: solving a self-contained problem, alone, under time pressure, with no documentation, while a stranger watches. The algorithm puzzle is the purest version of it. Can you spot that this is a graph traversal or a two-pointer problem and code it cleanly in twenty minutes, with someone judging your variable names as you go.
That is a real skill. It just isn’t the skill most of the job is made of, and it’s close to none of what a senior engineer is actually paid for.
What the Job Actually Needs
Picture a real workday. The problem is underspecified and half the work is figuring out what’s really being asked. You have documentation, a search bar, colleagues, and usually a lot more than twenty minutes. The coding is rarely the hard part. The hard part is deciding what to build, and more often what not to build. It’s noticing that the problem is simpler than it looks, or that it’s a shape you’ve seen before. It’s explaining a tradeoff clearly enough that other people will get behind it.
None of that is what the puzzle measures. The recognition piece is the interesting near-miss: interviews do test pattern-matching, they just test the wrong patterns. The shapes that matter at work are architectural and product-level, not “reverse this tree.” I’ve argued before that recognizing a problem’s shape is most of problem-solving, and the library of shapes that helps you on the job is not the one a whiteboard checks.
The Mismatch Runs Both Ways
There’s a flip side I don’t think gets said enough. When the interview doesn’t resemble the job, it doesn’t only misjudge the candidate. It misinforms them.
A friend of mine interviewed somewhere that grilled him on modern Java, best practices, architectural patterns, a full system-design round. He prepared hard, performed well, took the offer. Then he started, opened the codebase, and found spaghetti everywhere. His summary, roughly: for all the talk of modern Java and best practices in the interview, what they actually had was a mess.
The interview hadn’t described the job. It had described the job they wished they had. They were screening for engineers fluent in the practices they aspired to, while the actual day-to-day was untangling the code they’d really written. That isn’t just an inaccurate filter. It’s a quiet bait-and-switch the company probably didn’t even realize it was running, and one the candidate only discovers after he’s signed. An interview that looks nothing like the work fails everyone in the room, the new hire included.
The More Senior the Role, the Worse the Fit
This gets worse the more senior the role. A junior’s day really is close to “solve the well-defined problem in front of you,” so a problem-solving screen is a defensible proxy at that level. A staff engineer’s value is judgment, restraint, and making the people around them better. A whiteboard puzzle measures none of those, so the more senior the position, the more the standard interview tests a skill the job has largely moved past.
I’ve felt this as a candidate, too. The processes that evaluated me most accurately looked nothing like a timed puzzle, and the ones built around puzzles learned very little about whether I’d be any good at the role they were actually hiring for.
What Does Predict It
So what holds up as a signal? From both sides of the table, a few things.
How someone reasons about a messy, real problem out loud, with the freedom to say “I don’t know, here’s how I’d find out.” That sentence is a green flag, and a good interviewer should be fishing for it rather than penalizing it. The actual red flag is bluffing.
Whether they ask what not to build. The engineers I most wanted to hire pushed back on the premise of the question, scoped it down, and questioned whether the thing was worth doing at all. That restraint is most of what senior judgment is, and it’s invisible to a puzzle that hands you a fully specified problem and rewards you for solving exactly it.
Whether they’d make the team better or just themselves faster. A strong IC who quietly becomes the only person who understands anything is a bottleneck, not a multiplier, and you can usually hear which one you’re talking to within a few minutes of asking them to walk through a past decision.
The Interviews That Worked
The evaluations that told me something, again on both sides, barely felt like interviews. They felt like two people who already worked together talking through the craft. You pull on a thread, they pull one back, and you follow it into the corners where it gets interesting. No timer, documentation fine, nobody performing. Spin up a few of those threads over an hour, each one quietly probing what the other actually knows, and by the end you both have a real answer to the two questions that matter: does the other person know their craft, and is this someone you’d enjoy building with. A specific conversation about something they genuinely built, the decisions, the tradeoffs, what they’d change now, gets you there far faster than any puzzle. You learn more about an engineer in fifteen honest minutes on a real past decision than in an hour of watching them sweat through a data-structures problem they’ll never see again.
Set against that, the leetcode round looks like what it often is: a lazy way to put a number on a person. It hands the interviewer a tidy score to write down at the end, which feels like rigor and is mostly just convenient for whoever’s doing the rating. Convenience for the evaluator is a poor reason to learn almost nothing useful about the person across the table.
Why the Puzzle Persists Anyway
The standard interview scales, and it feels objective. A puzzle has a right answer, it’s relatively hard to fully fake, and at high volume, especially for junior roles with no track record to read, a problem-solving screen is a defensible cheap filter. The alternatives are harder. Judgment is harder to assess than correctness, more exposed to bias and gut feeling, and an interviewer who’s bad at reading it can do real damage while feeling completely fair. Part of why the puzzle survives is that it feels rigorous even when it’s measuring the wrong thing, and “measure the right thing, subjectively and imperfectly” is a genuinely hard sell to a hiring committee.
There’s a trap in the obvious alternative, too. “Just look at their open-source work” quietly punishes everyone without the free time to build in public, which disproportionately means caregivers and people whose jobs already own everything they produce. Grading people on their side projects is the wrong fix. Better to evaluate how they reason about real work, which every working engineer has whether or not they ship anything in their spare time.
What Held Up, On Both Sides
Both as someone who has hired and as someone who’s recently been hired, the same thing keeps being true. The evaluations that predicted anything looked like the work, and the ones built around artificial puzzles mostly predicted how good someone was at artificial puzzles. The most serious interest I’ve had lately didn’t come from acing a screen at all. It came from people having seen what I’d actually built and written.
So if you design interview loops, hold every stage up against whether solving it looks anything like the job. When it doesn’t, you aren’t measuring whether someone can do the work; you’re measuring whether they can interview. That’s usually the gap between the candidate who interviews well and the one who turns out to be good at the job.