HN Debrief

Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

  • AI
  • Hiring
  • Software Engineering
  • Developer Tools

Coinbase’s post says it spent a year rebuilding engineering interviews for an "AI era" where code generation is cheap, then shifted from classic coding screens toward exercises that let candidates use AI in a real repo and discuss design, debugging, and production tradeoffs. The stated goal is not to catch people using models, but to measure whether they can steer them well and still show engineering taste, understanding, and practical judgment.

If you are changing interviews for AI-assisted development, test judgment and code comprehension directly instead of treating tool usage as the skill. Keep a baseline check for unaided fluency, or you risk hiring people who can drive an assistant but cannot catch subtle failures, especially in security and maintainability.

Discussion mood

Mostly skeptical and eye-rolling. People accepted that AI belongs in modern engineering work, but they distrusted interview formats that center prompt choreography, questioned the quality of Coinbase’s metrics and rhetoric, and worried that AI-first hiring will deskill juniors while flattering management narratives about productivity.

Key insights

  1. 01

    Generated code fails in architectural ways

    The important mistakes are no longer obvious syntax or math errors. Models are increasingly fine at producing code that runs, but they still miss framework conventions, place logic in the wrong layer, add needless abstractions, and leave behind bloated, hard-to-maintain structure. That shifts interview value away from raw code production and toward asking whether a candidate can read code critically and explain why something that works is still wrong.

    Use interview tasks where candidates must review and improve plausible code, not just generate it. In your own team, treat AI output like junior code that may be functionally correct but structurally expensive.

      Attribution:
    • Levitz #1
    • ryanSrich #1
  2. 02

    AI amplifies senior judgment but not junior formation

    The sharpest concern was not current senior productivity. It was who becomes senior later. A junior learns by making bounded mistakes, then getting corrected by someone who already knows where the traps are, including domain-specific issues like timing attacks in login flows. AI can surface options, but it does not replace the prior mental model needed to rank them, so heavy reliance early in a career may speed output while starving the judgment formation companies depend on later.

    Protect apprenticeship on teams that are going all-in on AI. Keep real code review, postmortems, and ownership of production decisions in the loop for less experienced engineers.

      Attribution:
    • sarmadgulzar #1 #2 #3
    • bonoboTP #1
  3. 03

    The useful signal starts after the prompt

    People running AI-assisted interviews said the strongest evidence comes from what happens around the model, not from the prompt itself. A candidate who asks for a plan, reviews it, and can then productionize, test, and explain the result is showing engineering skill. A candidate who gets the code quickly but cannot reason about it is not. That makes the prompt phase mostly a setup step, not the core of the assessment.

    Design interview rubrics around explanation, debugging, and production tradeoffs. Treat the model as a way to create artifacts for discussion, not as the thing being graded.

      Attribution:
    • nitwit005 #1
    • mbil #1
  4. 04

    AI interview scoring is easy to distort

    Several comments pointed out that once candidates can choose different models and tools, you are no longer running a clean comparison. A strong result may reflect a better harness, a stronger subscription tier, or more patience with latency. The same problem shows up in internal metrics like "AI-generated code". If the classification depends on who opened the PR or who made the commit, the number is easy to game and says little about real effort or quality.

    Standardize the toolchain if you want comparable interview results. For internal dashboards, define AI-usage metrics tightly and pair them with outcome measures like defects, review churn, and cycle time.

      Attribution:
    • cheeaun #1
    • jacamera #1
  5. 05

    Productivity gains do not fix the pipeline problem

    A few comments tied two claims together that companies often keep separate. Yes, agents can occasionally produce a month of work in an hour. They can also waste an hour on a trivial fix. Meanwhile large organizations were rarely bottlenecked on typing speed in the first place. That matters because management can use impressive demos to justify cutting junior hiring, even though the long-term constraint is still people who understand systems well enough to ship and maintain them.

    Do not convert isolated AI wins into hiring policy without looking at throughput and quality at the team level. If you cut junior roles, be explicit about how you will replace the training function they used to provide.

      Attribution:
    • serial_dev #1
    • pards #1
    • throwatdem12311 #1
    • xmcqdpt2 #1
    • sarmadgulzar #1
  6. 06

    Basic unaided competence still predicts AI competence

    Teams that opened interviews to any framework and any assistant still found that basic fundamentals separated candidates fastest. People who could not answer simple technical questions or follow straightforward instructions also could not use AI effectively or debug its output. That undercuts the idea that allowing AI removes the need for baseline fluency. It mostly makes that baseline visible in a different way.

    Keep a short fundamentals check even if the main loop is AI-friendly. You need proof that candidates can reason without the tool before tool-augmented performance becomes meaningful.

      Attribution:
    • o0ower0o #1
    • Muromec #1 #2
    • pards #1

Against the grain

  1. 01

    Prompt habits are too recent to weigh heavily

    A minority view held that interviewing on AI workflow overfits to a skill people can pick up in days or weeks. On that view, a senior engineer who has not recently optimized their prompting may look worse than a less experienced candidate who has, even though the deeper engineering judgment still sits with the senior. That is a real risk if the interview gives too much credit to polished tool usage.

    Do not let recent familiarity with the current AI stack dominate hiring decisions. Weight durable engineering ability more than fluency with whichever assistant is hot this quarter.

      Attribution:
    • Muromec #1 #2
    • pmg101 #1
  2. 02

    No-AI or capped-token interviews may be cleaner

    Some commenters argued the better response is not to embrace AI in interviews, but to constrain it. A strict token budget or a no-AI segment forces candidates to show they can still write and reason independently. That camp sees unlimited assistant use as a path to hiring people who are dependent on models and cannot recover when the model is wrong, unavailable, or too expensive.

    Consider a mixed format with explicit limits rather than an all-or-nothing policy. A bounded-AI section can reveal whether candidates know how to use the tool sparingly and when to stop trusting it.

      Attribution:
    • rvz #1
    • dan_q #1

In plain english

AI ↩
Artificial intelligence, here mainly meaning software models that can generate code, text, or answers from prompts.
PR ↩
Pull request, a proposed code change submitted for review and possible merging into a shared codebase.
timing attacks ↩
Security attacks that infer secrets by measuring how long a system takes to respond to different inputs.
token ↩
A unit of text processed by an AI model that is often used for pricing and usage limits.

Reference links

Related article references

Company and executive context