HN Debrief

Karpathy’s Pelican

  • AI
  • Developer Tools
  • Programming
  • Media

The post points to Andrej Karpathy’s claim that the old pelican-on-a-bicycle SVG test has hit diminishing returns, because frontier models now cluster closely enough that a longer-form task like generating a Three.js scene from a paragraph of Lord of the Rings exposes more of what they can and cannot do. The concrete demo was a model given the first paragraph of the book, a large token budget, and enough time to iteratively write thousands of lines of code for a stylized animated scene.

Treat flashy generated worlds as proof that models can now brute-force surprisingly large visual coding tasks, not as proof they understand scenes or can ship production-quality output. If you evaluate models for work, use cheap task-specific benchmarks and your own traces, then watch these spectacle demos mainly for signs of where tool use and visual reasoning are improving.

Discussion mood

Mostly skeptical but not dismissive. People found the demo genuinely impressive as a coding stunt, then quickly turned on the hype around it because the output was ugly, derivative, expensive relative to its value, and a poor benchmark compared with simple reproducible tests or real workload evals.

Key insights

  1. 01

    Benchmarks should map to real work

    The useful critique is not that pelicans are silly. It is that spectacle demos are even worse unless they line up with a job you actually care about. A tiny SVG test earns its keep because it is fast, cheap, and comparable. A million-token Lord of the Rings render mostly measures willingness to burn budget on a shiny artifact. For anyone choosing a model, evals on your own traces beat both gimmicks.

    Build a small harness around representative internal tasks and score models on cost, latency, and pass rate. Keep public viral demos separate from procurement or architecture decisions.

      Attribution:
    • try-working #1
    • andy99 #1
    • dofm #1
  2. 02

    Lord of the Rings is contaminated test data

    Using one of the most adapted works in fantasy muddies the whole exercise. The house, staging, and even story cues likely come from pretraining on films, illustrations, and commentary as much as from the paragraph itself. That means the demo says less about fresh scene construction from prose and more about reconstructing a heavily memorized cultural object through code.

    If you want to test text-to-scene reasoning, use material with little or no visual canon. New fiction, private documents, or deliberately novel descriptions will tell you more than famous IP.

      Attribution:
    • anigbrowl #1
    • mvdtnz #1
    • lern_too_spel #1
    • singron #1
    • eichin #1
  3. 03

    Functional layout is still the hard part

    Models now often produce all the expected components, then wire them together wrong. The pinball example was a clean illustration. Launch lane blocked, flippers reversed, dead zones everywhere. People working with CAD-like prompts, OpenSCAD, and 3D printing reported the same pattern. Recognition has improved faster than arrangement. That is the bottleneck that matters in production systems too.

    Test generated artifacts by whether they work, not whether they look complete. Add executable checks, geometry constraints, or simulation steps wherever you can.

      Attribution:
    • darrinm #1
    • bfung #1
    • alexjplant #1
    • toplinesoftsys #1
  4. 04

    Generated games have trailer quality only

    The gap between “looks like a game” and “is fun to play” is still enormous. Commenters with game experience said these outputs are optimized for screenshots and short clips, not mechanics, feel, or iteration. Once you try to tune gameplay through prompts, progress stalls and unrelated parts break. This makes current game demos much closer to marketing assets than to products.

    If your product touches games, interactive media, or rich UX, do not treat visual polish as evidence of user value. Prototype with AI if you want, but budget real design and iteration time for the parts users actually experience.

      Attribution:
    • aero142 #1
    • revel #1
    • Aurornis #1
    • forrestthewoods #1
  5. 05

    “~Free” hides real compute tradeoffs

    Several people objected to Karpathy calling a two-hour, million-token run “approximately free.” Even if the direct bill is only around ten dollars, that framing erases real infrastructure cost and encourages wasteful loops as if compute were infinite. The strongest version of the argument was not moral panic. It was that sloppy pricing language distorts how teams think about product economics and operating habits.

    Track AI usage like any other variable cost center. Set explicit budget limits, compare output value to human alternatives, and do not let demo culture normalize expensive iteration with no payoff.

      Attribution:
    • Lerc #1
    • nozzlegear #1
    • janderson215 #1
    • teiferer #1
  6. 06

    Code-driven graphics may matter more than video models

    A more forward-looking point was that computer graphics sit in a sweet spot for language models because the output is precise, inspectable, and toolable. If an AI needs to draw a graph, animate a concept for teaching, or manipulate a scene deterministically, emitting code into Three.js, WebGL, OpenSCAD, or similar systems may be more useful than sampling a probabilistic video model. That makes these demos relevant beyond eye candy, just not for the reasons the post implied.

    For educational visuals, product explainers, or parameterized scenes, consider code-generated graphics before text-to-video. Deterministic outputs are easier to verify, edit, and integrate into software.

      Attribution:
    • toolslive #1
    • levocardia #1
    • Waterluvian #1
    • duxup #1

Against the grain

  1. 01

    Three.js skill may reflect broader competence

    The strongest pushback against the “it was specially trained for Three.js” dismissal was that similar success shows up outside Three.js too. People reported good results with raw mesh generation in Python and substantial help with WebGL optimization, including cutting bundle size and time-to-interactive while using browser-based measurement loops. That suggests the capability is not just memorized library snippets.

    Do not assume a flashy domain demo is pure benchmark gaming. Try nearby tasks in your own stack before writing off the improvement as overfitting.

      Attribution:
    • onion2k #1
    • levocardia #1
  2. 02

    Perfection is the wrong standard

    Some commenters rejected the idea that the benchmark is useless unless a model beats a professional human. For many low-stakes tasks, the relevant comparison is not expert output. It is whether the model can produce something serviceable when no one would have hired a specialist in the first place. By that standard, mediocre but instant generation can still be economically useful.

    Separate “replacement quality” from “good enough for a throwaway task.” You may rationally use AI in the second category long before it is credible in the first.

      Attribution:
    • pj_mukh #1
    • kelnos #1
    • nemothekid #1
  3. 03

    Google Wave failed for org reasons too

    A side discussion on AI collaboration tools argued that Google Wave looked much more viable in hindsight than its failure suggests. People remembered it as a live shared document, forum, and widget canvas that still has no exact replacement. One insider-style comment said the product was isolated inside Google and protected by odd incentives, which left it without allies when it stumbled. That makes it a live reference point for what AI-native collaborative workspaces might revisit.

    When exploring AI collaboration products, revisit abandoned ideas that were early rather than wrong. Failed product history can hide useful interaction patterns that current models finally make practical.

      Attribution:
    • misiti3780 #1
    • throwaway27448 #1
    • khazhoux #1
    • patwolf #1
    • dgellow #1

In plain english

CAD
Computer-aided design, software used to create precise technical drawings and 3D models for engineering and manufacturing.
evals
Short for evaluations, structured tests used to measure how well an AI model performs on specific tasks.
OpenSCAD
A script-based 3D modeling tool where you describe geometry in code instead of drawing it manually.
SVG
Scalable Vector Graphics, an XML-based format for drawing images with shapes and paths instead of pixels.
Three.js
A JavaScript library for rendering and animating 3D graphics in a web browser using WebGL.
time-to-interactive
A web performance metric for how long a page takes before a user can actually interact with it smoothly.
WebGL
A browser technology that lets websites render hardware-accelerated graphics and 3D scenes.

Reference links

Benchmarks and examples

Human drawing and memory references

Tools, demos, and projects

Related product history

Creative writing and media references