HN Debrief

GPT 5.6 Sol is the best "vision" model OpenAI ever released

  • AI
  • Computer Vision
  • Benchmarks
  • Developer Tools
  • Open Source

The post benchmarks GPT-5.6 Sol on vision tasks like OCR, object detection, counting, and a few harder image-understanding cases, and argues it is the best vision model OpenAI has shipped so far. That much landed. What did not land was the implied market position. People immediately noticed the author’s own charts showed Gemini 3.5 Flash beating Sol on nearly everything that matters for practical workloads, often at much lower cost, and the author agreed the piece was already stale because Gemini 3.7 Flash arrived a week later and now looks like the price performance leader. Qwen also came up repeatedly as the other serious contender, especially for local or lower-cost deployments.

If you are picking a production vision stack today, benchmark across vendors and separate general multimodal convenience from the narrow task you actually need. For extraction, counting, and detection at scale, cheaper specialized models or deterministic tools still look safer than betting on one frontier model.

Discussion mood

Impressed by how much better multimodal models have become, but mostly skeptical of the headline and unconvinced that Sol is the best practical choice. The dominant reaction was that Gemini Flash, and often Qwen or classic CV tools, look stronger on real workloads once cost, speed, and reliability matter.

Key insights

  1. 01

    Video captioning appears to be a real Sol win

    For short video clips and sub-second motion, Sol may be ahead in a way the benchmark barely captured. One commenter said older GPTs regressed once reasoning was added and often hallucinated actions, while Sol is the first model they have seen reliably describe quick movements and remain robust across frame sampling rates. That points to training or inference changes that matter for product teams doing clip understanding, not just still-image OCR or counting.

    If your workload includes short video or fast motion, do not infer model quality from still-image leaderboards. Run separate evals for clip captioning and frame-rate sensitivity before standardizing on Gemini or any other still-image winner.

      Attribution:
    • schopra909 #1
  2. 02

    Flash Lite breaks on larger structured batches

    The cheap Gemini tier is not just a smaller version of the stronger models. It can quietly stop applying instructions as batch size grows, and it may corrupt fields that should be copied exactly, like URLs or quoted text. Prompt tuning helps, and pushing constraints into JSON schema descriptions helps more, but one commenter said the error rate still climbed with larger JSON arrays while non-Lite Flash handled the same workload cleanly.

    Treat low-cost extraction models as batch-size sensitive systems, not drop-in commodities. Measure fidelity on your largest real inputs and watch for silent corruption, not just obvious failures.

      Attribution:
    • ComputerGuru #1 #2
    • MrBuddyCasino #1
  3. 03

    Tool use changes what vision benchmarks mean

    A line-following puzzle that looked like a vision failure became solvable once the model could use Python to trace the image. Another commenter had seen Sol crop, zoom, and enhance handwritten records before answering. That changes the interpretation of many comparisons. Some benchmarks are really measuring raw tool-less perception, while actual products often care about the combined system of perception plus code execution plus iteration.

    Define whether you need naked visual reasoning or agentic visual work before comparing models. A benchmark that bans tools can underestimate the best production setup for document and image pipelines.

      Attribution:
    • TeMPOraL #1
    • dllu #1
    • faxmeyourcode #1
  4. 04

    VLMs are becoming data engines for narrow models

    The strongest defense of using frontier vision models on simple tasks was not that they should replace detectors. It was that they can label messy niche data cheaply enough to bootstrap the detector you actually want. People pointed out that manual annotation is expensive and that VLM disagreement can flag examples for human review. That makes the expensive general model useful one step earlier in the pipeline, not necessarily at inference time.

    Use frontier vision models to accelerate dataset creation and QA when you lack labeled data. Then train or fine-tune a cheaper task-specific model for steady-state production.

      Attribution:
    • SkalskiP #1
    • bonoboTP #1
    • throwup238 #1
  5. 05

    Benchmark harness errors are still a live issue

    Readers found a ground-truth labeling mistake in one benchmark image, and another disputed failure may have been a rotation or image-processing problem rather than a model miss. The author fixed one issue and said another case involved large-image handling. That is a reminder that small benchmark sets can be swayed by annotation bugs, resolution choices, and coordinate post-processing errors, especially in vision where display transformations are easy to get wrong.

    Audit example-level failures before making vendor decisions from benchmark tables. For vision, store original images, transformations, and evaluation code together so you can separate model weakness from harness noise.

      Attribution:
    • defrim #1
    • SkalskiP #1 #2
  6. 06

    The winning setup may be a portfolio, not one model

    Several builders are already splitting work across vendors. Fable handles planning, Sol handles computer use and some visual persistence, Gemini handles image analysis, and Qwen is attractive for local or lower-cost deployment. That is less a temporary hack than a sign that multimodal capability is diverging by task. The single best model is often the wrong framing.

    Design your orchestration layer so you can route OCR, counting, planning, and UI analysis to different models. Vendor abstraction is now a product feature, not just procurement hygiene.

      Attribution:
    • lwarfield #1
    • criddell #1
    • weli #1
    • logicallee #1
    • iamleppert #1

Against the grain

  1. 01

    UI critique is becoming a credible vision use

    Against the broader skepticism about subjective visual judgment, some people said recent models are finally useful for screenshot-based UI review. The claim was not that they have perfect taste. It was that they can now break a screen into coherent components, spot consistency problems, and surface objective design principles well enough to help non-experts improve an interface. That is a narrower and more believable use than asking for originality.

    For internal design review, try models as structured critics rather than as designers. Ask for hierarchy, readability, and consistency feedback on screenshots, then have humans decide what to keep.

      Attribution:
    • weli #1
    • TeMPOraL #1
  2. 02

    General models may overtake fine-tuned counters

    One builder running a free counting service argued the combination of broad training data and user feedback loops may let general models beat many bespoke counters over time. They still admitted cost and latency are off for users today, but they were notably more bullish than the dominant “use OpenCV or a detector” view. The point is strongest where threshold tuning and edge cases dominate maintenance cost.

    Do not assume a custom vision model keeps its advantage forever on mid-complexity tasks. Re-test frontier models periodically if your bottleneck is maintenance and edge-case handling rather than pure throughput.

      Attribution:
    • ParanoidShroom #1 #2
  3. 03

    Consumer search tasks are already good enough

    A few practical examples cut against the production-readiness pessimism. People reported success using multimodal models to find products in grocery aisles, identify highlighted text in deep screenshot context, and work mainly through rough visual collages and scribbles. These are not safety-critical tasks, but they show that “good enough” vision has already arrived for casual search and assistance jobs where convenience matters more than exact coordinates.

    If your use case is low-stakes visual assistance, you may be able to ship now with human tolerance for occasional misses. Do not let hard industrial benchmarks obscure easier consumer workflows that are already viable.

      Attribution:
    • slibhb #1
    • 5555watch #1
    • bob1029 #1
    • 1saadcodes #1

In plain english

EXIF ↩
Exchangeable Image File Format, metadata embedded in image files that can include camera details, timestamps, and sometimes location.
JSON ↩
JavaScript Object Notation, a common text format for sending structured data between systems.
OCR ↩
Optical Character Recognition, software that tries to read text from images or pixels.
OpenCV ↩
Open Source Computer Vision Library, a widely used software toolkit for image processing and computer vision tasks.
UI ↩
User Interface, the visual and interactive parts of a software product that people use directly.
VLM ↩
Vision-language model, an artificial intelligence system that works with both images and text.

Reference links

Benchmark dashboards and comparisons

Vendor docs and model lifecycle

Model announcements and side-by-side posts

Examples of tools and specialized systems

Historical and technical references