HN Debrief

Why Large Language Models Fail at Tabular Prediction

  • AI
  • Machine Learning
  • Data Science
  • Developer Tools

The paper asks a simple question: if you give a frontier large language model a full tabular dataset directly in the prompt and ask it to predict outcomes, why does it perform so badly compared with standard tabular machine learning? The authors isolate the model to a pure inference setting with no tools, no fine-tuning, and no agent scaffolding, then test common explanations such as poor serialization. Their headline claim is that formatting is not the main issue. The bigger problem appears to be dimensionality. As the number of features grows, performance collapses, and decades-old tabular methods still dominate.

Do not treat a general-purpose LLM as a drop-in forecaster just because it can access your tables. For prediction on structured business data, default to established tabular models and watch specialized foundation models separately instead of assuming language-model progress will carry over.

Discussion mood

Mostly agreeing, mildly amused, and a bit exasperated. People saw the result as directionally obvious for practitioners, but worth publishing because it cleanly separates "LLMs are bad at this" from shallower explanations like CSV formatting or weak prompting.

Key insights

  1. 01

    Enterprise AI hype is outrunning model fit

    Pointing Claude or another chat model at a data warehouse is exactly the mistake this paper helps puncture. The useful frame is not "can the AI see the data" but "what model class is built for this prediction problem." That lands hard in companies where years of calling everything AI have blurred the line between language interfaces and actual forecasting systems.

    If stakeholders want predictions from warehouse data, force the conversation toward evaluation baselines and model choice before any LLM integration plan. Treat natural-language access to data as a separate product feature from the prediction engine itself.

      Attribution:
    • _joel #1
    • pjot #1
    • importantbrian #1
  2. 02

    The real benchmark is specialized tabular models

    The interesting comparison is not chat LLMs versus XGBoost alone. It is general language models versus systems built specifically for tabular learning, including TabFM and TabPFN. Even people bullish on newer transformer-based tabular models still framed them as competing with XGBoost, LightGBM, and CatBoost on bounded parts of the problem, not as evidence that chat-style LLMs suddenly became good at tables.

    Track tabular foundation models as a separate category from LLMs. If your team wants zero-shot or lower-touch modeling on structured data, evaluate TabPFN or TabFM against boosted trees instead of trying to repurpose a chatbot.

      Attribution:
    • tough #1
    • jeffreyrogers #1
    • woofwoofwoof #1
    • dist-epoch #1
    • sebastiansm7 #1
    • dill_1 #1
  3. 03

    Formatting may help interpretation, not core prediction

    Comments pushed on the paper's claim that serialization is not the main bottleneck because older models often handled markdown tables better than CSV. But the better read is that format can change how easily a model extracts signals from a result, while still not fixing the underlying weakness on tabular prediction. The Honeycomb MCP example points the same way. Teams are adding ASCII charts next to tables because richer representations may help LLMs interpret outputs, not because they turn the model into a strong tabular learner.

    Use presentation tricks like markdown tables or charts when an LLM needs to summarize results for humans. Do not confuse improved readability with improved predictive ability.

      Attribution:
    • gw32 #1
    • IneffablePigeon #1
  4. 04

    The stripped-down setup is the point

    Removing tools, fine-tuning, and agent scaffolding is what makes the paper useful. A controlled setup can expose whether the base model has any native capacity for this kind of reasoning. If performance only appears after adding retrieval, code execution, or a separate model pipeline, then the capability belongs to the surrounding system, not to the LLM itself.

    When you evaluate LLM-based analytics products, separate base-model ability from pipeline ability. Ask which component is actually doing the prediction before you credit the LLM for the result.

      Attribution:
    • scott_s #1
    • mwkaufma #1

Against the grain

  1. 01

    Pure prompting studies miss real deployments

    For builders using LLMs as part of agentic systems, a single-pass no-tools benchmark can feel academic to the point of irrelevance. In production, you would reach for code, model libraries, and external execution rather than asking a chat model to infer directly from raw rows in context. On that view, the paper proves a narrow limit of naked prompting more than it informs how practical systems should be designed.

    If your product already wraps LLMs with code execution and classical models, do not overgeneralize this paper to the whole stack. Use it to limit where the LLM sits in the workflow, not to rule out hybrid systems.

      Attribution:
    • heaney-555 #1
    • bonsai_spool #1
  2. 02

    Missing reasoning budget weakens the evaluation

    Leaving out the model's reasoning level or budget makes the result harder to interpret because inference settings can materially change performance. A negative result is more convincing when readers can see whether the model was given the expensive test-time compute that vendors increasingly treat as part of capability.

    When you read or run LLM benchmarks, insist on full inference settings, including reasoning mode and token budget. Otherwise you cannot tell whether you measured a model limit or a cost cap.

      Attribution:
    • WhitneyLand #1

In plain english

agent scaffolding ↩
Extra software around a model that gives it tools, memory, planning steps, or workflows beyond a single model response.
ASCII ↩
A basic character set covering English letters, digits, and symbols, commonly used as the narrow safe subset for protocol text.
CatBoost ↩
A gradient-boosted decision tree library that is especially strong on tabular data with categorical features.
CSV ↩
Comma-separated values, a simple text file format commonly used to store tabular data for spreadsheets or imports.
dimensionality ↩
The number of input features or columns in a dataset.
fine-tuning ↩
Training an existing AI model further on new data so it performs better for a specific task or domain.
Honeycomb MCP ↩
A tool integration for the Honeycomb observability platform using Model Context Protocol so models can access and work with telemetry data.
LightGBM ↩
A gradient-boosted decision tree framework designed for fast and accurate learning on tabular datasets.
LLM ↩
Large Language Model, a machine learning model trained to generate and analyze human-like text.
prompt ↩
The instruction text and context given to an AI model to guide its output.
TabFM ↩
A foundation model designed specifically for tabular data rather than general language generation.
TabPFN ↩
A probabilistic foundation model designed for tabular prediction tasks.
tabular prediction ↩
Using structured rows and columns of data, like a spreadsheet or database table, to predict an outcome such as churn, fraud, or sales.
transformer ↩
A neural network architecture commonly used for sequences like text, audio events, or notes, where each output depends on the surrounding context.
XGBoost ↩
A widely used gradient-boosted decision tree library that often performs very well on tabular data.

Reference links

Specialized tabular models

  • Google TabFM blog post
    Cited as a leading specialized foundation model for tabular data and contrasted with general LLMs.
  • TabPFN GitHub repository
    Mentioned as another transformer-based system focused on tabular prediction, with better results than general LLMs but slower inference than boosted trees.