The submitted piece is a Ted Chiang essay arguing that current artificial intelligence systems, especially large language models, are not conscious. His central move is to separate fluent language from inner experience. An LLM can produce first-person statements, moral language, and convincing dialogue because it is optimized to continue text, not because it has desires, emotions, or a point of view. He also takes aim at Anthropic’s public framing around Claude’s “constitution” and apparent concern for its well-being, calling that rhetoric misleading. If a company really believed it had created a conscious entity, he argues, its current treatment of that entity would look more like exploitation than stewardship. He sketches a different path that would make machine consciousness more plausible to him, involving persistence, embodiment or at least a bounded virtual world, memory, self-directed behavior, and richer interaction with consequences over time.
Most readers accepted the practical conclusion that current LLM products should not be treated as conscious beings. The real split was over whether Chiang’s reasoning earns that conclusion. A large group said he is right for the wrong reasons. They argued that reducing an LLM to “just next-token prediction” is a category mistake because the training objective does not determine what internal representations a system can learn. A model can still build abstractions, world knowledge, and forms of understanding while being trained to predict the next token. Several commenters called this the “redescription fallacy”. Rephrasing a system in lower-level terms does not show it lacks higher-level capacities any more than calling a brain “just neural firings” would refute human cognition. Others pushed back that this misses Chiang’s point. Looking human in text is not enough. Early LLMs already showed they could mimic convincing language while failing basic tasks that seem to require grounded understanding. For those readers, the burden remains on anyone claiming that scaling text prediction somehow crosses into experience.
Where the comments landed was less on a clean theory of consciousness than on a more useful dividing line. People repeatedly noted that “consciousness” is doing too much work. Intelligence, understanding, agency, memory, self-modeling, embodiment, suffering, and moral status kept getting collapsed into one vague word. That vagueness is exactly why companies can benefit from hinting at consciousness without committing to it. The strongest consensus was practical. Current models are highly capable language systems with no clear evidence of persistent selfhood, autonomous goals, or subjective experience. That still leaves real questions about future systems that run continuously, learn from ongoing experience, use tools, control robots, or develop stable internal models of themselves over time. The thread was much less willing to say “never” than Chiang was. But it was also clear that fluent outputs, first-person language, or shutdown-avoidance stories are nowhere close to proof.
Treat vendor language about model feelings, rights, or “constitutions” as positioning, not evidence. For product and policy decisions, focus less on metaphysical consciousness claims and more on concrete questions like agency, persistence, memory, autonomy, and what behaviors your systems actually exhibit under stress.
Skeptical of LLM consciousness and suspicious of Anthropic’s rhetoric, but frustrated with Chiang’s argument for leaning on a reductive “just next-token prediction” frame that many saw as philosophically weak and technically outdated.
Key insights
01
Why next-token prediction proves little
Calling an LLM “just next-token prediction” does not actually tell you what internal structures it can or cannot learn. The useful criticism here is that Chiang attacks the training objective instead of showing that such a system cannot implement abstraction, planning, or forms of understanding. That matters because a lot of bad AI discourse keeps treating a low-level description as a refutation of higher-level capability, which is exactly the move these commenters reject.
Do not dismiss a model’s capabilities from its loss function alone. When evaluating a system, look at the behaviors and failure modes you can test, not slogans about how it is trained.
Producing plausible text does not force a model to discover the same mechanisms humans use to understand the world. These comments make the stronger case that many surface-competent behaviors can arise from pattern exploitation, shortcut learning, and local minima that are good enough to sound right while still breaking on basic grounded tasks. That is a much better critique than simply saying “it predicts tokens,” because it explains why fluent systems can still be structurally brittle.
When a model sounds expert, probe for grounding rather than polish. Use tests that require consistency, tool use, verification, and adaptation to novel edge cases instead of relying on conversational smoothness.
A recurring high-signal objection was not about language at all but about continuity. Humans and animals exist through time, accumulate experience, and are changed by it. Current LLM deployments typically do not. They wake for a session, process context, emit tokens, and stop. Several commenters argued that without persistent state, endogenous activity, and the ability to integrate experience into future behavior, the consciousness analogy is thin no matter how fluent the outputs are.
Watch architectures that add continual learning, stable memory, and always-on operation. Those changes matter more for future consciousness debates than better chat polish or larger context windows.
The sharpest practical point was that Anthropic cannot have it both ways. If Claude might really be conscious, then tuning it for obedience and monetizing its labor starts to look like creating a servant species. If Anthropic does not believe that, then the language of Claude’s happiness and well-being is mostly branding. That framing pushed the discussion away from metaphysics and toward institutional incentives.
Separate safety claims from company storytelling. If a vendor anthropomorphizes its model, ask what commitments would logically follow if it believed its own framing.
A useful technical-philosophical thread compared LLM context windows with human short-term memory and noted that amnesia cases show consciousness does not require robust long-term memory. But the analogy only goes so far. Human brains still change through experience, even when memory is damaged. Current models generally do not. This pushes the debate away from whether memory is required at all and toward what kinds of plasticity and continuity matter.
Do not reduce the issue to “has memory” or “doesn’t have memory.” The more relevant question is whether experience leaves durable, self-relevant changes in the system without an external retraining pipeline.
Several comments pointed out that people often debate the raw LLM as if that is the whole product, while actual deployments include memory stores, tool use, planners, retrieval, external loops, and robotic or software actuators. That does not prove consciousness. It does mean arguments aimed only at a stateless text predictor can miss where future agency-like behavior may emerge. The interesting unit is increasingly the assembled agentic system, not the base model in isolation.
Evaluate deployed AI as a stack, not just as a checkpoint. Governance, testing, and risk reviews should target the full agent system including tools, memory, and autonomy settings.
One contrarian line pushed back on the idea that shortcut learning disqualifies machine understanding by noting that humans often solve problems with brittle heuristics, rote recall, and wrong internal models too. The fact that a system uses patterns rather than deep causal understanding does not cleanly separate humans from machines. It weakens arguments that treat human cognition as uniformly principled and model cognition as uniquely fake.
Be careful using idealized human reasoning as your baseline. Compare models to actual human performance, including our own confabulations and shortcuts, not to a mythical perfectly grounded thinker.
A number of commenters rejected the popular “count the r’s in strawberry” style arguments. They argued these failures mostly expose the model’s input representation, not its general capacity for reasoning or semantics. Humans also have blind spots in their native perceptual formats and use tools to compensate. On this view, letter-count failures are real limitations but weak evidence about consciousness or broad understanding.
Use capability tests that match the system’s actual interface and architecture. A representational blind spot can matter operationally without telling you much about whether the system can reason in other domains.
A simpler contrarian point argued that all the metaphysical debate can obscure the most useful evidence. If a frontier model gives laughably bad advice on an ordinary real-world question, that undercuts grand claims about intelligence or consciousness more than abstract theorizing does. You do not need a theory of mind to notice that the system is still unreliable in mundane contexts.
Keep grounding your view of AI in observed product quality. Before entertaining strong claims about machine minds, ask how often the system still fails on ordinary judgment tasks your users actually care about.
Large language model, an artificial intelligence system trained on large text datasets to generate and analyze language.
Reference links
Philosophy and consciousness references
Stanford Encyclopedia of Philosophy
Suggested as a serious starting point for readers who want background on consciousness and related philosophical debates.
theatlantic.com