The post argues that when a user query does not line up cleanly with a fixed product taxonomy, you can let an LLM "hallucinate" the category in the taxonomy’s language, then use embeddings to snap that invented label back to the nearest real class. The example is ecommerce search, where a phrase like "brown coffee table" may map better after the model rewrites it into the site’s own category vocabulary. The point is not that the hallucinated label is true. It is that it can be closer to the schema than the raw query.
Most of the useful reaction was practical. People recognized this as a close cousin of
HyDE and older retrieval tricks where a generated answer or summary embeds better than the original query. They also pushed on the missing benchmark. The obvious baseline is direct
embedding similarity between the user query and category labels, or a two-stage system that retrieves a shortlist with
BM25 or embeddings and lets a smaller model rerank. That framing largely won. The technique looks credible as a way to improve candidate generation when the taxonomy language is specialized, but not as a reason to skip evaluation against simpler methods.
A second thread grounded it in real production constraints. Several people said sending an entire taxonomy into the prompt is too expensive and still hallucinates, especially when you have tens or hundreds of thousands of labels. In that setting, using an LLM to compress the query into taxonomy-like language can be a reasonable compromise. But confidence stayed low on end-to-end accuracy. Commenters with hands-on experience said embedding sort often gives a decent shortlist and then falls apart under rigorous testing. Human review, manual overrides for important queries, or a heavier verification step still seem necessary. The broad takeaway was pragmatic rather than evangelical: this is another retrieval heuristic in the toolbox, not a new law of classification.
softwaredoug.com