Cactus introduced Needle 2 as a tiny 45 million parameter model compressed into a 14MB binary that runs in about 28MB of RAM. It is aimed at one narrow job: turning messy user language into tool calls or structured outputs on hardware that cannot run typical local LLMs, including sub-$200 phones, Raspberry Pis, wearables, and some ESP32-class devices. The company explicitly framed it as an action model, not a general-purpose chatbot, and said the model is designed to be paired with a confidence score so low-confidence cases can be rejected or escalated to a larger cloud model.
What came through most clearly is that this only makes sense if you accept the product shape. People who treated the demo like a general language model got absurd outputs, including unrelated prompts mapping to door locks or thermostats. That did not really undermine the core claim. It clarified it. Needle 2 is being sold as a local intent-to-schema mapper where reliability depends on the confidence threshold, tight tool definitions, and task-specific fine-tuning. Several examples showed that wording of tool descriptions materially changes results, and the author repeatedly pointed users to the bundled data synthesis and fine-tuning pipeline rather than claiming strong
zero-shot robustness.
The useful criticism landed on evaluation, not on whether tiny models are interesting. People wanted abstention quality and false-positive rates measured across real command sets, not just anecdotal confidence scores from single prompts. That is the gap between a fun demo and something an integrator can trust in a lock, thermostat, or robot. At the same time, the mood was broadly positive because the target category feels real. A few commenters pointed out that the industry has overfocused on running ever-larger models on laptops and desktops, while there is obvious demand for a much smaller local action layer in voice interfaces, Home Assistant setups, cheap consumer devices, and lightweight robotics. Training cost also did not look like a serious objection at this scale. The more credible concern is product reliability, especially on unsupported or ambiguous commands.