The more useful conclusion is that offensive security with LLMs is now very real, but still heavily shaped by harness design and human guidance. Several practitioners said one-shot "find the bug" prompting is the wrong frame. Better results come from multi-step workflows that decompose search, validation, and exploitation, often with humans steering exploration or a coordinator agent managing retries and confirmation. Without that structure, models hallucinate vulnerabilities, exhaust obvious attack paths from training, or get stuck. People who have used models for crackmes, reverse engineering, and
pentesting said current systems are most effective as force multipliers for experts, not autonomous auditors.
A second major theme was frustration with guardrails, especially from Claude. Multiple people reported refusals on benign biology, log analysis, malware explanation, decompilation, game overlays, forking MIT-licensed code, and even retrieving their own local documents. The complaint was not simply that safety exists. It was that the current implementation is broad, inconsistent, and costly. Users described hidden server-side prompt injections, extra tool-call churn, session terminations without refunds, and brittle behavior that changes depending on wording, account history, or whether the target looks local versus live. The practical effect is that legitimate defensive work gets harder, while determined users route around the blocks with prompt reframing, local proxies, clean sessions, other vendors, or open-weight models.
The dominant mood was that the bottleneck is shifting from model capability to who is allowed to access it and under what conditions. Some saw that as a safety tradeoff worth making for average users who should not hand agents secrets or let them attack live systems. But most of the high-signal commentary landed on a sharper point: if leading labs over-constrain their best models, security professionals and serious builders will migrate to providers that are cheaper, less restrictive, or easier to run locally. In that world, guardrails become less a meaningful barrier to abuse and more a tax on legitimate use.