A lot has been said about the rising cybersecurity capabilities of frontier LLMs. With each release, the most recent models seem to break to the top of the benchmarks. However, people may get confused, because one leaderboard may imply a completely different conclusion than another, and two equally reputable security researchers may have completely different opinions about the same model.
Of course, this is partly due to the different ways we evaluate these models (more about this here). But it is also about the way we use them. To take advantage of the agentic abilities of frontier LLMs, they need to be put in the proper harness. The harness defines roles and ways of communication between agents, manages context, introduces tooling, verification and safety systems, and arms the agentic system with the information relevant to its task. A very capable set of models can become useless, especially in complex applications, if it is not integrated in the proper harness.
Hackian, our agentic pentesting harness, is the product of continuous research and development, with the purpose of extracting the maximum capacity from the strongest frontier models. Since the beginning of the year (B.O.Y.) we have more than doubled our performance on (our private version of) EthiBench, not just because stronger models became available, but because we adapted our harness to how these models operate best.
This is clearest in our latest jump in performance. Our previous harness, Hackian 1.2, performed about the same with newer, supposedly more capable models (post-June 2026) as with older ones (pre-June 2026). Only when we updated it to Hackian 2.0, with capabilities better matched to the strengths of frontier LLMs, was the full potential of our agentic system unlocked. We worked from the hypothesis that recent models are increasingly developed “in tune” with the workflows of state-of-the-art coding agents, so we adapted some of those insights to the agentic pentesting context, rethinking how agents coordinate, share information and adapt their plans. The result is a harness that may underperform with outdated models, but makes the most of the latest AI advances.
Average F1 score on EthiBench (private version). With the latest models, Hackian 2.0 reaches 79.1%, while the previous harness stays around 64–67% with both old and new models. At the beginning of the year (B.O.Y.), we were at 33.45%. Error bars: ±1 standard deviation.The models set the ceiling, but the harness determines how close you get to it.
See what Hackian finds on your attack surface. Run Hackian on your own assets and judge the results yourself. Try now.