AI in cybersecurity: ready to protect or to attack?

Today’s most important AI news

The week starts with an uncomfortable question for any security team: how well do GPT-5.4, Claude Opus 4.6 or Gemini 3.1 Pro work when asked to find real vulnerabilities? The research published today offers the most rigorous answer yet, and it isn’t entirely reassuring. In addition, two separate studies document systematic failures in capabilities advanced models supposedly already master: strategic reasoning and high-stakes forecasting. It was a day of papers correcting inflated expectations.

Frontier LLMs in cybersecurity: real promise, real gaps

A new dual benchmark evaluated six frontier models — GPT-5.4, Codex 5.3, Claude Opus 4.6, Claude Sonnet 4.6, Gemini 3.1 Pro and Gemini Flash — in two radically different security scenarios. The first is vulnerability detection in source code (white box) in C, Java and Python. The second is security testing on real web applications (black box), with 118 verified vulnerabilities spread across more than 20 distinct CWE families. It is, to date, one of the benchmarks closest to real working conditions to have been published.

The results point toward something the researchers call “vertical foundation models”: the idea that a general LLM, however capable, doesn’t replace a model specialized in security. The best general models show acceptable performance in static code detection, but degrade notably in the black-box scenario, where they need to reason about the behavior of running applications without seeing the source code. The difference between white and black mode reflects exactly the difference between reading documentation and doing real pentesting.

For a developer or an AppSec team, the implication is concrete: using a frontier LLM as the first line of code review makes sense today, but automating penetration testing with those same models still requires dense human supervision. The gap between the two modes also suggests that whoever builds the first model specialized in offensive security — trained specifically on black-box scenarios — will have a significant competitive advantage in the security tools market.

More capable models, worse forecasts: the inverse scaling problem

A study on forecasting documents a phenomenon that runs against almost everyone’s intuition: on problems with superlinear growth and regime-change risk — exactly the kind of situation that matters most in finance and epidemiology — more capable models produce worse distributional forecasts than less capable models. The authors call this “inverse scaling” and validate it both on a contamination-free simulation benchmark and on real historical data.

The proposed mechanism is revealing. Larger models tend to anchor their probability distributions in historical patterns of linear growth, ignoring the possibility of abrupt jumps. When the time series they must forecast follows an exponential curve or has heavy tails, the more sophisticated model actually trusts its own prior knowledge too much and underestimates the problem’s structural uncertainty. A less trained model, paradoxically, produces more honest distributions about what it doesn’t know.

For a product manager or executive evaluating using LLMs in risk analysis or strategic planning, this is a direct warning sign. Using GPT-5.4 to generate normal growth scenarios can work well. Using it to model disruptions, crises or phenomena with regime-change potential can produce false confidence. The practical recommendation is to treat LLM forecasts in high-volatility scenarios as a working hypothesis, not a risk analysis.

LLMs’ strategic reasoning under the microscope: what current benchmarks don’t measure

A third paper attacks a different but related problem: existing benchmarks for evaluating strategic reasoning in LLMs are saturating as models improve, and worse, they don’t predict well how those models will behave in real economic situations. LLMs are increasingly used as agents in auctions, markets and automated negotiation systems, and evaluators lack reliable tools to anticipate their behavior.

The paper proposes GENSTRAT, a framework that makes it possible to generate new strategic games with controlled properties, instead of always evaluating on the same canonical games. The idea is that if a model really reasons strategically — and doesn’t just memorize game-theory patterns — it should generalize to games it never saw during training. The experiments show that the gap between performance on canonical games and new games is substantial across all models evaluated.

The practical angle here is for those building negotiation systems, dynamic pricing or any application where the LLM has to make decisions in competitive environments. A model that looks excellent on standard benchmarks can behave erratically when market conditions don’t resemble the training examples. Before deploying an agent in production in high-economic-stakes contexts, GENSTRAT offers a methodology for more robust evaluations.

In a nutshell

What connects today’s three topics is the same tension: the distance between benchmark performance and real-world performance grows as problems become more specific, more dynamic or more unpredictable. Cybersecurity, crisis forecasting and strategic reasoning are domains where that distance matters enormously and where the consequences of trusting a benchmark number too much can be serious. The pattern emerging is that the industry is starting to ask the right questions — not “how intelligent is this model?” but “under exactly what conditions does it fail, and how predictable are those failures?”. That’s the question that separates those who use AI in a sophisticated way from those who only read the press releases.

---

Sources used: https://arxiv.org/abs/2605.23243, https://arxiv.org/abs/2605.22672, https://arxiv.org/abs/2605.23238