AI in the real world: cybersecurity, CEOs and brands under pressure

Today’s most important AI news

The focus of AI research shifted today toward uncomfortable territory: how ready are frontier models to operate in contexts where an error has real consequences? Three studies published this week put LLMs to the test in cybersecurity, executive decision-making and product recommendation, and the results temper the optimism surrounding these systems. It’s not a day of launches or scandals, but of concrete evidence that should change how professionals deploy these tools.

LLMs versus vulnerable code: results that make security teams uncomfortable

A benchmark published on arXiv evaluated whether frontier models are ready to work autonomously in cybersecurity. The study used two test modes: vulnerability detection in source code (in C, Java and Python) and security testing on five production web applications with 118 real vulnerabilities spread across more than 20 CWE families. The models evaluated include GPT-5.4, Codex 5.3, Claude Opus 4.7, Claude Sonnet 4.6 and Gemini 3.1 Pro.

The findings indicate that no generalist model reaches the level of consistency needed to operate unsupervised in critical security environments. The models show strengths in detecting known vulnerability patterns at the function level, but their performance drops noticeably in black-box mode, where they must infer an application’s behavior without access to the code. This reflects an important gap: most real pentesting and audit scenarios are precisely black-box.

The practical implication is direct for security teams already using or evaluating LLM-based tools. These results reinforce the case for vertical models specialized in cybersecurity, trained on domain-specific data, rather than generalist models adapted with prompts. If your organization is considering integrating AI into its code review pipeline or its penetration testing, this study is required reading before making that decision.

Can an LLM be CEO? What the benchmark doesn’t say about real management

Another paper evaluated whether LLMs can make strategic decisions at an executive level. The benchmark, called CEO-Sim, simulates resource-reallocation scenarios where the model must integrate conflicting recommendations from multiple stakeholders under information asymmetry, time pressure and organizational ambiguity — conditions traditional reasoning benchmarks systematically ignore.

The models evaluated achieve reasonable performance when conflicts between stakeholders are explicit and information is well structured. The problem appears when information asymmetry is high and internal agents’ recommendations contradict each other without clear signals of which is more reliable. In those scenarios, LLMs tend to average positions instead of making a decision with judgment, a behavior known in real management as analysis paralysis or, worse, false consensus.

For product managers and entrepreneurs experimenting with AI agents to support business decisions, this has a concrete implication: LLMs are useful for synthesizing information and generating options, but delegating the resolution of real organizational conflicts to them remains risky. The model doesn’t know which stakeholder has more context, which political relationships are at play or which implicit commitments exist. That’s still human work.

When the LLM recommends products: the bias that favors big brands

A third study analyzed how brands compete within LLM-based recommendation systems. Using skincare products as the test category, the researchers evaluated GPT-4o-mini, Claude Sonnet and Gemini 3 Flash to understand which factors determine which brands get recommended and how often.

The central finding is that the models exhibit a systematic bias toward established brands, regardless of objective product quality. The authors call it “incumbent advantage”: brands that already have a high presence in training data receive disproportionate recommendations, which reproduces and amplifies existing market asymmetries. In categories where consumers can’t evaluate quality before buying, like cosmetics or supplements, this bias has a direct impact on purchasing decisions.

For entrepreneurs and challenger brands, this is a warning sign. If your customer-acquisition strategy assumes AI agents will recommend your product neutrally, the data says otherwise. And for product managers building recommendation systems on LLMs, the study raises a design question that can’t be ignored: how do you audit and correct brand bias in a model that has no explicit product list, but preferences embedded in its weights?

In a nutshell

What connects these three studies isn’t technical but strategic: LLMs are entering high-risk domains, and the industry still lacks robust mechanisms to audit them in those contexts. The day’s pattern is evidence arriving after adoption. Companies are already using these models in security, in business decisions and in product recommendations, and academic research is racing to document the limits product teams should have measured before launching. The field’s maturity isn’t measured in reasoning benchmarks, but in the ability to know when not to deploy.

---

Sources used: https://arxiv.org/abs/2605.23243, https://arxiv.org/abs/2606.17459, https://arxiv.org/abs/2606.17443