AI in production: real agents, new models and clinical limits

Today’s most important AI news

Today was a day dense with simultaneous moves: new model launches from Google, critical research on the limits of medical AI and a paper that systematizes for the first time the real challenges of deploying LLM agents in production. The common thread is clear — AI has stopped being a promise and now faces the frictions of the real world. What used to be a lab experiment is becoming a system that makes decisions with consequences.

Medical AI fails when information is missing — and nobody was measuring it properly

A new arXiv study evaluated four of today’s most advanced models — Claude Opus 4.8, GPT-5.5, Grok 4.3 and Gemini 3 — in open clinical conversations where information is missing. The finding is uncomfortable: safe behavior in those contexts means knowing when not to commit, asking for clarification and qualifying answers. No closed benchmark measures that, and most “production-ready medical AI” evaluations don’t take it into account.

The problem goes even deeper. The paper shows that the evaluator changes the result: when the model’s own provider acts as judge, the systems appear safer than they are. Human evaluators and judges from different models yield significantly different results. This isn’t a methodological detail — it’s a structural flaw in how the industry measures whether its models are safe for health settings.

For a product manager or developer building on AI APIs in medical or high-risk contexts, the implication is direct: don’t trust the provider’s benchmarks as a signal of clinical safety. You need to design your own stress tests with incomplete-information scenarios, and choose evaluation judges that have no conflict of interest with the model you’re testing.

LLM agents in production: the problems academic papers didn’t anticipate

A paper titled “Agents in the Wild” systematizes something many engineering teams already live with but nobody had rigorously documented: LLM-based agentic systems face completely different challenges when they leave the lab and enter real production in domains like software engineering, scientific discovery and finance. The benchmarks and algorithmic innovation that dominate the academic literature don’t prepare teams for the robustness, security and reliability problems that appear at scale.

“Objective drift” is one of the most documented phenomena: in long interactions, agents lose sight of the original goal. Hierarchical multi-agent systems, like the one proposed in the Multi² paper also published today, try to mitigate this with supervision layers between agents, but they add architectural complexity that has its own operating cost.

If your team is evaluating or already deploying LLM agents in real workflows, this is the kind of research worth reading before scaling. The gap between “works in a demo” and “works in production with real users for weeks” is bigger than benchmark results suggest.

Google launches three new Gemini Flash models and OpenAI opens ChatGPT to small businesses

Google DeepMind today announced three new models: Gemini 3.6 Flash, Gemini 3.5 Flash-Lite and Gemini 3.5 Flash Cyber. The Flash family has been Google’s bet on efficiency over raw power — faster, cheaper models than the flagships, designed for production integrations where latency and cost per token matter more than performance on deep reasoning benchmarks. The Cyber variant suggests a specialization toward cybersecurity use cases, although the technical details of the announcement aren’t complete yet.

In parallel, OpenAI launched the “ChatGPT for Small Businesses” program, aimed at helping entrepreneurs and small teams build AI skills, automate work and grow using ChatGPT Work. It’s a market-expansion move toward segments that until now adopted AI informally and without structure.

Taken together, both launches mark a trend: the big labs no longer compete only on technical capability, but on distribution and accessibility. Google lowers inference costs; OpenAI lowers the adoption barrier for companies without a technical team. For developers, this means more options for efficient, cheap models to build on. For entrepreneurs, it means enterprise AI tools are less and less exclusive to companies with big technology budgets.

In a nutshell

Today’s pattern is the clash between expectations built in the lab and the realities of deployment. Models are more capable than ever — GPT-5.5 leads the new Relay-Bench with just 43.3%, showing that benchmarks remain challenging — but technical capability alone doesn’t solve the problems of biased evaluation, objective drift in agents or silent failures in the face of incomplete information. The industry is learning, sometimes by stumbling, that scaling AI responsibly requires as much rigor in measurement as in development. Teams that understand that distinction will make better decisions than those that simply look at the model’s score on the provider’s benchmark.

---

Sources used: https://arxiv.org/abs/2607.18828, https://arxiv.org/abs/2607.19336, https://arxiv.org/abs/2606.03698, https://deepmind.google/blog/introducing-gemini-36-flash-35-flash-lite-and-35-flash-cyber/, https://openai.com/index/introducing-chatgpt-small-business-program, https://arxiv.org/abs/2607.18438