AI enters real work: finance, medicine and construction

Today’s most important AI news

Today’s dominant topic isn’t a spectacular launch or a controversy: it’s a practical and urgent question many professionals are already silently asking themselves. Does AI really work when you take it out of the lab and put it to work in my industry? Several studies appeared today that try to answer that rigorously, comparing models on financial analysis, medical diagnosis and construction drawing reading tasks. The results are revealing, and not always favorable to the most popular models.

How well do big models analyze markets — and why the answer matters

A new study compared five prominent LLMs — GPT-4 Turbo, Claude 3 Opus, Gemini 1.5 Pro, Llama 3 70B and the specialized FinGPT — on concrete technical analysis tasks for financial markets: candlestick pattern recognition, indicator interpretation, signal generation and risk assessment. The methodology is systematic and structured, which sets it apart from the generic benchmarks that usually circulate.

The most interesting finding isn’t which model “won”, but that FinGPT — a finance-specialized model, considerably smaller and less well known than GPT-4 Turbo — achieves competitive results on specific technical tasks in the domain. This reinforces a trend that has been gaining strength: specialization can make up for size when the problem is well bounded.

For a product manager or developer working at Latin American fintechs, this has a direct implication. Before integrating the biggest and most expensive model available into a financial analysis flow, it’s worth evaluating specialized alternatives. The cost per token of GPT-4 Turbo versus a model like FinGPT can differ by several orders of magnitude in production, and if performance is comparable on the relevant tasks, the choice should be obvious.

AI and construction drawings: a domain multimodal models don’t master

Researchers presented DrawingVQA, the first benchmark designed to evaluate multimodal language models on real construction drawings. And here “real” is the key word: these aren’t simplified diagrams or pretty architectural renders, but the technical documentation civil engineers and architects use on site — abstract geometry, symbolic notation, data tables, overlapping annotations and specialized text all in a single image.

Current models, including the most capable ones on general visual tasks, perform notably poorly on this benchmark. The structural reason is clear: models were trained mainly on photographs and simple diagrams, not on the particular semantic density of an engineering drawing where a misread number can have real physical consequences. DrawingVQA exposes a gap that doesn’t appear in the usual multimodal model rankings.

For technology teams in construction, infrastructure or architecture — sectors where digitalization in Latin America is still at early stages — this benchmark is a useful cautionary signal. The “AI that reads your drawings” demos circulating on LinkedIn are probably showing carefully selected favorable cases. Before incorporating any model into a workflow with technical construction documentation, this kind of structured evaluation should be the starting point.

Measuring AI where business professionals actually work

A third study tackled something that has long been uncomfortable in the corporate world: standard LLM benchmarks measure factual recall, code and math, but not what analysts, product managers and consultants actually do every day. The paper proposes an evaluation framework centered on “analytical knowledge work”: synthesis of complex information, judgment under uncertainty, strategic reasoning situated in real business cases.

The results show that frontier models perform well on recall and structured reasoning tasks, but their performance drops appreciably when the problem requires integrating multiple ambiguous sources, recognizing which information is relevant in a specific context, or exercising the kind of judgment that distinguishes a senior analyst from a junior one. The gap isn’t dramatic, but it’s consistent and statistically significant.

The implication for executives and PMs evaluating how much to delegate to AI in strategic analysis processes is concrete: current models are good assistants for speeding up structured work, but still show systematic failures precisely in the most valuable parts of professional cognitive work. Automating the easy part and keeping human supervision on the final synthesis isn’t conservatism: it’s what the data supports today.

In a nutshell

What these three stories have in common is more important than each one separately: the industry is moving from asking “what can AI do?” to asking “what can AI do in this specific professional context?”. That transition is mature and necessary. Generic benchmarks served to demonstrate general capabilities; domain benchmarks — finance, construction, analytical business work — are the ones that will determine where investment is justified and where there’s still too much risk. For Latin American professionals making adoption decisions right now, the lesson is simple: demand evaluations specific to your use case before committing to any solution, because the model that leads the general ranking may not be the right one for what you need.

---

Sources used: AI Trading: Evaluating Large Language Models for Technical Market Analysis (https://arxiv.org/abs/2607.15414), DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings (https://arxiv.org/abs/2607.15418), Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning (https://arxiv.org/abs/2607.16057)