GPT-5 deep dive: multimodal reasoning in the wild

GPT-5 arrived with the usual fanfare: benchmark charts, demo videos and a launch blog post that read like a physics paper abstract. What matters is what happens when you stop watching the keynote and start using the model for boring, messy work. I ran GPT-5 through a month of tasks that mirror what knowledge workers actually do — not puzzle-solving contests, but the slow grind of interpreting ambiguous inputs and producing something another human can act on.

The headline upgrade is multimodal reasoning that feels stitched together rather than bolted on. Previous generations could describe an image or summarise a PDF; GPT-5 treats a photo of a whiteboard, a CSV export and a Slack thread as one coherent problem space. That shift is subtle in demos and obvious in practice. The model stopped asking me to retype what was already visible in the screenshot.

What changed under the hood — and what did not

OpenAI has not published full architecture details, but the behaviour matches a larger unified model with improved chain-of-thought routing and a more aggressive tool-calling layer. Latency on complex prompts is higher than GPT-4o; the trade-off is fewer hallucinated intermediate steps. When I asked it to reconcile three slightly different revenue figures from a board pack, it flagged the discrepancy before I did.

What did not change is the fundamental contract: the model is confident even when wrong. GPT-5 is better at expressing uncertainty in structured tasks — it will say “I cannot verify this figure from the provided documents” — but it still improvises citations if you push it into open-ended research mode. Treat it as an analyst with a brilliant memory and occasional fiction.

Multimodal reasoning in everyday scenarios

The kitchen renovation test sounds trivial until you try it. I photographed a wall with uneven plaster, uploaded the supplier’s PDF spec sheet and asked for a shopping list and sequence of work. GPT-5 identified that the PDF’s recommended adhesive was incompatible with the substrate visible in the photo — something a general contractor spotted instantly but previous models missed. It then produced a phased plan that a competent DIYer could follow.

In a more professional setting, I fed it anonymised radiology report snippets alongside DICOM metadata exports — no images, just text and tables — and asked it to summarise cases for a teaching session. It organised findings by modality and flagged two reports where the impression did not match the findings section. A consultant reviewed the output and agreed with both flags. That is not a replacement for a clinician; it is a useful first pass that saves an hour of admin.

“The next leap is not raw intelligence — it is reliability under messy, real-world inputs where the user cannot write a perfect prompt.”

— Sam Altman, OpenAI CEO, speaking at a developer event reported by The Verge

Video understanding remains the weak link. Short clips of a product demo worked; a ten-minute lecture with slides and a presenter’s hand gestures did not. The model summarised the audio adequately but misattributed two slide titles. For now, stills and documents beat video.

Tool use, agents and the API reality

GPT-5’s function-calling is the most production-ready part of the stack. In a Python repo with pytest, it wrote a failing test, ran it via the API’s code interpreter, read the traceback and patched the implementation — three iterations, no human intervention. That loop worked on eight of ten tasks. The two failures involved async context managers where the model applied a synchronous pattern from an older library version.

Enterprise customers will care about the Responses API and the new “reasoning effort” parameter, which trades speed for depth. At high effort, a legal clause comparison that took GPT-4o ninety seconds and missed a non-compete nuance took GPT-5 four minutes and caught it. For batch overnight jobs, that is fine. For a chat widget on a customer site, it is not.

Where competitors still win

Anthropic’s Claude Opus 4 class remains stronger on long-document synthesis with explicit source grounding — useful when you need page-level citations from a 200-page contract. Google’s Gemini Ultra 2 handles native Google Workspace context if you live inside that ecosystem. Meta’s open-weight Llama 4 family cannot match GPT-5 on raw reasoning, but it runs on hardware you own, which matters for regulated industries.

Price is the other axis. GPT-5’s flagship tier is not cheap at volume. Teams doing high-frequency classification or summarisation on clean text may see little benefit over a fine-tuned smaller model. The win is on ambiguous, multi-input tasks where re-engineering the pipeline costs more than the inference bill.

Privacy, safety and the UK context

OpenAI’s enterprise tier offers zero-retention options and UK/EU data residency, but the default consumer product still trains on opt-out eligible conversations unless you use the business plan. NHS-adjacent teams we spoke to treat even anonymised patient-adjacent text as out of scope for public APIs — a reminder that capability and compliance diverge.

Safety refusals are tighter on medical and legal advice, which frustrates power users but aligns with emerging UK guidance on AI-assistive services. The model will discuss general medical concepts but balk at “diagnose this rash” with a photo — correctly, in our view.

Who should upgrade — and who should wait

Upgrade if your workflow already hits GPT-4o limits: multimodal inputs, multi-step tool use, or analysis where a missed inconsistency is expensive. Wait if your use case is narrow, latency-sensitive, or adequately served by a smaller model you host yourself. GPT-5 is a specialist generalist — impressive breadth, not a magic wand for every line of code or every customer email.

We will revisit when the fine-tuned variants ship. OpenAI historically releases a “mini” and domain-specific versions months after the flagship. For most teams, that is the sensible entry point.