The Financial Times reports that leading assistants failed many personal-finance questions in its newsroom test, raising concerns about answers that readers may act upon.

The Financial Times tested chatbots on financial questions and concluded that they were wrong most of the time. The finding matters because a plausible error about tax, borrowing or investments can lead directly to a costly decision. It should also be interpreted cautiously because the complete method was not accessible during this verification.

Why it matters

Personal-finance questions often depend on jurisdiction, date and individual circumstances. A correct general statement may become wrong when applied to a different tax year, residency status or account type. Language models can produce fluent answers without reliably recognizing which missing fact changes the result.

A newsroom test can reveal practical failures that abstract benchmarks miss. It can ask questions in ordinary language and judge whether the answer would help a reader. But its strength depends on the sample: which assistants were tested, how many questions were used, whether browsing was enabled and what counted as wrong.

Those details are essential for interpreting “most of the time.” Without a denominator, the phrase does not say whether a model missed six of ten questions or hundreds in a larger test. Without the prompts and scoring rules, another evaluator cannot reproduce the result or determine whether a response was entirely wrong, incomplete or insufficiently qualified.

A fluent answer is not advice

Financial assistants create a particular risk because the conversational format can hide uncertainty. The system may present a number confidently even when it has assumed a country, date or income level. Readers should treat the first answer as a starting point for questions, not as authority to move money.

A safer design asks for the relevant jurisdiction and time period, cites an official rule and identifies facts that could change the result. It should also distinguish educational information from regulated financial advice. These controls can reduce error, but they do not prove that the underlying model reasons correctly.

The strongest response to a poor result is not a generic warning label. Product teams should test the exact tasks their users perform, record error categories and route high-impact questions to official sources or qualified professionals. A system that reliably explains where to verify an answer may be more useful than one that attempts a complete recommendation.

The Financial Times article is the original publisher of the reported test, but its full text and methodology could not be inspected here. The headline conclusion is therefore attributed to the newspaper and marked as not independently reproduced. No precise model ranking or error rate should be inferred from the accessible material.

The next useful step would be publication of prompts, answer keys, model versions and scoring decisions. That would turn a warning into a repeatable evaluation and show whether failures cluster around calculation, outdated rules or missing context.

Verification

  1. UNVERIFIED — The Financial Times reports that chatbots failed most questions in its personal-finance test. The full method and results were not accessible during this review. Primary publisher: https://www.ft.com/content/c0cd359d-df84-4208-a789-ffa864b43666
  2. UNVERIFIED — The number of questions, tested model versions, browsing settings and scoring rules were not confirmed. Primary publisher: the Financial Times article above.
  3. ANALYSIS — Jurisdiction, date and personal facts can materially change a financial answer. This is an editorial explanation of the domain’s requirements.
  4. ANALYSIS — Users should verify high-impact answers with official sources or qualified professionals. This is practical risk guidance, not a result claimed by the test.

Glossary candidates

  • Denominator: The total number of cases behind a reported rate.
  • Reproducible evaluation: A test whose prompts, settings and scoring can be repeated.
  • Jurisdiction: The country or legal area whose rules apply.

Cold-reader sentence: The Financial Times reports frequent chatbot errors on finance questions, but inaccessible methodology prevents precise conclusions about models or failure rates.