Skip to content

Study method

How 50 tests and 450 AI answers were measured

The study follows the questions people ask as they move from understanding a financial need to finding and choosing an adviser.

We tested 13 guidance questions, 12 discoverability questions and 25 direct firm-selection questions. Every finding keeps the denominator that produced it; the answers are never pooled into one firm score.

Return to the findings

01 / Investigation design

Three kinds of buyer question

The 13 guidance questions examined how AI explained financial decisions and which sources supported the answer. The 12 discoverability questions measured which established firms became visible. The 25 demand-informed, pre-defined UK firm-selection questions measured which advisers entered the consideration set.

Five selection questions were deliberately repeated word for word, so the investigation contains 50 question tests and 45 distinct wordings. The direct-selection questions include 20 national questions across four buyer-need families plus five matched city questions. Search demand informed this set but is treated as a proxy rather than a representative survey of every UK buyer prompt.

Download the exact frozen corpus

02 / Capture

450 responses across two days

Questions tested
50
Providers
OpenAI, Gemini and Perplexity
Repetitions
3 per question and provider
Capture completeness
450 of 450 responses
Guidance and discoverability
25 questions · 225 answers · 30 July 2026
Direct firm selection
25 questions · 225 answers · 31 July 2026

Runs used fresh stateless sessions with no conversational carry-over. A valid answer was never rerun because it returned no firm or an inconvenient result. The direct-selection source records retain their original discarded-pilot label; the report treats the sealed export as a dated analysis capture without silently relabelling those records.

03 / Semantic review

A name string did not count as a recommendation

Every adviser-related entity in all 225 answers was reviewed by semantic role. Only a firm or adviser presented as an option for the buyer counts as a candidate. Comparison mentions, incidental references, warnings, directories, regulators and cited authorities remain separate.

The review also records shortlist position, rationale, panel status, own-domain citation and recoverable sentence-level source support. A final entity pass separated Nephos Group from Nephos Wealth Limited and excluded 12 invalid umbrella-identity candidate occurrences from valid adviser rankings.

04 / Scoring

Question shortlists before summary tiers

  • Each answer contributes total normalized shortlist mass of 1, so longer provider lists do not dominate.
  • Repetitions are averaged within each question and provider; the three providers receive equal weight.
  • The four national families receive equal weight in the cross-scenario breadth view.
  • Repeat-confirmed means a candidate appeared in at least two of three runs for the same question and provider.
  • The five matched local questions are reported separately and never enter the national summary.
  • The public output uses selection tiers rather than claiming a precise universal 1-to-150 ranking.

05 / Eligibility

Absence is limited to relevant opportunities

The open-universe view includes every candidate found, including firms outside the constructed panel. The opportunity-adjusted panel view includes only verified eligible firm-question cells. Unknown eligibility is never converted into ineligibility or used to penalise a firm.

The registry is intentionally partial. It does not support a claim that every panel firm was fully assessed against every client minimum, permission, specialism or local presence rule.

06 / Limits

What this edition does not claim

  • Results describe the exact dated 25-question, three-provider, three-run API capture only.
  • The constructed 150-firm panel is not the whole UK market and is not a quality or suitability ranking.
  • Search-demand evidence is a proxy for AI-question demand, not a representative survey of every buyer prompt.
  • A valid answer may name no candidate firm and instead direct the buyer to a directory or regulator.
  • The five matched local questions are reported separately and never enter the national breadth view.

An absence is evidence only for the exact capture date, 25 prompts, three tested API surfaces and applicable eligibility rule. It is not evidence of adviser quality, suitability, consumer-interface behaviour or total market visibility.

Evidence package

Frozen inputs and derived outputs

Derived output hash3e9f7b6cf852ee98c488c78d88eba34f403b4edc44c350eff3c6600910233366

How the findings connect

Guidance and discovery remain separate analytical modules

The immutable fa-queries-0.2 capture contains 225 answers across 13 guidance questions and 12 discovery questions. Its 117 guidance answers form the evidence-authority module; 99 valid UK discovery answers support the visibility finding; and FA-SN-04 remains a disclosed jurisdiction-drift example.

The guidance and discoverability questions supply the 93-of-150 visibility result and the evidence-without-credit finding. Those answers do not enter the direct-selection candidate scoring, breadth tiers or question shortlists.