Study method
How 50 tests and 450 AI answers were measured
The study follows the questions people ask as they move from understanding a financial need to finding and choosing an adviser.
We tested 13 guidance questions, 12 discoverability questions and 25 direct firm-selection questions. Every finding keeps the denominator that produced it; the answers are never pooled into one firm score.
Return to the findings01 / Investigation design
Three kinds of buyer question
The 13 guidance questions examined how AI explained financial decisions and which sources supported the answer. The 12 discoverability questions measured which established firms became visible. The 25 demand-informed, pre-defined UK firm-selection questions measured which advisers entered the consideration set.
Five selection questions were deliberately repeated word for word, so the investigation contains 50 question tests and 45 distinct wordings. The direct-selection questions include 20 national questions across four buyer-need families plus five matched city questions. Search demand informed this set but is treated as a proxy rather than a representative survey of every UK buyer prompt.
Download the exact frozen corpus02 / Capture
450 responses across two days
- Questions tested
- 50
- Providers
- OpenAI, Gemini and Perplexity
- Repetitions
- 3 per question and provider
- Capture completeness
- 450 of 450 responses
- Guidance and discoverability
- 25 questions · 225 answers · 30 July 2026
- Direct firm selection
- 25 questions · 225 answers · 31 July 2026
Runs used fresh stateless sessions with no conversational carry-over. A valid answer was never rerun because it returned no firm or an inconvenient result. The direct-selection source records retain their original discarded-pilot label; the report treats the sealed export as a dated analysis capture without silently relabelling those records.
03 / Semantic review
A name string did not count as a recommendation
Every adviser-related entity in all 225 answers was reviewed by semantic role. Only a firm or adviser presented as an option for the buyer counts as a candidate. Comparison mentions, incidental references, warnings, directories, regulators and cited authorities remain separate.
The review also records shortlist position, rationale, panel status, own-domain citation and recoverable sentence-level source support. A final entity pass separated Nephos Group from Nephos Wealth Limited and excluded 12 invalid umbrella-identity candidate occurrences from valid adviser rankings.
04 / Scoring
Question shortlists before summary tiers
- Each answer contributes total normalized shortlist mass of 1, so longer provider lists do not dominate.
- Repetitions are averaged within each question and provider; the three providers receive equal weight.
- The four national families receive equal weight in the cross-scenario breadth view.
- Repeat-confirmed means a candidate appeared in at least two of three runs for the same question and provider.
- The five matched local questions are reported separately and never enter the national summary.
- The public output uses selection tiers rather than claiming a precise universal 1-to-150 ranking.
05 / Eligibility
Absence is limited to relevant opportunities
The open-universe view includes every candidate found, including firms outside the constructed panel. The opportunity-adjusted panel view includes only verified eligible firm-question cells. Unknown eligibility is never converted into ineligibility or used to penalise a firm.
The registry is intentionally partial. It does not support a claim that every panel firm was fully assessed against every client minimum, permission, specialism or local presence rule.
06 / Limits
What this edition does not claim
- Results describe the exact dated 25-question, three-provider, three-run API capture only.
- The constructed 150-firm panel is not the whole UK market and is not a quality or suitability ranking.
- Search-demand evidence is a proxy for AI-question demand, not a representative survey of every buyer prompt.
- A valid answer may name no candidate firm and instead direct the buyer to a directory or regulator.
- The five matched local questions are reported separately and never enter the national breadth view.
An absence is evidence only for the exact capture date, 25 prompts, three tested API surfaces and applicable eligibility rule. It is not evidence of adviser quality, suitability, consumer-interface behaviour or total market visibility.
Evidence package
Frozen inputs and derived outputs
- Frozen 25-question corpus
- Method and provider settings
- Corrected derived report data
- Corrected findings and limitations
Derived output hash3e9f7b6cf852ee98c488c78d88eba34f403b4edc44c350eff3c6600910233366
How the findings connect
Guidance and discovery remain separate analytical modules
The immutable fa-queries-0.2 capture contains 225 answers across 13 guidance questions and 12 discovery questions. Its 117 guidance answers form the evidence-authority module; 99 valid UK discovery answers support the visibility finding; and FA-SN-04 remains a disclosed jurisdiction-drift example.
The guidance and discoverability questions supply the 93-of-150 visibility result and the evidence-without-credit finding. Those answers do not enter the direct-selection candidate scoring, breadth tiers or question shortlists.