Artificial Intelligence

How to Check Whether AI Actually Mentions Your Firm

In plain terms

No one can give a firm a precise count of how often ChatGPT or Google’s AI mentions it, and Semrush, which sells an AI visibility tool, says as much: “no platform can provide exact numbers on visibility.” The author’s answer is to measure in six layers, from most to least reliable, starting with Google’s Search Console reports and ending with paid dashboards.

The layer that answers “Do we show up in ChatGPT?” is structured prompt testing. That means ten to twenty client-style questions per practice area, run in temporary or logged-out sessions across six engines, recording whether the firm was named, whether a page was linked and which sources were cited. Because answers vary, the result is the share of runs in which the firm was mentioned. The trend in branded queries in Search Console is one of the better proxies for AI visibility, in the author’s view.

And what the numbers cannot tell you

By Dave Taillefer, Business Director, ICONA

Sooner or later a partner asks the question: "Do we show up in ChatGPT?" The honest answer is "sometimes, for some people, on some days." The second honest answer is that nobody, including the companies selling AI visibility dashboards, can give you a precise number. Semrush, which sells one of the larger such tools, says so in its own documentation: "no platform can provide exact numbers on visibility."

That is not a reason to skip measurement. It is a reason to measure in a way that respects what the data can and cannot do. Here is how I approach it, from the most reliable layer to the least.

Why this is harder than rank tracking

For twenty years a ranking was a fact. You searched, you saw position four, a tool recorded position four. AI answers do not work that way, for four reasons.

They are not deterministic. Ask the same question twice and you may get different sources. Parse, an analytics firm, ran more than 16,000 ChatGPT prompts repeatedly in spring 2026 and found that repeat answers shared only about 21 percent of their cited sources; Google's AI Overviews shared 31.5 percent. SISTRIX tracked 82,000 prompts across six countries over 17 weeks and found that roughly 74 percent of the domains cited in ChatGPT search were new from one week to the next, against about 56 percent in AI Mode and 5 percent in AI Overviews. Read those numbers as: AI Overviews behave like search results, and conversational assistants do not.

They are personalized. Since April 2025, ChatGPT's memory can draw on a user's past conversations when the feature is on, and OpenAI's current documentation says it also draws on files and connected apps. Two people asking the same question can get answers shaped by what they have asked before. Your test from your office is not your client's result from their kitchen.

They are location-aware, and legal questions are jurisdictional. A prompt about limitation periods answered in Calgary should not look like the same prompt answered in Toronto.

And they fan out. Google describes AI Mode as using a "query fan-out" technique that breaks a question into subtopics and issues many searches at once. The prompt you typed is not the query that found your page.

Layer one: what Google will tell you

Google and Bing are the only AI answer providers with any reporting at all, and both are thin. Google first.

In the standard Search Console Performance report, clicks and impressions from AI Overviews and AI Mode are included but blended into the "Web" search type with no filter. Google's counting rules are specific: a click on a link inside an AI Overview counts as a click; an AI Overview occupies a single position and every link inside it is assigned that same position; AI Mode positions follow the standard results-page methodology. In practice, if your average position improved while clicks fell, an AI Overview may be part of the reason, but the report will not say so.

In June 2026 Google added a separate Generative AI performance report, first in the UK and worldwide from August 31, 2026. It shows impressions only: how many times links to your site appeared in a generative AI feature on Google Search, broken down by page, country, date and device. AI Overviews and AI Mode are combined, there are no clicks, positions or queries, the report appears only once a site clears an undisclosed impression threshold, and the Search Console API has no field for it. Google has said additional metrics will follow. For a firm it answers exactly one question, "are our pages being shown inside AI answers at all," and that is worth knowing.

The same June 2026 update added the property-level control that opts a site out of these features. Confirm nobody has switched it on.

How often do legal searches trigger an AI Overview? It depends what you count. A widely quoted SE Ranking study from September 2024 found AI Overviews on 78 percent of 300 hand-picked informational legal queries, U.S. desktop only. Ahrefs' analysis of 146 million U.S. results pages in September 2025 found them on 23.6 percent of queries classified as legal. Both are correct; one measured curated "how does this work" questions and the other measured every query classified as legal, branded and unbranded. Similarweb reported in July 2026 that AI Overviews appeared on roughly 43 percent of all Google searches, up from 15 percent a year earlier, by a panel-based method that is not comparable with the keyword studies. None of these are Canadian figures. Treat them as direction, not magnitude.

Layer two: what your analytics will tell you

Since May 2026, Google Analytics 4 has had a default channel called AI Assistant. Google's definition: traffic arriving "from sources like ChatGPT, Gemini, Deepseek, Copilot, or Grok," and it "excludes Google's AI Overviews and AI Mode." That exclusion is the important part. Clicks from Google's own AI features arrive as google / organic and cannot be separated from ordinary organic search in GA4.

For the assistants, tracking is decent. ChatGPT appends utm_source=chatgpt.com to outbound links, by OpenAI's own account. Referrers from perplexity.ai, gemini.google.com, claude.ai and copilot.microsoft.com are identifiable. Expect the absolute numbers for a law firm to be small, and expect quirks: for part of May 2025, Google's AI Mode stripped referrers entirely and its traffic showed up as Direct until Google fixed it.

What to look at is not sessions but outcomes. Compare contact-form completions and calls per session for the AI Assistant channel against organic search. Google has said that clicks from results pages with AI Overviews are "higher quality," with users more likely to spend time on the site. That is Google's claim about Google's product; test it against your own conversion data before repeating it.

Layer three: Bing, which most firms ignore

Bing handled about 9.6 percent of Canadian searches in August 2026, and its index feeds Microsoft Copilot. In February 2026 Bing Webmaster Tools launched an AI Performance report in public preview, expanded in June, that shows how often your pages were cited across Copilot and Bing's AI summaries, a sample of the grounding queries that produced those citations, and breakdowns by intent and topic. It is the only engine currently showing query-level AI data, though still no clicks. Setting it up takes ten minutes. Most firms have not.

Layer four: your server logs

Before asking whether AI systems cite you, check whether they visit. A log extract filtered to OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Googlebot over thirty days tells you whether the retrieval systems are reaching your pages at all. Verify the user agents against the IP ranges OpenAI, Anthropic and Perplexity publish, because anyone can spoof a name. Zero visits from a crawler you have not deliberately blocked usually means something upstream is blocking it; I have written separately about where those blocks hide.

Layer five: structured prompt testing

This is the layer that answers the partner's question, and it has to be done with discipline or it produces noise.

Build a prompt set. Take each practice area and write ten to twenty questions the way a prospective client would type them, not the way a keyword tool would. Spread them across intents: who handles this ("which lawyers in Edmonton handle wrongful dismissal"), how to choose ("what should I look for in a family lawyer"), cost ("what does an uncontested divorce typically cost in Alberta"), process ("how long does probate take in Alberta"), and the specific procedural questions your lawyers actually get asked. Include the jurisdiction in some and leave it out of others.

Control the conditions. Use temporary or logged-out sessions so memory does not colour the results; OpenAI notes that temporary chats do not use memory. Set the location to the city you serve. Run each prompt across Google AI Mode, Google AI Overviews, ChatGPT with search enabled, Perplexity, Copilot and Gemini.

Record three things per run: was the firm named; was a firm page linked; and which sources were cited, by type (law society directory, legal directory, news outlet, government or court site, another firm, a public legal education organization). The third field is the most useful thing you will collect. The mix of sources an assistant leans on for your practice area in your province is a map of where your firm needs to be present and accurate.

Repeat. Run the panel monthly, or weekly if you can automate the recording, and report share of runs in which the firm was mentioned rather than any single result. The volatility figures above tell you why a single screenshot proves nothing.

A simple scoring sheet:

Prompt Engine Firm named Page linked Sources cited (by type) Date
"family lawyer in Calgary for a high-net-worth divorce" AI Mode No No Legal directory, news article, two firm sites 2026-09-03
"how long does probate take in Alberta" ChatGPT Yes Yes Government site, firm site (ours), legal directory 2026-09-03

Fifty prompts across six engines is three hundred runs a month. It is tedious. It is also the only measurement that resembles what your clients actually see.

Layer six: the dashboards

Semrush's AI Visibility Toolkit, Ahrefs' Brand Radar, Profound, Peec, Otterly and Similarweb's AI Search all sell versions of the prompt panel above, run at scale on their prompts with their sampling. They report mentions, citations and "share of voice" across engines, and some add sentiment. They are useful for trend and for competitor comparison. Their limits are the ones the vendors themselves disclose when you read the small print: the prompts are either the vendor's or ones you write, not your clients'; results generally come from sessions without your clients' history; Ahrefs notes that coverage is limited for brands with little search volume, which describes most law firms; Otterly notes that manual searches will differ from tracked results because of memory and personalization; Peec runs each prompt once a day. Buy one if you want the trend line automated. Do not mistake the trend line for the truth.

What the numbers cannot tell you

Whether the person who read the answer became a client. Most AI exposure produces no click, and the standard interfaces do not record who saw your name. Pew found people clicked a link inside an AI summary about 1 percent of the time.

Whether "impressions" means you. Google's Generative AI report counts a link to your site being shown. An assistant naming your firm in prose, with no link, is invisible to every report on this list except your own prompt panel.

Where the enquiry came from. A common path now runs: a prospective client reads about your firm in AI Mode, closes the tab, later searches your firm's name, and calls. That shows up as branded organic search, which is why I treat the trend in branded queries in Search Console as one of the better proxies for AI visibility we have.

What would have happened otherwise.

A quarterly page

Pull it together into one page every quarter: branded search impressions and clicks from Search Console; Generative AI report impressions, once you clear the threshold; AI Assistant channel sessions and conversions against organic search in GA4; Bing AI citations; your prompt panel's share of mention by engine and practice area; and, most importantly, the list of sources the assistants cite for your practice areas where your firm is absent or inconsistent. The last item is the to-do list. The rest is context.

Measurement in this environment is a sampling exercise, not a census. That is uncomfortable for a profession that likes evidence, but it is the honest state of the tools in 2026, and a firm that measures carefully and reads the results modestly is ahead of nearly everyone.

Sources

Filed under Artificial IntelligenceSEO
Related service Get found in search and AI answers

More in Artificial Intelligence

View all 20
Twenty-Five Years of "Search Is Dead" Is Your Firm Accidentally Invisible to AI? Why Ranking #1 No Longer Guarantees New Clients