Fundamentals · 17 September 2026

By Eddie Eastwood, Founder

ChatGPT Left Us Off the Shortlist: What Can We Learn?

How do I find out who ChatGPT put on the shortlist instead of us?

A prospect told you that they'd already asked ChatGPT to put together a shortlist before they picked up the phone, and your business wasn't on it. That is a direct piece of information about why you lost. This one came with an explanation attached, even if it's an incomplete one, and it's worth taking seriously rather than filing away as an odd comment on the way out.

You can't pull the prospect's conversation out of ChatGPT yourself. It's their chat history, not yours, and there's no request process that hands it over. What is available to you runs through the prospect. Ask them directly, since they already brought the subject up themselves, and ask for a screenshot or a shared link while you are there: OpenAI's shared links let someone view a conversation without giving access to the account that created it. Then put a version of the same question to ChatGPT yourself and see who it names today. Neither the prospect's account of the conversation nor your own re-run is a certified copy of what they saw. The two together tell you a great deal more than sitting with the comment and wondering.

Why this one comment is worth more than it feels like

A lost deal can leave the reasons a guess: price, timing, a personal connection, something about the pitch. The prospect goes quiet, picks someone else, and never says which it was. A prospect who volunteers "we'd already shortlisted with ChatGPT and you weren't on it" has handed you something rare: a specific, checkable mechanism instead of a vague feeling.

That specificity is the whole value here. It's not proof that AI recommendations are costing you business generally. It's one data point about one deal, stated plainly by the person who was actually in the room. Treat it the way you'd treat any unusually direct feedback from a lost prospect: follow it up while it's fresh, rather than either dismissing it or overreacting to it.

What you actually want to know from this one deal

"Find out who was on the list" is really three questions bundled together. Separate them before you do anything else:

• Who was named. Which competitors, if any, came up when the prospect ran their query.
• What was said about you, if anything. Whether your business was described inaccurately, described but ranked low, or simply never mentioned at all. Those are three different problems with three different fixes.
• Whether this is a one-off or a pattern. Whether other recent prospects have quietly done the same thing without saying so out loud.

You can make real progress on the first two by talking to the prospect. The third one needs a small amount of your own testing, described below.

Ask the prospect: what to actually say

The prospect opened the door by mentioning it unprompted. You're not trying to win the deal back at this point. That's a separate conversation. This one is about learning something useful for the next prospect. Two questions get you most of it:

• "Do you remember who else was on that list?" Someone who mentioned the shortlist may well name names, especially once the deal is decided and there's nothing left to lose by answering.
• "Did it say anything about us specifically, or did we just not come up?" This is the question that tells you which problem you actually have: wrongly described, described but outranked, or invisible.

Keep it short and low-stakes. A long interrogation about their AI habits is a different thing, and it can make them wish they hadn't mentioned it at all.

If they're willing to go one step further, ask them to send the conversation rather than describe it. A screenshot is the obvious ask. A ChatGPT shared link is the better one, because it carries the full wording instead of whichever part fitted on their screen, and OpenAI's own documentation describes a shared link as letting people view the conversation without giving access to the account behind it.

Run the same question yourself, without rebuilding a whole system around it

The second move is to put the buyer's question to ChatGPT yourself: the kind of thing someone in that position would type, in their words, for your category and market. That tells you who gets named today. It may not match what the prospect saw, since answers to the same question shift between two people asking on two different days.

This is a single, targeted check on one specific question, not the start of an ongoing monitoring habit. To check this kind of question regularly rather than only when a prospect mentions it, use the method on How to Check If AI Recommends Your Business. Use that guide once you've decided monitoring is worth doing; use the single check here to answer the immediate question about this one deal.

Neither source alone settles the question completely. Together, they get you from "a prospect mentioned something odd" to "here's roughly what happened and here's what it might mean."

What a shortlist question actually returns: our own test

Before publishing this page we ran the buyer's question at scale, to see what a shortlist-shaped B2B question produces when nobody is watching over its shoulder.

We put 32 questions to ChatGPT, Gemini and Perplexity through the DataForSEO LLM Responses API over 2026-09-16 and 2026-09-17, across four B2B categories: managed IT support, commercial office cleaning, payroll software and contract management software. Every response was saved as a rendered API response capture. Four results bear on the deal you have just lost.

A shortlist question does return a shortlist. 18 of the 20 first-run answers named at least one vendor. Of the two that named nobody, one answered with selection criteria instead of company names, and the other was cut off before it reached its list.

Two engines given one identical question agreed on nothing. Asked which managed IT support providers to shortlist, with no location and no other qualifier, ChatGPT and Gemini named seventeen companies between them and shared none. Across all eight questions both engines answered, mean overlap was a Jaccard of 0.147, on a scale where 1.0 is two identical lists and 0 is no shared name at all.

Adding a city replaces the list rather than filtering it. In all three location pairs we could test, the named set after adding a city shared zero vendors with the same question asked without one. Two extra words produced an entirely different answer.

Most of the field never appears at all. Of the 96 managed IT businesses in one captured Columbus business-listings field, 5 were named across the eleven Columbus answers combined. That 96 is a single capture on one category tag at a 15 km radius, taken once, rather than a census of the market: 25 further Columbus providers were named by the engines but absent from that particular capture. Phase 2 browser evidence captured 13 of those 25 at HTTP 200; the other 12 had no captured URL and remain UNVERIFIED. So the figure records that 91 businesses in the captured field were never named in those answers. It says nothing about which of them should have been named, and we captured no evidence about why any company was named or left out.

One limit applies to every count above. 18 of the 32 answers hit the 1,200-token output limit and stopped mid-sentence, so each count is a lower bound rather than a finished total. The per-engine figures also sit behind a second limit: each engine is reached through its own API endpoint and the three were not asked to work equally hard, with one returning roughly half the output length of the others, so the three are reported separately and never averaged into a single figure.

And the scope matters as much as the numbers. This measures what the engines put in their answers, on those questions, on those days. It does not measure what your prospect saw, and it does not measure how often buyers ask questions like these.

What that changes about your own check

Three things follow for the check described above.

Use the prospect's words, including their city. A location is not a filter applied to a national list: in every pair we could test it produced a different list. A check run without the city your buyer would have typed is answering a different question from the one they asked.

Check more than one engine. Two of them, given one identical question on the same day, produced seventeen names and no agreement. Whatever you find on ChatGPT alone is one engine's answer, not the market's view of you.

Read your own absence carefully. Not being named is the ordinary outcome for most businesses in a category, so a single answer that leaves you out is a data point rather than a verdict. Which of the three problems below you have is the useful question, and a single check will not settle it.

What this one deal doesn't tell you

A single lost deal is easy to over-read in either direction. One comment from one prospect doesn't tell you that AI-driven buying decisions are now routine in your market. Nor does it tell you how your business shows up across the full range of ways a B2B buying committee uses these tools: comparison queries, vendor research, feature questions. That fuller picture, including how buying committees use these tools and what a comparison query returns, is covered in full on What Is AI Visibility for B2B Companies?. This page stays with the one real event that triggered it. If what you learn here points toward a wider pattern, that guide is where to go next.

Which of the three problems you actually have

What you learn points toward one of three next moves:

• You weren't named at all. Your site, your listings and whatever else is public about you may not describe the business clearly enough for these tools to surface it. That's a foundational visibility problem, and it's the one worth fixing first.
• You were named but described inaccurately or thinly. Check whether the wording matches your business. A vague or dated description is a different problem than not being mentioned at all.
• A competitor was consistently named ahead of you. Note it without assuming you know why. A single ranking on a single question is a data point to track, not an explanation.

Whichever of these applies, one deal gives you a direction, not a finished diagnosis. If the same pattern shows up again on a second or third deal, that's the signal that it's worth treating as routine and building a regular check around, using the monitoring method above.

If this keeps happening

A single prospect mentioning it once is a lead. The same thing happening on several deals is a pattern. At that point, stop working each instance separately and start checking systematically, using the method on How to Check If AI Recommends Your Business. For the wider B2B picture, What Is AI Visibility for B2B Companies? covers it.

FAQ

How do I find out who ChatGPT put on the shortlist instead of us?

The transcript itself isn't yours to see; it sits in the prospect's account. The prospect can still show you, though: a screenshot works, and so does a ChatGPT shared link, which lets someone view the conversation without giving access to the account that created it. Ask them directly, since they volunteered that the shortlist happened, then run the same kind of question yourself on ChatGPT and see who gets named today. Neither the prospect's memory nor your own re-run reproduces exactly what they saw, but the two together tell you far more than guessing.

Can I get the exact conversation the prospect had with ChatGPT?

Not from OpenAI. A ChatGPT conversation belongs to the account that ran it, and OpenAI documents no route for a business to request another person's chat history. The conversation can still reach you from the prospect: ask for a screenshot, or for a shared link, which OpenAI describes as letting people view a ChatGPT conversation without giving access to the creator's account. Of those two, the shared link is the better ask, because it carries the wording itself rather than a cropped picture of part of it.

Is one lost deal enough to draw any real conclusion from?

One deal tells you this happened at least once. It doesn't tell you whether it's routine or a fluke. Treat a single instance as a lead worth following up, not proof of a pattern. Use the follow-up questions on this page to find out which it is before you change anything.

Should I actually ask the prospect what ChatGPT told them?

Yes. Someone who raised it themselves has already shown they don't mind discussing it, so ask plainly. A short, direct question works better than a vague one: ask who else was on the list, or what ChatGPT said about your business specifically if anything. Whatever they tell you, write it down before you lose the detail: the names they remember, and the wording if they can recall any of it.

How is this different from generally monitoring what AI says about us?

Ongoing monitoring is the broader habit: checking regularly what AI platforms say about your business across many buyer questions. How to Check If AI Recommends Your Business covers that method in full. Start here when one prospect has told you a specific shortlist left you out and you want to run that single thread down first.

What do I do if this turns out to happen on every deal, not just one?

At that point a single-deal diagnostic stops being enough. You need the fuller B2B picture: how buying committees use AI tools during evaluation, what a comparison query returns, and what a business can do about its position in one. What Is AI Visibility for B2B Companies? covers that ground; this page hands off to it rather than repeating it.

Today's answer about your own business is one you can read directly, without another lost deal to prompt it. Free AI Visibility Check for your business →