Why we never say "100% qualified": reading AI vendor claims

"95% accuracy." "98% qualified." "Every lead pre-qualified." Suppose those are three vendor pitches, and the reps at your 40-person firm spend their mornings sorting leads that go nowhere. You have budget for one tool. How do you choose?

You cannot, yet. None of those sentences is a measurement. Each is a number or a word standing where a definition should be. Below: how to put the definition back, what to ask before you sign, and why this site never claims that every lead is qualified.

We sell lead-response and outreach systems, so we have an interest here. We have tried to write the checklist we would want as the buyer, and to describe our own method precisely enough that you can find fault with it.

Why an accuracy figure tells you so little

Accuracy is a ratio: how many answers were right, out of how many were checked. A percentage on its own hides every part of that sentence. Four questions bring them back.

Right according to whom? For a lead, "right" means someone decided the lead was good or bad. A human labeled it, or an outcome (a booked call, a signed deal) labeled it, or the tool's own scoring labeled it. These give very different numbers, and only the last is circular.

Out of what? Imagine an accuracy figure measured on a hand-picked set of 200 clean records. It says little about the messy list you will upload. The FTC's account of one case makes the point. In 2025 it alleged that a company claimed its AI content detector was 98% accurate, while independent testing put its accuracy on general-purpose content at 53%, according to the FTC's administrative complaint.^[1] The FTC alleged the model had been trained or fine-tuned only to classify academic content effectively, not the wider material the company's claim covered.^[1] These are allegations in a settlement the FTC announced for public comment, not findings after a trial.^[1] The lesson for a buyer is not that vendors lie. It is that a figure can be true of one dataset and meaningless on yours.

Which kind of wrong? A tool that calls everything "qualified" never misses a good lead, and is useless. A tool that never calls anything qualified never wastes your time, and never finds you anyone. Any single percentage that does not state both kinds of error (good leads missed, bad leads passed) is half a claim.

At what base rate? Suppose 5 in 100 leads are truly good. A tool that says "not qualified" to everyone is then 95% accurate. That is not a joke about bad products. It is arithmetic, and it is why a headline percentage tells you so little.

"Qualified" is a judgment, not a fact

"Qualified" sounds like a property of the lead, like a postal code. It is not. It means "this lead meets the bar that someone set". Change the bar and the same lead flips.

A dentist with two chairs is a perfect lead for a seat-based booking tool and a poor one for a hospital-grade scheduling system. Neither tool is wrong. They hold different definitions of fit. Imagine a vendor who says "100% qualified". Either the definition is so loose that it excludes nothing, or it is a definition they have not told you.

There is also a regulatory edge to this. Announcing a set of enforcement actions in September 2024, then-FTC Chair Lina Khan said "there is no AI exemption from the laws on the books".^[2] Under the proposed order in the detector case, the company may not make claims about its products' effectiveness unless they are not misleading and it holds "competent and reliable evidence" at the time the claim is made.^[1] A marketing claim about AI performance is an ordinary advertising claim. It needs support you could show someone. That is a rule about the United States; we are not saying what it means for any particular vendor or for you. Check with counsel about the markets you sell in.

Seven questions for any vendor

You can put these in an email. A vendor with a real answer will enjoy them.

1. What exactly is being measured? "Qualified" as in meeting a written ICP? "Interested" as in replied? "Converted" as in paid? Ask for the one-sentence definition.
2. What was the sample? How many records, from what industries and countries, collected when? Was it a customer's real data or a test set the vendor built?
3. Who labeled the ground truth? A named person or team, a customer, or the tool itself? Were they blind to the tool's output?
4. What are the two error rates? Good leads missed and bad leads passed, not just one blended percentage.
5. What does the miss look like? When the tool is wrong, what happens next? Does a bad lead get an email sent in your name? Does a good one disappear silently, or sit in a queue a person can see?
6. Can I see why a given lead got its result? If the answer is "the model decided", you cannot challenge it, and neither can your team.
7. What would you refuse to claim? A seasoned vendor has a list. A vendor with no list has no discipline.

A vendor can answer the first five with a spreadsheet. If every answer arrives as another marketing page, that is also an answer.

The one legitimate perfect score

A perfect score has one defensible use: a coverage claim. Of the things the system was supposed to act on, how many did it act on?

Our version is: 100% of your ICP-fit leads that have a consent record get contacted.^[3] Look at what has to be true for that sentence to hold:

• Someone defined "ICP-fit" in writing (you did, with us).
• Every lead in the batch was scored against that definition.
• Every lead that cleared the bar and has a consent record was contacted. A lead with no consent record is not contacted at all. It is counted as skipped and listed, so the claim covers the leads we are allowed to write to, not every lead on the list.
• Anyone can count the contacted ones and compare them with the fit ones.

A person with a spreadsheet and ten minutes can check it. It also says nothing about whether those leads will buy, or whether the bar was set in the right place. That is deliberate. We would rather promise a smaller thing we can prove than a bigger thing we can only assert.

Set that beside a claim that every lead is qualified. It cannot be counted, because "qualified" is judged, not tallied. A claim you cannot audit is a claim you cannot be held to, which is its main attraction for the person making it.

How we score: bands with evidence, not a verdict

Our lead-scoring module is short enough to describe in full. What follows is taken from the code, not from memory.^[3]

You (the client) give us an ICP profile. It can include industries, job-title keywords, a company-size range, "must-have" signals, and "nice-to-have" signals. A profile with no criteria at all is rejected rather than quietly producing zeros.^[3]

Each lead is scored against it. Points are assigned per matched criterion: 15 for a must-have signal, 10 for a title match, 10 for company size in range, and 5 each for an industry match or a nice-to-have signal.^[3] The score is then expressed as a percentage of what could be assessed for that lead, not a raw total. That matters: an ICP with two criteria and one with five have different maximum points, so a fixed cut-off would punish the smaller profile. If a lead has no company size on file, size is simply left out of what is assessable. The lead is not marked down for missing data.^[3]

The percentage becomes a band: high at 70 or above, medium at 40 or above, low at 15 or above, and no fit below that.^[3] A lead in "no fit" is still in the batch and still visible. Nothing is hidden.

Every score comes back with its working: which signals matched, which did not, written out in plain language. There is no yes/no field called "qualified" anywhere in the module.^[3] The one boolean that exists, "worth engaging", is true for the high and medium bands and is documented in the code as a threshold decision about whether to contact, not a guarantee of quality.^[3]

Here is a small invented example, run through the real scoring code on October 2, 2026. It is a test of the method, not a result for any client. The ICP is dental practices with 5 to 50 staff, a title of owner or practice manager, the must-have signal "online booking", and the nice-to-have signal "hiring front desk".

If the ICP says "must have online booking" and a lead shows no evidence of it, the reason is printed.^[3] A salesperson can disagree, change the signal list, or override the call. That is the point.

The coverage number then comes from counts. If two leads clear the bar and we contact both, coverage is complete. If we contact one, it is 50%, and the report says so. A lead with no consent record sits outside the claim and is shown as skipped.^[3] If nobody clears the bar, the report shows zero worth engaging rather than dividing by it.^[3]

What this method does not do

The limits, in the same detail as the method.

• The scoring is matching, not prediction. It checks lead details and collected signals against your list of criteria using lowercase text comparison. It does not estimate who will buy.^[3]
• The bands are design choices. The thresholds of 70, 40, and 15 and the point values are ours.^[3] We have not validated them against sales outcomes, because we have no client outcomes to validate against yet.
• It is only as good as the signals gathered. Weak evidence in gives a weak band out. The "why" line shows exactly which evidence was used, so that weakness is visible.
• The ICP is yours. If the definition of fit is wrong, we will contact the wrong people very thoroughly.

If a vendor can tell you their equivalent list of limits, trust them a little more. If they cannot, ask.

A short checklist for your next evaluation

• Replace every "accuracy" or "qualified" figure with its definition, its sample, and its two error rates, or treat it as a slogan.
• Prefer coverage claims you can count to quality claims you have to believe.
• Ask for the working behind one individual result, chosen by you, not by them.
• Ask what happens on the miss, and who sees it.
• Run a small paid or free pilot on your own list, and decide in advance what result would make you stop.

If you are comparing tools, we are happy to be one of the options on the table: talk to us and we will tell you what our system measures, what it does not, and where it is the wrong fit.