AI sales coaching: how to check the scores before you coach with them

This guide is for sales managers, enablement leads and founders who are being sold AI sales coaching and want to know what it will and won't do for their team. It covers the three forms of AI coaching, the evidence, how to build a scorecard the AI can apply, how scores go wrong, the rules on recording and monitoring, and a calibration test you can run in a week. For AI that helps reps with admin rather than skills, see the guide to AI sales assistants.

The three forms of AI sales coaching

Most products now combine two or three of these. Buy for the one your team needs most and judge the others as extras.

What it does well, and what it doesn't

Does well: listens to every call, which no manager can; finds the moments worth a manager's time (a pricing question, a competitor mention, a missed next step); counts what's countable, such as questions asked, talk time and monologue length; and gives reps a safe place to rehearse a pitch twenty times.

Does badly: judges deal strategy, because the call is only part of the picture; knows what happened before and after the call unless it reads your CRM and email; tells a good silence from a lost prospect; and grades qualities that aren't in the words, such as credibility with a skeptical CFO.

The best use is triage plus practice: the AI points the manager at the three minutes that matter in a 40-minute call, and reps rehearse the hard moments with an AI buyer before they meet real ones.

What the evidence says

The strongest independent evidence comes from a neighboring job. Brynjolfsson, Li and Raymond studied 5,179 customer support agents given a generative AI assistant (NBER working paper 31161, published in the Quarterly Journal of Economics in 2025). Issues resolved per hour rose 14% on average, "including a 34% improvement for novice and low-skilled workers," with minimal effect on experienced, highly skilled agents. The authors suggest the tool spread the practices of the best workers to newer ones.

Two cautions. Support agents aren't sales reps, and the assistant prompted agents live rather than scoring them afterward. And most AI sales coaching statistics you'll see quoted come from vendors' own reports, without a control group. The reasonable expectation: the biggest gains are with new reps, and you should measure your own.

Build a scorecard the AI can apply

An AI applies a scorecard literally. Items like "built rapport" or "sounded confident" produce scores nobody can explain. Write each item as something a listener could point to in the transcript, and score it 0 (missing), 1 (partly) or 2 (clearly done).

Score each item 0 = missing, 1 = partly, 2 = clearly done. Quote the transcript line that earns the score.

1. Agenda: rep states the purpose and time, and asks if the buyer wants to add anything.
2. Current state: rep asks how the buyer handles [problem area] today, and the buyer describes it.
3. Quantified problem: the buyer states a number for the cost, size or frequency of the problem.
4. Impact: rep asks what happens if nothing changes, and the buyer answers.
5. Decision process: rep asks who else is involved and how a decision gets made.
6. Next step: a specific next meeting with a date, time and named attendees is agreed.

Total out of 12. Flag for manager review: any call scoring 0 on item 6, or any mention of a competitor or price.

Keep counts such as talk ratio as context, not targets. A rep told to hit a 40% talk ratio will hit it by asking more filler questions. If you run a named method such as MEDDPICC, turn each letter into one observable item.

How AI coaching goes wrong

• Transcript errors. Every score rests on the transcript. A 2020 study in PNAS tested five speech recognition systems from Amazon, Apple, Google, IBM and Microsoft and found an average word error rate of 0.35 for Black speakers against 0.19 for white speakers. Systems have improved since, but the lesson holds: check transcript quality for each rep, including those with strong regional or second-language accents, before you compare their scores.
• Literal scoring. An item like "quantified the problem" gets credit for any number on the call, including the rep's own pricing. Ask for the quote behind each score.
• Metric gaming. Whatever you score, reps optimize. Score behaviors that help the buyer decide.
• Easy role-play buyers. A simulated buyer who raises objections on cue and then agrees teaches reps that objections are scripted. Have your best rep play-test each persona.
• Live prompts as a crutch. New reps who read every suggestion sound like they're reading. Set prompts for facts (specs, compliance answers), not for conversation.
• Coaching the wrong calls. Polished calls with accounts that were never going to buy are still wasted calls. Coaching improves how reps run a conversation; list quality decides who they're having it with. Bebop handles that part: it finds companies and contacts that match a plain-language description and explains why each fits.

The rules on recording and monitoring

This is general information, not legal advice.

• Recording consent. Call scoring needs a recording. Federal law allows one-party consent, but some states, including California, require everyone on the call to agree. Announce recording at the start of every call, and make sure the tool's bot announces itself in video meetings.
• Telling employees. New York's Civil Rights Law section 52-c requires employers with a place of business in the state to give written notice on hiring to employees whose phone calls, email or internet use are monitored, to get it acknowledged and to post it. Penalties run up to $3,000 for repeat offenses. Other states have their own rules; give every rep a written notice anyway.
• Emotion inference in the EU. The EU AI Act prohibits AI systems that infer the emotions of people in the workplace, except for medical or safety reasons, and the European Commission says that prohibition has applied since 2 February 2025. If you have reps in the EU, switch off any feature that scores a rep's mood, stress or sentiment, and ask counsel whether buyer-side sentiment scores are covered too.
• Data use. Recordings contain customer information. Find out whether the vendor trains its models on your calls and how long it keeps them.

Worked example: a calibration test at a cybersecurity vendor

Illustrative example. The company, people and numbers are invented to show the method.

The team: Ironvale Security (ironvale.example) sells managed detection and response to companies with 200 to 2,000 employees. It has 8 SDRs, 6 account executives and 2 sales managers, who each spend about 5 hours a week listening to calls.

The test: the managers pick 20 recorded discovery calls (a mix of reps, won and lost deals, and two reps whose first language isn't English). Each manager scores all 20 on the six-item scorecard above without seeing the other's scores; then the AI scores them.

On the quantified problem item, the AI credited any number, including "we have 400 endpoints" said by the buyer as background and the rep's own price. The managers rewrote the item as "the buyer states the cost, size or frequency of the problem" and reran it: 16 of 20.

A transcript check on a five-minute sample per rep found the two second-language speakers' transcripts had roughly three times as many errors as the rest, mostly on product names. Ironvale added its product and competitor names to the tool's custom vocabulary and kept manager review on those reps' calls until a recheck matched the others.

The decision: keep the AI scoring on items 1, 2, 5 and 6, use the rewritten item 3, and drop item 4 (impact) to manager-only review. At an illustrative $50 a seat a month, 14 seats cost $8,400 a year. If the managers' listening time falls from 10 hours a week combined to 4, that's about 290 hours a year back, which is the figure Ironvale will check after a quarter.

A one-week calibration test

1. Write the scorecard as five to eight observable items, each scored 0, 1 or 2 with a quote.
2. Pick 20 recorded calls across reps, outcomes and accents.
3. Have two managers score them blind and count item-level agreement. That's your ceiling.
4. Run the AI on the same calls and compare with the items where the managers agree.
5. Read every disagreement. Most are fixable by rewriting the item.
6. Check transcripts with a five-minute sample per rep.
7. Go live on the items that pass, keep the rest for managers, and re-run the test each quarter or when the vendor changes models.

What it costs

Published prices checked on 26 October 2026, per user per month:

Add the hidden cost: someone has to own the scorecard and re-run the calibration. Budget a few hours a month for it.

What to ask a vendor

• Can I write my own scorecard items and see the transcript quote behind every score?
• What is your transcription accuracy across accents and languages, and can I add custom vocabulary?
• Does any feature infer emotions, sentiment or stress, and can I switch it off per region?
• How does your recorder announce itself, and how do you handle all-party consent states?
• Do you train models on my calls, and how long do you keep recordings?
• Can reps see and dispute their own scores?

Next step: give reps better answers to practice with, using the responses in the objection handling guide.