Quality and Call QA concepts - the mental model and key terms
Objective. Give Sales and CSM the mental model behind Quality and Call QA and a plain-language glossary, so every term they use with a customer carries the right meaning and the right caveat.
The mental model is a loop - evaluate, review, coach - whose trust lives in four supports: a rubric that defines 'good', a sample that represents the operation, calibration that keeps scoring consistent, and coaching that closes the loop. AI accelerates the loop; it does not own it. Get the terms right and the whole app reads cleanly; blur them and coverage gets mistaken for quality. This module defines the concepts and shows how they chain.
See it in the demo
The core concept - a trust-controlled quality loop
Quality and Call QA is built on one idea: quality is not a score, it is a loop you can trust. Interactions are evaluated against a rubric; evaluations are reviewed and kept consistent; findings close into coaching that changes behaviour; and recurring-error patterns feed the next cycle. The product's real work is not scoring more calls - it is making each step trustworthy, because a score that is not trusted changes nothing and a score that is trusted but wrong is dangerous.
Everything in the app serves that loop. Sampling decides what you look at. Scenarios & Checklists decide what 'good' means. Calibration and four-eye checks decide whether a score means the same thing whoever gave it. Disputes decide whether a score is defensible when challenged. Coaching and root cause decide whether findings turn into improvement. Ask Quality AI makes the loop faster and wider but stays under human review. Read the app as this loop, not as a dashboard of numbers.
Key terms (a plain-language glossary)
- Rubric (Scenarios & Checklists) - the written definition of a good interaction. Every score means only what the rubric encodes.
- Evaluation / score - one reviewer's scored read of one interaction against the rubric, backed by evidence. Not objective truth; a scored judgement.
- Sample - the set of interactions chosen for review. A QA score is a sample, not a census, so it generalises only as far as the sample is representative.
- Coverage - the share of interactions reviewed. A reach measure, not an accuracy measure.
- Calibration - having several reviewers score the same interactions to measure agreement. The trust metric that makes scores comparable; it proves consistency, not correctness.
- Four-eye check (maker-checker) - routing a selected high-stakes score to a second reviewer who confirms or corrects before it stands. Raises confidence on that item only.
- Dispute - an agent's challenge to a score. An open dispute makes a score provisional; a resolved one makes it defensible.
- Root cause - recurring error patterns clustered from findings. Hypotheses to validate, not proven causes, and not additive.
- Coaching - the action created from a finding. Assigned, completed and improved are three different states.
- Ask Quality AI - AI that accelerates first-pass evaluation and surfaces patterns, under human review. A suggestion, not the final word.
How the concepts chain together
The chain reads in one direction, and trust flows down it: the rubric defines 'good'; sampling decides which interactions are judged against it; evaluation produces scores; calibration and four-eye checks keep those scores consistent and confident on high-stakes items; disputes keep them defensible; root cause turns scattered findings into patterns; coaching acts on the patterns; analytics and reports summarise the whole; and the next cycle tests whether recurring errors shrank.
Because trust flows downhill, a weakness high in the chain contaminates everything below it. A wrong rubric makes every score measure the wrong thing. A biased sample makes every aggregate unrepresentative. Uncalibrated reviewers make every mixed aggregate a blend of incompatible judgements. This is why a CSM validates upstream (rubric, sample, calibration) before trusting anything downstream (analytics, reports, a Command Center tile), and why 'read across the loop' beats 'read one number'.
The one concept people most often get wrong
The single most common error is collapsing coverage into quality - believing that scoring more calls, especially with AI, is the same as knowing quality is good. It is not. Coverage is how many interactions you looked at. Quality trust comes from accuracy (proven by calibration agreement) and improvement (proven by recurring-error reduction and coaching that completes). '100 percent of calls scored' can sit on an uncalibrated rubric and a biased sample and be confident-looking noise.
The corollary, equally missed: an AI score is not self-validating. Confident wording is not accuracy; a human has to stand behind any score that carries a consequence. Whenever a customer or a teammate reaches for a coverage number as proof of quality, the correction is the same sentence every time - coverage is reach, accuracy is calibration, improvement is recurring-error reduction. Hold that distinction and the rest of the mental model stays honest.
Interpretation check
Quality and Call QA is a trust-controlled loop, not a score. Trust flows downhill from the rubric, sample and calibration, so coverage is reach, accuracy is calibration agreement, and improvement is recurring-error reduction - never collapse those into one number.
In the field
OVO (contact centre): the loop concept is concrete - a voice interaction is sampled, scored against the rubric, calibrated, possibly disputed, and coached, with the next sample testing whether the error declined. KnackGlobal (RCM): the chain concept is why a healthy aggregate over a rubric that skips compliance items is still an unmanaged risk.
Practical task
Without looking, write the loop from rubric to next-cycle check in order, then define five of the key terms in your own words. Finally, write the one-sentence correction you would use when a customer treats a coverage number as a quality result, and name the two upstream things you would validate before trusting any downstream aggregate.
Self-check
In one sentence, what is the core concept the app is built on?
Quality is a loop you can trust - evaluate, review, coach - whose value comes from making each step trustworthy, not from scoring more calls.
Define calibration and say what it does and does not prove.
Calibration has several reviewers score the same interactions to measure agreement. It proves scores are consistent and comparable (correctness of the standard is a separate, rubric-fit question it does not prove).
Why does a weakness high in the chain matter more than one low down?
Because trust flows downhill - a wrong rubric, biased sample or uncalibrated reviewers contaminate every score and aggregate below them, so upstream validation protects everything downstream.
What are the three distinct states of coaching, and why keep them separate?
Assigned, completed, and behaviour improved. Keeping them separate stops a completion rate being mistaken for improvement - only the recurring error shrinking next cycle proves the loop paid off.
What is the single concept people most often get wrong, and the correcting sentence?
Collapsing coverage into quality. The correction: coverage is reach, accuracy is calibration agreement, improvement is recurring-error reduction - scoring more calls is not knowing quality is good.
Verified vs demo. The screens that embody these concepts are demo-shown on seeded data. Exact controls, thresholds and figures are SME-validation-pending, and AI scoring, recording/transcription sources and dispute/calibration workflow specifics are entitlement-dependent - confirm with product.