Quality and Call QA: Scenarios & Checklists
Objective. Open Scenarios & Checklists and read it as the definition of 'good' - the rubric every downstream score inherits - so you can judge whether a customer's scores mean what they think.
Scenarios and checklists encode what a good interaction looks like. This screen answers 'what exactly are we scoring against?' - and because every call, queue, dispute and analytic inherits it, it is the most consequential screen in the app.
See it in the demo
What this screen answers
This is where the rubric lives: the scenarios and checklist items that define quality for this operation. It answers 'what does good look like, and is that written down consistently?' Sales uses it to show that scoring is not one reviewer's opinion but a shared standard; a CSM uses it to validate that the rubric actually reflects how this customer defines quality before trusting any score built on it.
The task
Open the scenarios or checklist for the work type in question and read the items as the standard against which interactions are judged. The task is to check fit and clarity: do the items match this customer's real definition of quality, are they unambiguous enough that two reviewers would read them the same way, and do they cover the compliance or handling points that matter. This is a validation task, not a scoring task.
How to read it
The classic misread is treating the rubric as neutral or universal. It is not - a score only ever means what the checklist encodes. If the rubric does not match how this customer defines quality, every downstream score is measuring the wrong thing, no matter how high coverage or calibration looks. Read the checklist as the denominator of 'good': change an item and you change the meaning of every score that inherits it. Ambiguous items are the root of most reviewer disagreement, so clarity here is what makes calibration achievable at all.
Honest limits
This screen defines the standard but does not prove it is applied consistently - that is calibration - nor that it is the right standard for the customer, which needs the customer's own sign-off. Confirm with product which scenario and checklist structures, weightings and auto-fail rules are configurable and entitled. Never present a demo rubric as the customer's agreed definition of quality.
Interpretation check
The rubric is not neutral - a score means only what the checklist encodes, so a rubric that does not match the customer's definition of quality makes every downstream score measure the wrong thing.
In the field
KnackGlobal (RCM): the checklist must encode the documentation and compliance points that carry real exposure - a rubric that skips them produces reassuring scores over an unmanaged risk.
Practical task
Open one scenario checklist in the demo and write a validation note: two items that are clear, one item that is ambiguous and could split two reviewers, and one check you would add for a compliance-heavy operation - then state why changing one item changes the meaning of every score.
Self-check
Why is this the most consequential screen in the app?
Because every call, queue, dispute and analytic inherits the rubric. If the definition of good is wrong or ambiguous here, all downstream scores are measuring the wrong thing or measuring it inconsistently.
A customer has high coverage and calibration but still distrusts the scores. Where do you look?
At the rubric fit - whether the scenarios and checklist actually match how this customer defines quality. Consistency against the wrong standard is still the wrong answer.
Why do ambiguous checklist items undermine calibration?
Because two reviewers read an ambiguous item differently, so agreement is impossible to achieve. Clarity in the rubric is what makes calibration attainable.
Can you present the demo rubric as the customer's standard?
No. The demo shows the mechanism; the customer's agreed definition of quality needs their own validation and sign-off before any score built on it is credible.
Verified vs demo. Screen is live in the demo at /#/scenarios; exact controls and figures are SME-validation-pending.