Golden set and calibration
Mark reference calls with the score you expect, re-score them against a rubric, and read the calibration report and fault-precision statistics.
Before an automated grade drives coaching or compliance decisions, you should know how far it sits from your own judgement. VoiceLens does this with a golden set, a small collection of calls you have scored by hand, and a calibration run that re-scores them with any rubric version and reports the difference.
Build a golden set
Pick representative calls
Choose 10 to 30 completed calls that span the range: clearly good, clearly bad, and the ambiguous middle. Include the call types your rubric branches on. Avoid calls with very poor line quality; they measure the line, not the rubric.
Score each one by hand
Decide, with your reviewers, what health score each call deserves on the 0 to 100 scale. Agree on the bands first (80 and above healthy, 50 to 79 needs work, below 50 critical); precise numbers matter less than consistent ones.
Mark them as golden
On each call, choose Mark as golden, enter the expected score and an optional note (up to 500 characters) explaining why. Members and above can do this. A workspace can hold up to 100 golden calls.
Run a calibration
Open the Golden set page and choose Calibrate. Pick either a saved template version or the unsaved draft you are editing. VoiceLens re-scores every golden call with that rubric, deterministically and without any generative enrichment, and produces a report. Nothing is saved to the calls; calibration is a what-if. Running it requires the Admin role.
Read the report
| Metric | Meaning | What to look for |
|---|---|---|
| Mean absolute error (MAE) | Average distance between predicted and expected scores. | Under 10 points is a well-fitted rubric for most teams. |
| Mean signed error (bias) | Average of predicted minus expected. | Positive means the rubric is lenient, negative means harsh. Fix with thresholds and weights before touching individual detectors. |
| Within tolerance | Share of calls predicted within ±10 points of the expected score. | The number to report upward. |
| Correlation | Pearson correlation between predicted and expected. | Tells you whether the rubric ranks calls in the same order you do, even if the scale is off. |
Below the summary, each golden call shows predicted versus expected, so you can open the outliers and see which fault or slot caused the gap.
Iterate
Typical adjustments, in order of impact:
- Weights: if capture-heavy calls are systematically over- or under-scored, move
captureWeightandcoverageWeight. - Thresholds: latency and dead-air thresholds change how many warnings fire on every call.
- Detector toggles: a detector that misfires on your call type can be disabled per rubric.
- Slots and questions: mark fewer slots as required, or add synonyms so captures are recognised.
- Phrases: loosen required phrases that your agents say in different words.
Re-run calibration after each change; it takes seconds and does not touch live scores until you save.
Fault precision
Calibration checks the score. Fault precision checks the detectors. Whenever a reviewer marks a fault as confirmed or false positive on any call, VoiceLens tracks the false-positive rate per fault code (and per language) and shows it on the fault precision dashboard, also available at GET /api/analytics/fault-precision.
A code with a high false-positive rate on your calls is a candidate for adjustment or disabling. False positives are also excluded from the health score of the call they were marked on, so reviewing faults improves both the statistics and the scores.
When to recalibrate
- After any rubric edit that changes weights, thresholds or required slots.
- When you add a new agent type (a new bot version, a new team) whose calls look different.
- Quarterly, with a few fresh golden calls, to catch drift in how callers talk.