Why does the same KPI produce different numbers?

Labels such as “conversion”, “active customer” and “on-time delivery” are not complete definitions. The measurement should be bounded by the management decision it is intended to support.

Identify the event being counted

A form click, a submission attempt, a CRM record and a qualified inquiry are different events. Orders and deliveries are different counting units, as are customers and contracts.

If one report counts customers and another counts contracts, trying to make their totals match will not resolve the issue. First describe what each row represents. Data consistency concerns whether records representing the same thing contradict one another. Government Data Quality Framework.

Create a short metric dictionary

Each KPI entry should contain:

• Decision: Which action would be reconsidered if the result changed?
• Definition and unit: What exactly is counted or measured?
• Numerator and denominator: Which records enter each part of a ratio?
• Time: Which date field, time zone and reporting period apply?
• Scope: Which products, teams or customers are included?
• Exclusions: How are tests, cancellations and duplicates handled?
• Source and freshness: Which system and last processing time are used?
• Owner and version: Who maintains the definition, and what changed most recently?

The dictionary needs to be accessible to the people reading the report, as well as to the person producing it.

Reconcile groups of records

Suppose report A shows 12 inquiries in a week and report B shows nine. Comparing the underlying records reveals two tests and a repeat submission about the same decision in A. That accounts for the difference of three.

Not every discrepancy will have this explanation. Build groups for “only in A”, “only in B” and “in both”, then classify the reasons for differences. If identifiers do not permit reliable matching, state the limitation. Treat uncertain matches as unresolved rather than confirmed.

Preserve the denominator and the period

Dividing this week's wins by this week's proposals may not describe conversion for the same group of opportunities. The wins could relate to older proposals. For conversion questions, follow records from the same starting group, or cohort, and show how many remain unresolved.

A zero denominator makes a ratio undefined. Missing data is also different from an observed zero. Reports should distinguish unavailable records, a measured zero result and a period whose outcomes have not yet matured.

Make definition changes visible

When a KPI definition changes, record the date and reason. If historical values can be recalculated, identify which periods use the new rule. Otherwise, mark the break in the series so that a definition change is not mistaken for a performance change.

A completed reconciliation should provide a shared definition, explained differences and a list of unresolved records. These outputs help identify the technical changes needed before commissioning another dashboard.

For a data or definition discrepancy that affects a management decision, Probe can frame an initial review. Describe the conflicting results and the decision they affect.