Measurement Framework · System Health
What gets measured
gets managed.
gets managed.
A capability without a measurement framework is a capability without accountability. This framework defines the leading indicators that detect problems early, the lagging indicators that confirm outcomes, and the domain-level KPIs that make System Health performance visible, trackable, and defensible at every governance level — from daily operations to executive review.
Indicator Architecture
Leading
Signal problems before they become failures
Lagging
Confirm outcomes and validate capability effectiveness
Composite
Synthesize signals into the System Health Index
Indicator Architecture — Leading vs. Lagging
Domain KPIs — The Four System Health Measurement Domains
Domain 01
Workflow Integrity
Monitored
Leading Indicators
SLA Trending Rate
Direction of SLA performance over the last 30 days — accelerating, stable, or declining
Stable or improving
Backlog Growth Rate
Week-over-week change in open items across workflow domains — leading signal for capacity pressure
≤ 0% week-over-week
Open Overdue Rate
Percentage of open items that have already breached their due date
≤ 5% of open items
Lagging Indicators
SLA Achievement Rate
Percentage of items closed within their defined SLA across all workflow domains
≥ 90% monthly
Average Cycle Time
Mean time from workflow initiation to closure, per domain — baseline and trend
Within domain baseline
Workflow Health Index
Composite score combining SLA, backlog, and cycle time performance across all workflow domains
≥ 80 / 100
Domain 02
Data Integrity
Monitored
Leading Indicators
Field Completeness Rate
Percentage of required fields populated across active records — by domain and system
≥ 95% per domain
Classification Accuracy Rate
Percentage of records with correct workflow type, risk category, and product classification assigned
≥ 98% per review cycle
Open Data Quality Gaps
Count of identified data quality issues awaiting remediation — leading signal for certification readiness
≤ 10 open per domain
Lagging Indicators
Data Quality Score
Composite data quality rating per domain — completeness, accuracy, consistency, and lineage combined
≥ 85 / 100
Gap Remediation Cycle Time
Average time from data quality gap identification to confirmed remediation
≤ 14 days average
Data Integrity Health Index
Composite score of data quality across all four dimensions and all active domains
≥ 80 / 100
Domain 03
User Behavior
Building
Leading Indicators
Training Currency Rate
Percentage of eQMS users with current, valid training on their assigned workflows and modules
≥ 95% at all times
Overdue Training Count
Number of users with training overdue by more than 7 days — early warning before currency breach
0 at any governance review
Access Anomaly Rate
Frequency of detected access anomalies — dormant accounts, role drift, permission escalations
0 unresolved > 5 days
Lagging Indicators
Training Compliance Rate
Percentage of required training completions achieved on time per reporting period
100% on-time compliance
Access Review Completion
Percentage of periodic access reviews completed within the defined review window
100% per review cycle
User Health Index
Composite score combining training currency, access health, and adoption rate across all eQMS users
≥ 80 / 100
Domain 04
Reporting Reliability
Building
Leading Indicators
Pipeline Success Rate
Percentage of scheduled data pipeline runs that complete successfully without error or manual intervention
≥ 98% per week
Data Freshness Age
Age of the most recently ingested data for each dashboard and report — measured in hours from source
≤ 24 hours for live views
Metric Definition Drift
Number of report metrics whose calculated definition has diverged from the approved business definition on file
0 undefined deviations
Lagging Indicators
Dashboard Uptime
Percentage of time dashboards are available and displaying current, accurate data during business hours
≥ 99% business hours
Report Accuracy Rate
Percentage of reported metrics that match source system data within defined tolerance, confirmed by periodic audit
100% at each audit
Reporting Health Index
Composite score combining pipeline success, data freshness, uptime, and metric accuracy
≥ 85 / 100
Capability-Level Indicators — Cross-Domain & Decision Confidence
Capability-Level Measurement
Indicators That Span All Four Domains
| Type | Indicator | Definition | Target | Cadence | Owner |
|---|---|---|---|---|---|
| Composite | System Health Index | Weighted composite of all four domain health scores — the single top-level indicator of overall capability health | ≥ 80 / 100 | Weekly + Monthly QMB | Capability Owner |
| Lagging | Decision Confidence Score | The percentage of data products and reports consumed by leadership that carry a High or Medium confidence tier | ≥ 85% High/Medium | Monthly QMB | Capability Owner |
| Leading | Alert Resolution Rate | Percentage of triggered health alerts resolved within their defined SLA window — leading indicator of operational health | ≥ 90% within SLA | Weekly Ops Review | Capability Owner |
| Lagging | Data Product Certification Rate | Percentage of active data products carrying a current, valid certification at any point in time | 100% current | Monthly | Data Product Owners |
| Leading | Governance Adherence Rate | Percentage of required governance reviews, triage sessions, and cadence meetings held on schedule | 100% cadences held | Monthly | Capability Owner |
| Composite | Capability Maturity Score | The current maturity level of the capability on the defined maturity model — advancing from Reactive through Predictive | L3 Managed by Q4 2026 | Quarterly | Capability Owner |
Measurement Catalog — Governed Records for All System Health Measurements
SH-M-001
Workflow Integrity
Active
Composite
Leading
Business Context
Business Definition
Measures the operational reliability and execution health of critical eQMS workflows — confirming that quality processes are executing within defined time, volume, and compliance parameters.
Business Purpose
Provides early warning of workflow degradation before it becomes a compliance issue, operational backlog, or leadership concern. Enables proactive intervention rather than reactive remediation.
Business Question Answered
"Are our quality workflows operating as expected — on time, within SLA, and without systemic backlog?"
Decision Supported
Process owner escalation decisions · Governance triage prioritization · QMB scorecard reporting · Inspection readiness assessment
Technical Specification
Calculation Logic / Business Rules
Placeholder — Author Calculation Here
Composite score combining: SLA Achievement Rate (%) + Backlog Growth Rate (%) + Open Overdue Rate (%) — weighted per domain priority.
Specific weights, thresholds, and domain inclusions to be defined by Capability Owner and approved via governance review.
Data Source
eQMS platform — workflow completion records, SLA timestamps, open item queue; ingested via BI data pipeline
Refresh Frequency
Daily automated ingestion · Weekly operational review · Monthly QMB scorecard
Thresholds
Target: ≥ 80 / 100 composite · Caution: 65–79 · Breach: < 65 — triggers Tier 2 escalation
Target State
Composite Health Index ≥ 80 sustained over rolling 90 days · Automated alert routing live for all workflow domains
SH-M-002
Data Integrity
Active
Composite
Leading
Business Context
Business Definition
Measures the accuracy, completeness, consistency, and lineage integrity of quality data across all eQMS domains — confirming that data entering reporting and decision workflows meets defined quality standards.
Business Purpose
Prevents decisions being made on incomplete or inaccurate data. Provides the documented evidence of data integrity required by 21 CFR Part 11, ALCOA+, and GxP data governance standards.
Business Question Answered
"Can leadership trust the data underpinning this report — and is that trust documented and auditable?"
Decision Supported
Data product certification decisions · Decision Confidence scoring inputs · Regulatory submission readiness · Quality risk assessments
Technical Specification
Calculation Logic / Business Rules
Placeholder — Author Calculation Here
Composite score combining: Field Completeness Rate (%) + Classification Accuracy Rate (%) + Open Data Quality Gap Count (inverse) — weighted per ALCOA+ dimension priority.
Lineage documentation completeness to be added as dimension once Data Product Catalog is operational.
Data Source
eQMS data extracts · Data quality rule engine · Data Product Catalog entries · Lineage documentation repository
Refresh Frequency
Per ingestion cycle (daily) · Gap resolution tracked continuously · Monthly certification review
Thresholds
Target: Data Quality Score ≥ 85 / 100 · Caution: 70–84 · Breach: < 70 — certification hold triggered
Target State
Data Quality Score ≥ 85 sustained · Lineage documentation complete for all active data products · Gap remediation ≤ 14 days average
SH-M-003
User Behavior
Building
Composite
Leading
Business Context
Business Definition
Measures the human dimension of system health — whether users are properly trained, actively using the system as intended, and whether access and permissions reflect current organizational needs.
Business Purpose
Ensures that the quality system is operated by qualified, trained individuals with appropriate access — the human governance layer that underpins all other system health dimensions. Directly addresses 21 CFR Part 11 access control and training requirements.
Business Question Answered
"Are the right people doing the right things in the right systems — and are they properly trained and authorized to do so?"
Decision Supported
Training compliance reporting · Access review decisions · Inspection readiness for personnel qualification · GxP audit preparation
Technical Specification
Calculation Logic / Business Rules
Placeholder — Author Calculation Here
Composite score combining: Training Currency Rate (%) + Access Anomaly Rate (inverse) + Adoption Rate (%).
Weighting and specific anomaly classification rules to be defined. LMS-eQMS integration required for full automation.
Data Source
LMS platform — training records and currency data · IAM system — access logs, role assignments · eQMS — user activity and adoption metrics
Refresh Frequency
Training currency: daily pull from LMS · Access anomalies: real-time alerting (target) · Adoption: weekly aggregate
Thresholds
Target: User Health Index ≥ 80 / 100 · Training Currency ≥ 95% · Breach: Any unresolved access anomaly > 5 days
Target State
User Health Index ≥ 80 sustained · Training currency automated alert routing · Access reviews completed 100% on schedule
SH-M-004
Reporting Reliability
Building
Composite
Lagging
Business Context
Business Definition
Measures the integrity and reliability of the reporting layer — confirming that the dashboards, reports, and data pipelines delivering intelligence to leadership are current, accurate, and functioning as designed.
Business Purpose
Ensures leadership is making decisions on accurate, current data. A reliable reporting layer is the final quality gate before intelligence reaches executive consumers — if it fails, all upstream health signals are invisible or misleading.
Business Question Answered
"Is the reporting infrastructure delivering accurate, current intelligence — or are leadership decisions being made on stale or incorrect data?"
Decision Supported
Executive dashboard trust decisions · Data product certification · Decision Confidence tier assignment · QMB scorecard validity
Technical Specification
Calculation Logic / Business Rules
Placeholder — Author Calculation Here
Composite score combining: Pipeline Success Rate (%) + Dashboard Uptime (%) + Data Freshness compliance (%) + Report Accuracy Rate (%).
Metric definition audit process and tolerance thresholds to be defined. Pipeline monitoring tooling selection pending.
Data Source
BI platform pipeline logs · Dashboard uptime monitoring · Data freshness timestamps · Periodic metric accuracy audits
Refresh Frequency
Pipeline success: per-run logging · Uptime: continuous monitoring · Freshness: per dashboard view · Accuracy: periodic audit schedule
Thresholds
Target: Reporting Health Index ≥ 85 / 100 · Pipeline success ≥ 98% · Dashboard uptime ≥ 99% business hours
Target State
Reporting Health Index ≥ 85 sustained · All metric definitions certified and versioned · Pipeline health alerts automated and routed
SH-M-005
Decision Confidence
Defined
Composite
Lagging
Business Context
Business Definition
A quantified, traceable measure of trust in any quality signal — synthesizing the health status of all four System Health domains into a scored tier (High / Medium / Conditional / Low) that leadership uses to calibrate decision confidence before acting.
Business Purpose
Transforms the abstract question of "can we trust this data?" into a governed, auditable score. Eliminates the risk of leadership making high-stakes quality decisions without visibility into the data's reliability status.
Business Question Answered
"How confident should leadership be in this quality signal — and exactly which dimension is limiting that confidence right now?"
Decision Supported
All executive quality decisions · Regulatory submission readiness · QMB agenda prioritization · Investment and resource decisions
Technical Specification
Calculation Logic / Business Rules
Placeholder — Author Calculation Here
Scoring inputs: Layer 1 (System Health Index) + Layer 2 (Data Readiness Score) + Layer 3 (Certification Status) + governance completeness.
Tier assignment rules: High = 85–100 | Medium = 65–84 | Conditional = 40–64 | Low = 0–39.
Weighting of each layer to be defined and approved via governance review.
Data Source
Aggregated from: System Health Index scores (SH-M-001 through SH-M-004) + Data Product Catalog certification status (SH-M-006) + Governance adherence records
Refresh Frequency
Calculated per report generation · Displayed at point of use on every executive report and dashboard · Monthly aggregate for QMB scorecard
Thresholds
Target: ≥ 85% of reports at High / Medium tier · Caution: Any Conditional-tier report presented to leadership without disclosure · Breach: Low-tier report reaching leadership without escalation
Target State
Decision Confidence scoring live on all leadership reports · Tier displayed at point of use · Drill-down to limiting domain available on demand
SH-M-006
Data Product Certification
Defined
Lagging
Governance
Business Context
Business Definition
Measures the certification coverage and currency of all governed quality data products — confirming that every data asset consumed by leadership carries a current, valid certification issued by its named Data Product Owner.
Business Purpose
Ensures the governance gate separating raw data from decision-ready intelligence is functioning. An uncertified or expired data product cannot be consumed at Decision Confidence tier above Conditional — this metric enforces that boundary.
Business Question Answered
"Are all active data products currently certified and owned — and has the certification been maintained without lapse?"
Decision Supported
Decision Confidence tier assignment (Layer 3 input) · Executive report release decisions · Audit and inspection evidence · Data governance compliance
Technical Specification
Calculation Logic / Business Rules
Placeholder — Author Calculation Here
Certification Rate (%) = (Active data products with current certification / Total active data products) × 100.
A certification is "current" if: sign-off is within the defined renewal window, Data Product Owner is named and active, and business definition version matches the current published definition.
Renewal window and version matching rules to be defined.
Data Source
Data Product Catalog (SharePoint / Dataverse — in development) · Certification sign-off records · Data Product Owner registry · Business definition version log
Refresh Frequency
Certification status: real-time from Catalog · Expiry alerts: 30 / 14 / 7 day advance warnings · Monthly certification summary for QMB
Thresholds
Target: 100% of active data products certified at all times · Breach: Any uncertified product consumed by leadership — triggers immediate escalation to Capability Owner
Target State
Data Product Catalog live with real-time certification status · Automated expiry alerts · 100% certification coverage sustained
Measurement Commitments
Governing Principles · Measurement Framework
Four commitments that keep measurement honest
01
Measure What Matters
Every KPI in this framework is directly traceable to a business outcome — decision confidence, compliance assurance, or operational efficiency. Metrics that do not connect to outcomes are removed at annual review.
02
Leading Before Lagging
The operating priority is leading indicator performance. If the leading indicators are healthy, the lagging indicators will follow. Chasing lagging indicators alone is reactive management — it reports on what already happened.
03
Every Metric Has an Owner
No KPI in this framework is orphaned. Every indicator has a named owner who is accountable for its performance, its target, and its escalation when the threshold is breached. Unowned metrics are ungoverned metrics.
04
Targets Are Reviewed Annually
Targets are set to challenge the capability toward the next maturity level — not to confirm the current state. Annual review recalibrates targets based on maturity advancement, operational learning, and strategic direction from Quality Leadership.