Short answer: First define the decision management will make; then record field events with a unique identity, time, and responsible party. Produce indicators through fixed granularity, a data quality gate, and human approval.
Why is field data not directly an indicator value?
Every row in a tablet form is a measurable event; but on its own it is not a management answer. Opening a work order, starting a field visit, completing a checkpoint, verifying a nonconformity, and publishing the report are different business events. Aggregating them in a single "completed" field conflates speed with quality and workload with outcome.
Microsoft's star schema guidance states that fact tables store observations or events, while dimensions provide filtering and grouping. The same guidance specifically requires fact tables to be loaded at a consistent grain. The technical conclusion is clear: work order, visit, checkpoint, and report must not be counted as if they were the same row type.
A good indicator first explains which decision it supports; then which events it counts, in which time window, and by which rule.
First step: choose a single management question
Dashboard design does not start with selecting charts. First the decision question is written: "At which stage do delayed jobs accumulate?", "What is the common cause of jobs requiring repeat visits?" or "Why does report publication deviate from the planned date?" Each of these questions requires different events, scope, and a responsible party.
Section Measure 2.5 of the NIST AI RMF Playbook addresses verifying that an indicator actually measures the concept it claims to measure, in the context of construct validity. The counterpart in Vizyon AI design is not to accept the result produced by the model or rule alone as truth. The indicator should be presented together with the source records, calculation rule, usage limits, and authorized human review.
Fill in the KPI card before calculating
| Field | Question to ask | Example |
|---|---|---|
| Decision | Which action does this indicator trigger? | The planning owner changes the job distribution. |
| Event | Which record is counted? | A field visit closed by an authorized user. |
| Granularity | What does one row represent? | A single field visit within a work order. |
| Time | Which start, end, and time zone are used? | Planned start and actual close, Istanbul time. |
| Scope | Which records are included or excluded? | Cancelled jobs are reported separately from the denominator. |
| Owner | Who approves the definition and threshold? | Operations manager and data owner. |
| Evidence | Which records can the result be traced back to? | Work order, visit, and transaction history identities. |
Build the event model on an immutable backbone
Every field event should be kept with at least a unique event identity, work order identity, event type, occurrence time, recording time, responsible role, source, and status history. When a correction is needed, a new version or correction link should be established rather than silently overwriting the old event. This way, why a management indicator later changed can be explained.
OpenTelemetry's stable metrics data model treats the event at recording time, the stored time series, and the transported metric stream as separate concepts. This distinction also applies to field operations: the raw event is evidence, the metric is a view calculated with a specific window and unit, and the dashboard is the decision presentation. Counter, instantaneous state, and duration distribution must not be used as if they were the same number.
Run the data quality gate before the KPI calculation
Missing data is not low performance; delayed synchronization is not an operational delay either. ISO 8000-8:2015 defines the concepts of information and data quality and the prerequisites needed for measurement within quality management. The standard was published in 2015 and its validity was confirmed in 2022. This basis supports placing a measurable data quality gate in front of every indicator.
Microsoft Purview documentation treats data quality in separate dimensions such as accuracy, completeness, conformity, consistency, currency, and uniqueness. Rather than squeezing all of these dimensions into a single artificial score on the operations screen, make the ones critical to the decision visible:
- Completeness: The presence of mandatory field entries and evidence links.
- Uniqueness: Ensuring the same event is not counted twice due to offline synchronization or retries.
- Consistency: Work order status not contradicting visit and report statuses.
- Currency: The time of the last successful data flow and the accepted latency window.
If the data gate does not pass, the Vizyon AI result should not be displayed like a normal KPI; it should state the number of out-of-scope records, the last update time, and the need for review. This way, a manager can distinguish poor operations from weak data coverage.
How should the Vizyon AI analytics layer be constrained?
Vizyon AI can be used to find patterns over defined business events, prioritize exceptions, and order potential causes for investigation. However, a prediction or recommendation does not replace a conformity decision or a published report. In particular, conditions such as a small sample, a changing field form, a new team, or a delayed data flow should be shown alongside the result.
This approach is consistent with the govern, map, measure, and manage functions of NIST AI RMF 1.0. In technical implementation, the following trail is kept for each analytics output:
- The data window and filters used,
- The version of the indicator dictionary,
- The model or rule version,
- The time the recommendation was generated,
- The reviewing role and the decision given,
- The justification for any subsequent correction.
To further evaluate the division of duties between artificial intelligence and human approval, you can consult the artificial intelligence and human approval in inspection processes guide. If integration identities need to be protected, the transfer of data to ERP and accounting without duplicate entry article completes the technical backbone.
Choose a small, auditable scope for the first dashboard
In the first release, instead of moving the entire operation onto a single screen, pick one decision loop. For example, track planned and completed field visits, waiting reasons, and data completeness together. Verify that from every card you can drill down to the source work orders and the event list.
Pre-publication checklist
- Is each KPI tied to a single decision question?
- Are the numerator, denominator, time window, time zone, and exclusions documented?
- Is event granularity the same across all periods?
- Are missing, duplicate, and delayed data shown separately?
- Can the user drill down from a total count to the source record?
- Is the AI recommendation presented with its source, scope, and confidence warning?
- Do technical approval and the operational decision rest with an authorized person?
- When a definition or model changes, can old results be reproduced?
This check turns the dashboard from a decorative report into an auditable decision tool. To evaluate a data model and analytics flow specific to your organization, you can explore VizyonTech artificial intelligence solutions or define a pilot scope based on your current field process.
Sources
- NIST — Artificial Intelligence Risk Management Framework (AI RMF 1.0), 26 January 2023.
- NIST AI Resource Center — AI RMF Playbook, full version 30 March 2023.
- ISO — ISO 8000-8:2015, Data quality — Concepts and measuring, November 2015; confirmed in 2022.
- OpenTelemetry — Metrics Data Model, current stable model documentation.
- Microsoft Learn — Understand star schema and the importance for Power BI, last updated 30 December 2024.
- Microsoft Learn — Data quality health report in Unified Catalog, current documentation.