Quality: rules, metrics, and how quality is ensured

What it is

Quality is the “can I trust this data?” layer of a governed data product. It includes validation rules (e.g. “order total must match sum of line items,” “email must be valid format”), completeness checks (e.g. “required fields present”), freshness metrics (e.g. “last event < 5 min ago”), and optional anomaly detection (e.g. “unusual spike in nulls”). When quality fails, the checks and results are surfaced to the product’s owners. In Loxtep, a governed data product can carry quality metadata so the context we deliver is not only rich but reliable.

How it’s made

Quality for a governed data product is defined and monitored as follows:

• Rule definition. You attach quality rules to the asset (or to specific entities/fields): e.g. “Order.total = sum(LineItem.amount),” “Customer.email matches regex,” “Payment.amount > 0.” Rules can be structural (schema, types) or semantic (business invariants). Each rule has a severity (info, warn, critical) and an optional remediation hint.
• Execution and scoring. Rules are run on ingestion (per event or batch) or on a schedule (e.g. daily over the last 24h of data). Results are stored as quality events: rule_id, pass/fail, sample of failures, count. An aggregate “quality score” (e.g. % of rules passing, or a weighted score) is computed and stored with the asset so consumers can see “this asset is 98% green” or “degraded.”
• Freshness and latency. Quality also tracks freshness: e.g. “last successful sync from Shopify at T” and “max event age = now - T.” If max age exceeds a threshold (e.g. 1 hour), the asset can be marked stale and that result is surfaced to the product’s owners.
• Surfaced to owners. Quality summary (score, freshness, last run) is part of the product’s metadata and is surfaced to owners so they can see fitness for use. The header and response shapes in examples are illustrative.

Example quality config and result for an Orders context asset:

{
  "quality": {
    "rules": [
      { "id": "q1", "name": "Order total consistency", "expr": "order.total == sum(line_items.amount)", "severity": "critical" },
      { "id": "q2", "name": "Email format", "expr": "matches(customer.email, email_regex)", "severity": "warn" },
      { "id": "q3", "name": "Payment amount positive", "expr": "payment.amount > 0", "severity": "critical" }
    ],
    "freshness": { "max_age_seconds": 3600, "alert_if_stale": true }
  },
  "quality_result": {
    "score": 98,
    "last_run": "2025-03-15T14:00:00Z",
    "rules_passed": 2,
    "rules_failed": 1,
    "failures": [{ "rule_id": "q2", "sample_count": 3, "hint": "Check email normalization" }],
    "freshness_ok": true,
    "max_event_age_seconds": 120
  }
}

In Loxtep, quality checks can be configured when you define a governed data product. Results are stored and surfaced to owners so the product’s health is visible.

Why it matters for governed data products and AI

Context that’s wrong or stale is worse than no context — it leads to bad decisions and lost trust. By making quality a first-class part of the governed data product, owners can see when the data is fit for use. In Loxtep, a governed data product can include quality rules and metrics so the context we deliver is not only complete but reliable.