Vord

Enhancement Proposal

Proposal: Observability — telemetry, SLOs, and the security-freshness view

§8.3 names three observability planes conceptually, but there is no instrumentation decision, no SLOs, and no operational/security view. This specs them within the existing architecture: OpenTelemetry is a dependency choice under the ADR-0008 criteria (not new architecture), and the one hard constraint — edge telemetry must not touch the hot path — is inherited from the constitution (rule 1) and the no-core-handle Vör contract (ADR-0010), not invented here.

Motivation

For an identity system, observability is also a security control: the security/freshness plane is how an operator answers "is revocation propagating, are keys fresh, is someone attacking right now?" — the Vör analog of Wyrd's "is my data meeting its durability policy?". Without it, the threat model's detect leg (ADR-0027) is blind, and the scale claims (write-rate, revocation lag) are unmeasured.

Design

  • Instrumentation: OpenTelemetry (traces, metrics, logs) — vendor-neutral and backend-agnostic (Prometheus/Grafana/Tempo/Loki/Datadog/… with no lock-in), CNCF / Apache-2.0, passing the ADR-0008 control-resilience test. A dependency choice, not a new runtime component.
  • Telemetry obeys the constitution. Edge (Vör) instrumentation is async, best-effort, and never a hot-path core read — a verification never blocks on, waits for, or fails because of telemetry export (entailed by rule 1 and ADR-0010). Edge export is fire-and-forget, batched, and sampled.
  • The three planes, concretely (§8.3):
    • Request plane — RED (rate / errors / duration) per endpoint and plane: authorize, token, verification, and authorization-decision latencies.
    • Security / freshness plane (the differentiator's telemetry): revocation-propagation lag, key-rotation lag, edge-cache (JWKS / status-list) staleness, failed-verification rate (a spike = attack or misconfiguration), login-failure rate (spike = credential stuffing), MFA challenge/abandon rates, status-list size.
    • Capacity plane — per-tenant metadata-op-rate (the binding write-scaling signal, Wyrd ADR-0018 §4), issuance write rate, directory size, Forseti query rate.
  • SLOs / SLIs as first-class (extending the §10 quality scenarios with budgets): verification latency (p99), revocation-propagation lag (V2), issuance write latency (V3/V4), verification-path availability through a core outage (ADR-0010), key-rotation lag (V5). The SLI set is fixed now; the budgets are set during implementation.
  • Distributed tracing across the planes: a trace spans relying-party → Modgud → core / Forseti (issuance) and, separately, relying-party → Vör (verification) — the trace context travels in the request, never via a core lookup, so tracing introduces no hot-path core dependency.
  • Structured logging: JSON with correlation / trace IDs, and no secrets, tokens, or PII in logs or telemetry — the canonical no-secrets rule lives in proposal 0004 (secret handling; privacy §12.8); this proposal applies it to telemetry, enforced by the CI check.
  • The security / freshness dashboard — the "are we secure right now?" board (revocation lag, key freshness, failed-verification and login-failure spikes, edge-cache staleness): the operator's at-a-glance posture and the alerting trigger source.
  • Alerting on the security/freshness SLIs — revocation-lag breach, key-rotation lag, a failed-verification or login-failure spike (which also feeds the future risk-based / adaptive-auth decision, §12.8), edge-cache staleness — the human side of the abuse-case loop (ADR-0027).
  • Tenant-scoped observability: per-tenant metrics for the operator and, tenant-isolated (ADR-0012), tenant-scoped series (a tenant's own login success/failure, MFA adoption, active users). Settled (formerly an open question of v1 vs roadmap): this proposal owns metric/event emission, including the tenant-scoped series; the presentation and reporting surfaces over them — dashboards and tenant-facing views via the admin API (ADR-0021) — are proposal 0018's.
  • Audit vs operational telemetry are distinct. The identity event stream (ADR-0022) is the durable, replayable audit / compliance / billing record (streamed out via proposal 0009); OTel metrics/traces are the ephemeral ops view. Same events, different consumers and retention.

Alternatives considered

  • A single vendor's agent — rejected: lock-in, fails the ADR-0008 control-resilience test.
  • Logs-only — rejected: no metrics/traces, so no SLOs and no distributed tracing.
  • Synchronous edge telemetry — rejected: it would put work on the verification hot path and violate rule 1.

Graduation criteria

OpenTelemetry traces/metrics/logs are emitted from every plane; the security/freshness dashboard and alerting are live; per-tenant metrics exist; SLOs are defined with budgets; a "no PII/secrets in telemetry" check passes; and a telemetry-export failure neither fails nor slows verification (an abuse/chaos test proving the edge stays non-blocking).

Backward compatibility

OpenTelemetry is a dependency adoption under ADR-0008; the hot-path constraint is inherited from rule 1 / ADR-0010; SLOs extend §10. No new architecture. (If a binding instrumentation-and-SLO decision is wanted, as Wyrd records in its own telemetry/OTel ADRs, that would be an ADR — but it is not architecturally required here.)

Open questions

  • Default SLO budgets — deferred to measurement.
  • Edge-trace sampling strategy at hyperscale.