Streaming context updates are event-driven changes that continuously transform source-system events into validated, resolved, time-stamped, lineage-complete graph updates so consumers can reason over the current enterprise situation with known freshness and quality.
Design for decision freshness, not maximum speed #
The first stage is to classify decisions by how quickly their context loses value. Fraud authorization may have a half-life measured in seconds; workforce planning may tolerate a daily refresh; legal-entity reference data may change monthly. Streaming everything raises cost and operational complexity without improving every decision. Define a freshness objective per context domain and tie it to a consumer outcome.
Create a freshness contract with source, event type, acceptable source-to-graph lag, maximum stale duration, recovery objective, and downstream decisions affected. This turns “real time” into an engineering and governance commitment rather than a slogan.
Stage 1: choose the right event source #
Change data capture is useful when existing systems cannot emit domain events. It provides comprehensive row-level change, but business meaning must be reconstructed from tables and transactions. Domain events express intent more clearly, such as OrderShipped or EntitlementRevoked, but require application ownership and reliable publication. Many enterprises use both: CDC for broad coverage and domain events for high-value workflows.
Every event needs an immutable identifier, source, entity key, event time, record time, schema version, operation, payload, and trace context. Avoid publishing database snapshots as unbounded messages; publish the smallest change that can be interpreted independently or retrieved deterministically.
Stage 2: establish contracts, partitions, and replay #
A schema registry or equivalent contract process is essential. Producers must evolve schemas compatibly, and consumers must know when a field is added, deprecated, or reinterpreted. Partition by the entity or aggregate whose ordering matters. Global ordering is expensive and usually unnecessary; local ordering is enough when the graph can reason temporally across entities.
| Dimension | Batch | Streaming | Design implication |
|---|---|---|---|
| Latency | Minutes to days | Milliseconds to minutes | Choose from decision half-life |
| Ordering | Implicit within extract | Partition-local and imperfect | Preserve event time and sequence |
| Recovery | Rerun the batch | Replay from durable log | Handlers must be deterministic |
| Schema change | Detected at run time | Continuous compatibility risk | Use versioned contracts |
| Cost | Bursty compute | Always-on operations | Stream selectively |
| Best use | Reference, reconciliation, repair | Operational state and triggers | Run a hybrid portfolio |
Stage 3: resolve, sequence, and upsert correctly #
Before an event becomes graph context, validate it against the ontology, resolve its entities, type its relationships, and stamp both event time and record time. Duplicate events should be harmless; handlers must be idempotent. Out-of-order events should update the correct temporal interval rather than simply replacing the latest value. When identity resolution is uncertain, quarantine or create a provisional entity rather than merging silently.
Use atomic graph writes for all facts that represent one business change. If a shipment event updates status, location, custody, and expected arrival, partial application creates an impossible situation. Record lineage from each graph fact back to the event and processing version.
Stage 4: publish changes without bypassing governance #
Graph subscriptions should be purpose-scoped and entitlement-aware. A consumer may subscribe to changes in supplier risk, customer status, inventory, or workforce availability, but the Context Harness still filters which entities, attributes, and inferred facts it can receive. High-volume consumers may need materialized views or compacted state topics; interactive consumers may use context APIs that assemble the current situation on demand.
Three industry patterns
Banking. Transaction, device, account, and watchlist events refresh fraud and AML context. Manufacturing. Equipment, quality, inventory, and supplier events propagate production risk. Telecommunications. Network alarms, customer-impact events, and entitlement changes update service situations in near real time.
A realistic enterprise scenario #
A telecommunications provider wants service-assurance decisions to reflect network topology, customer impact, maintenance windows, and premium service commitments within minutes of a fault.
Understand. CDC captures ticket and asset changes; domain events carry alarms and maintenance actions. Events are partitioned by network element, validated against versioned schemas, resolved to graph entities, and applied with event time, lineage, and idempotency. The graph propagates each affected dependency to services and customers.
Decide. The Decision Layer evaluates impact, contractual priority, available field capacity, and policy. It distinguishes a duplicate alarm from a new failure and explains which relationships created the customer-impact estimate.
Execute. The Execution Grid opens or updates incidents, proposes dispatch, and notifies approved channels. Outcomes return as events. Batch reconciliation runs nightly to detect missed changes and repair drift, making streaming and batch complementary rather than competing designs.
Common mistakes to avoid #
- Streaming every source without a decision-level freshness requirement.
- Assuming broker delivery guarantees remove the need for idempotent business handlers.
- Ignoring event time and overwriting state based only on arrival order.
- Using CDC payloads as the ontology instead of translating them into stable context concepts.
- Operating without replay, dead-letter handling, and reconciliation.
- Publishing graph changes directly to consumers without Context Harness enforcement.
How OpenKnowra approaches this #
The architecture above is platform-neutral. OpenKnowra operates streaming context updates as part of the Context Graph Engine: CDC and event ingestion, schema validation, entity resolution, bitemporal graph writes, lineage, subscriptions, and continuous quality scoring. The Context Harness applies purpose and entitlement policy before context reaches consumers; the Decision Layer evaluates situations; and the Execution Grid returns outcomes through the same governed event loop.
Implementation starts with one decision whose value materially depends on freshness, then adds the minimum sources, events, and recovery controls required to meet a measurable source-to-decision SLO.