Testing event-driven workflows without timing guesses

An acknowledgement is not a business result

A producer acknowledgement proves that Kafka accepted a fixture under the configured acknowledgement policy. It does not prove that a downstream service consumed, processed or emitted anything.

Event-driven tests must account for batching, retries, rebalances, shared traffic and variable dependency latency. Fixed sleeps hide these properties and still fail when processing takes slightly longer.

Define a bounded test contract

A trustworthy test defines the finite input, observed topics, correlation rule, timeout, record limit and assertions before publishing. dFlow~IQ uses a manual same-cluster model so every run has an explicit observation boundary.

• Fixture boundary: exact input topics, keys, headers and values.
• Observation boundary: exact output topics, timeout and record/byte limits.
• Correlation boundary: a unique run ID preserved by the application.
• Assertion boundary: required, forbidden and cardinality conditions.

Capture baseline offsets before publishing

For every output partition, capture the current end offset before fixtures are sent. Observation starts at that position, which excludes retained historical records.

Baselines do not exclude unrelated records written concurrently after capture. A unique correlation ID performs that second stage of isolation. If output partitions change during a run, the boundary is no longer reliable and the run should stop.

new record       = offset >= baseline[topic, partition]
candidate record = new record AND correlation matches
asserted record  = candidate record AND assertion applies

Propagate a unique run identity

Generate a unique value for every run and place it in a stable header such as x-test-run-id. Include it in the payload as well when downstream components do not preserve headers.

Do not correlate only by timestamps or a reused business ID. Producer clocks differ, and retained records may contain the same entity from an earlier run.

key: it-order-763ab295
headers:
  x-test-run-id: 763ab295-8c4e-4557-9f70-a243e3ef4740
value:
  {"orderId":"it-order-763ab295","totalMinor":12500}

Make assertions eventual and explicit

A missing required event fails or times out at the deadline. A forbidden event fails immediately. Exact cardinality should not pass the instant the first result arrives because a duplicate may follow.

Derive timeouts from the processing budget

Build the deadline from producer delivery, consumer poll cadence, application batching, processing, dependency latency, output production and an observation margin. Poll repeatedly within one hard deadline instead of sleeping and reading once.

Expected absence is always temporal: it means no correlated match was observable during a stated interval, not that the event can never appear.

Separate decoding, schema and business checks

• Wire decoding asks whether the bytes can be interpreted using their writer schema.
• Instance validation asks whether this record satisfies the intended schema.
• Contract identity asks whether the correct schema or acceptable compatible version was used.
• Business assertions ask whether the decoded values represent the expected result.

Run the test in dFlow~IQ

1. Open Tests and select the target workspace.
2. Add one or more UTF-8 fixtures with input topic, optional key, headers and value.
3. Add assertions for exact count, key, header, JSON field, schema validity or expected absence.
4. Set a timeout from 1 to 120 seconds and save the definition.
5. Select Preview test and review fixture bytes, topics and assertions.
6. Type the exact test name to confirm, then run.
7. Review PASS, FAIL or TIMEOUT per assertion while the report remains on screen.