Design logistics event integrations for duplicates and late arrivals

A carrier reports delivery at 16:02. At 16:05, a delayed hub-departure message arrives for an event that occurred at noon. The integration overwrites the shipment status with “in transit”, and customer service sends an unnecessary escalation. A retry then creates the same escalation again.

Reliable event processing separates what happened from when the message arrived. Store the source event with its identity and provenance, then apply explicit rules to derive the operational view. Arrival order alone is not a dependable business sequence.

Keep the event log and current view separate

The event log preserves accepted source evidence. The current view answers a business question such as “what is the latest supported delivery state?” Keeping these roles separate makes it possible to investigate a result and rebuild it after a mapping change.

Define the minimum event contract: source/tenant, event identity, shipment and leg, event type, event time and timezone, receipt time, actual/estimated status, location where relevant, schema/version and correction relationship. Names and structures depend on the partner interface.

GS1 EPCIS and DCSA Track and Trace offer standardised event structures for their respective scopes. Agree supported versions and business semantics with partners rather than assuming every carrier uses the same model. GS1 — EPCIS and CBV artefacts; DCSA — Track and Trace standards.

Walk through a late event and a retry

This illustrative event log uses custom labels for explanation; it is not a conformant payload for either standard.

The derived view should remain delivered under the example's agreed rules. The departure event is useful evidence, but its later arrival does not make it a later physical event.

Now add a genuine correction from the authorised source stating that E-200 was associated with the wrong shipment. A rule that always keeps the most advanced status would fail here. The correction needs its supported relationship, authority and effect on the derived state.

Define duplicates without discarding legitimate changes

A reliable source event identifier, scoped to the source and account, is preferable to guessing from the shipment number. One shipment can have several legitimate events of the same type, including repeated attempts.

If a partner lacks a dependable event identity, agree a deduplication method and its limits. A fingerprint built from event fields may identify repeated messages, but a changed timestamp or corrected location can make a legitimate correction resemble a new event. Preserve uncertain candidates and route them instead of silently deleting evidence.

Keep business-event identity separate from transmission attempt identity. The latter helps diagnose retries; the former prevents repeated business effects.

The check must survive concurrent delivery. Two workers can both see “not processed” before either writes its result. Use a durable uniqueness rule for the source/account/event identity and make the accepted processing transition atomic with the local business change. An in-memory list or a separate check-then-write is insufficient. Retain deduplication evidence for the supported retry and replay horizon; expiry must not turn old deliveries into new business effects.

Design state transitions for the real journey

Do not reduce all events to one universal linear ranking. A delivered consignment can enter a separate return journey. A delivery attempt can fail without undoing an earlier valid event on another leg. An estimated departure can be superseded by an actual departure with a different time.

Model shipment, leg and relevant subprocess separately where needed. Specify which sources can change each fact and what happens when they disagree.

A schema-valid event can still be wrong for the business. Validate shipment ownership, units, location scope and supported event meaning before applying an effect.

Prove replay without repeating external actions

Rebuilding the current view from the event log is different from resending emails, issuing invoices or creating warehouse work. Separate state reconstruction from side-effect delivery. Replaying a month's history should not send a month's alerts again.

Use stable action identities and the destination's supported duplicate controls for external effects. Preserve whether an action was prepared, attempted, acknowledged or uncertain. When an API request times out, check the destination outcome before creating another business action.

Test recovery after processing stops between recording an event and completing its downstream work. One design option is a transactional outbox: commit the local state change and a pending-action record together, then deliver that action separately. AWS documents this pattern and the continued need for idempotent consumers. It does not remove uncertainty at an external destination. AWS Prescriptive Guidance — Transactional outbox pattern.

In a proposed recovery test, interrupt processing after the local commit but before the action is sent. The pending record must survive. Then interrupt after the destination accepts the action but before local acknowledgement is saved. Recovery must reconcile or safely repeat that same action under the destination's supported contract, without creating another one.

For webhook ingestion, acknowledge according to the agreed interface only after the input is durably retained or definitively rejected. Acknowledging success before retention risks losing the event if processing stops. Durable receipt is still distinct from business acceptance.

Make failures observable to the right owner

IT owns transport errors, unsupported schemas and processing recovery. Operations owns ambiguous business events and disputed commitments. The partner account owner resolves missing or contradictory source evidence.

Measure event delay, unsupported-message age, duplicate attempts, unresolved corrections and the time needed to rebuild a reviewed sample. Show the last valid event and its freshness to users. A healthy API connection does not prove fresh shipment information.

Recovery flow: retain input → validate authority and meaning → recognise duplicate or correction → update the derived view → deliver approved effects → reconcile outcomes. Unsupported inputs remain available for investigation with an explicit affected scope.

Recover the explanation as well as the status

A robust integration can explain why the current shipment view follows from the available events. Stable identity, explicit correction rules and controlled replay let teams recover that explanation after delays or failures.

Next actions

1. Collect real examples of duplicate, late and corrected events from an authorised source.
2. Define event identity, time semantics, ownership and supported transitions.
3. Rebuild expected views from an independently reviewed event set.
4. Test interruption between event processing and downstream acknowledgement.
5. Measure unresolved source and processing cases before broadening the integration.

For an event-recovery assessment, identify the source, affected shipment view and downstream action at risk of repetition.