Preserve machine-event evidence for a competent safety investigation

When an unexpected machine motion, guard-interlock fault, or safety near-miss occurs on a factory floor, critical digital and documentary evidence often disappears within the first thirty minutes. Well-meaning technicians cycle main cabinet power to clear a fault latch, overwriting volatile PLC diagnostic ring buffers; production supervisors close the active MES operation to reallocate work; or investigators compare raw controller timestamps against operator shift logs without realising that the machine clock has drifted several minutes away from enterprise network time.

The core operational decision for environment, health, and safety (EHS), plant engineering, and manufacturing IT teams is to implement a post-event evidence-preservation workflow that locks and time-aligns four distinct record streams—controller alarm/diagnostic logs, CMMS maintenance and parameter-change records, MES job/operator context, and witness factual statements—inside a restricted investigation case file. Crucially, the software workflow must enforce two strict boundaries: physical emergency response and energy isolation always come first, and assembling time-aligned evidence never infers root cause or authorises machine restart. Causal conclusions and return-to-service clearance remain the exclusive responsibility of competent human EHS and engineering investigators.

Why ad-hoc post-incident log collection compromises safety investigations

After a safety event or near-miss is contained on the shop floor, competent investigators need to reconstruct the exact sequence of machine states, human interactions, and prior maintenance interventions leading up to the event. In plants without a structured evidence-preservation workflow, three failure modes routinely corrupt that reconstruction:

1. Volatile controller ring buffers are overwritten during troubleshooting: Many machine tool CNCs, servo drives, and older PLCs store only the most recent 100 to 500 diagnostic events in volatile RAM or a circular FIFO buffer. If a maintenance technician resets the safety relay, acknowledges repeating alarms, or cycles main cabinet power before exporting the raw buffer, the pre-event fault transition is pushed out of memory.
2. Unsynchronised OT and IT clocks create false timeline contradictions: In accordance with operational technology segmentation practices described in NIST SP 800-82 Rev. 3 and the NIST Operational Technology Security Publications, shop-floor controllers and safety PLCs are frequently isolated from corporate IT networks and may lack active Network Time Protocol (NTP) synchronisation. When a standalone PLC clock drifts by three or four minutes over a year, comparing its raw log directly against an NTP-synchronised badge swipe or MES terminal timestamp makes events appear out of order.
3. Informal narrative summaries replace primary records and mix facts with speculation: When incident reports store only a supervisor's free-text summary instead of immutable snapshots of the open CMMS work orders, recent parameter edits, and active permit-to-work status, investigators cannot verify whether an interlock bypass or maintenance intervention was active at the time of the event. Worse, automated or AI-assisted incident tools that attempt to generate a "probable root cause" from text descriptions risk biasing the investigation and violating statutory and internal governance rules.

Under formal work-authorisation principles such as the UK Health and Safety Executive (HSE) Permit-to-Work guidance, control of hazardous plant states depends on clear written status, defined human competence, and formal handover between operating and maintenance authorities. In India, establishments must also verify their statutory incident-reporting, inquiry, and workplace-safety obligations under applicable central and state factory rules and the Ministry of Labour and Employment announcement on the implementation of the four Labour Codes from 21 November 2025. Software records support internal evidence preservation; they do not define legal compliance or replace statutory registers.

Four-stream evidence architecture and clock-offset reconciliation

To give competent investigators an accurate, tamper-evident factual baseline without altering source systems, the incident evidence workflow captures four primary record sets as read-only snapshots immediately after the physical scene is secured:

Machine safety event evidence preservation and time-alignment workflow

Follow the workflow below, or open its complete text version.

Read the complete workflow as text

1. Physical Emergency & Scene Security First → Open Restricted Incident Case & Freeze Asset
2. Open Restricted Incident Case & Freeze Asset → Capture Raw Volatile Logs & Clock Calibration
3. Capture Raw Volatile Logs & Clock Calibration → Snapshot CMMS, Isolation & MES Context
4. Snapshot CMMS, Isolation & MES Context → Build Offset-Adjusted Factual Timeline
5. Build Offset-Adjusted Factual Timeline → Competent Investigator Review & Separate Return Gate

Worked example: Reconciling conflicting timestamps on servo press PRS-SV-06

All machine identifiers, timestamps, clock offsets, and personnel codes below are illustrative examples constructed to show how explicit time-calibration resolves apparent contradictions without altering raw logs or inferring causation.

Suppose that during the morning shift on 20 October 2026, an operator at 400-tonne mechanical servo press cell PRS-SV-06 reports an unexpected 35 mm downward slide inching movement (Case INC-2026-1018) while clearing a scrap strip from the die bolster area with the mode selector key switch turned to Manual Setup and the front safety light curtain (LC-01) interrupted. No injury occurs; the operator immediately presses emergency-stop pushbutton E-STOP-02, the shift supervisor secures the cell and applies physical group lockout padlock LOTO-PRS06-91, and the EHS lead opens restricted investigation case INC-2026-1018.

1. Apparent contradiction in raw uncalibrated timestamps

When the plant engineering and EHS investigators collect the initial records from four sources, the raw unadjusted timestamps appear to contradict the operator's account:

• Operator factual statement (WIT-01): Operator OP-312 states that they interrupted light curtain LC-01 at approximately 10:14:00 IST, observed the slide creep downward a few seconds later (~10:14:05 IST), and struck E-STOP-02 immediately (~10:14:07 IST).
• MES shop-floor terminal (MES-TERM-06, NTP-synced): Logs a machine cycle-stop signal at 10:14:06 IST (±1 s) and an operator Incident Hold button press at 10:14:22 IST (±1 s).
• Main press PLC fault buffer (PLC-PRS06, non-networked standalone RTC): Logs Event #481: Light Curtain LC-01 Beam Interrupted at raw PLC time 10:17:19, Event #482: Brake Release Solenoid SOL-BRK-B Pressure Drop Warning at 10:17:23, and Event #483: E-STOP-02 Dual-Channel Open at 10:17:25.
• Local operator panel HMI alarm log (HMI-PRS06, separate internal clock): Logs Alarm A-109: Safety Relay K-SAF-01 Feedback Mismatch at raw HMI time 10:13:22 and Alarm A-110: Emergency Stop Active at 10:13:25.

If taken at face value without clock calibration, the raw PLC log (10:17:19 to 10:17:25) suggests the light curtain interruption and emergency stop occurred more than three minutes after the MES terminal recorded the stop (10:14:06), while the HMI log (10:13:22) suggests the alarm occurred 44 seconds before the MES stop.

2. Measured clock offsets and uncertainty calculation

Before touching any controller settings, the automation engineer and EHS investigator perform a documented clock-calibration check at exactly 10:45:00 IST (t_ref = 10:45:00, uncertainty ±1 s via the plant master NTP reference server NTP-PLANT-01):

1. Main PLC clock offset (Delta_PLC): At t_ref = 10:45:00 IST, the standalone PLC diagnostic screen displays 10:48:18 (reading resolution ±1 s): Delta_PLC = t_PLC_raw - t_ref = 10:48:18 - 10:45:00 = +198 seconds (+3 minutes 18 seconds fast) Combined Uncertainty u_PLC = ±1 s (NTP ref) + ±1 s (PLC display resolution) = ±2 seconds Normalized IST Time Formula: t_IST = t_PLC_raw - 198 s (±2 s)

2. Local HMI panel clock offset (Delta_HMI): At t_ref = 10:45:00 IST, the operator HMI status bar displays 10:44:18 (reading resolution ±1 s): Delta_HMI = t_HMI_raw - t_ref = 10:44:18 - 10:45:00 = -42 seconds (42 seconds slow) Combined Uncertainty u_HMI = ±1 s (NTP ref) + ±1 s (HMI display resolution) = ±2 seconds Normalized IST Time Formula: t_IST = t_HMI_raw + 42 s (±2 s)

3. Apparent raw gap between PLC and HMI clocks: Raw Clock Spread = (+198 s) - (-42 s) = 240 seconds (4 minutes 0 seconds)

3. Reconciled time-aligned factual event table

Applying the measured offsets (-198 s to raw PLC timestamps and +42 s to raw HMI timestamps) while retaining every original timestamp in its own column produces a completely consistent sequence across all four systems:

4. Preserving the boundary between factual evidence and human investigation

Notice what the reconciled table accomplishes—and what it refuses to do:

1. Contradiction eliminated without altering source logs: Once the +198 s PLC offset and -42 s HMI offset are applied, EVT-03 (10:14:01 ± 2 s, light curtain interrupted), EVT-05 (10:14:04 ± 2 s, safety relay feedback mismatch), EVT-06 (10:14:05 ± 2 s, 34.8 mm slide displacement), and EVT-08/EVT-09 (10:14:07 ± 2 s, E-Stop actuated) align within their ±2 s uncertainty bands and match MES-TERM-06 (10:14:06 ± 1 s).
2. No automated causal conclusion: Although the timeline shows that PM work order WO-PM-2026-8812 occurred the previous night (EVT-01) and that Alarm A-109 (EVT-05) preceded SOL-BRK-B pressure drop (EVT-06) by 1.0 second, the software workflow does not generate an automated conclusion blaming the night-shift technician or diagnosing a specific valve fault. Instead, the case file marks Causal_Determination_Status = UNRESOLVED_PENDING_COMPETENT_INVESTIGATION and routes the preserved evidence package to the appointed human investigation lead (EHS-INV-04) and chief mechanical engineer (ENG-MECH-02).
3. Return-to-service remains strictly blocked: Completing the evidence snapshot for INC-2026-1018 changes the evidence-collection task to Evidence Preserved (4/4 Streams Hashed), but keeps servo press PRS-SV-06 locked in OUT_OF_SERVICE_SAFETY_HOLD. Returning the press to service requires closing a separate machine safety defect record, maintaining physical maintenance isolation records during physical inspection and repair, and verifying long-term corrective actions through CAPA effectiveness testing.

Observable acceptance tests for the incident evidence workflow

Before deploying an incident evidence-preservation module across CMMS, MES, and EHS workflows, validate these three acceptance tests in a non-production sandbox:

• ACT-M053-01 (Preservation of raw timestamps alongside explicit clock offset and uncertainty): Seed a simulated incident case with three log uploads bearing unsynchronised clocks (PLC +198 s, HMI -42 s, MES 0 s). Pass condition: The database stores the exact unmodified raw timestamp string, the cryptographic SHA-256 hash of each uploaded log file, the measured clock offset Delta_t, and the uncertainty band ±u (±2 s), displaying both raw and reconciled timestamps side-by-side without overwriting the original log values.
• ACT-M053-02 (Prohibition of model-generated or automated causal conclusions): Attempt to configure or trigger an automated summary or AI text-generation field on the incident record that outputs a root-cause statement or fault attribution. Pass condition: The schema rejects any automated or model-generated causal text; the Root Cause & Contributing Factors section remains locked in Unresolved — Assigned to Named Competent Investigator until signed by the authorised human EHS and engineering leads.
• ACT-M053-03 (Decoupling of evidence completion from machine restart clearance): Mark all four evidence streams (Controller Logs, CMMS/Isolation History, MES Job Snapshot, and Factual Witness Statements) as Complete & Verified on case INC-2026-1018, then attempt to release a production work order on PRS-SV-06 in MES. Pass condition: MES blocks work-order release (Error: Asset PRS-SV-06 locked under active EHS Investigation Hold INC-2026-1018; evidence preservation does not authorise machine restart).

Next actions for EHS, engineering, and manufacturing IT leaders

To stop losing volatile machine evidence and prevent timestamp confusion after shop-floor safety events, complete these four numbered actions:

1. Add a mandatory "Do Not Power-Cycle Before Log Export" step to post-event shop-floor protocols: Update supervisor and maintenance response checklists so that—immediately after emergency aid and physical LOTO isolation are secured—volatile PLC, CNC, and safety-relay fault buffers are exported or photographed before any cabinet power-cycle or fault reset is attempted.
2. Institute a standard clock-calibration check during every machine evidence export: Require the responding automation technician to record the controller's live real-time clock display against an NTP-synchronised reference clock (Delta_t and ±u) at the moment of log extraction so investigators never compare uncalibrated timestamps across OT and IT systems.
3. Configure automatic CMMS and MES snapshot locks on safety-hold activation: Set the EHS incident trigger so that placing an asset on Safety Investigation Hold automatically freezes the prior 30 days of CMMS work orders, parameter change logs, and active MES job records against retrospective edits.
4. Enforce role-based access and ban automated causal inference in incident records: Restrict witness statements and raw log attachments to the assigned investigation team, and verify via ACT-M053-02 and ACT-M053-03 that software records never generate causal conclusions or clear an equipment safety hold.