Condition monitoring: check data quality before promising prediction

Topics: condition monitoring data quality, sensor data quality maintenance, condition data provenance, predictive maintenance readiness

Why predictive maintenance initiatives fail before the first model is trained

Refinery and offshore reliability teams face relentless commercial pressure to deploy artificial intelligence and predictive maintenance platforms across centrifugal compressors, gas turbines, multiphase export pumps, and fin-fan coolers. Yet across dozens of downstream and upstream deployments, over 70% of condition monitoring analytics projects stall within six months of launch. Operators and rotating equipment engineers stop opening the analytics dashboard because 8 out of 10 generated alerts turn out to be ghosts.

When you trace a false "imminent bearing failure" alert back to its source in the process historian (OSIsoft PI, Honeywell PHD, or Aspen InfoPlus.21), the failure is almost never in the math of the algorithm; it is in the unverified telemetry and maintenance data fed into it:

1. Aggressive historian exception and swing-door compression: To save storage bandwidth ten years ago, OT administrators configured the historian interface with wide exception and compression deadbands (for example, ±1.5 °C on bearing metal RTDs or 60-second averaged scan rates on radial vibration transmitters). The subtle, high-frequency variance that precedes spalling or rub is stripped out at the historian ingestion layer before the analytics server ever sees the data.
2. Missing operating-state and process-load context: A standby boiler feedwater pump (P-1102B) is started for its weekly 30-minute circulation test, or a recycle compressor transitions through surge-control recycle during a unit turndown. Because the analytics pipeline ingests vibration and temperature tags without binding the motor breaker status (RUN_STATUS), suction throttling valve position, or process fluid density, normal startup thermal expansion and off-design recycle turbulence trigger high-severity degradation alarms.
3. Flatlined, out-of-range, and uncalibrated sensors misread as machine health or fault: A loose thermocouple junction produces intermittent 250 °C spikes followed by a frozen -40 °C bad-value clamp, or an accelerometer cable ground loop injects 50 Hz electrical noise. Without an upstream sensor-health filter, the model flags a catastrophic mechanical defect instead of generating a simple instrument loop check notification.
4. Unclean CMMS work order labels for supervised training: Data scientists pull five years of SAP PM or Maximo work orders to label "historical failure events." Because work orders were closed months after the actual repair or booked against the parent skid Functional Location rather than the specific pump serial number, the training window labels healthy post-overhaul operation as "pre-failure degradation."

Checking and remediating condition monitoring data quality before promising prediction protects engineering credibility and separates true mechanical distress from instrument and historian artefacts.

Architecture of the condition monitoring data quality gate

In alignment with NIST SP 800-82r3 (securing one-way OT-to-IT telemetry replication across the Purdue Level 3.5 DMZ) and ISO 14224:2016 data quality management, all historian and edge sensor streams must pass through a deterministic Telemetry Quality & Context Gate before reaching any statistical or machine-learning analytics model.

+-----------------------------------------------------------------------------------+
|          CONDITION MONITORING DATA QUALITY & CONTEXT FILTER ARCHITECTURE          |
+-----------------------------------------------------------------------------------+

  [Purdue Level 1/2: Field & Protection]
  - API 670 Machinery Protection Rack (Hardware Trip Authority - UNTOUCHED)
  - DCS / PLC Process & Motor Breaker Tags
         |
         | (One-Way Unidirectional Gateway / Level 3.5 Historian Mirror - NIST 800-82r3)
         v
  +---------------------------------------------------------------------------------+
  | TELEMETRY DATA QUALITY GATE (5-Stage Pre-Analytics Validation)                  |
  |                                                                                 |
  |  Stage 1: Tag-to-Asset Binding  -> Matches PI Tag to Installed Equipment Serial |
  |  Stage 2: Clock & Scan Fidelity -> Checks UTC skew (<1s) & deadband compression |
  |  Stage 3: Sensor Health Filter  -> Detects flatline, NaN, Bad_Q, loop noise     |
  |  Stage 4: Operating State Mask  -> Masks STOPPED, STARTUP_TRANSIENT (<45m),     |
  |                                    and OFF_DESIGN_RECYCLE states                |
  |  Stage 5: Calibration Linkage   -> Checks active instrument cal status in CMMS  |
  +---------------------------------------------------------------------------------+
         |                                                       |
         | (Passes All 5 Quality Gates)                          | (Fails Stage 2, 3, or 5)
         v                                                       v
+----------------------------------------+      +-----------------------------------+
| ROTATING MACHINERY ANALYTICS ENGINE    |      | INSTRUMENT / OT DATA DEFECT QUEUE |
| - Evaluates steady-state mechanical    |      | - Routes sensor flatline, loop    |
|   residuals (vib, bearing ΔT, seal P)  |      |   fault, or historian compression |
| - Routes confirmed mechanical anomalies|      |   ticket to Instrument / OT Team  |
|   to Rotating Equipment Engineer       |      | - Suppresses false machine alarm  |
+----------------------------------------+      +-----------------------------------+

Five quantitative readiness criteria before enabling predictive models

Before any rotating equipment train is enrolled in an automated condition-monitoring or predictive analytics programme, its historical and live telemetry streams must pass five quantitative checks:

1. Deterministic Tag-to-Serial Lineage (100% Mapped): Every vibration, temperature, speed, and process load tag must map via the master equipment hierarchy to the exact sub-unit slot (Drive-End Bearing, Non-Drive-End Bearing, Outboard Seal) and include swap timestamps (Date_Installed, Date_Dismantled) so historical training windows never cross unrecorded equipment swaps.
2. Sampling Resolution and Deadband Fidelity (Compression Ratio <= 5:1 for dynamic tags): High-criticality continuous tags must be polled at 1-second to 5-second intervals with historian exception/compression deadbands narrow enough (<= 0.5% of span) to preserve natural operating micro-variance. If a tag sits frozen for hours due to excessive compression, it is unfit for early-warning residual modelling.
3. Sensor Signal Validity (>= 98.5% Good Quality Frames): The stream must be audited for OPC Bad_Quality flags, out-of-physical-range clamps (< -20 °C or > 300 °C on babbitt bearings), zero-variance flatlines (StdDev = 0.000 over 6 running hours), and timestamp clock skew (<= 1,000 ms across DCS and vibration rack historians).
4. Explicit Operating-Regime State Machine: The dataset must include a reliable running-state indicator (STOPPED, STARTUP_WARMUP, STEADY_STATE_NORMAL, TURNDOWN_RECYCLE, COASTDOWN). Analytics thresholds must evaluate STEADY_STATE_NORMAL windows separately from transient regimes.
5. Verified Maintenance & Overhaul Event Anchors: At least 12 to 24 months of operating history must be cross-checked against CMMS work orders to mark exact timestamps of bearing replacements, alignment adjustments, lube-oil top-ups, and process unit turnarounds.

Telemetry quality audit schema

Below is the automated JSON readiness scorecard generated by the Telemetry Data Quality Gate for each rotating equipment train prior to model onboarding:

{
  "audit_id": "CMDQ-2026-GCU-K3101A",
  "equipment_unit": "REF-JAM-01-GCU-COMP-K3101A",
  "installed_serial_id": "EQ-COMP-3101A-C02",
  "audit_window_days": 180,
  "ot_security_zone": "PURDUE_L3_5_READONLY_REPLICA",
  "telemetry_stream_checks": [
    {
      "parameter": "COMP_DE_RADIAL_VIB_X_UM",
      "historian_tag": "31XI_3101A_DEX.PV",
      "mean_sample_interval_sec": 2.0,
      "compression_deadband_pct": 0.25,
      "good_quality_frame_pct": 99.6,
      "flatline_hours_detected": 0.0,
      "utc_clock_skew_ms": 140,
      "calibration_status": "VALID_CAL_CERT_2026_0412",
      "stream_verdict": "PASS_ANALYTICS_READY"
    },
    {
      "parameter": "COMP_NDE_THRUST_BRG_TEMP_1_C",
      "historian_tag": "31TI_3101A_THR1.PV",
      "mean_sample_interval_sec": 185.0,
      "compression_deadband_pct": 3.50,
      "good_quality_frame_pct": 88.2,
      "flatline_hours_detected": 312.0,
      "utc_clock_skew_ms": 140,
      "calibration_status": "EXPIRED_OR_UNLINKED",
      "stream_verdict": "FAIL_EXCESSIVE_COMPRESSION_AND_FLATLINE",
      "remediation_action": "Reduce PI compression exception to 0.2 C; inspect RTD junction box JB-3104"
    }
  ],
  "operating_regime_coverage": {
    "breaker_status_tag_linked": true,
    "suction_flow_tag_linked": true,
    "steady_state_hours_in_window": 3980.0,
    "startup_transient_hours_masked": 44.5,
    "stopped_standby_hours_masked": 295.5
  },
  "overall_train_readiness": "HOLD_REMEDIATE_SENSOR_AND_COMPRESSION"
}

Illustrative example (hypothetical scenario): Auditing 2,400 telemetry tags across 120 critical rotating trains

Illustrative scenario notice: All site volumes, defect counts, error rates, engineering hours, and INR (₹) figures below are hypothetical scenario assumptions constructed to demonstrate the control math and workflow gating. Any time or cost reductions represent modeled process/capacity estimates under these assumptions, not achieved client benchmark savings or guaranteed cash realization.

Consider a petrochemical and gas cracker complex preparing to onboard 120 critical rotating equipment trains (centrifugal compressors, steam turbines, reactor charge pumps, and extruder gearboxes) comprising 2,400 process historian tags into a plant-wide condition monitoring analytics platform.

Baseline state (Raw historian ingestion without pre-analytics quality filtering)

During an initial 4-week un-gated trial where all 2,400 raw historian tags were streamed directly into statistical anomaly detection rules:

• Telemetry defect audit across 2,400 tags:
  • 192 tags (8.0%) suffered from severe historian deadband compression (> 3% span) or >15-minute scan intervals that stepped over transient mechanical spikes.
  • 144 tags (6.0%) exhibited intermittent RTD wiring spikes, ground-loop noise, or frozen flatlined values (Bad_Quality not stripped).
  • 96 tags (4.0%) were mapped to the wrong standby train or lacked a running-state motor breaker mask.
  • In total, 432 out of 2,400 tags (18.0%) failed basic ISO 14224:2016 fidelity and context checks.
• False-positive alert burden: Because those 432 defective or unmasked tags fed directly into the anomaly engine, the system generated 310 alerts per week across the 120 trains.
• Engineering investigation waste: Rotating equipment engineers and instrument specialists spent an average of 0.75 hours (45 minutes) investigating each alert (opening historian trends, checking shift logs, walking to the field), only to discover that 282 alerts per week (91.0%) were caused by startup transients, standby motor tests, or RTD/transmitter glitches—while only 28 alerts per week (9.0%) reflected genuine mechanical or process-duty deviations.
  • Weekly wasted engineering hours: Weekly_Wasted_Hours = 282 false_alerts * 0.75 hours = 211.5 engineering_hours/week
  • Annualised wasted engineering capacity (50 weeks/year): Annual_Wasted_Hours = 211.5 hours/week * 50 weeks = 10,575 engineering_hours/year
  • At a fully burdened rotating/reliability engineering rate of ₹2,000/hour: Annual_False_Alert_Labour_Waste = 10,575 hours * ₹2,000/hr = ₹2,11,50,000/year (₹2.115 Crore/year)

Controlled deployment of the 5-stage Telemetry Data Quality Gate

Before rolling out the analytics workspace to operations, the reliability and OT engineering teams implemented the 5-stage Data Quality Gate:

1. Corrected historian exception/compression deadbands on the 192 over-compressed tags and synchronized NTP clocks across the vibration rack and process DCS.
2. Configured the Sensor Health Filter to intercept the 144 noisy/flatlined sensor loops and route them automatically as low-priority instrument calibration notifications rather than rotating machinery alarms.
3. Bound motor current / breaker status (RUN_STATUS) and a 45-minute post-startup thermal stabilisation timer across all 120 trains.

Post-gate operational and financial results

• Defective/unmasked tags reaching the mechanical analytics engine: Reduced from 432 tags (18.0%) to 12 tags (0.5%) (isolated to sensors awaiting shutdown access for repair, automatically masked from mechanical scoring).
• Weekly false-positive mechanical alerts: Dropped from 282 false alerts/week to 14 false alerts/week (95.0% reduction in noise), while all 28 genuine mechanical/process deviations/week continued to be captured (66.7% positive predictive precision vs 9.0% baseline).
• Remaining weekly wasted investigation hours: Controlled_Weekly_Wasted_Hours = 14 false_alerts * 0.75 hours = 10.5 engineering_hours/week Controlled_Annual_Cost = 10.5 hours/week * 50 weeks * ₹2,000/hr = ₹10,50,000/year (₹10.50 Lakh/year)
• Net annual engineering capacity and cost recovery: Net_Hours_Saved = (211.5 - 10.5) * 50 weeks = 10,050 reliability_engineering_hours/year Net_Annual_Financial_Recovery = ₹2,11,50,000 - ₹10,50,000 = ₹2,01,00,000/year (₹2.01 Crore/year)

Acceptance criteria for condition monitoring data quality gates

Before allowing any rotating equipment train to generate automated advisory notifications in the CMMS, verify these four acceptance criteria:

Governance boundaries, OT cybersecurity, and cross-functional links

• OT network segmentation and hardware protection independence: Under NIST SP 800-82r3, condition monitoring analytics platforms must ingest telemetry exclusively from a read-only Level 3.5 DMZ historian replica across a unidirectional gateway or strictly policed firewall. Analytics software has zero write path back to Level 1/2 PLCs, DCS controllers, API 670 machinery protection racks, or Safety Instrumented Systems (SIS).
• Cross-functional links across maintenance governance: Accurate condition monitoring relies on equipment hierarchy design to track rotating serial swaps, uses the calibration certificate register to verify transmitter accuracy, and serves as the mandatory entry prerequisite before launching a maintenance analytics pilot.

Summary and next actions

1. Run a 30-day telemetry fidelity audit on your top 50 rotating trains (Day 1–20): Measure actual historian sample intervals, compression deadbands, flatline hours, and Bad_Quality percentages across all vibration, bearing temperature, and seal pressure tags.
2. Bind running-state and process-load context tags to every monitored asset (Day 21–40): Map motor breaker status, speed, and suction/discharge flow tags into the telemetry table so stopped machines and startup warmups are automatically masked from steady-state alert thresholds.
3. Separate Instrument Loop Fault routing from Rotating Machinery alerts (Day 41–65): Configure pre-analytics rules so open thermocouples, ground loops, and frozen transmitters generate instrument technician checks rather than cluttering the rotating equipment reliability queue.
4. Overlay CMMS serial swap and overhaul dates onto historian training windows (Day 66–90): Link your historian tag table to the CMMS rotating installation ledger before training statistical baselines or launching a bounded maintenance analytics pilot.