Detect potential duplicate safety cases without deleting evidence

Similar reports may describe one event—or different events

A clinician emails a possible adverse event. A distributor later calls about a patient with a similar product and symptom, but the name and date are recorded differently. The safety team needs to determine whether the communications concern the same case.

Pharmacovigilance duplicate detection should make that assessment easier while preserving the evidence from both sources. Similarity is useful for prioritising review, but it is not sufficient to establish identity. An incorrect merge can conceal a separate report; an unresolved duplicate can create inconsistent processing.

The Indian Pharmacopoeia Commission (IPC) guidance for marketing authorisation holders (MAHs) identifies duplicate detection among critical PV processes. The organisation's approved case-management procedures determine how potential matches are assessed and recorded. IPC pharmacovigilance guidance for MAHs.

Distinguish duplicate delivery from duplicate case content

A technical retry can deliver the same message twice with the same source identifier. That problem differs from two independent communications that may describe the same event. The first concerns safe message processing; the second requires case-level assessment.

Keep both concepts in the design. A stable intake identifier can prevent repeated creation from an identical replay, while related but separately received communications retain their own source records for review. Do not apply a technical deduplication rule to all messages sharing a patient name or product.

The integration exception guide explains safe retry behaviour. The PV intake guide explains the source and receipt information that matching depends on.

Compare evidence without hiding uncertainty

Define the fields and context available to qualified reviewers. Relevant information may include product, reported event, dates, person details, reporter relationship and source references, subject to the approved process and access controls. Missing values should remain missing rather than being filled through guesswork.

Preserve the original representation alongside any normalised search value. A date may be approximate, a name may be transliterated, or an age may have been reported differently. A matching tool should make those differences visible rather than silently treating them as established equivalence.

Where the tool produces a score, show the evidence contributing to the suggestion in a form the reviewer can inspect. The score should not be presented as a probability of identity unless that interpretation has been properly established for the intended use.

Worked example: email and distributor call

Consider an illustrative clinician email and a distributor call received on different days. Both mention the same product and a similar event. The email gives an approximate event date; the call records a different spelling of the person's name and a date one day later.

The matching service proposes the pair for review and displays the supporting similarities and unresolved differences. The reviewer can inspect the original communications, associated intake references and any available follow-up under the approved procedure.

The review may conclude that the communications concern one case, that they concern separate cases, or that the evidence is insufficient to decide. Each outcome needs an attributable rationale and a defined next step. An unresolved match should not disappear because the algorithm's score is below a default threshold.

If the authorised decision links the reports, both source histories remain accessible. The later communication may provide additional information rather than being a useless duplicate. Preserve what was newly learned and its receipt history for subsequent processing.

If later evidence shows the linkage was wrong, the system needs a controlled correction path. The original decision and the revised conclusion should remain understandable, including any affected downstream records or submissions.

This example concerns evidence reconciliation. It does not make a clinical assessment or define a universal threshold for linking safety cases.

Give the reviewer a complete comparison view

A useful screen places the relevant information side by side with clear source attribution. Avoid a merged preview that hides the differences before the reviewer has decided whether merging is appropriate.

Let the reviewer record a rationale without forcing a premature binary choice. The available outcomes and follow-up process should match the approved procedure, including how unresolved candidates are monitored.

Preserve case and source relationships

Define the difference between an intake record, a case record and a relationship between records. A case may draw on several communications. Keeping those identities distinct helps the organisation explain how the current case information was assembled.

When information conflicts, retain provenance for the value selected by the authorised reviewer. Do not apply a general “latest message wins” rule to all fields. A later communication can be less precise than an earlier one, and the correct interpretation depends on the evidence.

Record the linkage decision, responsible role, time and affected records. If the software performs a technical consolidation after approval, verify that it preserves the required relationships and does not overwrite receipt history or prior review evidence.

Assess downstream consequences explicitly

A linkage or correction can affect follow-up, quality checks, reporting and reconciliation with partners. The responsible PV roles should determine the required actions under the applicable process. The software should surface those dependencies rather than assume that changing a case identifier completes the work.

Track pending actions until the relevant acknowledgements or decisions are available. A message sent to a partner is not the same as a reconciled partner record. Where a prior submission may be affected, route the issue through the authorised reporting process.

The PV software selection guide includes these scenarios in a broader evaluation. Demonstrating a merge button alone does not establish a complete duplicate-management capability.

Evaluate false matches and missed matches

Build an approved test set containing known related reports, clearly separate reports and ambiguous examples. Use appropriately protected or synthetic data. Include spelling differences, approximate dates, repeated product names and common events so the assessment reflects realistic ambiguity.

Review both false positives and false negatives. A system that sends every pair to reviewers may avoid some misses but create an unmanageable workload. A system that shows only obvious matches may overlook important candidates. The PV owner should assess the trade-off for the intended use.

Keep the evaluation separate from tuning data where feasible, and examine difficult categories rather than reporting one attractive average. Changes to matching rules, dictionaries or models need an assessed impact and verification approach.

Operate the queue with accountable ownership

Define who reviews candidates, how urgent work is prioritised and how unresolved items are escalated within the approved PV process. Monitor queue age and the reasons decisions remain open. Missing source evidence and insufficient reviewer capacity require different responses.

Periodically sample completed decisions for quality review. Look for recurring confusion, incorrect source mappings or inconsistent use of outcomes. The purpose is to improve the evidence and workflow, not reward a high percentage of merged reports.

Summary and next actions

Duplicate detection is useful when it brings related evidence together for qualified assessment while preserving every source history. Reliable linkage includes an explanation, downstream reconciliation and a controlled way to correct mistakes.

• Separate technical message replay from case-level similarity in the design.
• Define the evidence and outcomes reviewers need for uncertain matching decisions.
• Test false matches, missed matches and reversal of an incorrect linkage.
• Monitor unresolved candidates and downstream actions with named PV and integration owners.