Skip to content

Privacy-First Measurement: Server-Side Tagging, Consent Mode and Conversion APIs · Data quality, deduplication and QA · lesson 10 of 14 · 8 min

Data quality: event IDs, deduplication and a tracking plan

Why data quality is a performance lever

Every automated bidding system learns from your events. Duplicates inflate performance and make the system overspend; missing values make it undervalue good traffic; inconsistent names split data across events; wrong currencies wreck value-based bidding. In a hybrid browser-plus-server setup, the risk of duplicates and mismatches increases — unless you design for it.

The tracking plan: one source of truth

A tracking plan is a living document (often a spreadsheet or a schema file in your repository) that defines every event:

| Field | Example | |---|---| | Event name (canonical) | purchase | | Trigger | Payment confirmed by payment provider webhook | | Source of truth | Server (order service) | | Browser duplicate? | Yes — thank-you page tag, for context | | Event ID rule | order_{order_id} | | Required parameters | transaction_id, value, currency, items[] | | Value definition | Revenue excl. VAT and shipping; margin value sent to ads separately | | Platform mapping | GA4 purchase; Meta Purchase; TikTok CompletePayment; Google Ads conversion "Purchase"; LinkedIn rule "Purchase" | | Consent purposes | analytics (GA4), advertising (ads platforms) | | Owner | Growth engineering |

Event ID design

  • Deterministic: derive from a business key (order ID, lead ID + stage) so browser and server generate the same ID independently, or generate once and pass it (e.g., via the data layer and a hidden form field).
  • Unique per real-world conversion and stage: lead_8812_sql differs from lead_8812_opportunity.
  • Stable across retries: if your server retries a failed send, it must reuse the same ID.
  • Not personal: do not use emails as event IDs.

Deduplication rules by platform (summary)

| Platform | Dedup key | Notes | |---|---|---| | Meta | event_name + event_id (server) / eventID (Pixel) | Also uses fbp/external_id in some cases; best practice is matching IDs | | TikTok | event + event_id | Same ID in Pixel and Events API | | Google Ads | transaction_id (order ID) for web conversions; one upload per click/conversion action/time | Avoid uploading the same offline conversion twice | | GA4 | transaction_id for purchases | Decide browser or server as source | | LinkedIn | eventId | Shared between Insight Tag conversion and CAPI |

Validating event quality in code

Use a schema to validate events before they leave your server. For example, with JSON Schema in Python:

from jsonschema import validate, ValidationError

PURCHASE_SCHEMA = {
    "type": "object",
    "required": ["event_id", "transaction_id", "value", "currency", "items", "consent"],
    "properties": {
        "event_id": {"type": "string", "pattern": "^order_[0-9]+$"},
        "transaction_id": {"type": "string"},
        "value": {"type": "number", "minimum": 0},
        "currency": {"type": "string", "enum": ["AED", "SAR", "PKR", "GBP", "USD", "EUR"]},
        "items": {"type": "array", "minItems": 1},
        "consent": {"type": "object", "required": ["analytics", "ads"]},
    },
    "additionalProperties": False,
}

def check(event: dict) -> list[str]:
    try:
        validate(event, PURCHASE_SCHEMA)
        return []
    except ValidationError as e:
        return [e.message]

additionalProperties: False is a privacy feature: it stops developers from accidentally adding an email or free-text field to an event.

Monitoring data quality

Build a daily reconciliation job:

  • Backend vs GA4 vs each platform counts and value for key events (with expected ranges).
  • Duplicate rate: events with the same ID sent more than once, per platform.
  • Missing parameter rate: purchases without currency or value.
  • Match-quality indicators: Meta EMQ, Google enhanced conversions coverage.
  • Latency: time from conversion to server send.

Alert when a metric leaves its range. Most tracking incidents are caused by website releases, CMP changes or vendor API changes — so tie alerts to release calendars.

Worked example: a Karachi-based agency's client onboarding

The agency standardizes: every new client gets a tracking plan template, event ID conventions, a JSON schema per key event, a reconciliation dashboard and alerts. Onboarding takes longer, but "why don't the numbers match?" tickets fall sharply, and clients trust reports.

Pitfalls

  • Letting each platform integration invent its own event names.
  • Generating random IDs separately in browser and server.
  • Sending test events to production pixels without test codes.
  • No owner for the tracking plan.

How to measure success

Duplicate rate near zero, reconciliation within agreed tolerances, alerts firing within hours of incidents, and a tracking plan updated with every release.

Video lecture: Data quality: event IDs, deduplication and a tracking plan

Lecture coming soon · 14 chapters · about 8 minutes. Read the full transcript below.

  1. Data quality
  2. Why it matters
  3. What bad data does
  4. The tracking plan
  5. Event ID design
  6. Simple example: Jeddah pharmacy (illustrative)
  7. Dedup keys
  8. Validate in code
  9. Monitor daily
  10. Example: Karachi agency (illustrative)
  11. Mistakes + try this now
  12. Quick self-check
  13. Watch me do it: daily reconciliation (illustrative)
  14. Recap and next step

Lecture transcript

Data quality

Here's something most marketers don't realize. The single biggest cause of bad automated bidding isn't the algorithm. It's messy events. Duplicates that make performance look twice as good, missing values, the same purchase named four different ways. In this lecture you'll learn to build a tracking plan, design event IDs that deduplicate correctly everywhere, validate events in code, and monitor data quality so problems are caught in hours, not months.

Why it matters

Why does this matter? Because every decision downstream, bidding, budgets, reporting, is only as good as your event data. Here's an analogy. A bank that sometimes records a deposit twice, sometimes forgets the currency, and sometimes files it under the wrong account name would be shut down. Yet many marketing stacks do exactly this with conversions. Data quality isn't glamorous, but it's the accounting system of your marketing. Get it right and every tool on top of it gets smarter. Get it wrong and every tool confidently amplifies the error.

What bad data does

Think about what duplicates do. If every purchase is counted twice, your cost per purchase looks half what it is. The system thinks it's winning, scales spend, and you lose money. Missing values make it undervalue good traffic. Inconsistent names split learning across events. Wrong currencies make a hundred rupees look like a hundred dollars. And a hybrid browser-plus-server setup increases these risks, unless you design for it.

The tracking plan

The fix starts with a tracking plan, one source of truth for every event. For each event: the canonical name, what triggers it, which system is the source of truth, whether a browser duplicate exists, the event ID rule, required parameters, how value is defined, how it maps to each platform, which consent purposes apply, and who owns it. It can be a spreadsheet or a schema file in your code repository. What matters is that everyone uses it.

Event ID design

Now event IDs, the key to deduplication. Make them deterministic: derive them from a business key like the order ID, so the browser and server produce the same ID independently, or generate once and pass it along. Make them unique per conversion and per stage, so a lead reaching the qualified stage has a different ID from the same lead becoming an opportunity. Keep them stable across retries. And never use personal data like an email address as an ID.

Simple example: Jeddah pharmacy (illustrative)

Here's a simple worked example. A Jeddah pharmacy chain's online store has three developers. One names the purchase event Purchase, another uses order complete, and the third uses checkout success in the server integration. Meta sees two event names and can't deduplicate. GA4 gets three different events. Reports disagree. The fix takes one afternoon: a tracking plan row that defines the canonical event, purchase, maps it to each platform's name, sets the event ID rule to order underscore the order number, and lists required parameters. The developers update their code to match, and a schema check blocks anything that doesn't.

Dedup keys

Each platform deduplicates slightly differently. Meta and TikTok match on the event name plus the event ID. Google Ads uses the transaction ID for web conversions, and you must avoid uploading the same offline conversion twice. GA4 uses the transaction ID for purchases. LinkedIn uses the event ID shared between the Insight Tag and the Conversions API. Put these rules in your tracking plan so every developer knows them.

Validate in code

Validate events before they leave your server. The lesson shows JSON Schema in Python: required fields, types, a pattern for the event ID, a list of allowed currencies, and one setting that's a quiet privacy hero, additional properties set to false. That stops anyone accidentally adding an email or a free-text field to an event. If validation fails, log it, alert, and don't send.

Monitor daily

Then monitor. Run a daily reconciliation comparing backend, GA4 and each platform for key events, with expected ranges. Track the duplicate rate, the share of purchases missing currency or value, match-quality indicators like Meta's Event Match Quality and Google's enhanced conversion coverage, and latency from conversion to send. Alert when anything leaves its range. Most incidents come from website releases, consent platform changes or vendor API updates, so link alerts to your release calendar.

Example: Karachi agency (illustrative)

Here's an illustrative example. A Karachi agency standardized onboarding for every client: a tracking plan template, event ID conventions, a JSON schema for each key event, a reconciliation dashboard and alerts. Onboarding took a little longer, but tickets asking why the numbers don't match fell sharply, and clients started trusting the reports. That trust is worth more than any single campaign optimization.

Mistakes + try this now

Common data quality mistakes. Each integration inventing its own event names. Random IDs generated separately in browser and server. Retries that create new IDs. Test events sent to production pixels without test codes. And nobody owning the tracking plan. Try this now: pick your most important conversion and trace it end to end. Where is the ID created? Which systems send it? What name does each platform receive? Draw it on one page. If the ID or name changes anywhere along the path, you've found a deduplication bug.

Quick self-check

Quick self-check. Your server sends a purchase to TikTok, the request times out, and your retry logic generates a new event ID before sending again. TikTok actually received the first request. What happens? Pause. TikTok now has two purchases with different IDs, so it can't tell they're the same order, and your conversions are inflated. The fix is simple: create the event ID once, store it with the order, and reuse it on every retry. It's one line of code, and it's the difference between trustworthy and misleading numbers.

Watch me do it: daily reconciliation (illustrative)

Watch me do it. Let's set up the daily reconciliation for an illustrative Karachi online grocer. Step one, the query: yesterday's backend paid orders, GA4 purchases from the BigQuery export, and each platform's reported purchases from their APIs, all in one table. Step two, expected ranges from the last eight weeks: GA4 typically sees seventy-five to eighty-five percent of backend orders, Meta sixty to seventy-five, Google Ads sixty-five to eighty. Step three, the duplicate check: count purchase event IDs sent more than once per platform from our outbox table; expected zero. Step four, missing parameters: purchases without currency or value; expected zero. Step five, today's run: GA4 at eighty percent, fine. Meta at one hundred and thirty percent of backend orders. That's outside the range and above one hundred, which almost always means duplicates. Step six, the duplicate check confirms it: after yesterday's release, the thank-you page tag fires twice. The alert posts in the team channel with the numbers and the likely cause. The fix goes out the same day, and we annotate the data so nobody misreads yesterday's ROAS.

Recap and next step

Recap. Data quality is a performance lever. Write a tracking plan with a source of truth for every event. Design deterministic, stage-specific, retry-stable, non-personal event IDs. Know each platform's dedup key. Validate with strict schemas and reconcile daily. Your next step: write tracking plan rows for your three most important events, including the ID rule, source of truth, platform mappings and consent purposes.

Key takeaways

  • A tracking plan defines each event's trigger, source of truth, ID rule, parameters, values, mappings, consent and owner.
  • Event IDs must be deterministic, unique per conversion and stage, stable across retries and non-personal.
  • Know each platform's dedup key: Meta/TikTok event_id, Google transaction_id, LinkedIn eventId.
  • Validate events with schemas; additionalProperties: false prevents accidental PII.
  • Reconcile daily against the backend and alert on drift.

Try it

Write a tracking plan row for your three most important events, including event ID rule, source of truth, platform mappings and consent purposes.