Synthetic media, deepfakes and content provenance

0:00 Synthetic media and provenance

A finance manager hears the CFO's voice asking for an urgent transfer. It sounds exactly right. It isn't the CFO. Synthetic media makes stories like this possible, and it also powers incredible creative work. In this lesson you'll learn the risks, how provenance and watermarking work, what the rules say, and how to build a practical policy.

0:25 Opportunity vs risk

The opportunity is real: product imagery, localisation, dubbing, personalised video, all in minutes. So are the risks: impersonation scams, fake endorsements, election disinformation, non-consensual imagery, and a general erosion of trust in genuine content. The response is developing on three fronts: provenance, which proves where content came from; watermarking and detection; and regulation and platform policy.

0:49 Why it matters

Why does this matter to every organisation? Because synthetic media touches you whether or not you create it. Your marketing team may use AI images and voices. Fraudsters may clone your executives' voices to request payments. Fake videos or images about your brand can spread faster than corrections. And regulations, like the EU AI Act's transparency rules, now set expectations for labelling. Understanding provenance, detection and disclosure lets you use the creative upside while protecting your people, customers and reputation.

1:24 Food label and seal

Here's an analogy. Content Credentials work like a food label combined with a tamper-evident seal. The label tells you where the product came from and what went into it. The seal shows whether anyone opened it on the way. But a product with no label isn't necessarily poisoned. It might just come from a market stall that doesn't print labels. Provenance helps you trust labelled content more. It can't, on its own, prove unlabelled content is fake.

1:57 Simple example: an AI product image

A simple example. Your marketing team generates a product image with an AI tool that attaches Content Credentials. You edit it in a design tool that preserves them, then post it. Someone checks it with a verification tool and sees: created with an AI image generator, edited in a design application, published by your brand. Transparent and trustworthy. Now imagine you export it with a setting that strips metadata. The history is gone, and you've lost that trust signal for no reason.

2:33 C2PA and Content Credentials

Provenance first. The Coalition for Content Provenance and Authenticity, known as C2PA, publishes an open standard for attaching cryptographically signed manifests to media. A manifest can record who made the content, with what tool or device, what edits were made, and whether AI was involved. For consumers, these are called Content Credentials, often shown with a small CR icon.

2:59 Adoption and limits

Adoption is growing across the pipeline. Some phones and cameras sign images at capture; Google's Pixel 10 phones, for example, attach Content Credentials to photos from the Pixel Camera app. Major creative tools and several AI image generators add credentials to outputs. But many platforms and messaging apps strip metadata on upload, so credentials don't always survive. Remember: provenance proves history when it's present. It cannot prove that content without credentials is fake.

3:31 Watermarks and detection

Next, watermarks and detection. Invisible watermarks, like Google DeepMind's SynthID, are embedded in AI outputs and can be detected by the provider's tools, even after some edits. Detection classifiers try to spot AI content without a watermark. They give probabilities, can be fooled, and have false positives. Never use a detector alone to accuse someone of fakery.

3:56 The rules

Now the rules. Under the EU AI Act, Article 50 applies from the second of August 2026. Providers of generative AI must mark outputs in a machine-readable way as artificially generated, and deployers who publish deepfakes must disclose it, with lighter obligations for evidently artistic or satirical work. Major platforms require creators to label realistic AI content. Advertising regulators like the ASA and FTC apply existing rules, so fake testimonials are risky. And always get written consent before cloning anyone's voice or likeness.

4:32 Five-part policy

Put it together in a five-part policy. Consent: written consent for any real person's likeness or voice. Disclosure: label realistic AI-generated or altered content. Provenance: preserve Content Credentials in your pipeline and switch them on where tools allow. Verification: check provenance and source before sharing sensitive media, and use call-backs for any voice or video request involving money. Incident response: a plan for deepfakes of your executives or brand. The lesson text also shows how to inspect credentials with c2patool.

5:07 Worked example: voice-clone fraud

Back to that voice message. A bank's finance team in Dubai received it, apparently from the CFO, asking for an urgent transfer. Policy required a call-back on a known number and a second approver above a threshold. The call-back exposed the fraud. The bank then trained staff on voice-clone scams, added a code-word for urgent requests, and prepared a deepfake response plan covering its executives' public images and voices.

5:37 Provenance in your workflow

If you create content, make provenance part of your workflow. Switch on Content Credentials in tools that support them. Avoid export settings that strip metadata. Keep originals and project files. And when you publish realistic AI-generated or altered content, add the platform's AI label as well as your own disclosure. Brands that do this consistently build a reputation for honesty that becomes more valuable as synthetic content floods feeds.

6:07 Three mistakes

Three common mistakes. First, relying on AI detectors as proof, which can wrongly accuse genuine creators. Second, stripping Content Credentials in your own asset pipeline through careless export settings. Third, cloning a voice or likeness without documented consent, which creates legal, ethical and reputational risk, even for internal or light-hearted uses.

6:29 Try this now

Try this now. Find three images: one you know was generated with an AI tool that adds Content Credentials, one photo straight from a recent phone, and one downloaded from social media. Check each with the Content Credentials verify site or c2patool. Note what history each one shows, or doesn't. Then draft a one-page synthetic media policy using the five parts from this lesson: consent, disclosure, provenance, verification and incident response. Finally, agree a call-back rule for any urgent money request that arrives by voice or video.

7:07 Recap

To recap. Synthetic media brings value and real risk. Content Credentials prove history when present, but absence proves nothing. Watermarks help; detectors alone don't. The EU AI Act's Article 50 applies from August 2026, and platforms and ad regulators have their own rules. Build a policy on consent, disclosure, provenance, verification and incident response. Your next step: inspect three images and draft that policy. Next module: AI search and the future of work.