Governing AI: human review, audit trails and data quality

0:00 'The AI said so' is not an answer

Imagine a steering committee has released funding based on a forecast. Six months later, it's badly wrong. Someone asks: how was this number produced, who checked it and on what basis was it accepted? If the answer is 'the AI said so', you have a governance failure, whatever the model's accuracy. In project controls, outputs influence funding, contracts, claims and sometimes public accountability. So AI use has to be safe, explainable and auditable. In this lecture you'll learn seven principles for governing AI in controls, a risk-tiered review model, what an audit trail should record, AI-specific data quality checks including leakage, and how to validate a model before trusting it. By the end, you'll be able to draft a one-page AI use policy for your team.

0:55 Why it matters

Why does governance matter more in controls than in many other uses of AI? Because the outputs have consequences. A forecast date can trigger a contractual notice. An EAC can release or withhold millions. A risk assessment can shape a claim. When those outputs are wrong, someone must be able to reconstruct what happened. And the expectation of that record is growing. Auditors increasingly ask how AI-assisted outputs were produced and reviewed, and in some sectors and jurisdictions, regulation adds formal obligations. Good governance isn't there to slow you down. It's what lets you use AI in serious work at all, because it gives leaders a reason to trust the result.

1:43 The concept: seven principles

Here are seven principles. One, human accountability: a named person owns every forecast, report and decision. AI assists. It doesn't sign. Two, review proportional to risk: the higher the impact, the more rigorous the review. Three, traceability: inputs, prompts or model versions, outputs and human edits are recorded. Four, data quality and fitness: models use governed, validated data. Five, confidentiality and privacy: only approved tools, sensitive data protected, personal data handled lawfully. Six, transparency: readers know when content was AI-assisted, according to company policy. And seven, monitoring: model performance is tracked over time, and models are retrained or retired when they drift. Think of it like the controls you already apply to a junior analyst's work, made explicit and written down.

2:36 Risk-tiered review

Now the review model, with four tiers. Low: formatting, or summarising meeting notes. A spot check is enough. Medium: a draft variance narrative, or schedule quality suggestions. The controls lead does a full review and checks facts against the source. High: an EAC or a forecast date used for funding decisions, contractual notices or claims. That needs an independent review, a documented rationale and approval by the accountable manager. And prohibited: fully automated contractual or financial decisions without human approval. Here's the key idea. The tier depends on what the output is used for, not on how clever the tool is. The same model can produce a low-tier summary and a high-tier forecast. The review follows the use.

3:27 Worked example one: an audit trail entry

Let's look at a simple audit trail entry. Output ID: the September monthly executive narrative for project fourteen. Purpose: monthly executive narrative. Tool and version: the approved enterprise assistant, with its version. Input data: EVM table version three, data date the thirty-first of August; the risk register export and change log from the first of September. Prompt: the standard narrative template, reference NARR two. AI draft stored: yes, with a link. Human reviewer and date. Changes made: corrected the cause attributed to control account one point three, and removed an unsupported claim about recovery. And approval by the accountable manager. It takes two minutes to complete. And if anyone asks, a year later, how that narrative was produced, the answer is on one page.

4:21 Worked example two: a leakage failure

Now a realistic failure, from the lesson. A fictional developer built a model to forecast EAC across its portfolio. Back-tests looked excellent. Almost too good. In live use, forecasts were far too optimistic. The problem? The training data had inadvertently included final-cost fields, information that wouldn't have been known at the time of each forecast. That's called leakage, and it makes a model look brilliant on history and useless in practice. Worse, because no audit trail recorded which model version produced which forecast, it took weeks to work out what had gone wrong. The fixes: explicit leakage checks, versioned models, back-testing using snapshots as they were at historical data dates, and a rule that portfolio forecasts are always compared with simple CPI-based methods.

5:14 Watch me do it: validating a forecast model

Here's how I'd validate a forecasting model before anyone relies on it. I take completed projects and use the snapshots stored at historical data dates: say, at twenty-five, fifty and seventy-five per cent complete. At each snapshot, I compute two forecasts using only information available at that date. The model's forecast, and a simple benchmark: budget divided by CPI. Then I compare each with the final actual cost and calculate the error. If the model doesn't clearly beat the simple benchmark, it doesn't justify its complexity. If it does, I deploy it as a third opinion in forecast reviews, alongside formulas and the bottom-up estimate. And I keep monitoring its error on new projects, because models drift as prices, productivity and contract types change.

6:08 Regulation and standards, briefly

A brief word on regulation, which varies by jurisdiction and keeps evolving. The EU AI Act takes a risk-based approach, with obligations phasing in over several years, so check the current timelines for your use case. The UK has taken a principles-based approach led by existing regulators. The US has a mix of federal guidance and state laws. The UAE and Saudi Arabia have published national AI strategies, principles and guidance, and Pakistan has developed national AI policy work. On the management side, ISO slash IEC forty-two thousand and one sets out requirements for an AI management system, and frameworks such as the NIST AI Risk Management Framework are widely used. Always follow your organisation's legal and compliance guidance. This course doesn't provide legal advice.

7:02 Common mistakes, recap and try this now

The common mistakes: 'the AI said so' as a justification, no record of model versions or prompts, using public tools with confidential commercial data, training on data that isn't representative of current projects, and no monitoring after deployment. Also check for bias: if some project types or contractors are under-represented in the history, predictions for them can be systematically wrong. So, to recap. Apply seven principles, led by human accountability. Tier your review by how the output will be used. Keep an audit trail for every AI-assisted output. Check data for representativeness, leakage, bias and freshness. And validate models by back-testing against simple benchmarks. Your try-this-now: draft a one-page AI use policy for your controls team, covering tiers, review requirements, approved tools, audit trail fields and prohibited uses.