AI for Data Analysis & Decision Making · Writing formulas, SQL and Python with AI, and verifying them · lesson 9 of 16 · 11 min

Python analysis with AI, and verifying the results

Why Python (even if you are not a programmer)

Many AI assistants can write and run Python in a sandbox. For analysts who don't code, this unlocks powerful analysis: joining large files, statistical tests, charts, and repeatable scripts. You do not need to write Python yourself, but learning to read it at a basic level and to verify its results is a valuable skill.

A good analysis request

Using Python (pandas), load sales.csv and customers.csv.
- Join on customer_id (check for customers missing in either file and report counts).
- Compute monthly revenue and number of active customers per region.
- Show the first 5 rows after each major step.
- Plot monthly revenue by region as a line chart.
Keep the code simple and commented so a non-programmer can follow it.

Asking for intermediate outputs ("show the first 5 rows after each step") and counts makes each step inspectable.

Reading AI-generated Python: what to look for

You don't need to understand every line. Focus on:

Filters: df[df["status"] == "completed"] shows which rows are kept.
Joins: merge(..., how="left") or how="inner": inner joins drop unmatched rows silently.
Grouping: groupby(["month", "region"]) shows the grain of results.
Aggregations: .sum(), .mean(), .nunique(): is the right one used?
Missing values: dropna() or fillna(0) change results; know when they are used.
Dates: parsing and time zones; how months are defined.

Built-in checks

Ask the AI to include assertions that fail loudly if assumptions are violated:

# Checks the analysis relies on
assert sales["order_id"].is_unique, "Duplicate order IDs"
assert (sales["revenue"] >= 0).all(), "Negative revenue found"
merged = sales.merge(customers, on="customer_id", how="left")
assert len(merged) == len(sales), "Join changed row count"
print("Unmatched customers:", merged["region"].isna().sum())

These turn silent errors into visible ones.

Verification strategies

Reconcile totals with a trusted source.
Trace one entity: pick one customer and manually check their rows through each step.
Recompute key numbers differently: a pivot table in a spreadsheet, or a separate simple script.
Look at the charts critically: sudden spikes or drops often reveal data issues.
Rerun on fresh data: if the script is reused, check it still behaves when the data changes.

Statistical functions: ask what and why

When the AI runs statistical tests (t-tests, regressions, correlations), ask:

"Which test did you use and why is it appropriate for this data?"
"What assumptions does it make, and do they hold here?"
"What does this result mean in plain language, and what does it not mean?"

A p-value or regression coefficient reported without context invites misinterpretation (module 5 covers common traps).

Worked example: marketing mix exploration

A marketer asks for "the impact of each channel on sales" using weekly spend and sales data. The AI runs a regression and reports channel coefficients. The marketer asks about assumptions. The AI notes: only 52 weeks of data, strong correlation between channels (spend often rises together before holidays), and no control for seasonality. The coefficients are unstable. Conclusion: useful for hypotheses, not budget decisions; the team plans geo-based tests instead. The AI's willingness to list limitations was only triggered by asking.

Reproducibility

Save the final code with the data version it ran on. Ask the AI to consolidate the chat's code into a single clean script or notebook. This lets a colleague rerun it, and lets you rerun next month.

Hands-on: a complete, reviewable analysis notebook

Ask the assistant to produce a single notebook in this structure. Every section prints something you can check:

# 0. Setup ---------------------------------------------------------------
import pandas as pd
import matplotlib.pyplot as plt
DATA_VERSION = "exports/2026-09-01"          # record which files the results came from

# 1. Load and inspect -----------------------------------------------------
sales = pd.read_csv(f"{DATA_VERSION}/sales.csv", parse_dates=["order_date"])
customers = pd.read_csv(f"{DATA_VERSION}/customers.csv")
print(sales.shape, customers.shape)
print(sales.head())

# 2. Assumptions as assertions -------------------------------------------
assert sales["order_id"].is_unique, "Duplicate order IDs"
assert (sales["revenue"] >= 0).all(), "Negative revenue: handle refunds explicitly"
assert customers["customer_id"].is_unique, "Duplicate customers would fan out the join"

# 3. Join with a row-count guard -----------------------------------------
merged = sales.merge(customers, on="customer_id", how="left", validate="many_to_one")
assert len(merged) == len(sales), "Join changed row count"
print("orders without a customer match:", merged["region"].isna().sum())

# 4. Analysis --------------------------------------------------------------
merged["month"] = merged["order_date"].dt.to_period("M")
monthly = (merged.groupby(["month", "region"], dropna=False)
                 .agg(revenue=("revenue", "sum"), customers=("customer_id", "nunique"))
                 .reset_index())
print(monthly.tail(10))

# 5. Reconciliation --------------------------------------------------------
assert abs(monthly["revenue"].sum() - sales["revenue"].sum()) < 0.01, "Totals do not reconcile"

# 6. Chart -----------------------------------------------------------------
pivot = monthly.pivot(index="month", columns="region", values="revenue")
pivot.plot(title="Monthly revenue by region (source: " + DATA_VERSION + ")")
plt.ylabel("Revenue (GBP)"); plt.show()

validate="many_to_one" makes pandas raise an error if the customer table has duplicate IDs, which is exactly the fan-out problem from the SQL lesson, caught in code.

Tracing one entity

After any join or aggregation, pick one customer and print their rows at each stage:

cid = merged["customer_id"].iloc[0]
print(sales[sales.customer_id == cid][["order_id", "order_date", "revenue"]])
print(merged[merged.customer_id == cid][["order_id", "region", "revenue"]])

Second worked example: a Riyadh subscription app

An analyst asks for revenue by acquisition channel. The notebook's join assertion fails: the channel table has duplicate customer IDs because customers who reinstalled the app were recorded twice. Instead of silently inflating revenue, the analysis stops. The team decides to use each customer's first recorded channel, documents the rule in the notebook, and reruns. The assertion that "failed" saved a board slide.

Going further

Learn a handful of pandas operations by reading: read_csv, merge, groupby, agg, pivot_table, query, plot. With these, you can follow most AI-generated analysis. Ask the assistant to explain any unfamiliar line; it is an excellent tutor for this.

Video lecture: Python analysis with AI, and verifying the results

Lecture coming soon · 15 chapters · about 8 minutes. Read the full transcript below.

Python with AI, verified
Why it matters
A good request
Read six things
Assertions
Verify further
Example 1: channel regression
Example 2: Riyadh subscription app
Watch me do it, part 1
Watch me do it, part 2
Six pandas operations
Common mistakes
Trustworthy notebooks
Recap
Try this now

Lecture transcript

Python with AI, verified

You don't need to be a programmer to benefit from Python anymore. Many AI assistants write and run it for you: joining large files, running statistical tests, drawing charts. But here's the catch. If you can't read the code at a basic level, you can't tell whether it did what you asked. In this lecture you'll learn how to request analysis the AI can't fake, the handful of pandas operations worth recognizing, how assertions turn silent errors into loud ones, and a complete notebook structure you can reuse.

Why it matters

Why does this matter? Because code is where AI analysis is most powerful and most hidden. A single line can drop unmatched customers, fill missing values with zero, or join tables in a way that duplicates rows. The chart that comes out looks just as confident either way. Learning to read a few key operations, and to demand checks in the code, turns AI from a black box into a colleague whose work you can review.

A good request

Here's how to ask. Using pandas, load sales and customers. Join on customer id, and report how many customers are missing in either file. Compute monthly revenue and active customers per region. Show the first five rows after each major step. Plot revenue by region as a line chart. And keep the code simple and commented, so a non-programmer can follow it. Asking for intermediate outputs and counts makes each step inspectable. It's like asking a builder to show you the foundations before they pour the concrete.

Read six things

Now reading the code. You don't need every line. Focus on six things. Filters, like keeping rows where status equals completed. Joins, where merge with how equals inner silently drops unmatched rows, and how equals left keeps them. Grouping, which shows the grain of results. Aggregations: sum, mean or count of unique values. Missing values: drop N A or fill N A with zero both change results. And dates: parsing, time zones and how months are defined. Six things, and you can review most AI-generated analysis.

Assertions

Then assertions. An assertion is a line that says: this must be true, or stop. Ask the AI to include them for every assumption the analysis relies on. Order ids are unique. Revenue isn't negative, or refunds are handled explicitly. After a join, the row count hasn't changed. And pandas can do even better: merge with validate equals many to one raises an error if the customer table has duplicate ids, which is exactly the fan-out trap from the SQL lesson, caught automatically. Assertions turn silent errors into loud ones.

Verify further

Verification strategies, beyond assertions. Reconcile totals with a trusted source. Trace one entity: pick one customer and check their rows through each step. Recompute key numbers differently, like a pivot table in a spreadsheet. Look at charts critically, because sudden spikes or drops often reveal data issues. And rerun on fresh data to check the script still behaves. When the AI runs statistical tests, ask which test it used and why, what assumptions it makes, and what the result does and doesn't mean in plain language.

Example 1: channel regression

First example, a simple one. A marketer asks for the impact of each channel on sales, using weekly spend and sales data. The AI runs a regression and reports channel coefficients. The marketer asks about assumptions. The AI notes: only fifty-two weeks of data, strong correlation between channels because spend rises together before holidays, and no control for seasonality. The coefficients are unstable. Conclusion: useful for hypotheses, not budget decisions, so the team plans geo-based tests instead. The limitations only appeared because she asked.

Example 2: Riyadh subscription app

Second example, a business case. A Riyadh subscription app wants revenue by acquisition channel. The notebook's join assertion fails. The channel table has duplicate customer ids, because customers who reinstalled the app were recorded twice. Instead of silently inflating revenue, the analysis stops. The team decides to use each customer's first recorded channel, documents that rule in the notebook, and reruns. The assertion that failed saved a board slide.

Watch me do it, part 1

Watch me build the notebook. Section zero, setup, including a data version string that records which export folder the results came from. Section one loads both files and prints their shapes and the first rows. Section two lists assumptions as assertions: unique order ids, no negative revenue, unique customer ids. Section three joins with a left merge and validate many to one, asserts the row count didn't change, and prints how many orders had no customer match. Every section prints something I can check.

Watch me do it, part 2

Section four does the analysis: a month column, then revenue and distinct customers by month and region, keeping missing regions visible with drop N A set to false. Section five reconciles: the sum of monthly revenue must equal total revenue within a cent, or the notebook stops. Section six draws the chart, with the data version in the title and currency on the axis. Then I trace one customer through the raw and merged tables. Finally, I save the notebook with the data it ran on, so a colleague can rerun it next month.

Six pandas operations

A few pandas operations are worth learning by sight, because they appear in almost every AI-generated analysis. Read C S V loads data. Merge joins tables. Group by, with agg, summarizes. Pivot table reshapes. Query filters with a readable condition. And plot draws. With those six, you can follow most analysis code line by line. And whenever you see an unfamiliar line, ask the assistant to explain it in plain English. It's an excellent, patient tutor, and a few minutes of questions builds real fluency.

Common mistakes

Common mistakes. Accepting code you haven't skimmed. Inner joins that silently drop unmatched rows. Filling missing values with zero without noticing. Charts without units or sources. Statistical results without assumptions. And leaving the analysis as a long chat instead of a single reproducible notebook.

Trustworthy notebooks

How do you know your Python workflow is trustworthy? Every notebook has assertions for its key assumptions, a reconciliation to a trusted total, a traced entity, and a recorded data version. And a colleague can rerun it on next month's data without asking you anything. Those five properties matter more than elegant code.

Recap

Recap. AI-run Python unlocks powerful analysis, so learn to read the six key operations. Request intermediate outputs, counts and simple commented code. Add assertions, including validated joins. Reconcile, trace an entity and recompute differently. Ask which statistical method was used and what it doesn't mean. And save one reproducible notebook with its data version.

Try this now

Try this now. Ask an AI assistant to perform a join and aggregate analysis in Python on data you know, using the notebook structure from the lesson, with assertions and a reconciliation. Then trace one customer through each step by hand. If any assertion fails, don't delete it. Find out why.

Key takeaways

AI-run Python unlocks powerful analysis; learn to read the key operations and verify results.
Request intermediate outputs, counts and simple, commented code.
Add assertions for assumptions (unique IDs, row counts after joins, valid ranges).
Ask which statistical method was used, why, and what the result does and doesn't mean; save reproducible scripts.

Try it

Ask an AI assistant to perform a join-and-aggregate analysis in Python with assertions. Trace one customer through each step manually to verify.