Designing system prompts and roles

The mental model: a brief for a brilliant new colleague

A system prompt is not a magic spell. It is the standing brief you would give a highly capable colleague on their first day: who they are serving, what good work looks like, what they must never do, and what to do when they are unsure. Models are strong generalists with no knowledge of your business, your customers or your taste. Every gap you leave, the model fills with a plausible average. Advanced prompt engineering is mostly the discipline of removing those gaps deliberately.

A useful test: if you handed your system prompt to a smart contractor with no other context, could they do the job well? If they would need to ask five questions first, the model needs those answers too.

The five layers of a strong system prompt

1. Role and audience. Not "You are an expert marketer" (which adds little on its own) but a role tied to a situation: "You are the support assistant for a B2B invoicing product. Your readers are finance managers at small companies who are busy and often frustrated."
2. Objective and success criteria. What outcome matters? "Resolve the question in one reply where possible; if not, collect the three details a human agent needs."
3. Behavioural rules with reasons. "Never quote prices, because pricing varies by contract and wrong quotes create legal disputes." Explaining why lets the model generalise to cases you did not list.
4. Knowledge and tools. What reference material is available, which tools exist and when to use them.
5. Output contract. Format, length, tone, language and what to do when information is missing.

Role design: when personas help and when they hurt

Role prompting works because it sets defaults for vocabulary, depth and priorities. "You are a senior data engineer reviewing a pipeline for reliability" produces a different review than "You are a helpful assistant." But roles have failure modes:

• Persona theatre. Elaborate backstories ("You are Max, 47, who loves hiking") consume tokens and rarely improve task quality.
• Overclaiming authority. "You are a licensed doctor" can push the model to sound more certain than it should. Prefer "You help clinicians by summarising literature; you do not diagnose."
• Conflicting roles. "Be a ruthless critic" plus "be warm and encouraging" produces mush. Pick one primary stance and specify how to handle the tension: "Be candid about problems, framed constructively."

Instructions: specific, positive, reasoned

Models follow instructions more literally than many people expect, especially newer ones. Three rules of thumb:

• Say what to do, not only what to avoid. "Write in flowing paragraphs" beats "Don't use bullet points."
• Quantify vague words. "Short" means different things to different people; "under 120 words" does not.
• Give the reason. "Avoid jargon because readers are first-time founders" lets the model judge edge cases.

Avoid SHOUTING and stacked "CRITICAL!!!" markers. With capable modern models, aggressive emphasis tends to cause overreaction, such as refusing reasonable requests near a forbidden topic. Calm, precise language with reasons works better.

A worked example

Weak:

You are a helpful assistant for our agency. Be professional. Don't make things up.

Strong:

You draft first-pass replies to inbound leads for a small digital agency
that serves e-commerce brands in the UK and the Gulf.

Goal: a reply the account manager can send after light edits, which
(1) acknowledges the lead's specific situation, (2) asks at most two
qualifying questions, and (3) proposes a 20-minute call.

Rules:
- Only mention services listed in the services section; if the lead asks
  for something else, say we will confirm whether it is a fit.
- Do not quote prices or timelines; these depend on scoping.
- Match the lead's language (English or Arabic) and formality.
- If the message is spam or unrelated, reply with the single word SKIP.

Output: plain text email body, 80-150 words, no subject line.

Notice what changed: a real audience, a measurable goal, rules with reasons, an explicit escape hatch (SKIP), and an output contract.

System prompt vs user turn

Put stable, reusable instructions (role, rules, format) in the system prompt. Put the task-specific material (this lead's email, today's data) in the user turn. This separation makes prompts easier to version, cache and test, and it keeps untrusted content out of the highest-authority position.

Failure modes to watch

• Instruction overload. Past a certain point, adding rules makes the model worse at all of them. If you have forty rules, group them, delete the ones the model already follows, and test.
• Hidden contradictions. "Always be concise" in one section and "explain thoroughly" in another. Read your prompt as a hostile reviewer.
• Stale rules. Rules written to fix an old model's quirk may now cause over-correction. Re-test rules when you change models.

What changed with the newest models

Frontier models released through 2025 and 2026 follow instructions more literally and more completely than their predecessors. Three practical consequences:

• Say everything you want. If you want the assistant to go beyond the obvious (suggest alternatives, add edge cases), ask for it. Newer models tend not to guess that you wanted more.
• Remove legacy scaffolding. Rules such as "ALWAYS double-check", "think step by step before every answer" or "you MUST use the search tool" were written for older models. On current models they often cause over-triggering: excessive tool calls, verbose hedging or refusals. Rewrite them calmly with the reason, or delete them and re-test.
• Some old tricks no longer apply. For example, several current APIs, including recent Claude models, no longer accept a pre-filled start to the assistant's reply ("prefill"). Use structured outputs or clear format instructions instead.

Provider migration guides are the best source for these shifts; read them whenever you change model generation.

Hands-on: a system prompt as a versioned template

Store the system prompt as a file with explicit variables, render it in code, and keep the stable part byte-identical between requests so it can be cached (see the prompt caching lesson).

# pip install anthropic
import os
from string import Template
import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment
MODEL = os.environ.get("CLAUDE_MODEL", "claude-opus-5")  # check the current models page

SYSTEM = Template("""You draft first-pass replies to inbound leads for $agency, a small
digital agency serving e-commerce brands in the UK and the Gulf.

<goal>
A reply the account manager can send after light edits: acknowledge the lead's
situation, ask at most two qualifying questions, propose a 20-minute call.
</goal>

<rules>
- Mention only services in <services>; if asked for anything else, say we will
  confirm fit, because we lose trust when we over-promise.
- Do not quote prices or timelines; they depend on scoping.
- Match the lead's language (English or Arabic) and formality.
- If the message is spam or unrelated, reply with exactly: SKIP
</rules>

<services>$services</services>

<output>Plain-text email body, 80-150 words, no subject line.</output>""")

system_prompt = SYSTEM.substitute(agency="Northwind Digital",
                                  services="Paid social; Shopify CRO; email automation")

def draft_reply(lead_message: str) -> str:
    try:
        resp = client.messages.create(
            model=MODEL,
            max_tokens=1024,
            system=system_prompt,
            messages=[{"role": "user", "content": f"<lead_message>\n{lead_message}\n</lead_message>"}],
        )
    except anthropic.APIStatusError as err:
        raise RuntimeError(f"API error {err.status_code}") from err
    return "".join(b.text for b in resp.content if b.type == "text").strip()

print(draft_reply("Salam, we launch our abaya store in Riyadh in March. Can you run TikTok ads?"))

The same structure ports to other providers: in the OpenAI Responses API the system-level brief goes in instructions; in the Gemini API it goes in the system instruction field. Keep the brief provider-neutral where you can and adapt only the call.

How to test a system prompt

Before shipping, run it against 20 to 50 realistic messages, including out-of-scope, Arabic, rude and spam examples, and check each rule is followed. When you edit a rule, re-run the whole set: a fix for one behaviour frequently breaks another.

Going further

If you work through an API, treat the system prompt as code: store it in version control, template variables explicitly (for example {{customer_tier}}), and keep a changelog of why each rule exists. A rule with no recorded reason is a rule nobody will dare to delete.