FDE Times dataFDE jobs open 924▲33 Change in open roles over 7 days, like-for-like: roles from companies we started tracking this week are left outNew in 7 days 80Companies hiring 196Remote-friendly 25%Median US pay $200kTop hirer Databricks 107
中文VI

The newspaper of the Forward Deployed Engineer

Guides

The system prompt that survives the second department

Split into role, policy, client context and department layers, each new team costs one config block rather than a forked copy to maintain forever.

ShareLinkedInFacebookX

In brief

  • Each layer gets its own XML tag. Keep tag names the same across prompt versions, and nest tags when the content is hierarchical.
  • Give every policy a reason. Models generalise from explanations, not just from the commands themselves.
  • The department layer holds that department's role and output format. Every other layer is shared across the company.
Diagram comparing two approaches. On the left, copying the prompt: three equal-height blocks for CS, Finance and Team 3, each a full copy of the prompt, so every team keeps its own version. On the right, layering: one large dark blue block holding three shared layers, with three small config blocks beneath it for CS, Finance and Team 3. The Team 3 block is coloured orange to show that adding a team needs only one more config block.
Adding a third team: if you copy, you gain one more prompt to maintain. If you split into layers, the role, policy and customer-context layers stay as they are, and you only write one more department_config block.

Picture this. The internal assistant you built for the customer service team has been running smoothly for a few weeks. Then the head of accounting stops by and asks: “Can my team use it too?”

You have two options. The first is to copy the prompt, edit the copy, and maintain two separate versions forever. The second is to reorganise the prompt into layers, so that each new department only costs you one extra configuration block.

Forward Deployed Engineers run into this early in customer work, and only the second option holds up once a third and fourth department join the queue. This guide shows how to build that kind of prompt, one step at a time.

All you need is a text editor and any chat interface or playground to test in. The sample code is plain, simplified Python and does not call any provider’s API. The prompt snippets are in Vietnamese, as in the original deployment; the text around each one explains what it says.

What are you building?

The customer is a fictional logistics company with two departments sharing one internal assistant: customer service and accounting. The prompt will have four layers.

The role layer says who the assistant is. The policy layer sets out what it may and may not do. The customer context layer describes the company, and the department configuration layer changes depending on who is using the assistant.

DAIR.AI’s Prompt Engineering Guide notes that not every prompt needs every component, and that how you organise one depends on the task. Treat the four layers as a starting framework, then, and drop any layer your task does not need.

Step 1: write the role layer as if you only had one sentence

Anthropic’s prompting guidance says that a single sentence assigning a role in the system prompt is enough to steer Claude’s behaviour and tone. Learn Prompting defines the role component as giving the AI a persona or a specific perspective. Write the most important sentence first:

<role>
You are the internal assistant of a logistics company, helping staff look up procedures and draft responses.
</role>

(In English: “You are the internal assistant of a logistics company, helping staff look up procedures and draft replies.”)

Check: ask the same question (“What do we do when an order is late?”) twice, once with the <role> tag and once without. The answer with the role should sound like someone inside the company, not like a generic consultant.

Keep the company-level role broad. Each department’s detailed role goes in step 4.

Step 2: write policies with a “because”

The usual temptation here is to write a long list of “Do NOT…” rules with no explanation. Anthropic recommends spelling out the context or motivation behind each instruction, meaning why the behaviour matters, so that Claude understands what you are actually trying to achieve.

<policies>
  <policy>Do not include customers' bank account numbers in answers, because chat content may be forwarded outside the company.</policy>
  <policy>When unsure about a procedure, say clearly that you are unsure and point to the person to ask, because staff will follow the answer word for word.</policy>
</policies>

(The two policies: do not put a customer’s bank account number in an answer, because chat content may be forwarded outside the company; and when unsure about a procedure, say so and point to the right person to ask, because staff will follow the answer word for word.)

Check: ask something the list does not cover, such as a customer’s personal phone number. If the model has understood the reason (“may be forwarded outside”), it will be careful with the phone number even though you never wrote a rule for it. The reason is what lets a policy generalise.

Step 3: keep customer context separate from instructions

Anthropic recommends putting each kind of content in its own tag, such as <instructions>, <context> and <input>, so the model is less likely to mix them up. Customer context is the part most easily confused with instructions. A sentence like “the company prioritises business customers” may be a fact about the company, but the model may also read it as an order.

<customer_context>
  The company has two customer groups: individuals and businesses.
  The internal order management system is called the "waybill system".
</customer_context>

(The context states two facts: the company has two customer groups, individuals and businesses, and the internal order management system is called the “hệ thống vận đơn”, or waybill system.)

Check: reread every line in this tag. If a line reads like an instruction, move it to <policies>. The context tag should contain only facts.

Step 4: the department layer is the only part that changes

This layer holds everything that differs between departments. Anthropic recommends consistent, descriptive tag names across all your prompts, and nesting tags when the content is hierarchical.

According to Learn Prompting, each role produces different output that reflects its own perspective, while the output format is a separate component of the prompt whose job is to force the answer into a set structure.

<department_config>
  <department_role>You help accounting staff reconcile shipping invoices.</department_role>
  <main_instruction>Answer reconciliation questions based on the procedures in the context.</main_instruction>
  <output_format>Table with: order ID, amount, reconciliation status, notes.</output_format>
</department_config>

(For accounting: the role is helping staff reconcile shipping invoices; the main instruction is to answer reconciliation questions based on the procedures in the context; the output format is a table with order ID, amount, reconciliation status and notes.)

The <main_instruction> tag is not decoration. Learn Prompting warns that without a clear instruction, the AI tends to give generic or off-topic answers. For customer service, you keep the same three child tag names and change only their content. Its output format, for example, would be a short, polite reply to the customer.

The table below shows the only places where the two departments differ:

Child tag Accounting Customer service
department_role Helps reconcile invoices Helps draft replies to customers
main_instruction Answer reconciliation questions Draft responses to complaints
output_format Four-column table Short paragraph to the customer

The three layers above stay the same for both departments. To add a third department, you write one more <department_config> block.

Step 5: add examples and department policies, then assemble it in code

Anthropic advises that examples should closely match the task and be varied enough, and that they should be wrapped in <example> tags (grouped in <examples> when there are several) so Claude can tell examples apart from instructions. Put the examples inside <department_config>, because a model answer for accounting is no use to customer service.

Policies that apply to only one department belong in the same place, in a child tag such as <department_policies>, still with their “because” as in step 2. The code below is simplified. It only shows how one department’s pieces are assembled onto the shared part:

# Illustrative, simplified: combine the shared layers with one department's config
SHARED = "<role>...</role>\n<policies>...</policies>\n<customer_context>...</customer_context>"

DEPTS = {
    "accounting": {
        "department_role": "You help accounting staff reconcile shipping invoices.",
        "main_instruction": "Answer reconciliation questions based on the procedures in the context.",
        "output_format": "Table with: order ID, amount, reconciliation status, notes.",
        "department_policies": [
            "Do not write 'Reconciled' when the waybill system has no status yet, because accounting records entries based on the answer."
        ],
        "examples": [
            "Q: Has order VD-0001 been reconciled? A: | VD-0001 | ... | Reconciled | |"
        ],
    },
}

def build_prompt(dept: str) -> str:
    c = DEPTS[dept]
    policies = "\n".join(f"    <policy>{p}</policy>" for p in c["department_policies"])
    examples = "\n".join(f"    <example>{e}</example>" for e in c["examples"])
    return f"""{SHARED}
<department_config>
  <department_role>{c['department_role']}</department_role>
  <main_instruction>{c['main_instruction']}</main_instruction>
  <output_format>{c['output_format']}</output_format>
  <department_policies>
{policies}
  </department_policies>
  <examples>
{examples}
  </examples>
</department_config>"""

(accounting is the accounting department. Its department policy says: do not write “Đã đối soát” (reconciled) when the waybill system has no status yet, because accounting records entries based on the answer.)

Check: run print(build_prompt("accounting")) and read the output. Every opening tag needs a closing tag, and accounting’s own policy must not leak into the shared <policies> block.

Now look at what the output looks like. Suppose an accountant asks, “Has order VD-1023 been reconciled?” With a prompt that has only the three shared layers, a hypothetical answer might be a generic paragraph like this:

To check whether an order has been reconciled, you can look in the system
or contact the relevant department.

(Roughly: “To check whether the order has been reconciled, you can look in the system or contact the relevant department.”)

Once the accounting block is added, the same question (still a hypothetical illustration) produces exactly the four-column table the department needs:

| Order ID | Amount | Reconciliation status | Notes                                                                 |
| VD-1023  | —      | Undetermined          | Waybill system has no status yet, ask the person in charge of reconciliation |

The cell to look at is “Trạng thái đối soát” (reconciliation status). The model writes “Chưa xác định” (not determined) rather than filling in “Đã đối soát” to complete the table, because accounting’s own policy forbids that. The amount cell is left empty instead of holding an invented number. The “Ghi chú” (notes) column, which says the waybill system has no status yet and to ask the person in charge of reconciliation, comes from the shared policy about saying so when unsure.

Further check: if answers start copying an example almost word for word, your examples are too similar. Replace one with a completely different case.

Step 6: the colleague test

Anthropic offers a “golden rule”: give the prompt to a colleague who knows almost nothing about the task and ask them to follow it. If they are confused, Claude will be too.

Ask someone who is not on the project to read the whole prompt and then play the accounting department’s assistant. Every time they ask “what does this mean?”, you have found a line to rewrite.

Common mistakes

The first mistake is renaming tags between versions, for example writing <context> today and <background> next week. That cancels out the consistency Anthropic recommends.

The second is putting one department’s policy in the shared <policies> instead of <department_policies>, which saddles the other department with constraints that make no sense for it. The third is writing a department role so long that it overrides the company-wide role.

How does this request show up at a customer?

At a customer, nobody calls this a “layered prompt”. They say: “My team wants the assistant to answer a bit differently.”

Start by sitting down with that department and asking three questions: what role they need the assistant to play, what format answers must follow, and what it must never say, and why.

The answers to those three questions fill the <department_config> block, including <department_policies>.

On your CV, don’t just write “wrote prompts”. Say that you designed a single prompt shared by several departments, where adding a new department takes only one more configuration block.

Once a second department starts using the assistant, the hard part is no longer writing good sentences. It is deciding what belongs to the whole company and what belongs to a single department.

Was this article useful?

Use with your AI assistantAsk Claude ↗Ask ChatGPT ↗
4 sources
Read next on the roadmap · Stage 3: Applied AIThe temperature knob is disappearing, but tuning is notMax tokens, stop sequences and penalties still decide whether output is usable, and each client task, from extraction to support bots, needs its own settings.