If your repo went public today, whose keys would leak?
When one codebase serves many customers, configuration and secrets need separate homes, and a secret has to be revocable on the day a contract ends or leaks.
In brief
- The code is identical for every customer; only the configuration differs. An if-branch on a customer's name means the two have been mixed.
- A ConfigMap cannot keep a secret, and by default a Kubernetes Secret is also stored unencrypted in etcd.
- Secrets need an expiry and must be revocable: once a lease is revoked, the secret stops working immediately, whether you are leaving a customer or suspect a leak.
Your solution is running smoothly at the first customer. Then a second customer signs, and to hit the deadline you add a line, if customer == "B":, to switch the CRM URL. By the third customer the repo has gained two files, config_a.yaml and config_b.yaml, plus a token someone committed at 11pm.
Nobody sets out to wreck a codebase. It decays one rushed handover at a time. For an FDE this is routine, not an edge case: the same product goes into different infrastructure, different data and different security rules.
The skill required is to tell code, configuration and secrets apart, then put each where it belongs. Get that right and the tenth customer costs one new set of values, not one more code branch.
Code does not change by customer; configuration does
The config principle of the Twelve-Factor App starts from an observation: configuration varies substantially between deploys, while code does not. For an FDE, a “deploy” means a customer. Each customer is a deploy with its own API address, model name, batch size and access keys.
To check whether the separation is complete, 12-factor offers a test: could the codebase be made open source at any moment without exposing a single credential? For FDEs the test needs a second clause. The repo must also not reveal customer names, internal URLs or how a customer organises its systems.
12-factor chooses environment variables to hold configuration. The practical reason for an FDE is plain: when you are standing in a customer’s server room, allowed to change parameters but not to rebuild the image, an environment variable can be changed on the spot without touching a line of code.
Don’t multiply “environments” by the number of customers
By reflex, many people name configuration sets after environments: dev, staging, prod. With three customers the list becomes prod-a, staging-a, prod-b, and before long nobody remembers how one file differs from another.
A tidier approach is to describe each customer as a flat list of variables, typed and documented. Picture an agent that reads tickets from a customer’s CRM, deployed in two places:
| Variable | Customer A (hypothetical) | Customer B (hypothetical) | Type |
|---|---|---|---|
TENANT_ID |
acme |
beta |
Configuration |
CRM_BASE_URL |
https://crm.a.example |
https://crm.b.example |
Configuration |
BATCH_SIZE |
not set, uses the shared default in code | 20 |
Configuration |
CRM_API_TOKEN |
in the secret store | in the secret store | Secret |
The last column matters most. Customer-specific values in the first three rows must not live in the repo; the code may hold only a shared, harmless default, such as the batch size.
If those three values leak, the damage is mostly information about the customer’s systems. If the fourth leaks, it is a security incident with the customer, and possibly a breach of contract.
Example: a loader that stops as soon as a variable is missing
The most dangerous configuration errors usually come from a default nobody noticed. The application starts normally and, three days later, calls the wrong customer’s CRM. The loader below does the opposite: a missing required variable stops it at once, and secrets are never printed.
# config.py: read configuration from environment variables, stop at once if any is missing
import os
from dataclasses import dataclass, field
class ConfigError(RuntimeError):
pass
def _required(name: str) -> str:
value = os.environ.get(name)
if not value:
raise ConfigError(f"Missing required environment variable: {name}")
return value
@dataclass(frozen=True)
class Settings:
tenant_id: str
crm_base_url: str
batch_size: int
crm_api_token: str = field(repr=False) # secret: never appears in logs
def load() -> Settings:
return Settings(
tenant_id=_required("TENANT_ID"),
crm_base_url=_required("CRM_BASE_URL"),
batch_size=int(os.environ.get("BATCH_SIZE", "100")), # shared default, belongs to no customer
crm_api_token=_required("CRM_API_TOKEN"),
)
Three points stand out. Nothing in the code names a customer. Defaults in code are allowed only for harmless parameters such as BATCH_SIZE: 100 is a safe shared value, not any one customer’s number, and a customer that needs something else sets the variable, as customer B sets 20.
Anything that decides where data goes must be declared explicitly, with no default. And repr=False means a debugging print(settings) will not leak the token into the customer’s logs.
On Kubernetes: ConfigMap for configuration, Secret for keys
If the customer runs Kubernetes, a ConfigMap is the natural home for the first three variables, since it is meant for non-sensitive settings that vary by environment. One image, many customer clusters.
envFrom:
- configMapRef:
name: app-config-customer-b
- secretRef:
name: app-secrets-customer-b
The trap is in the second entry. The Kubernetes documentation states plainly that a ConfigMap provides no secrecy or encryption, so sensitive data must go into a Secret, which keeps passwords and tokens out of application code. Yet even a Secret is, by default, stored unencrypted in etcd, and anyone with access to the API or to etcd can read it.
A Secret is therefore safe only with extra protection. At the kickoff with the customer’s infrastructure team, ask directly: is Secret encryption enabled on this cluster, and who currently has read access to Secrets in this namespace?
Secrets need an expiry date
A token that lives forever is a dormant risk waiting for the day it leaks. OWASP advises rotating secrets regularly, so that a stolen credential is usable only for a short time, and centralising the storage, provisioning, auditing and rotation of secrets in one place.
Vault takes the idea further with leases. When a lease is revoked, the secret becomes invalid immediately and can no longer be renewed.
That transforms the two hardest situations an FDE faces: when a contract with a customer ends, you revoke that customer’s leases rather than combing through every machine; when you suspect a leak, you revoke first and investigate afterwards.
The boundaries between customers must be just as clear. In a multi-tenant system, each tenant’s secrets need strict isolation, for example a separate store per tenant, and no single encryption key shared across customers, so that an incident at one does not spread to another.
In a CI/CD pipeline, that means a separate service account per customer: the pipeline deploying for customer B has no reason to be able to read customer A’s token.
Four mistakes that break configuration on site
The most common is letting a customer’s name slip into the code, even through a “temporary” if branch. Next is using defaults for variables that decide where data goes. Then comes putting a token in a ConfigMap because “it’s inside the customer’s cluster anyway”.
The last is subtler: your dev machine runs SQLite, the customer runs Postgres, and the configuration breaks only once you are on site. 12-factor recommends keeping development and production as similar as possible, so that the only differences lie in configuration. Run the same kind of database and the same kind of queue as the customer on your own machine.
Customer B is leaving and the token may have leaked: what do you do?
This is a question worth rehearsing before a customer asks it, and a good scenario to practise before an FDE interview. OWASP lists rapid response to exposed secrets among its key concerns.
A good answer works through each layer in turn: revoke B’s leases, confirm that B’s CI service account cannot touch any other customer’s secrets, issue new secrets for whatever is still running, record everything in the audit log, and change no code at all.
If you have done this on a real project, put it in your CV as one concrete sentence: the number of customers, how secrets were separated, how long revocation took.
By the handover to the tenth customer, your job is simply to fill in a list of variables and create a new lease. The codebase stays clean enough to open-source at any moment.
Was this article useful?
Thanks for the feedback!
8 sources
- The Twelve-Factor App – Config
- The Twelve-Factor App – Dev/prod parity
- Secrets | Kubernetes
- ConfigMaps | Kubernetes
- Lease, renew, and revoke | Vault | HashiCorp Developer
- Secrets Management Cheat Sheet – OWASP Cheat Sheet Series
- How to Implement Tenant-Specific Secret Stores with Dapr · 2026-03-31
- Establish multi-tenant security for B2B SaaS applications · 2026-06-18