Keep logs for 12 months, delete data without delay: an API architecture problem

One standard makes you keep audit logs for at least 12 months. A law makes you erase personal data “without undue delay” when there are grounds to do so. Put a customer’s email address into a log store that must live for 12 months and you have created a place every future deletion request will have to reach.

That is the problem facing anyone building APIs for payments, healthcare or an e-commerce marketplace with users in Europe and the United States. Read GDPR, CCPA/CPRA, HIPAA and PCI DSS clause by clause and most of it comes down to three engineering decisions: which fields the API accepts, what the logs record, and where data lives and for how long.

For anyone aiming to work as an FDE, this is worth understanding at the design level. Once you touch a customer’s real data, these are the decisions you will have to make, or at least be able to explain.

Two forces pulling on the same log table

The first force says “remember”. According to OneUptime’s analysis of PCI DSS, Requirement 10.5.1 calls for at least 12 months of audit history, with the most recent three months immediately available.

Wiz Academy also classes logging that tracks every API call and response as a compliance control, because it records when and in what context data was accessed. Traceable lists “insufficient logging and monitoring” among the top compliance risks for APIs.

The second force says “forget”. GDPR, which has applied in every EU member state since 25 May 2018, includes Article 25, requiring systems to be designed to implement principles such as data minimisation. Article 17 obliges controllers to erase personal data without undue delay when there are grounds to do so.

GDPR is not only for European companies. It governs the processing of data belonging to people in the EU, so a team in Hanoi serving German customers falls within its scope.

The two forces only collide, however, when logs contain personal data. A log line reading “user 84213 read record 5521 at 09:14, result 200” can sit untouched for 12 months.

A log line that copies a full response body with a name, email and phone number turns the log store into a second personal database, and every deletion request has to reach into it.

The leak is usually in the “details” field

“Log every API call and response” sounds safe, but it is also the easiest way to leak data. OneUptime points to a very common failure pattern: a generic details object in the audit record becomes the leak path for PANs, CVVs and authentication tokens. A developer adds it to make debugging easier, then one day someone stuffs the whole request body into it.

Under PCI DSS the consequences are starkest. Card verification codes and other sensitive authentication data must not be retained after a transaction is authorised. If a CVV slips into the logs, and those logs must be kept for 12 months under 10.5.1, you have put yourself in violation for 12 months.

Picture a POST /payments endpoint. The “convenient” log looks like this:

{"event":"payment.create","user":"[email protected]","details":{"pan":"4111...","cvv":"123","amount":250000}}

A log designed for compliance keeps only what an auditor needs:

{"event":"payment.create","actor_id":"u_84213","resource":"pay_9f2a","card_ref":"tok_7c1e","result":"authorized","ts":"2026-10-08T09:14:02Z"}

The difference lies in how the data is blocked. OneUptime advises stopping sensitive payloads at ingest rather than relying on every developer remembering to mask data. In practice, the most durable approach is an allowlist schema: the logger rejects any field not on the list, and no free-form field exists at all.

Deletion is a workflow, not a DELETE statement

If logs are where the law makes you remember, user rights are where it makes you prove you know where the data lives.

Wiz Academy summarises that under CCPA/CPRA, APIs must support four rights: to know, to delete, to opt out of sale or sharing, and to correct, while being transparent about collection, processing and retention.

The California Attorney General’s office gives a concrete figure: requests to know, delete or correct must be answered within 45 calendar days, extendable by a further 45 days. On a calendar, a request received on 1 March has a default deadline of 15 April, and the end of May at the latest if extended.

Your system needs to store the date received, the status and the deadline for each request. That means a privacy_requests table with a state machine, not a Jira ticket drifting around.

The CCPA right to delete also comes with “certain exceptions”. So the deletion flow cannot be absolute: it must check which records need to be kept for legitimate reasons, delete the rest, and record that decision.

GDPR Article 17 adds pressure on speed, and that pressure spreads to every place the data has a copy: the primary database, replicas, caches, log stores and backups.

On risk, the CCPA generally does not let individuals sue on their own, except in the case of data breaches, where statutory damages are assessed per incident, up to USD 750 per incident. Other violations are handled by the Attorney General or the state’s privacy protection agency (CPPA).

A breach is the rare case in which users can sue directly, so a log store full of personal data is exactly the slice of risk engineers can cut out through design.

Which provision touches which layer

Sorting the requirements into three technical layers makes the engineer’s share of the work clear:

Regulation	Key provisions	API	Logs	Storage
GDPR	Article 25: data protection by design; Article 17: right to erasure	Accept only the fields you genuinely need	Do not copy personal data into logs	Erasable in every copy, including backups
CCPA/CPRA	Four rights: know, delete, opt out of sale/sharing, correct; 45-day deadline	An endpoint or flow for each right	Record how each request was handled	Deletion with an exception check
HIPAA	Privacy Rule, Security Rule, Breach Notification Rule	Controls on receiving and processing ePHI	Enough of a trail to detect and report breaches	Security controls applied to ePHI
PCI DSS 4.0	API-specific requirements; Requirement 10.5.1	Authentication, authorisation, encryption	Kept 12 months, 3 months immediately available	No CVV retained after authorisation

Read down the columns and the log layer is where requirements overlap most densely, which makes it the layer most worth designing carefully from day one.

HIPAA and PCI DSS 4.0 are blunter than you might think

Most regulations never mention the word “API”. Wiz Academy notes the exception: PCI DSS 4.0, which sets compliance requirements specifically for APIs, covering authentication, access control, encryption and vulnerability management.

HIPAA comprises three rules: the Privacy Rule, the Security Rule and the Breach Notification Rule. As Wiz interprets them, APIs must control both the intake and the processing of ePHI, carry the accompanying security measures, and be able to support breach notification.

The third rule brings us straight back to logs: without a complete access trail, you cannot say whose data was exposed.

So if you can do only one thing first in a customer’s system, map the flow of sensitive data: which endpoints it enters through, which logs it is written to, which stores it is copied into. That map answers a PCI audit, a GDPR erasure request and a HIPAA notification duty at the same time.

For an FDE, the sensible moment to do this is the first week on site, during the data-flow review before connecting any agent or pipeline to the customer’s systems. Finding a details field full of card numbers then is far cheaper than finding it during an audit.

A skill worth putting on your CV

If the project you are targeting serves users in the EU or California, or handles card or health data, these regulations follow the project wherever you happen to sit.

When reading job descriptions, look for terms such as PCI, HIPAA, GDPR, audit logging, data retention or PII handling, and treat them as a signal that you will be making design decisions, not just writing CRUD.

On your CV, do not write “knowledge of GDPR”. Write what you built: an allowlist logger that blocks PANs and tokens at ingest, a data-deletion workflow with a 45-day SLA and an exception branch, a log retention policy of 3 months hot and 9 months in cold storage to reach the full 12 months.

An interviewer can dig into any line of that, and you can answer with the decisions you actually made.

Data law will keep changing, but the design principles are stable: logs record actions by ID, and personal data lives in a single place that knows how to delete itself. Engineers who design that way from day one will not have to rewrite the system when the auditors come knocking.