Audit trail & integrity

Every request against RedScribe is recorded in an audit log built around a hash chain: editing a stored entry, including directly through the database rather than the app, is detectable, because it invalidates every hash that comes after it. That’s a real, useful property, and this page explains how it works. It’s also a narrower guarantee than “tamper evident” suggests — see Limitations below for the ways a person with direct database access can alter history without it being caught. For the operational side, viewing entries, running a purge, setting up cron, see Audit log.

What gets logged

Almost every request is logged, not just the ones that change data. Ordinary page views and API reads are recorded too. A small number of high-frequency or non-sensitive endpoints are excluded by design: the report live-preview endpoint (which would otherwise flood the log on every keystroke of report configuration), the health check, the CSP violation-report endpoint, and the once-a-minute session check each open tab makes. Every login attempt, successful or not, is recorded separately; see Authentication & sessions for how that feeds into lockout.

Each entry records who did it (user, username, and their role at the time, kept even if the account is later deactivated, which the audit log marks with “(deactivated)” beside the name; an erased account’s personal data is removed but the entries stay, with the username as it was then), what they did (the action, HTTP method, path, and status code), which engagement it touched, the client IP, query string, referer, user agent, and how long the request took.

What’s redacted before it’s ever written

A handful of fields are sanitized before an entry is written, so a sensitive value is never at rest in the log in the first place, rather than being redacted after the fact:

• Password-reset and account-setup links have their token stripped out of the stored path.
• Any query string parameter whose name looks like token, password, secret, key, otp, auth, signature, or credential has its value replaced, regardless of what the parameter is actually for.
• A referer URL that contains something that looks like a bearer token gets that segment redacted the same way.
• Query string, referer, and user agent are all capped at a few hundred characters each, redacted or not, so a single request can’t bloat an entry.

The hash chain

Each audit entry’s integrity hash is computed by folding together its own fields, plus the hash of the entry immediately before it, using a keyed HMAC (not a plain hash) so the hash can’t be recomputed by someone without access to the server’s own signing key. The example below shows what goes into that computation, in order, for a single entry:

canonical = "|".join([
    prev_hash, pk, created_at, actor_id, actor_username, actor_role,
    action, method, path, status_code, engagement_id, object_ref,
    ip_address, query_string, referer, user_agent, duration_ms,
])
entry_hash = HMAC-SHA256(key=<server signing key>, msg=canonical)

The very first entry ever written chains onto a fixed, well-known starting value. Because each entry’s hash folds in the one before it, changing any field on any row, including by editing the database directly rather than going through the app, changes that row’s hash. That, in turn, breaks the chain for every row written after it. The tamper is detectable anywhere downstream of the edit, not just on the row that was actually changed.

Rotating the server's secret key breaks verification of old entries

The hash chain is keyed with the same secret Django uses for other signing purposes. If you rotate that secret on a running instance, every previously written entry will fail verification, because verification always checks against the current key. That’s expected, not a bug, but it means you should think about the effect on historical audit verification before rotating it.

Concurrency safety

Appending a new entry takes a database-level lock first, so that two requests arriving at the same time can’t both read the same “last entry,” compute a hash against it, and insert two entries that both claim to follow the same predecessor. Writes to the chain are serialized, even under concurrent load.

Verifying the chain

docker compose exec web python manage.py verify_audit_log

This walks every entry in order, recomputing each one’s expected hash and comparing it to what’s stored. It fails loudly, with a non-zero exit code, the moment a stored hash doesn’t match, and names the first entry where that happened.

The oldest surviving row is a special case, since there’s nothing earlier to check its prev_hash against. The command reports it as an anchor rather than a failure, and tells you which of two things is going on:

• If its hash matches what you’d get from the fixed starting value, it’s genuinely the first entry ever written.
• Otherwise, it’s flagged as an oldest surviving row whose earlier history may have been purged, which is expected after a legitimate retention purge, not a sign of tampering.

Limitations

The hash chain catches one specific thing well: an edit to a row that’s still in the table, wherever it happens in the chain, including an edit made directly through the database rather than the app. It’s weaker against a few other things a person could do with direct database access, worth understanding plainly rather than assuming away:

• Deleting the newest entries leaves no trace. Nothing in a later row depends on rows that were never written, so deleting, say, the last ten entries right after doing something and having verification run afterwards reports the (now-shorter) chain as fully intact. There’s nothing downstream of a deleted tail to invalidate.
• Deleting a contiguous span from the front looks identical to a legitimate retention purge. As covered above, the oldest surviving row is always trusted as a new anchor, precisely so routine retention purging doesn’t get flagged as tampering. That design choice means an attacker who deletes, say, the first 10,000 rows gets the same “anchor, not a failure” result a real purge would.
• Verification uses the same secret the app signs sessions and CSRF tokens with (SECRET_KEY). Anyone who has that secret, which in practice usually means anyone with meaningful access to the running instance or its environment, can recompute a fully self-consistent chain after editing any row, not just the newest or oldest ones.
• Login-attempt records aren’t part of the chain at all. LoginAttempt rows are retained and purged on the same schedule as the audit log (see Audit log), and it’s easy to assume they carry the same integrity guarantee as a result. They don’t — they’re a plain table, with no hash chain of their own.

None of this makes the hash chain pointless. It’s a real deterrent against casual or accidental edits, and against anyone who can reach the database but not the app’s signing secret. It just isn’t a defence against a determined actor who already has both — for that threat, the honest answer is infrastructure controls outside RedScribe itself: restricting who holds direct database credentials, shipping audit events to a separate system in near-real-time, or a database configured so the application’s own role can’t delete from this table at all.

Partitioning and retention

The audit log is stored as one partition per calendar year, created ahead of time:

python manage.py create_audit_partition            # creates next year's partition
python manage.py create_audit_partition --year 2028 # or a specific year

That structure makes retention cheap: purging drops an entire year’s partition at once when it’s fully past the retention cutoff, rather than deleting rows one at a time, and only falls back to row-by-row deletion for a partition that’s partially expired. The same purge also removes old login-attempt records on the same schedule. By default, entries are kept for 6 years, matching HIPAA’s general documentation-retention guidance. See Audit log for tuning that to your own compliance obligations.