Tamper-evident audit logs
Detect (or prevent) someone editing the audit trail
A security audit log is only useful if you can trust it. If an attacker gains DB access, they might also try to scrub their tracks from the audit log. Tamper-evident logs ensure you'd at least know if someone tried.
Threats
- Attacker deletes audit rows showing their actions.
- Attacker rewrites rows (changing actor / target / outcome).
- Insider with DB admin runs
DELETE FROM security_audit.
Levels of defense
Level 1: Append-only DB table
-- Postgres: rules to prevent UPDATE/DELETE
CREATE RULE audit_no_update AS ON UPDATE TO security_audit DO INSTEAD NOTHING;
CREATE RULE audit_no_delete AS ON DELETE TO security_audit DO INSTEAD NOTHING;Now even the application DB user can't modify rows. But a superuser can drop the rules.
Level 2: Separate audit DB
Audit logs in a separate Postgres instance. App writes via a narrow API (insert-only).
Compromised app DB → audit DB unaffected. Different credentials, different host ideally.
Level 3: Append-only stream
Stream audit events to Kafka, AWS Kinesis, or S3 (object lock). These services don't allow updates.
# S3 with object lock, write once, never overwrite
aws s3api put-object \
--bucket olympus-audit \
--key 2026/05/15/12-30-45.json \
--body event.json \
--object-lock-mode COMPLIANCE \
--object-lock-retain-until-date 2033-05-15S3 with compliance mode: even root can't delete until retention expires.
Level 4: Cryptographic chain (Merkle / hash chain)
Each row contains a hash of the previous row:
ALTER TABLE security_audit ADD COLUMN prev_hash TEXT;
ALTER TABLE security_audit ADD COLUMN this_hash TEXT;On insert:
const prev = await db`SELECT this_hash FROM security_audit ORDER BY created_at DESC LIMIT 1`.first();
const data = `${row.actor_id}:${row.action}:${row.target_id}:${row.timestamp}`;
const thisHash = sha256(prev?.this_hash + data);
await db`INSERT INTO security_audit (... , prev_hash, this_hash) VALUES (... , ${prev?.this_hash}, ${thisHash})`;To verify: re-compute the chain. Mismatch → tampering detected.
Caveats:
- Chain breaks if a row is deleted (next row's prev_hash doesn't match).
- Inserts must be serialized (race condition if parallel).
Level 5: Notarized
Periodically anchor the latest chain head to an external system:
- Public blockchain (Bitcoin, Ethereum), tamper-evident anchor.
- Notary service (Cloudflare AOC, RFC 6962 Certificate Transparency).
- Hash to your status page (publicly).
Daily, post the latest chain head's hash. If attacker tampers, they can't change the publicly-posted hash.
What Olympus does
Default level: 1 (rules to prevent UPDATE/DELETE on security_audit).
For level 3-5, build custom, Olympus doesn't include out of the box.
Recommended baseline
For most:
- Level 1 + Level 3 (S3 object lock for backup).
Daily backup of security_audit to S3 with 7-year retention, compliance mode. Even if DB is wiped, audit history survives.
# /etc/cron.daily/audit-backup
psql -d olympus -c "COPY (SELECT * FROM security_audit WHERE created_at > NOW() - INTERVAL '24 hours') TO STDOUT" \
| gzip \
| aws s3 cp - "s3://olympus-audit/$(date +%Y/%m/%d).csv.gz" \
--object-lock-mode COMPLIANCE \
--object-lock-retain-until-date $(date -d "+7 years" -I)Detection vs prevention
You can never prevent tampering by a sufficiently-privileged attacker (root/DBA). What you CAN do: ensure tampering is detected.
Level 4-5 give you that. Even if attacker scrubs DB, the externally-anchored chain reveals discrepancy.
Operationalizing
Daily verify
Cron job verifies the chain:
node verify-audit-chain.js > /tmp/verify.log
if grep -q "MISMATCH" /tmp/verify.log; then
alert-oncall "Audit log tampering detected"
fiQuarterly anchor
# Get latest chain head
psql -c "SELECT this_hash FROM security_audit ORDER BY id DESC LIMIT 1" -tA > head.txt
# Post publicly (gist, status page, blockchain)
gh gist create head.txt --public --desc "Audit log anchor $(date -I)"What gets logged
Audit only events that prove or refute claims of malicious behavior:
- Authentication events (success, failure, MFA enroll/disable).
- Authorization grant/revoke.
- Settings changes (especially security settings).
- Admin actions (impersonation, password reset by admin).
- Data exports (GDPR DSR).
- Account changes (email change, role change).
Don't audit:
- Every API call (volume too high).
- Read access (unless required by regulation).
- Internal system events.
Right granularity: every event that, in retrospect, you'd want to ask "did X happen?"