TroubleshootingAuth issues
Locked out of admin
You can't log into Athena because there's no working admin identity
You're locked out of Athena. Either:
- The seeded admin password no longer works.
- Every admin's identity has been locked out via brute-force protection.
- The admin database state is somehow corrupt.
Diagnostic
Can you reach Athena at all?
curl -I https://iam.<domain>/api/health
# Expected: 200If 5xx, Athena itself is broken, see Incident Response. If 200, Athena's running, you can't auth.
Is your IAM admin identity present?
You need SSH or pgAdmin access to the IAM Kratos DB:
ssh prod 'podman exec olympus-postgres psql -U postgres -d iam_kratos -c "
SELECT id, traits->>'email' AS email, state
FROM identities;
"'Find your row. If state is anything other than active, that's the problem. If your email isn't in the list, the seeded admin was never created (or was deleted).
Is your account locked?
ssh prod 'podman exec olympus-postgres psql -U postgres -d olympus -c "
SELECT identifier, locked_until, reason
FROM lockouts
WHERE domain = '\''iam'\'' AND locked_until > NOW();
"'If your email is here with locked_until in the future, brute-force protection blocked you.
Recovery
Case A: account is locked, you remember password
ssh prod 'podman exec olympus-postgres psql -U postgres -d olympus -c "
DELETE FROM lockouts WHERE identifier = '\''you@yourdomain.com'\'' AND domain = '\''iam'\'';
"'Now try to log in.
Case B: you don't remember your password
Use the Kratos admin API to reset it via the recovery flow:
ssh prod 'podman exec iam-kratos kratos identities patch <your-identity-id> --endpoint http://localhost:7001 \
--set "/state=active" \
--set "/credentials/password/config/hashed_password=$(podman exec iam-kratos kratos hash-password "newpass123!")"'This sets your password directly. Then log in normally.
Case C: no IAM admin identity exists at all
This shouldn't happen if you deployed via Daedalus's Accounts step, but if for some reason there's no admin:
ssh prod 'cat > /tmp/admin.json <<EOF
{
"schema_id": "default",
"state": "active",
"traits": { "email": "you@yourdomain.com", "role": "admin" },
"credentials": {
"password": {
"type": "password",
"identifiers": ["you@yourdomain.com"],
"config": { "password": "newpass123!" }
}
},
"verifiable_addresses": [{
"value": "you@yourdomain.com",
"verified": true,
"via": "email",
"status": "completed"
}]
}
EOF'
ssh prod 'podman exec iam-kratos kratos identities import --endpoint http://localhost:7001 --schema-id default /tmp/admin.json'Prevention
- Always have at least two admin identities. Backup admin is locked-out insurance.
- Document the SSH and DB access paths. This page assumes you have them. Write a runbook for who on your team has them.
- Set the brute-force lockout high enough. A normal user typing a wrong password three times shouldn't lock the only admin.
- Don't rotate
ENCRYPTION_KEYwithout testing recovery first. A failed rotation can leave admins unable to log in if Athena depends on encrypted settings during startup.