Make ApiCallTracker.save() atomic to prevent corrupt history files

Cin observed that database/api_call_history.json was occasionally
landing on disk truncated mid-write — `_load()` would log
`History file is not valid JSON, starting fresh` and 24h of metrics
would be lost.

Root cause: `save()` opened the file in 'w' mode (which truncates to
0 bytes immediately) and then streamed JSON via `json.dump`. Any
SIGINT/SIGTERM/crash between truncate and final write left the file
half-formed — exactly Cin's symptom of the JSON cutting off mid-array.

Switch to the standard atomic pattern: write to a sibling .tmp file,
flush + fsync, then `os.replace` (atomic on every platform we run on).
Failed writes also clean up the leftover .tmp file. The canonical
file is now either the previous good copy or the new good copy —
never a partial one.
This commit is contained in:
Broque Thomas 2026-04-22 22:15:09 -07:00
parent e66af77ff6
commit 02f26bf338

View file

@ -267,7 +267,12 @@ class ApiCallTracker:
def save(self):
"""Persist 24h minute history to disk. Call on shutdown."""
"""Persist 24h minute history to disk. Call on shutdown.
Uses an atomic write (write to a sibling .tmp file, fsync, rename) so
a SIGINT/SIGTERM/crash mid-write can't leave the JSON file truncated.
Without this, an interrupted shutdown corrupts the history file and
the next startup loses 24h of metrics."""
try:
now = time.time()
cutoff = now - 86400
@ -283,10 +288,22 @@ class ApiCallTracker:
if entries:
data[key] = entries
events = [dict(e) for e in self._events if e['ts'] >= cutoff]
with open(_PERSIST_PATH, 'w') as f:
json.dump({'ts': now, 'history': data, 'events': events}, f)
payload = {'ts': now, 'history': data, 'events': events}
tmp_path = _PERSIST_PATH + '.tmp'
with open(tmp_path, 'w') as f:
json.dump(payload, f)
f.flush()
os.fsync(f.fileno())
os.replace(tmp_path, _PERSIST_PATH)
except Exception as e:
logger.error(f"[ApiCallTracker] Failed to save history: {e}")
# Best-effort cleanup of stale tmp file from a failed write.
try:
if os.path.exists(_PERSIST_PATH + '.tmp'):
os.remove(_PERSIST_PATH + '.tmp')
except Exception:
pass
def _load(self):
"""Restore 24h minute history from disk. Called on init."""