4aee18dcbe
Self-contained HTML status page served from the ingester's FastAPI app.
Polls /health, /sources, /stats, /jobs?status={claimed,dead,succeeded}
every 3s from the browser and renders queue chips, sources with
last-poll/skip-reason/circuit state, active jobs with cancel, recent
failures with retry, and recently-completed feed with op badges so
DELETE rows are visually distinct from UPSERTs. Zero external deps —
single static HTML, no CDN, no fonts, no images. Works offline.
To support the dashboard:
- New /stats endpoint exposing rolling throughput (5m/30m/1h), worker
occupancy, oldest-queued age, and per-source DLQ + queue-depth
breakdowns. Each field is a single SQL aggregation against the queue.
- JobRepo gains count_succeeded_since, oldest_queued_age_seconds,
counts_by_source.
- SourceSummary gains last_skip_reason. BasePoller now records the
reason the most recent sweep attempt was skipped ("pending_work" /
"circuit_open"), cleared on the next successful poll. Closes the
gap where operators couldn't tell from /sources alone why a source
wasn't picking up new work.
Auth: dashboard route is unauthenticated (markup only). The JS attaches
the bearer to its own JSON fetches; on 401 it prompts once and stashes
the token in localStorage.
Two Logfire fixes that landed alongside:
- Drop logfire.instrument_fastapi() and the [fastapi] extra. The control
plane is polled frequently (dashboard + docker healthcheck), so every
endpoint became a span and drowned the useful traces. logfire itself
stays — pulled in transitively via pydantic-ai-slim[logfire] — so
ingester.poller.* / ingester.job / document.* spans keep emitting.
- Wrap FSPoller._handle_watch_change in an ingester.poller.watch_event
span and pass _enqueue_extra. Without this, the watchfiles callback
ran with no active context, the _otel carrier in job.extra was empty,
and the worker's ingester.job span surfaced as an orphan trace root
instead of nesting under the FS event that caused it.
38 lines
1.3 KiB
Python
38 lines
1.3 KiB
Python
from fastapi import APIRouter, Depends
|
|
|
|
from haiku.rag.ingester.api.schemas import (
|
|
StatsResponse,
|
|
ThroughputStats,
|
|
WorkerStats,
|
|
)
|
|
from haiku.rag.ingester.api.server import APIState, get_state
|
|
|
|
router = APIRouter(tags=["stats"])
|
|
|
|
|
|
@router.get("/stats", response_model=StatsResponse)
|
|
async def stats(state: APIState = Depends(get_state)) -> StatsResponse:
|
|
"""Dashboard summary: rolling throughput, worker occupancy, backlog age,
|
|
and per-source DLQ / queue depth. Each field is a single SQL aggregation
|
|
against the queue file — cheap to call every few seconds."""
|
|
jobs = state.job_repo
|
|
|
|
counts = await jobs.counts_by_status()
|
|
worker_total = (
|
|
state.config.ingester.workers.worker_count if state.pool is not None else 0
|
|
)
|
|
|
|
return StatsResponse(
|
|
throughput=ThroughputStats(
|
|
succeeded_5m=await jobs.count_succeeded_since(300),
|
|
succeeded_30m=await jobs.count_succeeded_since(1800),
|
|
succeeded_1h=await jobs.count_succeeded_since(3600),
|
|
),
|
|
workers=WorkerStats(
|
|
busy=counts.get("claimed", 0),
|
|
total=worker_total,
|
|
),
|
|
oldest_queued_age_s=await jobs.oldest_queued_age_seconds(),
|
|
dlq_by_source=await jobs.counts_by_source("dead"),
|
|
queue_depth_by_source=await jobs.counts_by_source("queued", "claimed"),
|
|
)
|