Add the anchor-free, center-heatmap detector CLAUDE.md §3.3 specifies:
- targets.py: encodes ground-truth boxes into a per-class Gaussian
heatmap + wh regression target + center mask, using the standard
CornerNet/CenterNet gaussian-radius formulation.
- model.py: a small conv backbone (stride 4) with heatmap (sigmoid)
and wh regression heads - exactly the two outputs CLAUDE.md
specifies, sized with the 66ms/tick budget in mind.
- losses.py: modified focal loss (heatmap) + masked L1 (wh), combined
with the standard CenterNet wh_weight=0.1.
- client.py / dataset.py: pull a promoted dataset version + its set's
class list from the labeling backend and turn it into a
torch.utils.data.Dataset, reading images from local disk (same
machine as FRAMES_ROOT).
- train.py: wires it into a basic DataLoader -> train loop ->
per-epoch checkpoint.
The exact loss weighting/architecture sizing is a reasonable, standard
v1 default, not a tuned final answer - CLAUDE.md's own roadmap flags
the exact formulation as still open; this is the starting point to
iterate from.
16/16 unit tests pass on CPU with synthetic data (target/model/loss
correctness, dataset-version parsing). Beyond that, ran a real
end-to-end smoke test: live labeling backend -> promoted dataset
version -> `spelunkai-train` actually training one real epoch against
it and writing a checkpoint. Not verified: multi-epoch convergence on
real data, which needs jai's GPU and real labeled frames.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add DatasetVersion/DatasetVersionFrame/DatasetVersionLabel and a
promote endpoint (POST /sets/{id}/dataset-versions) that freezes a
set's currently-reviewed frames (or an explicit frame_ids selection)
into a named, immutable snapshot: it copies each label's data at
promotion time rather than referencing the live rows, so later edits
or deletes to those labels can't retroactively change an already
-promoted version. GET /dataset-versions/{id} returns the frozen
frames+labels - this is what the training pipeline will eventually
pull from.
This was the labeling backend's last deliberately-deferred piece from
the original data model (needed its own design pass for snapshot
semantics). 31/31 backend tests pass, including one that promotes a
version, edits and deletes the live label afterward, and asserts the
snapshot is untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Registering thousands of extracted frames one-by-one via POST /frames
doesn't scale, so add `spelunkai-labeling-backend ingest-session`: it
writes Frame rows directly against the database (no server needs to
be running) for all frame_*.png files under a directory already
placed under FRAMES_ROOT. Idempotent per (session_name, frame_index),
so re-running after copying more frames only inserts the new ones.
Restructured the CLI into subcommands (serve / ingest-session) while
keeping `spelunkai-labeling-backend` with no arguments working exactly
as before (defaults to serve), verified against a live run. 22/22
backend tests pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add a framework-free HTML/CSS/JS labeling UI: set/session/status
filters, a frame browser, drag-to-draw/move/resize bounding boxes with
a per-set class picker, per-frame status control, and Left/Right frame
navigation. No login - a locally cached username is sent for
attribution only, matching the backend's get-or-create user model.
Backend additions the frontend needed: serve frame images from a
configurable FRAMES_ROOT via a /images static mount, permissive CORS
(internal tool, not publicly exposed), and a GET /sets/{id}/frames
endpoint returning frames joined with their per-set label status
(defaulting missing rows to unlabeled) for the frame browser.
Verified the full call chain end-to-end against a running backend +
static frontend server (set/class creation, frame ingest, image
serving, label CRUD, status updates, CORS preflight) - every field
name the JS reads matches the API responses. 18/18 backend tests
pass. Not yet verified: actual interactive browser use (no browser
tooling available here) - try drag-to-draw/resize locally.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add the core labeling data model (label sets, ad-hoc Main->Sub class
hierarchy, frames, bounding-box labels, per-frame/per-set label
status) behind a FastAPI app, with SQLite as the default swappable
DATABASE_URL. Multi-user support is attribution-only for now
(get-or-create by username, no login flow yet). Dataset
versioning/promotion is intentionally deferred - it needs its own
design pass around snapshot semantics.
Each test gets a fully isolated app+DB via create_app(database_url=...)
rather than relying on process-global state. 13/13 tests pass; also
verified live end-to-end against a running uvicorn instance.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Slice a recorded session's lossless video into individual frame PNGs
(ffmpeg -vsync 0, no drop/dup) via a new `extract-frames` subcommand,
with frame count cross-checked against the manifest and input log so
any capture-rate drift surfaces immediately instead of silently
misaligning frames and logged input later. Includes a real end-to-end
test against an ffmpeg-generated synthetic video (skipped when ffmpeg
isn't installed, e.g. bare WSL).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add ffmpeg/x11grab-based video capture (libx264rgb, qp=0, true lossless
RGB) and an evdev-based keyboard state logger, orchestrated by a single
frame-tick loop so each JSONL input row lines up 1:1 with its video
frame. Unit-tested with fake keyboard/video components (no real device
or ffmpeg needed); real hardware capture still needs validation on the
recording PC.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add top-level README, .gitignore, and per-component directories
(recording, labeling backend/frontend, training, inference, control)
with READMEs and pyproject.toml/package skeletons per CLAUDE.md §2-3.
No implementation yet, just structure to build against.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
