History

Jonas a7c370b3d4 Implement recording tool: lossless capture + frame-synced input log

Add ffmpeg/x11grab-based video capture (libx264rgb, qp=0, true lossless
RGB) and an evdev-based keyboard state logger, orchestrated by a single
frame-tick loop so each JSONL input row lines up 1:1 with its video
frame. Unit-tested with fake keyboard/video components (no real device
or ffmpeg needed); real hardware capture still needs validation on the
recording PC.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-07-16 08:12:05 +02:00

src/spelunkai_recording

Implement recording tool: lossless capture + frame-synced input log

2026-07-16 08:12:05 +02:00

tests

Implement recording tool: lossless capture + frame-synced input log

2026-07-16 08:12:05 +02:00

pyproject.toml

Implement recording tool: lossless capture + frame-synced input log

2026-07-16 08:12:05 +02:00

README.md

Implement recording tool: lossless capture + frame-synced input log

2026-07-16 08:12:05 +02:00

README.md

Recording Tool

Captures Spelunky Classic HD gameplay on the dedicated recording PC (Ubuntu 24 Desktop, fullscreen 1280x720) as a lossless 30fps MP4, alongside a separate input log mapped frame-accurately to the video timeline (frame index + timestamp + key state). Output must be easy to slice into individual frames for labeling.

See CLAUDE.md §3.1 for full requirements.

Status: core capture + input logging implemented; not yet run against a live recording session on real hardware (developed off the recording PC — see Testing below).

How it works

Video (capture.py): shells out to ffmpeg -f x11grab ... -c:v libx264rgb -qp 0 to record the screen region losslessly (true RGB, no chroma subsampling) at a fixed framerate. Requires an Xorg session — x11grab does not work under Wayland, so the recording PC must log in via "Ubuntu on Xorg".
Input (input_logger.py): reads a raw /dev/input/eventX keyboard device via evdev directly (bypasses the window system entirely), so key state is captured reliably even while the game holds exclusive fullscreen focus.
Sync (session.py): a single frame-tick loop, paced off one monotonic clock, starts both video and keyboard logging together and writes one JSONL row per video frame ({"frame": i, "t": seconds, "keys": [...]}) plus a _manifest.json with fps/ resolution/frame count. Because both are ticked from the same start time at the same fixed rate, frame i in the log lines up with frame i of the video without post-hoc alignment.

Setup

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Reading /dev/input/eventX requires the input group (or root):

sudo usermod -aG input $USER   # then log out/in

Usage

# find the keyboard device path
spelunkai-record list-devices

# record until Ctrl+C
spelunkai-record record --input-device /dev/input/event3 --output-dir recordings

# fixed-length session
spelunkai-record record --input-device /dev/input/event3 --duration 120 --name run01

Testing

Unit tests (tests/) cover the frame-tick pacing, JSONL/manifest output, and the ffmpeg command construction using fake keyboard/video components — no real device or ffmpeg binary required, so they run anywhere (including WSL):

pytest

Actual capture (VideoCapture/KeyboardState against real hardware) still needs to be validated end-to-end on the real recording PC with Spelunky Classic HD running.