spelunkai/recording

Recording Tool

Captures Spelunky Classic HD gameplay on the dedicated recording PC (Ubuntu 24 Desktop, fullscreen 1280x720) as a lossless 30fps MP4, alongside a separate input log mapped frame-accurately to the video timeline (frame index + timestamp + key state). Output must be easy to slice into individual frames for labeling.

See CLAUDE.md §3.1 for full requirements.

Status: core capture + input logging implemented; not yet run against a live recording session on real hardware (developed off the recording PC — see Testing below).

How it works

• Video (capture.py): shells out to ffmpeg -f x11grab ... -c:v libx264rgb -qp 0 to record the screen region losslessly (true RGB, no chroma subsampling) at a fixed framerate. Requires an Xorg session — x11grab does not work under Wayland, so the recording PC must log in via "Ubuntu on Xorg".
• Input (input_logger.py): reads a raw /dev/input/eventX keyboard device via evdev directly (bypasses the window system entirely), so key state is captured reliably even while the game holds exclusive fullscreen focus.
• Sync (session.py): a single frame-tick loop, paced off one monotonic clock, starts both video and keyboard logging together and writes one JSONL row per video frame ({"frame": i, "t": seconds, "keys": [...]}) plus a _manifest.json with fps/ resolution/frame count. Because both are ticked from the same start time at the same fixed rate, frame i in the log lines up with frame i of the video without post-hoc alignment.
• Frame extraction (frames.py): slices a session's video into individual frame_%06d.png images (via ffmpeg -vsync 0, so no frames are silently dropped or duplicated during extraction) — the bridge to the Labeling Tool, which works on frame images rather than video. Cross-checks the extracted frame count against the manifest's frame_count and the input log's line count, and warns if they disagree — the one place actual capture drift (if the recording PC couldn't sustain 30fps) would surface, since it's not otherwise possible to detect from the video alone.

Setup

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Reading /dev/input/eventX requires the input group (or root):

sudo usermod -aG input $USER   # then log out/in

Usage

# find the keyboard device path
spelunkai-record list-devices

# record until Ctrl+C
spelunkai-record record --input-device /dev/input/event3 --output-dir recordings

# fixed-length session
spelunkai-record record --input-device /dev/input/event3 --duration 120 --name run01

# slice a recorded session into individual frame images for labeling
spelunkai-record extract-frames --output-dir recordings --name run01

Testing

Unit tests (tests/) cover the frame-tick pacing, JSONL/manifest output, and the ffmpeg command construction using fake keyboard/video components — no real device or ffmpeg binary required, so they run anywhere (including WSL):

pytest

test_frames.py additionally runs a real end-to-end extraction against an ffmpeg-generated synthetic test video (no display or game needed) when ffmpeg is on PATH; it's skipped automatically otherwise (e.g. a bare WSL install without ffmpeg).

Actual capture (VideoCapture/KeyboardState against real hardware) still needs to be validated end-to-end on the real recording PC with Spelunky Classic HD running.