Jonas
a7c370b3d4
Add ffmpeg/x11grab-based video capture (libx264rgb, qp=0, true lossless RGB) and an evdev-based keyboard state logger, orchestrated by a single frame-tick loop so each JSONL input row lines up 1:1 with its video frame. Unit-tested with fake keyboard/video components (no real device or ffmpeg needed); real hardware capture still needs validation on the recording PC. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
69 lines
2.4 KiB
Markdown
69 lines
2.4 KiB
Markdown
# Recording Tool
|
|
|
|
Captures *Spelunky Classic HD* gameplay on the dedicated recording PC (Ubuntu 24
|
|
Desktop, fullscreen 1280x720) as a lossless 30fps MP4, alongside a separate input log
|
|
mapped frame-accurately to the video timeline (frame index + timestamp + key state).
|
|
Output must be easy to slice into individual frames for labeling.
|
|
|
|
See CLAUDE.md §3.1 for full requirements.
|
|
|
|
**Status:** core capture + input logging implemented; not yet run against a live
|
|
recording session on real hardware (developed off the recording PC — see Testing
|
|
below).
|
|
|
|
## How it works
|
|
|
|
- **Video** (`capture.py`): shells out to `ffmpeg -f x11grab ... -c:v libx264rgb -qp 0`
|
|
to record the screen region losslessly (true RGB, no chroma subsampling) at a fixed
|
|
framerate. Requires an **Xorg session** — `x11grab` does not work under Wayland, so
|
|
the recording PC must log in via "Ubuntu on Xorg".
|
|
- **Input** (`input_logger.py`): reads a raw `/dev/input/eventX` keyboard device via
|
|
`evdev` directly (bypasses the window system entirely), so key state is captured
|
|
reliably even while the game holds exclusive fullscreen focus.
|
|
- **Sync** (`session.py`): a single frame-tick loop, paced off one monotonic clock,
|
|
starts both video and keyboard logging together and writes one JSONL row per video
|
|
frame (`{"frame": i, "t": seconds, "keys": [...]}`) plus a `_manifest.json` with fps/
|
|
resolution/frame count. Because both are ticked from the same start time at the same
|
|
fixed rate, frame `i` in the log lines up with frame `i` of the video without
|
|
post-hoc alignment.
|
|
|
|
## Setup
|
|
|
|
```
|
|
python -m venv .venv
|
|
source .venv/bin/activate
|
|
pip install -e ".[dev]"
|
|
```
|
|
|
|
Reading `/dev/input/eventX` requires the `input` group (or root):
|
|
|
|
```
|
|
sudo usermod -aG input $USER # then log out/in
|
|
```
|
|
|
|
## Usage
|
|
|
|
```
|
|
# find the keyboard device path
|
|
spelunkai-record list-devices
|
|
|
|
# record until Ctrl+C
|
|
spelunkai-record record --input-device /dev/input/event3 --output-dir recordings
|
|
|
|
# fixed-length session
|
|
spelunkai-record record --input-device /dev/input/event3 --duration 120 --name run01
|
|
```
|
|
|
|
## Testing
|
|
|
|
Unit tests (`tests/`) cover the frame-tick pacing, JSONL/manifest output, and the
|
|
ffmpeg command construction using fake keyboard/video components — no real device or
|
|
ffmpeg binary required, so they run anywhere (including WSL):
|
|
|
|
```
|
|
pytest
|
|
```
|
|
|
|
Actual capture (`VideoCapture`/`KeyboardState` against real hardware) still needs to be
|
|
validated end-to-end on the real recording PC with Spelunky Classic HD running.
|