- name
- anymove-capture
- description
- Turn a two-camera AI clip into a game-ready animation with anymove - read every joint with the creature's trained reader, check that the two views agree, fit the rig in both views at once, and export a GLB with renders and a side-by-side comparison. Covers the quality gates (paw agreement, reprojection error), what to do when a fit fails, and three-quarter showcase renders. Use when someone has an anymove clip to capture, asks whether a capture is good, wants the GLB or renders of a move, or a fit looks wrong.
- license
- Apache-2.0
- compatibility
- NVIDIA GPU for reading (a few GB), CPU or GPU for the fit, Blender 5.x for the GLB and renders, Python 3.11 venv from requirements.txt. Needs runs/<c>/reader.pt (anymove-reader) and runs/<c>/ref/ref.png (anymove-rig).
anymove-capture: clip in, animation out
tools/capture.sh <c> runs/<c>/videos/roar/s1/h3max.mp4 roar
Writes runs/<c>/clips/roar/: det.npz (every joint in both views, every frame), overlay.mp4 (the joints drawn on
the clip), motion.json (the fit: root path and rotation, Euler angles per bone, joints in metres),
render/animation.glb (the rig with the animation, for any engine), render/persp and render/side frames, and
showcase.mp4 (the AI clip with joints, the rig in 3/4, and clip vs rig through the same side camera).
What happens
- Read. The clip is aligned to the reference frame (video models resize or crop the canvas: an affine fit by ECC; "ECC did not converge" means a plain stretch was used, so check the overlay), split into its two views, and the reader finds every joint in each (about 10 s for 5 s of video).
- Check the views agree. A joint has the same height in both halves. Per foot (creature.json
"feet"), the correlation of its height between the views over the clip is the paw agreement. If the halves run out of step in time, the front view is time-warped onto the side view (only when that cuts the height gap by 15% or more). - Fit. Blender's forward kinematics in torch, Euler angles bounded by the joint limits, fitted to both views at once. Heights come mostly from the side view (the front view foreshortens leg lifts), left-right from the front view. It starts from an analytic IK pose aimed at the triangulated joints and anneals a robust loss from 40 px to 6 px, so big poses (sit, rear, slam) are found. Under a minute for 5 s (20 s for the dragon's roar on a CPU).
- Export. Blender keys the rig, writes the GLB and renders the views.
Is it good? Read the solve output
The dragon's roar, for example:
paw height agreement between views: {'fpaw_L': 1.0, 'fpaw_R': 1.0, 'hpaw_L': 0.78, 'hpaw_R': None, 'score': 0.93}
sync as generated: mean height gap 4.4 px, worst 8.6 px, best constant lag 0 frames
reprojection error px: {'front': 5.92, 'front_x_only': 2.55, 'side': 7.99}
- Paw agreement score: 0.8 or more, the views tell one story: keep it. 0.5-0.8: look at
showcase.mp4, often one limb is off. Under 0.5: the video model animated the views differently; reroll the clip (anymove-film). for a foot means it barely moved (planted feet in a roar); judge that clip by the reprojection error instead.