Pose and motion tracking

2D/3D Pose Tracker (BlazePose) ↗

Tracks 33 body landmarks frame by frame – head, shoulders, arms, hands (wrist, thumb, index and little finger), hips, legs.

Input

mp4 video, one person

Output

CSV or Excel per frame: 2D pixel coordinates and model-based 3D coordinates (metres, origin between the hips), visibility

In Healthy Strings

The standard tool for posture and arm movement from the 30R and 90R videos.

Watch out

3D is a model estimate per camera, not a calibrated lab measurement; depth (z) is the least reliable axis. A mirrored video swaps left and right.

The pose tracker and audio features of the soundtrack in one pass, on the same time axis.

Input

mp4 video with sound

Output

landmarks and audio features per frame (CSV)

In Healthy Strings

Relating bowing movement to loudness or brightness of the sound.

2D Multiple Person Pose Tracker ↗

Tracks up to six persons at once with the MoveNet model (17 points each).

Input

mp4 video with several persons

Output

17 2D keypoints per person and frame (CSV)

In Healthy Strings

Ensembles, lessons with teacher and student.

Watch out

No hands or fingers; 2D only.

2D/3D Hand Tracker ↗

21 landmarks per hand, frame by frame (MediaPipe Hands).

Input

mp4 video; best a close-up of the hand

Output

21 points per hand in 2D/3D (CSV)

In Healthy Strings

Bow hold and left-hand finger movements – needs a close camera.

Watch out

The whole-body view is usually too small for reliable finger points.

Hand tracking together with audio features of the soundtrack.

Input

mp4 video with sound

Output

hand landmarks and audio features (CSV)

In Healthy Strings

Finger movement and intonation or articulation on one time axis.

2D Face/Body/Hand Tracker ↗

Body, both hands, face and gaze in one pass (human.js).

Input

mp4 video

Output

2D landmarks of body, hands and face (CSV)

In Healthy Strings

A quick overview when head, face and hands all matter.

Watch out

2D only.

Point Tracker ↗

Follows up to eight points you place yourself (OpenCV.js, Kanade–Lucas–Tomasi tracker).

Input

mp4 video

Output

x/y of each point per frame (CSV)

In Healthy Strings

Bow tip, frog, scroll or chin rest – points the body models do not know.

Watch out

Points can drift when the object turns or is hidden; check the trace.

Video Motion Analysis Tool ↗

Quantity of motion, motion centroid, periodicity and optical flow – without a skeleton model (after RITMO VideoAnalysis).

Input

mp4 video

Output

time series, images and motion videos

In Healthy Strings

Overall amount of movement per task or outfit; works where pose models fail.

Watch out

Lighting changes and camera movement count as motion – use a tripod.

Video Motion Amplification ↗

Magnifies very small movements and colour changes in a video (Eulerian video magnification).

Input

mp4 video, camera fixed

Output

amplified video

In Healthy Strings

Making tremor, vibrato micro-movements or tension in the shoulder visible for teaching.

Watch out

Shows, does not measure; noise is amplified too.