Model
Designed the intra-frame encoder and 8-layer causal temporal Transformer over structured RVQ tokens.
Preprint · arXiv · October 2026
EMG-GPT: Predictive Pretraining on EMG Tokens
A causal Transformer that learns from wrist EMG alone, then transfers to continuous 3D hand-pose estimation on people it has never seen.
01 · See the result
Hand pose from muscle signals
Choose a sequence
Top: motion-capture target. Middle: EMG-GPT. Bottom: vEMG2Pose baseline. Drag the timeline to inspect any frame.
- Model size
- 40.5M parameters
- Regression
- 12.80° error on new users
- Tracking
- 8.89° error on new users
- Pretraining
- 400k steps on one GH200
02 · How it works
Tokenize, predict, transfer
- 1
Tokenize
A frozen NeuroRVQ tokenizer turns 16-channel EMG into discrete codes.
- 2
Encode
Each 40 ms frame of electrode × frequency codes becomes one state.
- 3
Predict
A causal GPT predicts the codes 200 ms ahead, coarse to fine.
- 4
Transfer
Learned states feed a pose decoder for 20 joint angles.
03 · Results
Close to supervised state of the art
12.80° = 12.80°
Re-running the released vEMG2Pose weights on exactly the same test samples gives the same new-user error.
Better than 2 of 3
Lower error than SensingDynamics and NeuroPose in every generalization setting.
04 · Why pretraining matters
More pretraining, better pose
Same pose decoder with the GPT frozen; only the pretraining checkpoint changes (0 = random GPT). Validation set, seed 123.
Scaling observation
Scaling capacity as high as 97.4M parameters did not materially improve transfer. For this problem, preserving the right structure mattered more than indiscriminate scale.
05 · My contribution
First author: model, training, evaluation, release
Systems
Built reproducible PyTorch and Slurm workflows for large experiments on Isambard-AI.
Evidence
Ran matched baselines, held-out-user evaluation, multiseed checks and deterministic visualisation.
Release
Open-sourced an installable inference package and both checkpoints on Hugging Face.
Cite
@misc{magni2026emggpt,
title={EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation},
author={Ettore Magni and Rolandos Alexandros Potamias and Stefanos Zafeiriou and Konstantinos Barmpas},
year={2026},
eprint={2610.05235},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2610.05235}
}Comparisons with published baselines are descriptive, not claims of benchmark superiority. Weights are released under CC BY-NC-SA 4.0 and the inference code under CC BY-NC 4.0; the NeuroRVQ tokenizer is by Barmpas et al. Public redistribution of the qualitative video clips is still subject to a final licence check.