The authors of a new arXiv paper on instruction-guided video editing say their motion-aware reward model cuts the rate of selecting motion-suppressed candidates to 0% on the VE-Bench motion ladder, while reaching a Spearman correlation of 0.57 with human quality ratings (arXiv). This matters because a reward that favors frozen or sluggish edits can steer editing models toward clips that look clean but barely move; the figures below are the authors’ own, not an independent test.
The failure: rewards that prefer frozen edits
The paper starts from a measured weakness in existing scoring. On VE-Bench, the field composite reward, which combines appearance and temporal scores, selects a motion-suppressed candidate in 97% of cases (arXiv). Two published video-quality scorers fare little better: VideoScore-v1.1 and VideoReward have capture rates of 80% and 76%, and VideoReward’s motion-quality head reaches 98%. In Table 4, the Cap. column is the ladder capture rate, and its down arrow marks lower values as better.
Agreement with people is also weak for these baselines: the field composite and VideoReward’s motion-quality head match the human majority on only 32% and 23% of non-tied predictions. On the deliberately suppressed VE-Bench ladder variants, a reward that favors low-motion clips could steer ranking or training toward them, but the paper’s best-of-N test over configuration-varied UniVideo outputs showed no comparable motion collapse in the tested candidate pool, and the authors say this does not establish general robustness.
Table 4: capture and preference rates
| Reward | Cap. ↓ | h2 | segFrz | burstFrz | MOS Sp. ↑ | Source page |
|---|---|---|---|---|---|---|
| Field composite | 97% | 6% | 11% | 12% | +0.55 | https://arxiv.org/html/2611.22495 |
| VideoScore | 80% | 58% | 40% | 48% | +0.18 | https://arxiv.org/html/2611.22495 |
| VideoReward | 76% | 35% | 57% | 53% | +0.46 | https://arxiv.org/html/2611.22495 |
| MQ head | 98% | 11% | 57% | 69% | +0.09 | https://arxiv.org/html/2611.22495 |
| Ours | 0% | 100% | 91% | 100% | +0.57 | https://arxiv.org/html/2611.22495 |
The paper’s Table 4 compares rewards on VE-Bench. For the authors’ own model, capture rate is 0%, and it prefers the original edit over a hold-2 suppressed variant in 100% of cases. The same row shows 91% for segment freeze and 100% for burst freeze, patterns the paper describes as unseen in training. VideoReward, by contrast, has a capture rate of 76% and prefers the original over hold-2 in only 35% of cases. The paper’s 76–98% baseline range and the authors’ 0% result come from the same Cap. column, so readers can trace both in Table 4.
Analysis: the baselines are not uniformly bad on every pattern, since VideoScore reaches 58% on hold-2 where VideoReward reaches 35%, yet neither comes close to the 100% in the last row. The motion-quality head shows how uneven the picture is: 98% capture and 11% on hold-2, but 69% on burst freeze. Analysis: the new model’s 91% on segment freeze is its lowest preference rate among the pattern columns, yet it is still far above the field composite’s 11% on that pattern.
Why a base reward misses mild holds
The authors probe the base reward’s internal features to explain the gap. A linear probe on the base preference reward separates originals from frozen clips with 96.2% accuracy, but reaches only 52.9% on both hold-2 and hold-4. Mild suppression is therefore close to invisible in the base reward’s representation.
An ablation points to the motion input as the key ingredient: removing the motion input block from the final motion-aware reward model drops hold-2 detection from 100% to 47%. The paper also reports that scores for originals and hold variants overlap under the base model, while the new model lowers suppressed-candidate scores and keeps a broad distribution on originals, with a standard deviation of 0.36. A caveat from the authors: hold-2 still has a high-score tail with a maximum of 0.98, so discrimination relies on within-source rankings rather than a single threshold.
Scalar add-ons versus input modality
The authors also compare against an external variant that adds a motion-faithfulness scalar to the base preference reward, with a weight of about 0.85 chosen on held-out folds. That variant lowers capture from 51% to 31%, with MOS Spearman moving from 0.49 to 0.46. The paper calls the remaining 31% a ceiling that only the input-modality treatment breaks, down to 0%.
The paper adds that matching the source is the right objective rather than maximizing motion: on VE-Bench, motion-faithfulness MOS Spearman is +0.40 while raw motion presence is -0.09. Analysis: for practitioners building preference data or reward-guided training for editing, this argues for checking edits against the source clip’s motion, not for scoring how much movement an output contains.
How we built this
Published resultsTable 4 of the paper: Reward robustness and quality alignment on VE-Bench
Rows of Table 4 of the paper (arXiv HTML version), copied cell for cell by our code; a two-row header is shown as "group: column"; an empty name cell repeats the name above it. These are the authors' own figures, not an independent test. The table has 9 columns; 6 shown, with every column that holds a figure this article gives. Not shown: frzn, flowAtt, Corr(mot.).
Sources
The sources this article was written from. Links open the original page.