Part 1 — detailed analysis and methodology

← Back to the findings · All 100 comparisons

One matched seed, 100 A/B pairs / 200 validated clips. MotionCache OFF.

Open the paired results · Blind scoring panel · Signal measurements · Detailed signal checks · Current completion count

Main finding

The tested prompt format is more flexible than a rigid schema, but the results do not establish that retention instructions or special wording are useless. Removing a retention block can leave the main scene intact; changing a single retention marker can also produce a clearly relevant difference. Concrete instructions sometimes work and sometimes fail under both phrasings.

Treat this as an exploratory, unblinded visual/signal analysis, not a universal revision of the guide. Human blind scores, audio-semantic review and cross-seed replication are still pending. Seed 2 is blocked until explicitly resumed.

The examples worth opening first

Retention: neither a magic switch nor pointless

F01, R01 and R02 preserve the central handoff when removing the whole block, removing marker words, or retaining markers without explanations. Most required scene information remains repeated elsewhere, and all image references remain connected. These are redundancy ablations, not removal of every source of retention guidance.

An important qualification for R05/R06/R08: the guide defines preservation relative to the role assigned to the reference, not every attribute in its source file. In these tests the subject definitions already permit replacing the top, omit the camera's original body color, or request only the cafe's atmosphere. Success with fully_preserved is therefore compatible with preserving that narrowed role. These tests do not demonstrate that the model ignores a genuinely contradictory preservation instruction. Guide, section 4.

R07 is the strongest candidate benefit in this pass: the man's blue facial cast is easy to see under attribute_transfer, while the competing fully_preserved arm largely misses it. Both receive the same transfer instruction in prose. This is closer to the requested color transfer, not proof of naturalistic lighting or better overall quality. One seed establishes an example, not a reliable success-rate advantage. It does not isolate a special parser mechanism from ordinary language-conditioning effects.

Shot wording and timing

Explicit cut prose still produces a wide/close/wide sequence when later shot tags are omitted (S01). Changing section conventions is therefore not equivalent to disabling shot control.

In S02, the largest adjacent-frame changes are at 4.208 and 8.042 s in A, and 4.042 and 6.833 s in B. The latter transitions were checked in boundary frames. These are observed timings, not frame-exact guarantees from the model. S05's contradictory decreasing times still produce shots, but with altered timing; syntactic tolerance does not imply good timing adherence.

In S07, denser inspection resolves what five widely spaced stills cannot: B really shows a blended transition, not just a different close-up. Inspect the transition strip.

Audio references: objective failures, with limits

The procedural source contains quarter-second tone bursts at 1, 3, 5, 7 and 9 seconds, at 330, 440, 550, 660 and 770 Hz. Both U01 arms retain approximately this pattern, with A/B zero-lag PCM correlation 0.99897. Neither waveform is identical to source PCM: audio-VAE reconstruction and delivery encoding are included.

U02 requests silence outside 2–6 seconds. Both arms nevertheless retain the 1-, 7- and 9-second events; outside-window RMS is about −31 dBFS, far above their quiet inter-event intervals. U03 requests a new lower-pitched pattern but both retain approximately the original ascending frequencies. These are failures of the stated requests in this preset, not listening-based judgments of sound quality.

A02's two silence variants have RMS −52.56 and −51.58 dBFS, and peaks −35.73 and −34.51 dBFS. They are low-level, not digital silence. Audibility has not been judged.

U07 is not a clean positive control. Its B arm selects Audio 2 in one field, but the unchanged definitions and retention text still demand Audio 1 and prohibit reusing Audio 2. Both outputs follow the tone track. That cannot settle whether audio-label indexing works: the test needs a noncontradictory replacement before drawing that conclusion.

Coverage and remaining judgments

No overall A-win/B-win percentage is reported. Several cases compare alternative valid instructions rather than competing ways to satisfy one target, and numerical difference is not a quality score. D12 also fails to establish a clear visual cut, weakening its intended speech-across-cut test.

Protocol and integrity

• Seed 28510931, matched within every pair; sequential fresh submissions, not batched mixed seeds.
• Native output 960 × 544, 243 frames at 24 fps = 10.125 seconds. No upscale, SelfLift or interpolation.
• Full INT8 convrot H3 Ref2VA for 94 cases; six base-model cases: one I2VA, two FL2VA, two last-frame-only, one T2VA.
• Hyperflow 8-step v1.0, full variant, strength 1, LoRA mode bypass; Euler, BasicGuider/CFG 1, Comfy Kitchen attention. MotionCache is absent from submitted graphs.
• Five core references: woman, man, cafe, camera and puppy. Extra probes use two source clips, a procedural cue track, a synthetic voice fixture and source endpoint images.
• Each packaged pair verifies output file hash, successful execution, fresh sampler-node execution, saved submitted graph versus history, and effective seed. Prepared A/B graphs differ only in prompt and output filename, except the repeat control, which differs only in filename. A runtime zero-valued nonce dependency forces fresh sampling without changing the seed.
• Presentation MP4s retain native encoded streams, remuxed without re-encoding and with embedded prompt/workflow metadata removed. Sampled JPEGs are review aids, not source media.
• Visual survey uses 0.5, 2.5, 5, 7.5 and 9.5 s; every pair also has a 2-fps timeline. Targeted checks use all-frame RGB differences, exact endpoint frames and denser transition strips. Audio is decoded for signal analysis, not listened to or transcribed in this pass.
• The blind panel has 99 prompt contrasts + 1 same-prompt rerender + 30 identical-video decoys, not 130 independent tests. No blind verdicts have been collected. Different-seed controls await a later seed and are not included in Part 1.

Repeated identical baseline graphs also reproduce identical metadata-stripped MP4s within their groups, despite fresh sampling. Validation details include group membership and hashes/counts. This strengthens the within-run repeatability check, but repeated baselines are still not new independent scenes or seeds. Static links and JavaScript syntax are validated. The redesigned findings page has been checked in an isolated headless browser; its comparison controls have also passed simulated ready/loading/error checks. Full browser video playback has not been certified.

What this cannot establish

This is one seed and mostly one common cafe scene, not 100 independent scenes. The 200 clips contain only 144 unique prompt texts, including only 48 distinct A texts; repeated baselines must not be counted as independent supporting evidence. Image content, prompt redundancy, low-resolution output and the Hyperflow/quantized inference recipe may influence every result.

There is no ordinary-schedule, unquantized or alternative-scene baseline, no multi-rater reliability estimate and no cross-seed uncertainty estimate. Exact remote weight hashes, node-code revisions and tokenizer-level truncation were not fully instrumented. The synthetic voice fixture is not a natural-human voice benchmark. Findings do not isolate core-model behavior from the ComfyUI implementation.

The practical takeaway is to treat the guide as useful prompting guidance whose necessity and strength need empirical testing, not a guaranteed parser specification. Keep concrete, consistent instructions; do not replace every retention mode with fully_preserved based on these results, and do not treat a format violation as automatic generation failure.

Next discriminating checks — proposed, not launched

1. Replicate R07, F01 and R05/R06/R08 on seed 2 and on genuinely different reference sets.
2. Preserve the useful S02/S07 timing/transition probes, with blind temporal judgments.
3. Repair the U07 contradictory selection control; give V03 a target action clearly different from its source; make D12 produce an unambiguous cut.
4. Listen and transcribe dialogue/voice cases before drawing conclusions about delimiters, speaker IDs, voice copy or singing.
5. Only then test whether strong failures depend on Hyperflow versus an ordinary schedule. Do not change the current study mid-seed.

Sources and attribution

The hypotheses were derived from the official Ref2VA prompting guide and base-mode guide, plus the user-supplied H3 Chatter discussion. Community statements are hypotheses, not accepted ground truth.

The blind/decoy review approach was informed by fblissjr's H3 explorations. This is an independent local implementation, not a claimed run of that repository's grader. Reference provenance and hashes are included; the full guide is not redistributed.

All generated clips are AI-generated research examples, not recordings of the depicted people performing these actions. This is an independent exploratory study, not endorsed by MiniMax, the reference photographers, or depicted people. See licensing and attribution for model and reference terms, including the model license's territorial restrictions on generated outputs. This presentation does not grant additional rights or waive those restrictions.