H3 prompt-guide challenge · Part 1

What did the A/B tests actually show?

You can simplify some prompts. That does not mean the wording is ignored.

The main scene survived without the retention block in one comparison, but details changed. In another comparison, changing just the retention mode changed the requested lighting effect. Shot wording also changed transitions.

Practical conclusion: keep the guide as guidance, not a guarantee. These results do not justify deleting every retention block or replacing every mode with fully_preserved.

How far can we trust this? 100 pairs / 200 clips, but only one seed and mostly one shared scene. Hyperflow 8-step, 960×544, MotionCache off. These are observed examples, not success rates. Dialogue, voice quality, music and human blind scoring are still unassessed.

The three clearest findings

01 · Retention block · Main action survived, details changed

The retention block was not needed for the main action in this example.

We changed: removed the whole retention_analysis block. Everything else, including the reference images, stayed the same.

We saw: both clips show the camera handoff. The dog and composition differ: A places a larger dog on the bench; B places a smaller dog on the floor mat.

Meaning: the rest of this prompt already supplied enough information for the main action. Removing the block was not the same as changing nothing. This does not tell us it is safe to remove it from every prompt.

See the evidence — with vs without retention (F01)

A · Retention block present

B · Retention block removed

Exact prompt change and sampled frames →

02 · Retention mode · A relevant difference appeared

A mode-word change affected the requested lighting transfer.

We changed: attribute_transfer → fully_preserved, with the same prose asking to transfer blue lighting from the woman's reference to the man.

We saw: A gives the man's face a blue cast; B remains much more neutral. The difference is visible in sampled frames from 2.5 to 9.5 seconds.

Meaning: “mode words are always ignored” is not a conclusion we can draw. Here, attribute_transfer was closer to the requested color effect. That is one useful example, not proof that it consistently wins or makes a better looking video. It also does not prove these words are special parser tokens.

See the evidence — lighting transfer (R07)

A · attribute_transfer
Blue cast on the man's face

B · fully_preserved
Much more neutral face

Exact prompt change and sampled frames →

03 · Shots and timing · Visible instruction response

Transition words and timestamps changed how the shots unfolded.

We changed: “cuts” → “cross-dissolves” in one pair; removed cut timestamps in another.

We saw: the dissolve version visibly blends views around 4.125–4.5 seconds, where the cut version changes abruptly. In the timestamp test, removing the times moved the return to the wide shot from about 8.04 to 6.83 seconds.

Meaning: these instructions had a visible effect in this seed. Keep explicit transition and timing instructions when they matter, but do not expect guaranteed frame-exact timing.

See the evidence — cut vs dissolve (S07)

A · “cuts”
Abrupt transition

B · “cross-dissolves”
Blended transition

Dissolve frame strip · Transition prompt change · Timestamp comparison

An important correction: what “fully preserved” means

It refers to the role you assigned to the reference, not necessarily every pixel or attribute in the source image. If you define a reference as “this person's identity,” changing their clothes is not automatically a contradiction.

Our jacket, camera-color and room tests already narrowed the reference roles. Both mode choices allowed the requested edits. Those tests do not prove that fully_preserved is ignored, and do not disprove that part of the guide.

Jacket test · Camera-color test · Room test · Official guide, reference retention

What failed — and what we cannot judge yet

Question	What Part 1 actually tells us	Evidence
Does correct wording guarantee the requested result?	No. Both versions missed the requested first-person viewpoint, selective audio copying, and video continuation in their respective tests. The cause is not isolated.	POV · Audio window · Continuation
Can changing text undo a connected keyframe?	The supplied endpoint composition remained even when alignment prose was removed or contradicted. The actual image inputs still matter.	Endpoints · Contradictory prose
Are dialogue, voice and music instructions reliable?	Not assessed yet. Audio signal measurements are not a substitute for checking the words, voices, music or lip-sync.	Review coverage
Did every test cleanly answer its question?	No. Some tests contain conflicting instructions or weak controls. They cannot settle those claims.	U07, V03 and D12 limitations

What should you do differently now?

Keep clear, consistent instructions. Removing a section may preserve the action while changing details you care about.
Do not replace every retention mode with fully_preserved. There is at least one relevant difference worth testing again, not a universal winner.
Specify the transitions and timing you want. These affected the result here; still inspect the output.
Treat failures as things to test, not instant proof the guide is wrong. Results may depend on references, seed, Hyperflow, quantization or implementation.

Why there is no “A wins 70%” score

We have not collected blind human scores or repeated the study across seeds. Many pairs reuse the same baseline, and a visible difference is not automatically an improvement. The same-prompt repeat produced identical decoded video and audio after fresh sampling, which is a useful pipeline check—not a quality score. Repeat control.

The next useful step is replication, not stronger claims. Seed 2 remains paused. Full measurements, test limitations and proposed follow-ups are in the detailed analysis.