LTX 2.3 Video to Video: IC-LoRA and Retakes

Video-to-video is where people expect a slider labeled "denoise strength" and find something else entirely.

LTX 2.3 has no single v2v mode. It has four different mechanisms that all take a video in and produce a video out, and they are built for different jobs: restyling with structural guidance, regenerating a time window, producing gradeable HDR, and re-voicing a speaker. Choosing the wrong one produces disappointing results that look like model weakness and are actually a routing mistake.

There is also one hard constraint that catches everyone: the official v2v pipeline only works with a distilled checkpoint. Load dev and you are done before you start.

Sourced from the LTX-2 pipeline documentation, the conditioning docs, and the official ComfyUI tutorial, checked July 2026.

The four video-to-video mechanisms

Pick by intent and the rest of the setup follows.

ICLoraPipeline: the main v2v path

The documentation describes it as two-stage generation with IC-LoRA support that can condition on reference videos (video-to-video) or on images at specific frames, with CFG guidance in stage 1 and upsampling in stage 2.

Two rules define how you use it:

1. Distilled only. The docs state plainly that ICLoraPipeline can only be used with a distilled model. This is not a recommendation. ComfyUI's IC-LoRA Union Control template follows the same rule — it loads ltx-2.3-22b-distilled-fp8.safetensors, while the T2V and I2V templates load the dev build.

2. It needs an IC-LoRA. "In-context" is the mechanism: the reference video's information enters as context tokens, and the IC-LoRA is what teaches the model to use them. No IC-LoRA, no in-context conditioning.

Under the hood, the conditioning documentation names the mechanism — video conditioning uses VideoConditionByKeyframeIndex, and it is listed as ICLoraPipeline only. That is the clearest possible statement that reference-video conditioning is not a general capability you can attach to any pipeline.

Which IC-LoRA to use

• Union Control (LTX-2.3-22b-IC-LoRA-Union-Control) — structural guidance from depth, pose, or edges. The general-purpose choice.
• Motion Track Control (LTX-2.3-22b-IC-LoRA-Motion-Track-Control) — motion-track based control.
• Pose Control / Detailer — 19B-era LTX-2 adapters; match the generation.

In ComfyUI, Union Control also needs a geometry estimator, moge_2_vitl_normal_fp16.safetensors, in models/geometry_estimation/. That model is what converts your reference clip into the depth and normal signals the LoRA consumes — without it there is nothing to condition on.

Testing whether v2v is even the right approach? A lot of "I need video-to-video" turns out to be "I need a better first frame plus a better prompt." ltx23.app lets you test the generation side quickly before you assemble a control pipeline.

RetakePipeline: the most underused tool in the stack

If one second of a ten-second clip is wrong, rerolling the whole clip is the expensive habit. Retake regenerates only a specified time region while preserving everything else.

How it works, per the docs: it encodes the source video and audio into latents, applies a temporal region mask marking [start_time, end_time] for regeneration, and denoises only the masked region from a text prompt. Content outside the window is preserved.

Two capabilities worth knowing:

• Independent video and audio control via regenerate_video and regenerate_audio flags. You can fix the audio in a window and leave the picture untouched, or vice versa.
• Works with either checkpoint — the full model with CFG guidance, or the distilled model with a fixed sigma schedule.

CLI arguments: --video-path, --start-time, --end-time, all required.

Constraints that will reject your file: the source video's frame count must satisfy 8k+1 (97, 193, …) and its resolution must be a multiple of 32. Footage from a camera or another model will usually need re-encoding first. Budget for that step.

Retake is also exposed in LTX Desktop as "video edit generation," so you do not need the CLI to use it.

HDRICLoraPipeline: for people who grade

This one is niche and genuinely differentiated. It runs two-stage video-to-video on the distilled model with an HDR IC-LoRA, then passes decoded latents through an HDR inverse transform (ARRI LogC3, auto-detected from LoRA metadata) to produce a linear HDR float tensor [f, h, w, c].

Practical notes from the documentation:

• Video only — audio is skipped
• Text embeddings are pre-computed externally and loaded from a .safetensors file, not encoded inline
• Tonemapping and EXR saving are the caller's responsibility — you get linear float data, not a finished file
• --spatial-tile controls tiled VAE decode tile size; reduce it on lower-VRAM GPUs
• --exr-half writes float16 EXR; --skip-mp4 suppresses the H.264 preview
• --high-quality generates 2x frames internally for smoother output and is roughly 2x slower
• --offload {none,cpu,disk} disables FP8 quantization when set to anything other than none

If your pipeline ends in DaVinci Resolve or Nuke rather than an MP4 upload, this is the only official path that hands you gradeable data.

LipDubPipeline: v2v for dialogue

Also technically video-to-video, but shaped by its own constraints. It uses IC-LoRA on a distilled checkpoint with a single lip-dub IC-LoRA applied in both stages. The reference clip supplies video and audio reference tokens whose VAE latents are appended to the target audio sequence as frozen reference tokens.

The consequence that trips people up: frame count and frame rate are derived from the reference video — the frame count is silently snapped to the nearest 8k+1 — so the CLI does not accept --num-frames or --frame-rate. Required: --reference-video. Optional: --reference-strength. And --lora must be passed exactly once.

Choosing, in one pass

• Keep the motion and composition, change the look → ICLoraPipeline + Union Control
• Fix a specific bad window → RetakePipeline
• Need EXR / grading → HDRICLoraPipeline
• New dialogue on an existing performance → LipDubPipeline
• Extend rather than transform → video extension is a supported conditioning mode in the trainer's config set (video_extend_lora.yaml)

And the setup checklist that prevents most failures:

1. Distilled checkpoint for IC-LoRA, HDR, and LipDub paths
2. Resolution divisible by 32, frames on 8k+1 — re-encode source footage if needed
3. The right IC-LoRA, matched to model generation (22b vs 19b)
4. Geometry estimator present for Union Control in ComfyUI
5. Two-stage means the spatial upscaler is still required

FAQ

Does LTX 2.3 support video to video? Yes — through ICLoraPipeline with an IC-LoRA, plus retake, HDR, and lip-dub variants.

Why does my v2v run fail with the dev checkpoint? ICLoraPipeline only works with a distilled model. ComfyUI's IC-LoRA template uses the distilled FP8 build for the same reason.

Is there a denoise-strength slider? Not in the image-model sense. Control comes from the IC-LoRA, the reference conditioning, and guidance parameters.

Can I regenerate just part of a video? Yes — RetakePipeline takes --start-time and --end-time and preserves everything outside the window, with separate video and audio regeneration flags.

Why is my source video rejected? Almost always the constraints: frame count must be 8k+1, resolution a multiple of 32.

What resolution does HDR output come at? It returns linear float frames; tonemapping and EXR writing are yours to do, with --spatial-tile controlling decode tiling on smaller GPUs.

Bottom line

LTX 2.3 video-to-video is four purpose-built mechanisms, not one mode with a strength dial. Match the mechanism to the job, load the distilled checkpoint where it is required, bring an IC-LoRA, and make your source footage legal (÷32, 8k+1) before anything else.

And when you want to check output quality without building a control rig, ltx23.app is the quick answer.

Sources

Verified July 2026 against primary documentation:

1. LTX-2 available pipelines documentation — ICLora, Retake, HDR, LipDub behavior, flags and constraints
2. LTX-2 conditioning documentation — video conditioning via VideoConditionByKeyframeIndex, ICLoraPipeline-only
3. Lightricks/LTX-2 — official GitHub repository — LoRA catalog and pipeline overview
4. ComfyUI official LTX-2.3 tutorial — IC-LoRA template, distilled FP8 checkpoint, geometry estimator
5. LTX-2.3-22b-IC-LoRA-Union-Control — depth/pose/edge control
6. LTX-2.3-22b-IC-LoRA-Motion-Track-Control — motion-track control
7. LTX-2.3-22b-IC-LoRA-HDR — HDR LoRA and pre-computed embeddings
8. LTX-2.3-22b-IC-LoRA-LipDub — lip dubbing LoRA
9. ltx-trainer configs — v2v and extension conditioning modes
10. Lightricks/LTX-2.3 — Hugging Face model card — ÷32 and 8k+1 constraints
11. Lightricks/LTX-Desktop — Retake exposed as video edit generation

Pipeline flags and constraints are those documented at the time of writing and can change between releases.