• Diffuse thousands of frames at full fps-30 with 13B models using 6GB laptop GPU memory.
• Finetune 13B video model at batch size 64 on a single 8xA100/H100 node for personal/lab
  experiments.
• Personal RTX 4090 generates at speed 2.5 seconds/frame (unoptimized) or 1.5 seconds/frame
  (teacache).
• No timestep distillation.
• Video diffusion, but feels like image diffusion.

So we have many input frames and want to diffuse some new frames.

The idea is that we can encode the input frames to some GPU layout like this:

This chart shows the logical GPU memory layout – frames images are not stitched.

Or, say the context length of each input frame.

Each frame is encoded with different patchifying kernel to achieve this.

For example, in HunyuanVideo, a 480p frame is likely 1536 tokens if using (1, 2, 2) patchifying
kernel.

Then, if changed to (2, 4, 4) patchifying kernel, a frame is 192 tokens.

I