comfy-h3-render-core

ComfyUI H3 Render Core (H3 Physical Rendering & Orchestration Stage)

Stage Role: Core physical generation stage for short dramas. Core Principles: Do not hardcode ComfyUI node numbers; strictly prevent Windows WDDM driver deadlocks; enforce single-speaker lock and latent space size safety checks.

---

1. Run Contract & Input/Output

• Input:
  1. plan.json (containing camera movement, dialogue, and sequence timing of 175 frames);
  2. Fixed-frame base images from Stage 2: segments/{seg_id}_keyframe.png;
  3. style_bible.json.
• Output:
  1. H3 natively generated video: segments/{seg_id}.mp4 (with native audio and video);
  2. VRAM status monitoring log and rendering time ledger.

---

2. Hardcore SOP: Three Anti-Crash Physical Laws (with numeric characteristics and error counterexamples)

2.1 Law 1: Latent Space Width 32-Divisibility Rule (Prevent VAE Latent Space Crash)

• Symptom/Error: RuntimeError: shape '[1, 24, 1, 1, 15, 2, 26, 2]' is invalid for input of size 38160
• Root Cause: MiniMax H3's 3D Patch size requires width to be divisible by 16, and the quotient must be even (i.e., width % 32 == 0).
• Criteria & Handling:
  • Forbidden: 848×480 (848/16=53 is odd, cannot be divided by patch=2, will crash!);
  • Forced Lock: Standard widescreen is always locked to 832×480 (832/16=52 is even, 52/2=26 gives perfect divisibility!).

2.2 Law 2: Windows WDDM Driver Mutex Lock Defense (Prevent 15x Slowdown)

• Symptom: On Windows, enabling cudaMallocAsync or multi-stream async offloading (async-offload) causes single-step sampling to drop from 40 seconds to 600 seconds (single segment takes up to 70 minutes).
• Root Cause: Windows' WDDM driver model causes severe kernel mutex lock contention when transferring large weights across CUDA streams.
• Handling (ComfyUI Golden Startup Arguments):

  python <COMFYUI_PATH>/main.py --disable-auto-launch --disable-mmap --disable-cuda-malloc --disable-async-offload --listen 127.0.0.1 --port 8188
  

  Using native caching and single-stream scheduling bypasses driver locks, keeping single-segment generation time stable at 350 seconds (5.8 minutes).

2.3 Law 3: Single Speaker Lock & Silence Suppression Instruction Assembly (Eliminate All-Characters Talking)

When constructing [Subject 1] and [Action 1], strictly enforce separate assembly:

• [Subject 1]: Primary speaker: [SpeakerName], alongside [ListenerName] (silent listener).
• [Action 1] Core Primitive:

  Only [SpeakerName] is actively speaking with natural, synchronized Mandarin lip movement, [Action]. 
  Crucial requirement: [ListenerName] remains strictly silent with lips tightly sealed and mouth completely closed, neither speaking nor making any mouth motions, showing a tense glare. 
  In the final 1.2 seconds, all characters firmly close their mouths, ending on a tense silent glare.
  

---

3. Hardcore SOP: VRAM Anti-Fragmentation & Checkpoint Resume Protection

1. Inter-Shot VRAM Cleanup: After rendering each shot, call the ComfyUI /free endpoint:

   payload = {"unload_models": False, "free_memory": True}
   

   Clear PyTorch intermediate 3D latent space tensor cache to prevent cumulative OOM during long production cycles.
2. Checkpoint Resume to Prevent Re-rendering: Check if segments/{seg_id}.mp4 exists and its size is greater than 100KB. If so, print [Reuse] and skip, enabling second-level checkpoint resume.

---

4. Dedicated Tool Script

• H3 Execution Graph Assembly & Orchestration Engine: python scripts/h3_render_orchestrator.py --plan plan.json --server 127.0.0.1:8188