qiwen-bot/gif-background-removal

GIF / Animated GIF Transparent Background Removal

Inputs to Collect

• Source file: GIF, PNG sequence, or video. Videos go through ffmpeg frame extraction (script handles this automatically).
• Viewing background: What background is the user viewing the image on? A dark interface or a white background will expose completely different residual artifacts. If unknown, default to running full background verification (script handles this automatically); do not skip this.
• Does the subject have light-colored parts close to the background color (e.g., white gloves, white boots, white edges)? This is a capability ceiling; see "Solvability Pre-check" below.

Solvability Pre-check (Skipping This Will Waste Tens of Minutes)

Background removal is an information recovery problem, not an algorithm tuning problem. When light-colored subject parts are close in color to the background, post-processing cannot separate them — chrominance, smoothness, temporal median, and background surface fitting will all damage the subject.

Don't estimate colors by eye; run this:

.venv-bg\Scripts\python scripts\cutout.py --input <source_file> --work work --stage solvability

It uses the assumption "camera is stationary + subject moves" to separate the background plate and subject, outputting three numbers:

• Median background color / Brightest subject hue / Difference between them
• Percentage of subject pixels falling within 25/255 of the background color — this is the most interpretable number

Testing on this robot asset: background (212,207,210), brightest subject (250,251,250), difference 40.7, but 32.6% of subject pixels fall within 25 of the background color → prints AMBIGUOUS.

When you see AMBIGUOUS, tell the user first:

The light-colored parts of this asset are very close to the background color. Pure background removal cannot achieve pixel-level cleanliness; at best, it will have very fine residual artifacts when magnified. If you have the source project, re-render with a high-saturation solid green/magenta backdrop, and I can achieve completely residue-free results. With only the final file, I can only get close to clean.

Then ask the user if they want to continue; don't silently run a 20-minute process.

Note: This check is a warning signal, not a verdict. The final quantification of residual artifacts after the full run is what matters — "solvable" might still produce residuals due to model edge inaccuracy, and "ambiguous" might still yield very clean results in practice.

Procedure

1. Create workspace and install environment python -m venv .venv-bg && .venv-bg\Scripts\python -m pip install rembg onnxruntime pillow scipy (scipy comes from the rembg dependency chain; guided filtering requires it — see below)

2. Run the pipeline

   .venv-bg\Scripts\python scripts\cutout.py --input <source_file> --outdir out --scale 2
   

   --input accepts video files, GIF/PNG files, or a pre-extracted PNG sequence directory. --scale 2 is not optional: gaps between limbs and torso are only 5–15px, and the segmentation model can't see them at 720p. If individual frames remain stubborn, rerun them with --stage matte --scale 3.

   On pure CPU, 70 frames at 720p at 2× scale takes approximately 20 minutes (about 17.6s per frame, onnxruntime without GPU). Inform the user of this time estimate beforehand to avoid assuming the process is stuck.

3. Verification (see section below); if it fails, rerun the problematic stage only — do not rerun the entire pipeline.

4. Delivery: Provide all three formats — GIF + WebP + APNG — and explain their respective use cases.

Hard Rules (Violating These Has Had High Costs)

• Use only opening for topological processing, never closing. binary_closing will fill gaps ≤2px, welding narrow spaces between arms and torso shut; no amount of further processing can recover them. This single rule has caused more rework than all other issues combined.
• Hole-filling threshold: use a small value (around 60 pixels, not hundreds). Real limb gaps are thousands of pixels; pinholes are only tens.
• Guided filter radius: ≤2, and default to no median filter. A 3×3 median filter or a guided filter with radius 4 will erase gaps only a few pixels wide.
• Do not apply nonlinear compression to alpha. A mapping like clip((clip((a-0.04)/0.92)-0.06)/0.88) intended to tighten edges actually raises alpha around 0.7, effectively making residuals more solid.
• Do not use ordered (Bayer) dithering for GIFs. Static frames look softer-edged, but the dither pattern is locked to the screen coordinate system; when the outline moves through it, it "boils" and flickers — worse in animation. Use hard thresholding for GIFs after cleanup; reserve soft-edge alpha for WebP/APNG.
• Use BiRefNet as the model. isnet/u2net may interpret ground reflections as part of the subject and cut them in; switching to birefnet-general-lite makes alpha drop to zero directly at the boot sole.
• Do not "remove background color" to make it cleaner. Aggressively removing residuals using background surface/chrominance criteria will also erase light-colored subject parts. The correct solution is to increase matte resolution, not change thresholds.

Output Contract

• out/<name>.gif — Transparent index 255, transparency=255, disposal=2, 1-bit hard edge. Topological cleanup (speckle removal) applies only to GIF.
• out/<name>.webp — Preserves continuous alpha, no topological cleanup, so a thin layer of soft warm fringe remains. Prefer this for complex backgrounds; this difference is intentional, not a bug.
• out/<name>.png — APNG, best browser compatibility.
• Preserve the original canvas size, frame count, per-frame duration, and looping settings; list these measured values in the reply. (PIL cannot read per-frame timing for animated WebP; if asked, transparently say "refer to the GIF timing" — don't fabricate.)
• Deliver files using the <media> tag, and explain the appropriate use cases for GIF vs. WebP.

Failure Handling

• GIF displays with a gray background in browsers: Don't modify the file first. Check the VP8X ALPHA flag and ANIM background color (verify.py prints these). If both are normal, the viewer is simply filling the transparent area a background color. If verify.py reports ANMF ... blend=0, that is a real problem — cutout.py exports with blend=1, so if this file has blend=0, it was not produced by this script or was processed by an external tool.
• Subject is eaten away: Almost always caused by overly broad background criteria. Revert to the final/ layer and re-export; do not continue tuning the erasing logic.
• Gaps in certain frames not clean: --stage matte --scale 3 to rerun only those frames; per-frame cost is about 20 seconds.
• rembg model download is slow: rembg 2.x stores models at ~/.rembg/models/<model>/<model>.onnx (not the old ~/.u2net). birefnet-general-lite is about 214MB. The first run will need to download it.

Verification (This Step Is Not Optional)

Must inspect every frame on the actual background color the user will see; otherwise, the verification is meaningless:

.venv-bg\Scripts\python scripts\verify.py out\<name>.gif --backdrop grey --ref work\final

It outputs: speckle count, per-frame opaque pixel ratio (to detect flickering), lowest opaque row, color quantization error of GIF relative to source matte, WebP container flags, and generates a four-backdrop comparison chart, a full-frame contact sheet, and a specified region zoomed view.

Acceptance criteria:

Metric	Pass Condition
Speckle count	0
Opaque ratio range across frames	< 0.05
Lowest opaque row	Consistent across frames (reflection residuals cause it to fluctuate or hit the bottom of the frame)
Average color quantization error	< 3/255 (thumbnails can't show hue loss; this number is the only indicator)
Four-backdrop comparison	Each backdrop is visible; no uniform light gray
Contact sheet	No residual shadows per frame, no noise, limb gaps clear

Inspecting only a single frame or only on a black background will miss issues — warm-color residuals are only obvious when overlaid on a dark gray background.

Windows (win32) Platform Notes

• Use py or python as the interpreter; python3 does not exist on Windows. Do not use python from PATH (it might point to a venv without pip, like hermes). Prefer the venv's .venv-bg\Scripts\python.exe.
• When paths contain Chinese characters, use the Write/Read tool to write script files; do not use Set-Content for content construction.
• PIL's ImageFilter.GaussianBlur does not support mode "F" or float RGB; use scipy.ndimage.gaussian_filter for blurring float images (scipy is already installed by rembg).
• Image.fromarray(float32_array, "RGB") will immediately error; first .astype(np.float32), then handle per channel.
• Do not use Remove-Item for cleanup; cutout.py --clean already encapsulates directory cleanup.

Examples

Input: "Make the background of this GIF transparent" → Run cutout.py --input action.gif --outdir out --scale 2, then verify.py, → Deliver all three formats GIF/WebP/APNG, listing: 720×720 / 70 frames / 100ms per frame / total 7000ms.

Input: "Clean it up more, there are still noise around the edges and uncleaned spots between the arm and body" → Do not adjust thresholds. First run verify.py --zoom <selected coordinates> to confirm defect locations, → If speckles, add --median 3; if gaps are incomplete, rerun with --stage matte --scale 3 on the relevant frames, → Then verify again. In the reply, clearly describe the defect type and which fix was applied; don't just say "optimized".