Communitygithub.com

boyang-hu/website-rebuild-skill

An agent skill that mirrors a website, rebuilds it from the minified code, and verifies the result with automated diffs.

What is website-rebuild-skill?

This agent skill provides a forensic methodology for rebuilding award-winning creative websites (WebGL, scroll narratives, portfolio sites) into independently runnable, verifiable engineering projects. It treats the source site's bundle as the sole specification, performing byte-level extraction and reconstruction rather than visual imitation. The workflow encompasses six constitutional disciplines ensuring mirror integrity, strict provenance-based porting, and quantitative verification gates. It supports multiple endpoint levels: L1 (mirror archive for preservation), L2 (engineering rebuild with verifiable 1:1 fidelity), and L3 (source-code reconstruction for study). The skill classifies websites into categories A (fully supported static sites), B (conditionally supported), C (reconstructable RSC/declarative frameworks), D (server-side dominant, not supported), and X (archival rescue from Internet Archive). It enforces legal boundaries through user-input tools, requiring explicit user decisions before public deployment while maintaining forensic completeness.

Works with~Claude Code~Codex CLI~Cursor
npx skills add boyang-hu/website-rebuild-skill

Ask in your favorite AI

Open a new chat with this agent skill pre-loaded.

Documentation

Website Rebuild (Award-Winning Site 1:1 Replication)

Replicate an award-winning creative website (WebGL / scrollytelling / portfolio) using a forensic method into an independently runnable, verifiable engineering project. Not a "looks-like" clone — it's a line-by-line port treating the source bundle as the sole specification, with quantitative verification gates as the closure criterion.

This methodology is distilled from six consecutive practice projects (timeline converging from 6.5 weeks to 1 day), subsequently backfilled with 22 complete rebuilds + 5 dead-site archive rescues, and calibrated with 43 site boundary exploration probes (see verified sites list in the repo README).

Prerequisites and Authorization ⛔ Must Read

This skill is intended for learning and research purposes of fidelity replication, used to study the implementation techniques of award-winning creative sites. It applies to websites that are your own, authorized, or publicly accessible and allow learning/emulation. It is not a tool for unauthorized collection of protected content, circumventing access controls, or commercial misappropriation of others' work.

Execute with the following boundaries:

  • Respect target site rules: Abide by its robots.txt, terms of service, and copyright; keep crawling low-frequency, single-session, and do not impose abnormal load on the target site. ⛔ robots.txt is a per-path permission statement, not an all-site switch — evaluate each URL individually (select group → longest match → no match means allowed), do not judge "entire site forbidden" just because there is any Disallow line (almost every commercial site has Disallow for /cart, /checkout, /admin); classify prohibitions by behavior type, only prohibitions against "crawling" affect the mirror scope, prohibitions against transactions only mean "don't click checkout." ⭐ "Unable to read / unsure" does not equal "forbidden": Escalate, do not halt, and certainly do not self-narrow the crawl scope. Interpretation method in references/legal-and-deploy.md §0.3.
  • Do not touch protected boundaries: Do not collect content requiring login, paywall, or authorization to access; this skill only handles anonymously publicly accessible resources. If the target site explicitly prohibits such copying, stop and inform the user — "explicitly prohibited" is defined by the four hard thresholds hardcoded in legal-and-deploy.md §0.3.6; for all other uncertainty, escalate, do not halt.
  • Output private by default: Default noindex, not publicly deployed. Before any public release, complete per-asset copyright verification and prominently mark as "unofficial replica" with original author attribution (see references/legal-and-deploy.md).

⛔ Legal judgment rests with the user; the skill only collects evidence and presents facts (three rules, valid throughout):

  1. Decision belongs to the user: The skill collects facts (per-asset attribution, license status, third-party rights holders, whether the source site is still operational, third-party identifiers in the output), lists options with their risk boundaries, and provides recommendations with reasoning; for anything involving "can it be public / deployed / redistributed / shown externally," must explicitly return to the user using the User Input Tools below; the agent must not make legal conclusions and proceed.
  2. Before receiving the user's explicit decision, execute the safe default: Private repository + noindex + not publicly deployed + no further distribution. When writing to the user, explain that this is a default action ("I won't publish it until you decide"), not an agent-made legal conclusion — the responsibility distinction is fundamentally different. The agent can only execute conservative defaults; moving toward public requires an explicit user decision.
  3. ⛔ Legal considerations must not reduce mirror completeness or gate coverage: The mirror is the evidence base; completeness is a technical invariant (the four-pass method, closure gate, GAP=0 all depend on it). Not crawling can only have technical reasons (not a file / server doesn't provide / requires authorization or login / source site explicitly prohibits) — always document; do not use legal reasons like "will never be public" or "shouldn't store an extra copy" to leave gaps. Empirical evidence: In one project, with "output never public" as justification, a class of assets was "documented, not re-crawled," missing about 60% of assets while all five gates were constantly green, hiding four milestones [objectarchive]. Legal decisions affect how the output is used, not whether the evidence base is complete.

Scope of Application ⛔ Must Read

Primary (Class A): Sites with statically hosted content, where all signature behavior (animations/interactions) resides entirely in client-side static assets — imperative WebGL/Canvas scenes, GSAP timelines, baked data files (GLB/.buf/.riv), minified or unobfuscated bundles. The vast majority of Awwwards-style creative sites fall into this category.

Conditionally Supported (Class B): Where the pipeline works but requires additional scenario handling (Shopify platform layer removal, third-party storage bucket assets, runtime API snapshots, SSG payload expansion). The current version's guides cover most Class B scenarios; for uncovered cases, explicitly inform the user of the risk.

Explicitly Rejected (Classes C/D):

  • C1 (Doable since v0.3: Reconstructive Reverse Engineering): Server-side component source is indeed not delivered, but its complete output (flight stream) is inlined in each page's HTML and is a comparable specification. Route: flight-decode builds coordinate system → reconstruct a buildable Next project (client-side party components translated verbatim per C2, server-side components reverse-engineered from flight tree into explicitly documented inferences) → verify-flight semantic gate closes (module id global bijection). Tested on rauchg.com (Next 16/Turbopack): 18/18 routes semantically consistent; blind reverse-engineering against answer key: structure ≈95%/behavior ≈98%. ⚠ C1 combines L2/L3 — the first artifact is "human-written source + gate-proven equivalence." Full workflow in references/rsc-reconstruction.md.
  • C2 (Doable, run as Class A): ⭐ Declarative in style but source is delivered (R3F / Theatre / Vue SFC compilation output). Tested on one R3F site: useFrame callback contained ordinary imperative code like MathUtils.damp(x, y, 7, t), sliced verbatim across 18 modules, swapped into page — CLEAN, all 8 canvases present, cross-side 99.5%. The slicer doesn't care about paradigm — it slices bytes. The renderer serves as a platform layer from the mirror. ⛔ The discriminator is not the library name, but "does the client hold the behavior source" (scope-and-fingerprint.md §4.0.1).
  • D: Behavior is primarily server-side (CMS content sites, e-commerce cart/inventory, A/B experiment bucketing, personalization injection) — the client has no portable target, and deterministic verification has no baseline.

Class X (Rescuable): The original site has disappeared (domain sold / platform reclaimed / path removed / replaced in place), but Internet Archive typically has captures — scripts/wayback-mirror.mjs uses CDX index to select a coherent moment by anchor + time window, fetches as standard mirror (downstream gates work unchanged), holes documented as-is in wayback-holes.txt (read method and workflow in references/archival-rescue.md). ⭐ Rescue output is a standard mirror — Class X can complete the full L3 workflow (tested first-launch: Strategy A shell + numeric gate 9,856 samples all equal + within pixel bandwidth + self-contained gate, no gate required semantic change). ⛔ "CDX has no coverage" means truly impossible, read at asset layer, not site layer: IA crawlers don't execute JS; sites driven by manifests/concatenation can have 100% code coverage but zero visual coverage (tested: 157/160 assets had zero captures across any era; offline browser shows white screen) — Step 0 first do layered coverage reconnaissance (deduction + CDX prefix query) to predict rescue depth, see archival-rescue.md §1.9. Historical award-winning sites have a disappearance rate of about 29% — which is why "mirror first, immediately" is this skill's first discipline.

Classification is determined by Step 0 fingerprint reconnaissance; full decision tree in references/scope-and-fingerprint.md. When rejecting, explain the reason and state which class the site belongs to, do not force execution.

User Input Tools

When needing to ask the user (confirm scope, legal decision, external dependency decision): Prefer the current runtime's built-in questioning tool (e.g., AskUserQuestion); if unavailable, output a numbered question list for the user to reply with numbers. Support combining multiple questions into one query when possible. Legal questions follow the five-paragraph format from legal-and-deploy.md §0.1: facts / what cannot be determined / options / risk boundary for each option / recommendation and current default action.

Constitution (Six Disciplines, Valid Throughout)

The following six are called "constitutional" in the six source projects; violation of any will be repaid as bugs in later stages:

  1. Mirror Must Never Be Contaminated: The mirror/ disk files are never modified; any local adaptation (CDN rewriting, external stub) is done dynamically at the service layer response.
  2. Source Code Is the Sole Arbiter, Never Fix by Eye: Each change first finds its line number in the bundle/CSS/mirror HTML before being implemented. Do not tune visuals, motion, or interaction by eye.
  3. Everything the Source Has, Let It Have; Don't Add What It Doesn't: Do not invent compensatory CSS/JS. Prefer looking wrong over inventing rules — self-created patches will invert into bugs after mechanism alignment.
  4. Copy Bugs, Dead Code, and Weird Syntax Verbatim: Every weird syntax in minified code could be behavior itself. A "helpful fix" of a no-op bug once caused a transition crash (empirical evidence in porting-discipline.md).
  5. Intentional Deviations Must Be Documented: Write down "how the source does it / how we do it / why / under what conditions to reconsider." Any undocumented difference is considered a bug.
  6. Code and Documentation in the Same Commit: Each milestone is committed as a pair (Port xxx + Update rebuild plan: xxx), with log entries always containing output / verification / lessons learned / next breakpoint (with line numbers).

⭐ Discipline 3's boundary at M(n+1): src/ is an explicitly documented derivative, not an assertion about the source, so renaming, splitting modules, and adding comments in src/ are not "invention" — Discipline 3 constrains "self-creating behavior to make it look right," not "making already proven equivalent code readable." But two hard boundaries remain inviolable: ① Structural rewriting is prohibited by default (merging duplicates, extracting common functions, changing algorithms — they make equivalence undecidable); ② Speculation in comments must be explicitly marked as speculation, do not present reverse-engineering notes as statements of fact. port/ and mirror/ still cannot be touched by a single byte. See references/readable-source.md §3.4 and §5.

Workflow

Progress Checklist

[ ] Step 0  Fingerprint Reconnaissance and Scope Gate ⛔ (Classify A/B/C/D/X; for C/D/X reject or guide, do not proceed)
[ ] Step 1  Pre-build Rating (falsify architecture, star difficulty per item, estimate timeline, confirm scope with user + endpoint L1/L2/L3)
[ ] M0      Mirror Forensics ⛔ (BFS crawler + CDP supplemental capture + manifest ledger; GAP=0)
[ ] M0.5    Mirror Works Offline ⛔ (zero 404 / zero console errors / zero external calls; serve via serve.mjs) ← L1 Mirror Archive endpoint
[ ] M1      Reverse Engineering Build Coordinate System ⛔ (pin version in _pretty; engine-notes before any code; freeze tech stack; create REBUILD_PLAN)
[ ] M2+     Strict Provenance-Based Porting (proceed by dependency-ordered milestones; first slice one end-to-end chain; cold launch test at each milestone + CLEAN gate)
[ ] M(n-1)  Diff Verification (select gate type per verification-gates.md decision tree; root-cause fix, do not tune parameters to smooth out differences)
[ ] M(n)    Close-out ⛔ (cold-start review / module list reconciliation; copyright evidence collection + submit to user for decision — must complete before public deployment) ← L2 Engineering Rebuild endpoint
[ ] M(n+1)  Source-ification (port/ → src/: split modules, deobfuscate rename, add comments, self-contained) ← L3 Source-ification endpoint

⛔ = Blocking gate: If verification criteria are not met, must not proceed to the next stage.

Flow

Step 0 — Fingerprint Reconnaissance and Scope Gate. Load references/scope-and-fingerprint.md, execute the detection protocol against the URL provided by the user (GET at path granularity, final URL identity, double-fetch diff, species/era verification, bundle initial inspection), output classification and rationale. Continue for Class A/B; For Class C/D/X, explain to the user then stop or guide.

Step 1 — Pre-build Rating. Load references/recon-and-rating.md. First falsify architecture assumptions (dependency tables can lie), star difficulty per item (assets/3D/scroll orchestration/private format/platform layer), confirm scope with the user (entire site or specific pages) and expectations.

⭐ In the same query, let the user choose the endpoint (three-level ladder, present with classification conclusion and graded cost estimates, not a dry list):

EndpointQuestion AnsweredStops AtTypical Use
L1 Mirror ArchiveWhat it looks likeM0.5Archiving, offline appreciation (award-winning site disappearance rate ~29% per year)
L2 Engineering RebuildWhat it doesM(n)Deployable, verifiable 1:1 replica (including copyright evidence collection and deployment assessment)
L3 Source-ificationHow it worksM(n+1)Research and study of implementation techniques

Ladder is monotonic; choosing lower is not a loss: Each level is a prefix of the next; the mirror discipline ensures time-sensitive parts are always done first — choosing L1 today allows upgrading later (explain this to the user). "Using it as my own project" (scaffolding) is not a stage of this skill — when asked, point to references/beyond-the-rebuild.md for handoff; that is their engineering, the skill ends at "human-readable truth."

M0 / M0.5 — Mirror Forensics. Load references/mirroring.md. Use scripts/mirror-site.mjs for BFS crawling + scripts/netcapture.mjs for real browser supplement capture, manifest documents each file with sha256, redirect: manual discipline, make individual decisions for external dependencies. scripts/verify-mirror.mjs is the mirror's own gate (five assertions, run before the offline gate — all downstream gates ask "can it render?" a wrong mirror can make them all green; an HTTP 200 is also not evidence that you "got that resource"). scripts/serve.mjs serves the mirror, offline verification. This step is always done first — the original site can disappear or be updated at any time; the mirror is the sole evidence baseline for the entire project and the reference server for all subsequent comparisons.

M1 — Reverse Engineering Build Coordinate System. Load references/reverse-engineering.md. ⛔ First action is to identify bundle morphology (flat concatenation / modular bundling / multiple chunks), then select tools — layered table scans top-level declarations, while webpack bundle top-level declaration count is 0, boundaries and dependency edges are given by the bundler (tested: 24,378 lines → 569 ready-made modules, using scripts/module-map.mjs). If the container cannot be recognized, FATAL, do not fall back to layered tables (§0.5). scripts/beautify-bundle.mjs (js-beautify pinned at 1.15.1) expands bundle into _pretty/, after which line numbers are the sole traceability coordinate system for the entire project. Write docs/engine-notes.md (template: assets/templates/engine-notes.md) before any code. Freeze exact version of the tech stack from bundle evidence. For data-driven animations, first dump numeric ledger. Create REBUILD_PLAN.md (template: assets/templates/rebuild-plan.md).

M2+ — Strict Provenance-Based Porting. Load references/porting-discipline.md, and load corresponding scenario guides from the branch routing table. Each ported file header annotates the source line number range; extract GLSL/magic numbers/data verbatim; data asset scripts are dumped into the repository, not hand-typed.

M(n-1) — Diff Verification. Load references/verification-gates.md and references/determinism.md. Full-site rendering breadth uses scripts/sweep-routes.mjs (all routes in one browser, per-route zero errors/zero failures/zero external calls + interaction hooks and per-route collection), single-route depth uses probe.mjs — ⛔ do not manually write Chrome-launching loops per route; cost is counted by browser launch count, and concurrent probes will reap each other's orphans. ⚠ Before attributing residuals, first establish self-comparison bandwidth (pixelcompare --self, per side ≥4 runs, interleaved) — residuals without bandwidth are always UNCLASSIFIED, and UNCLASSIFIED is failure, not pass. Gate type selection: If SSR/static HTML artifacts exist, first build byte gate → For frozen entropy source in DOM static scenes, go for byte-equal → For live scenes (WebGL/video/random phase), downgrade to quantitative metrics + noise classification → For data-driven animations, add numeric probe gate → CLEAN gate as final catch-all. Before judging timing bugs, calibrate probes first (references/environment-traps.md).

M(n) — Close-out. Cold-start review: Check each top-level class/module in the bundle against its landing point one by one (functional testing cannot detect full-block omissions; only list-based reconciliation can). Load references/legal-and-deploy.md to complete copyright evidence collection and submit the decision to the user — before user decision, follow the safe default (private + noindex + not deployed); before public release, must complete per-asset evidence collection and prominently mark as unofficial replica.

M(n+1) — Source-ification. Load references/readable-source.md. Up to M(n), the artifact is already proven correct but human-unreadable (tested: 14,271 lines crammed into one file, e appears 2962 times, comments account for 0.2%). This stage rewrites port/ into src/: split modules → scope-safe deobfuscation renaming → add tiered comments → copy assets to achieve self-containment. ⛔ Split granularity is not a free choice — for flat scripts, declaration order equals evaluation order; granularity is determined by three hard constraints (mutual references / evaluation order / import bindings cannot be assigned), first produce a splitting plan for review, then cut; when encountering a giant module, first measure the benefit curve of "lazy binding of a few trailing singletons" before deciding (tested: 6 bindings reduced from 11,246 lines to 1,013, while switching module systems would require paying for the entire toolchain to achieve the same granularity). ⭐ The judgment "this cannot be done" is extremely unreliable — twice it was judged structurally impossible, and both times the culprit was a single bug in the measuring tool; first suspect the tool measuring it, then the target (readable-source.md §3.1–3.1.3). ⛔ Prerequisite non-negotiable: Must first have fully green gates. Refactoring without a referee is blind alteration; with a meanAbsDiff 0.00 referee, every step can be proven — this is the best possible condition for refactoring and why it must come last. All existing gates are reused unchanged (target changed to src/ build output, tolerances must not be relaxed), plus symbol mapping gate and self-containment gate added. ⛔ Structural rewriting (merging duplicates, extracting common functions, changing algorithms) is prohibited by default — it degrades the gate from "proving equivalence" to "didn't detect inequivalence." ⭐ Discipline 4 is still valid in this stage: you can now read it, the urge to say "this is clearly a bug" will be stronger than at any other stage, yet it may still be behavior itself.

⭐ For container-free scope-hoisted artifacts (Vite/esbuild, sites with tier-by-tier line-level delivery): take a different path: do not rewrite, cut — concatenation-based decomposition (scripts/census-bundles.mjs produces chunk graph & coordinates → scripts/slice-esm.mjs cuts by declaration into semantically named parts, reassembled in order, byte-for-byte equal to original → scripts/verify-reassembly.mjs one gate settles; when byte equivalence holds, the ruling of all runtime gates is transferred for free). Execution side unchanged, browser continues running original chunks. See readable-source.md §3.0.6 for details.

Branch Routing Table

Step 1 reconnaissance results determine which scenario guides to load (as needed, not all at once):

Reconnaissance FindingLoad
Next.js App Router / RSC(self.__next_f flight stream) — C1 Reconstructive Reverse Engineeringreferences/rsc-reconstruction.md
WebGL / Canvas scenes (three.js, custom engine, GLSL)references/webgl-scenes.md
GSAP / Baked animation data / CSS variable animations / Custom input state machinereferences/animation-recovery.md
Private binary formats (.buf / .sog / VAT / GLB timeline / .riv)references/binary-formats.md
Shopify store (fingerprint: cdn/shop, Shopify.theme, cdn.shopify.com)references/shopify-platform.md
Sanity CMS (fingerprint: cdn.sanity.io/images/<projectId>/, *.api.sanity.io, payloads with chunks of _key/_type/_ref) — ⛔ classification depends on content baking timing, not library name; and auto=format assets negotiate different bytes by Accept headerreferences/sanity-platform.md
DOM Layer Strategy Selection (required for all sites; Webflow export / static single-page / framework SSR branches differ, plus orthogonal constraint "DOM read as coordinate source by 3D engine")references/dom-shell-strategies.md
Large assets (hundreds of MB media / licensed fonts)references/asset-management.md
Headless probe behavioral anomaly / suspected environment issuereferences/environment-traps.md

Step Summary

StageKey ActionBlocking Gate VerificationOutput
Step 0Fingerprint detection & classificationClassification clear & user informedClassification conclusion and rationale
Step 1Falsify + rate + confirm scopeUser confirmationDifficulty rating table, scope agreement
M0/M0.5Mirror + ledger + offline runverify-mirror all five green; GAP=0; zero 404/zero errors/zero external callsmirror/ (read-only), manifest, serve.mjs reference server
M1Expand bundle, reverse engineering notes, freeze stackengine-notes complete; version freeze table complete_pretty/, docs/engine-notes.md, REBUILD_PLAN.md
M2+Provenance porting, milestone pair commitsCold launch test at each milestone + CLEAN gate greenSource code with line number annotations, three registration tables incrementally updated
M(n-1)Diff verificationSelected gate type all green or all differences documentedVerification scripts + diff artifacts archived (docs/compare/)
M(n)Cold-start review + copyright evidence collection + submit to userList reconciliation zero gaps; user has made deployment decision (if pending, maintain safe default)Audit record, DEPLOY.md
M(n+1)Split modules + deobfuscate + add comments + self-containedExisting gates all green and tolerances not relaxed; symbol gate bidirectional bijection zero orphans; self-containment gate (copy out, disconnect network, build) passsrc/ (readable project), docs/rename-map.json, src/README.md

Script Directory

Node 22+, paths relative to this skill directory. Usage and maturity level in scripts/README.md.

⭐⭐ Dependency discipline is stage-based, not directory-based: before source-ification, the entire pipeline has zero dependencies.

From Step 0 to M(n), nothing is installed; the rebuild project only gets devDependencies at M(n+1) (scope-safe renaming requires a real parser). scripts/ (zero dependencies) and tools/ (allows devDeps) are just the projection of this stage boundary onto the directory — scripts/ houses probes and gates, tools/ houses source-ification stage refactorers.

⛔ No gate may import any tool (verification-gates.md §2.1.2) — the inspector cannot be the producer.

⭐ What if earlier stages need a real parser? Spawn, do not import. beautify-bundle.mjs (js-beautify) and module-map.mjs (acorn) both spawn a pinned-version npx, the script itself is still zero-dependency and still independently reviewable. ⛔ Do not hand-write a lexer — this skill tried it, a regex literal containing a quote knocked it off by 16,177 lines (F27). Bracket matching on the token stream is precise; bracket matching on text is guesswork about strings/regexes/comments.

⚠ This line was discovered after it was violated: module-map.mjs depends on @babel/*, yet lived in scripts/ for eight full versions, while the same discipline's original text sits three lines above it. A rule only written in documentation, with nothing checking it, will silently fail.

ScriptPurposeStage of Use
scripts/fingerprint.mjsCross-platform equivalent of Step 0 six-step detection protocol (GET alive + redirect chain & destination domain identity + double-fetch diff + species/era + HTML tech fingerprint + bundle initial inspection; appearance count built-in; <1KB Referer retry built-in; Sanity CMS evidence collection — projectId/dataset/API host/auto=format/_key, three spellings normalized, hit routes to sanity-platform.md). Only collects evidence, does not output classification — classification still follows scope-and-fingerprint.md §3 decision treeStep 0 (when no POSIX toolchain available)
scripts/mirror-site.mjsBFS crawler mirror (asset whitelist + iterate to fixed point; redirect:manual + triple ledger, including per-file sha256; ⭐ redirects.tsv accumulates across runs, like manifest — --scope page supplement once truncated it to header only; payload gate hit 404 on /work before discovery). --scope <prefix> limits the page queue to within the target path (mandatory when a microsite is mounted under an enterprise CMS domain; ⛔ limits pages only, not assets). Ledger accumulation (--seeds supplement does not truncate previous rows) + off-host census (count and warn for each not-followed host — silently discarding once caused 827 media references to disappear while report read "57 files saved")M0 first pass
scripts/netcapture.mjsReal browser CDP capture reconciliation & runtime resource supplement (CDN sites must pass --hosts, otherwise only same-origin traffic is observed, leading to false GAP=0)M0 second pass
scripts/verify-mirror.mjsMirror's own gate, run before offline gate. Five assertions: injection single-ity / ledger vs disk sha256 / authenticity (challenge page body + declared type vs magic bytes — a 200 is not evidence that "you got that resource") / closure / optional sample re-fetch from source. All downstream gates ask "can it render?" a wrong mirror can make them all greenM0 before closing, after each re-mirror
scripts/gapfill-video.mjsHLS/DASH streaming tiered supplement (master → rendition → segment), structural blind spot of static crawlerM0 (when streaming media exists)
scripts/reconcile-gaps.mjsRuntime gap reconciler: Takes GAP lines recorded by netcapture and the complete set list from byte deduction (e.g., next/image srcset tier enumeration — tested: 1,078 vs browser encountered 217), supplements each into the mirror. ⭐ Request header ladder (standard profile 4xx → bare profile retry: same 403 has two opposite cures, landonorris needs Referer, video.twimg hates Referer) + per-URL fault tolerance + batch accounting (one crash won't leave files unaccounted). ⛔ Standard profile for image URLs sends browser-like image Accept (lib/negotiate.mjs) — auto=format type CDNs negotiate format via Accept; */* gets fallback bytes (basement D5: all 391 variants fell back while gate stayed green); ledger records profile+varyM0 (sites with many runtime resources)
scripts/flight-decode.mjsC1 coordinate system: Decodes each page's self.__next_f.push stream into module reference table (I-line export name = free tier-1 naming evidence), HL preloads, de-referenced element tree + JSX-style outline. T-lines sized by declaration bytes; :HL zero-id lines must not break chainM1 (C1)
scripts/verify-flight.mjsC1 semantic gate: Build artifact flight tree ≟ mirror flight tree. Has its own parser (⛔ does not import flight-decode — inspector cannot be producer); normalization only normalizes build hash namespaces that carry no behavior (chunk name/css-module class/media hash/promotable resource mount point/encoding freedom), module id does global bijection (one-to-many is red); site-specific entries via --normalize-props (ISR epoch fields) and --normalize-class (library rendering subtrees); everything else is redM(n-1) (C1)
scripts/serve.mjsZero-dependency static server (MIME/Range/service-layer rewrite/redirect replay), also serves as source site reference server. --rewrite FROM::TO is registered literal substitution, for items not reachable by localization — source program branches by its own domain (location.hostname=="x.com" && (CDN=...), mirror is not on that domain, so the entire block is dead)M0 onwards
scripts/sweep-routes.mjsFull-route breadth verification: Launch one browser, iterate all routes, collect per-route log + console errors + external calls + interaction hooks (if configurable). ⛔ One browser loop, not per-route Chrome launch; concurrent probes will reap each other's orphansM(n-1) breadth
scripts/probe.mjsSingle-route depth probe: Pixel-compare + console error line numbers + numeric asset dumpM(n-1) depth
scripts/self-bandwidth.mjsPixelCompare self-comparison bandwidth: >4 runs per side, interleaved, output mean + varianceM(n-1) before attribution
scripts/verify-reassembly.mjsFor scope-hoisted artifacts that are cut and reassembled: verify byte equivalence between original and reassembled codeM(n+1) after slice
scripts/census-bundles.mjsChunk registry + cross-chunk reference count + declaration root & shadow treeM1 (scope-hoisted chunk identification)
scripts/slice-esm.mjsSplit ESM artifacts by declaration: produce semantically named modules, maintain import/export chainM(n+1)
scripts/wayback-mirror.mjsInternet Archive CDX index — anchor + time window selection → standard mirror output; holes documented in wayback-holes.txtStep 0 (Class X reconnaissance and mirroring)
scripts/artifact-summary.mjsSummarize artifact dimensions, origin, production chain, physical files, per-sha256 origin; spot-check 5 examples, flag "third-party origin copy" re-im portsM(n) close-out review
scripts/copyright-audit.mjsPer-asset attestation trace: sha256 → URL → page attribution (HTML meta/JSON-LD/explicit) → third-party rights; free-standing assets mark "unattached"; output acting as DEPLOY.md section placeholderM(n)

Reference Documents (references/)

DocumentContentLoaded When
references/scope-and-fingerprint.mdStep 0 full decision tree; fingerprint registry (species detection algorithm); scope classification criteriaStep 0, before judgment
references/recon-and-rating.mdArchitecture falsification methodology (dependency table lies, domain permutation test); per-item difficulty estimation; timeline estimation formulaStep 1
references/mirroring.mdMirroring discipline: BFS boundary conditions; external dependency decision tree; ledger structure; CDP protocol parameters; verify-mirror method; streaming media gap-fillM0
references/reverse-engineering.mdBundle morphology identification; beautifier version calibration; module-map boundary conditions; engine-notes template; tech stack freeze procedureM1
references/porting-discipline.mdProvenance porting rules; file header format; line number traceability discipline; data asset extraction procedure; deviation registration table; milestone pause conditionsM2+
references/verification-gates.mdGate type selection decision tree; per-gate threshold; self-bandwidth analysis protocol; residual classification (structural/timing/numeric/noise/UNCLASSIFIED); CLEAN gate definitionM(n-1)
references/determinism.mdEntropy source freeze list; Math.random mock; requestAnimationFrame normalization; audio clock sealing; Date.now traceability; environment fingerprint asset procedureM(n-1) before any comparison
references/environment-traps.mdEnvironment behavioral trap correction (GPU / high-DPI / font / audio context / full-screen API); calibration procedureAnytime probe abnormality is suspected
references/legal-and-deploy.mdPer-asset copyright evidence chain; legal threshold hard-coded definition; deployment decision tree; attribution requirement; DEPLOY.md template; local-first handover procedureM(n) close-out review
references/readable-source.mdM(n+1) source-ification workflow: split granularity hard constraints; scope-safe rename procedure; module boundary dependency graph; re-output gate reuse; structural rewriting prohibition; validation protocolM(n+1)
references/webgl-scenes.mdWebGL/Canvas scene porting: GLSL verbatim extraction; uniform/attribute mapping; render loop equivalence; multi-viewport canvas managementWhen WebGL/Canvas is detected
references/animation-recovery.mdAnimation recovery: GSAP timeline capture; CSS variable animation snapshot; custom state machine reverse engineering; data-driven animation numeric ledgerWhen GSAP/baked animation is detected
references/binary-formats.mdPrivate binary format decoder: .buf header parsing; .sog container extraction; GLB timeline interpolation; VAT data reconstruction; .riv state machine playbackWhen private binary format is detected
references/shopify-platform.mdShopify platform layer separation: asset separation; theme liquid static cache; storefront API response snapshot; cart/checkout dead-end handlingWhen Shopify store is detected
references/sanity-platform.mdSanity CMS handle: content baking timestamp determination; CDN URL normalized fetch; image pipeline (auto=format) negotiation; GROQ query result snapshotWhen Sanity CMS is detected
references/dom-shell-strategies.mdDOM layer strategy selection: Webflow export / static single-page / framework SSR branching; Orthogonal resource layer: DOM coordinate reading by 3D engine; Strategy shift when the entire site is a single canvasAll sites pass through
references/asset-management.mdLarge asset management: hundreds-of-MB GIF/MP4 license; licensed font extraction + replacement; lazy-loading conversion; size optimization (do not change sha256)When more than 100MB of assets exist
references/beyond-the-rebuild.mdHandoff after skill completion: fork independently; remove external dependencies; switch to real Vite/Next build; re-architect with authorization; the skill's terminal point ("human-readable truth")User asks "can I make it my own project"
references/archival-rescue.mdDead site rescue: Internet Archive CDX index query; anchor + time window selection; standard mirror output; wayback-hole.tsv format (asset layer, not site layer, reading); rescue depth estimation from layered coverage reconnaissanceStep 0 (Class X before judgment)
references/rsc-reconstruction.mdReact Server Component full reconstruction: flight stream decoding → coordinate system → reconstructable Next project → verify-flight semantic gate. Covers both App Router and Pages RouterWhen self.__next_f flight stream is detected

Templates Directory (assets/templates/)

All templates are standard format, no additional dependencies required. Loading each template before use is essential but must not be confused with "automatically populating" — the template provides structure, the content must be written from evidence.

TemplateFormatWhen to Use
assets/templates/engine-notes.mdReverse engineering notesM1, before any code
assets/templates/rebuild-plan.mdRebuild planM1, after engine-notes completion
assets/templates/port-header.mdPorted file headerM2+, each new file
assets/templates/deviation-log.mdIntentional deviation log entryM2+, when deviation is decided
assets/templates/compare-report.mdPer-gate verification reportM(n-1), after each gate run
assets/templates/DEPLOY.mdDeployment decision + attribution statementM(n), after user decision

Note on template usage: Do not pre-load all templates; load only when needed. Templates are for structure, not automation; manual filling ensures each entry comes from evidence.

Changelog

  • v0.1: Initial method from first two practice projects
  • v0.2: Added BFS mirroring, CDP supplemental capture
  • v0.3: Added C1 RSC reconstruction, Class X archival rescue
  • v0.4: Added branch routing table, domain-specific scenario guides
  • v0.5: Added self-bandwidth protocol, CLEAN gate definition
  • v0.6: Added source-ification (M(n+1)), scope-hoisted artifact path
  • v0.7: Added legal deployment framework, user input tools, scope decision tree

Related Skills