O0000-code/roam-to-logseq-db

Roam Research → Logseq DB (v2) migration

What this is and why it's not obvious

Logseq split into two products: the old file/Markdown version (OG), which has a built-in "Import Roam JSON" — and the new DB/SQLite version (v2), which is now the default and has no Roam importer at all (only SQLite / File→DB / EDN imports). There is also no community tool targeting DB v2. So a naive attempt flails. This skill encodes a path that works.

The method — do not deviate without reason:

Roam JSON → [scripts/roam_to_logseq.py] → Logseq FILE graph (.md + assets/ + logseq/config.edn)
          → [official File→DB importer] → DB graph

We produce a file graph (the universal intermediate every Roam→Logseq tool emits) and feed it to Logseq's own File→DB importer. We do NOT hand-write the DB (the tempting import-edn / upsertNodes path) — see the first gotcha for why that silently loses every image.

Prerequisites

• The Roam export must be JSON (not Markdown). Roam Markdown export drops block references; only JSON carries them. If the user has Markdown, have them re-export as JSON.
• The converter assumes Roam's default English daily-note titles ("January 25th, 2026"). A graph using a different daily-note display format will silently have its daily notes become ordinary pages (journal detection fails) — adjust DATE_RE in the converter and confirm via the journal-count QC check. Watch for this when migrating your other graphs, which may differ.
• The File→DB importer must be runnable. Two ways (pick per environment — see references/toolchain-setup.md):
  • Headless / fully scripted (best when a Logseq source checkout is available): the nbb script deps/graph-parser/script/db_import.cljs. One-time toolchain setup required.
  • GUI fallback (works for anyone with Logseq desktop, no source): after producing the file graph, the user clicks Import → File to DB graph in the app. Same engine, one click.

Workflow

Read references/conversion-rules.md (what the converter does to each Roam construct, and why) and references/qc-and-troubleshooting.md (verification queries + failure modes) before running — they carry the load-bearing detail this hub omits.

1. Inventory the export first. Skim the Roam JSON for scale and which constructs are present (block refs ((uid)), {{[[table]]}}, images, {{[[TODO]]}}, code fences, daily notes, /-in-title pages). This tells you what to spot-check later. The converter handles all of them, but knowing the counts lets you verify nothing silently vanished.
2. Convert: python scripts/roam_to_logseq.py --input <roam.json> --out <file-graph-dir>. It downloads+localizes images (this can take a few minutes; re-runnable — it skips already- fetched files), converts every construct, and writes a Logseq file graph. Read its --help.
3. Sanity-sweep the file graph (cheap, catches converter regressions before a slow import): no leftover {{[[table]]}} / {{ macros, no raw firebasestorage URLs, block-ref id:: present, file counts ≈ (pages − dropped) + journals. See references/qc-and-troubleshooting.md.
4. Import to a STAGING graph name first (never straight to the user's target). This proves scale + fidelity with zero risk. Headless: db_import.cljs <file-graph> <staging-name> -c -V. The -V runs Logseq's own validator; -c continues past a bad page so one glitch doesn't abort a 15-min import (then check its "ignored files" report).
5. QC the staging graph against the source (per references/qc-and-troubleshooting.md): counts reconcile, dangling-ref scan = 0, images copied to <graph>/assets/, journals/tasks/assets present, no created-at:: leaked into block content. Spot-check a table page and an image page.
6. Final placement. A DB graph cannot be renamed and the importer creates the graph by name, so to land it under the user's exact target name you must import directly to that name — which requires the app to release it. See the quit-Logseq gotcha. Confirm with the user before this step (it's the one action that touches their target).

Scale it to the task: a 100k+ block graph deserves the full staging→QC→final dance; a tiny graph can go straight in. But the gotchas below are non-negotiable regardless of size.

Gotchas (read these — they are why this took real effort to get right)

• build-EDN / direct-DB-write CANNOT copy image files. import-edn and the HTTP API are pure DataScript transactions that never touch the filesystem, so a hand-built #Asset node renders as a broken image (bug #31023). Only the File→DB importer physically copies assets/ and wires up #Asset nodes. This is the reason the whole method routes through a file graph. If you're ever tempted to "just write the DB directly," you will lose every image. Don't.
• Block references resolve as [[uuid]], NOT Roam's ((uuid)). In DB v2's file importer, ((uuid)) stays literal text; the target block needs id:: <uuid> and the reference site must be [[<uuid>]]. (The converter does this.) Source-code reading suggested ((uuid)) works — it does not in v2.0.1; only empirical testing caught it. Trust the dangling-scan, not the docs.
• Quit Logseq before importing to a graph the app holds open. The desktop app locks the open graph's SQLite via its db-worker; a second writer corrupts it. To place content under the user's exact target name: have them fully quit Logseq (Cmd+Q, not just switch graphs), then move the empty placeholder graph dir aside (don't delete — mv to .bak), import to that name, and tell them to reopen. Verify the app is actually gone (pgrep) before writing.
• A stale db-worker serves OLD data and will lie to your QC. The logseq CLI keeps a per-graph worker alive; after a re-import into the same name, that worker still holds the previous graph in memory, so your verification queries read stale numbers. Kill the worker (pkill -9 -f "logseq_db_<name>") before every QC pass and after every re-import. When a fix "didn't take," suspect this first.
• A DB graph cannot be renamed — copying/mv-ing the graph dir yields a graph whose worker won't start (identity is baked in). So the target name is chosen at import time; there is no cheap rename. Plan the double-import (staging → final) accordingly.
• The import is non-idempotent. Re-running appends duplicate blocks. Import once into a fresh graph; on any failure, drop the graph and re-run from scratch. Deterministic UUIDs (the converter uses uuid-from-roam-uid) make re-runs reproducible.
• Blockquotes, code fences, and multi-line blocks swallow trailing property lines. mldoc treats the indented created-at:: after a > quote / ``` / multi-line block as content, so the property leaks into the block text. The converter skips timestamps on such blocks and forces inline-opened code fences onto their own line — but if you edit the converter, re-check for created-at:: leaking into :block/title after import.
• roam-to-logseq (the pip package cited by blogs) does not exist — it's AI-generated SEO spam. Real references: Logseq OG's own roam.cljs (conversion semantics), sebastianpech/ roamtologseq and romilly/logseq-migration (asset download) — all target the file version.

Files in this skill

• scripts/roam_to_logseq.py — the converter (Roam JSON → Logseq file graph). Self-documenting; every conversion rule has a comment explaining the empirical reason. Reuse it; don't rewrite it.
• references/conversion-rules.md — per-construct conversion table + the reasoning behind each.
• references/toolchain-setup.md — set up the headless File→DB importer (deps, babashka, better-sqlite3, the nbb invocation) + the GUI fallback.
• references/qc-and-troubleshooting.md — the verification queries (dangling scan, counts, construct checks) and how to diagnose the common failures.