kylefritz/photo-skill

What does photo-skill do?

Scripts live in scripts/ next to this file; write it as $S below, but spell the absolute path out in each command, because shell variables don't carry between Bash calls. The library is ~/Pictures/Photos Library.photoslibrary unless PHOTOS_LIBRARY says otherwise.

The core idea: code ranks and fetches, the model looks and judges. SQL, osxphotos and the embedding index narrow a library of 100k photos to a few hundred candidates; contact sheets put those in front of you as labeled grids; you (or subagents) judge every tile and write a verdict per id; a script checks that every id got exactly one answer.

First time on a Mac

Run sh $S/setup.sh yourself (add --mcp for the optional MCP server, --index for the nightly index job). It checks uv, sqlite3, the Command Line Tools and the library, warms osxphotos and Pillow, and proves Full Disk Access by taking a snapshot. Full Disk Access is the one thing only a person can grant: System Settings → Privacy & Security → Full Disk Access → turn on the app Claude runs in (Terminal, Ghostty, the Claude app...), then quit and reopen it.

Rules

• Read-only unless the owner asks for a change in this conversation (import, date, hide, favorite, album). Read-only also rules out osxphotos query --add-to-album and osxphotos import, batch-edit, timewarp, push-exif, sync, add-locations.
• Never open the live Photos.sqlite, not even read-only: SQLite writes its -shm file. Query an APFS clone made by sh $S/snapshot.sh <dir> (instant, free). Never write inside the .photoslibrary.
• Downloading originals from iCloud is fine. With "Optimize Mac Storage" most originals are only in iCloud (in one ~100k-item library, 1.4% were on disk), but every item has a local preview. Previews are fine for contact sheets and yes/no calls, too small for reading text or faces; get the original with photokit download (~1.7 s each).
• Photos are private. Work in the session scratchpad or a gitignored folder, never commit them, and don't publish someone's photos (a web page, a PR, an upload) unless they ask. Strip EXIF (GPS!) from anything that leaves the machine: re-saving through Pillow does.

Pick the tool

Need	Use	Speed
Counts, dates, albums, people, trashed/hidden, GPS boxes, Apple's scores	sqlite3 on the snapshot (reference.md has the queries)	instant
What's in the picture ("a red torii gate"), more like these, near-duplicates	photo_index.sh search / like / query (below)	~15 s per call
Apple's own labels, AI caption, place names, named people, aesthetic score	photo_meta.py once → grep the JSONL	30 s + ms/item
Place, label, text-in-photo, GPS radius from a resident process	optional apple-photos MCP (setup.sh --mcp). Always pass library	30-40 s first call, then <1 s
The same filters from a shell, JSON metadata	uvx --from osxphotos osxphotos query --library "$HOME/Pictures/Photos Library.photoslibrary" --mute ... --json	~30 s per call
Many osxphotos lookups	one script: uv run --no-project --with osxphotos, osxphotos.PhotosDB(dbfile=LIB) loaded once	30 s once
Look at photos	ids → uuid_sheets.sh → read the sheets	seconds
Full-resolution files	photokit download (reuses a file already on disk)	~1.7 s each from iCloud
A plain folder of images, not Apple Photos	sheets.py DIR, and photo_index.sh build DIR
Change the library	import.py, set_dates.py, set_flags.swift (below)

Always give osxphotos and the MCP the library path (--library / "library" / PhotosDB(dbfile=...)). Without it they look up the "last opened library" in Photos' preferences inside its app container; macOS blocks that read behind an "access data from other apps" prompt that never shows in a terminal, and the call hangs forever at 0% CPU. Pointing --library at the snapshot also works, but then labels are missing and file paths are wrong.

Which items count

• ZTRASHEDSTATE = 0 always. ZKIND = 0 photos, 1 videos.
• ZSAVEDASSETTYPE: 6 = the owner's iCloud library (their phone), 3 = imported, 4 = someone else's photo in a shared album, 12 = Shared with You from Messages. "Photos I took" means 6 or 3. Shared items show up in date ranges, so say so if you include them.
• ZHIDDEN = 1 items are in counts unless filtered; mention them if any turn up.
• Dates: >= 'YYYY-MM-DD' and < 'next day' on datetime(ZDATECREATED+978307200,'unixepoch','localtime'). between '…-20' and '…-22' drops most of the 22nd. Same day every year: strftime('%m-%d', …) = '10-31'. 'localtime' is this Mac's zone; add ZADDITIONALASSETATTRIBUTES.ZTIMEZONEOFFSET for the zone where it was taken.
• No location is -180.0, so a GPS filter silently drops those; say how many were left out.
• No detected face: not exists (select 1 from ZDETECTEDFACE f where f.ZASSETFORFACE = a.Z_PK). It misses distant crowds and backs of heads, so still look before publishing anything.

Looking at photos

1. sh $S/snapshot.sh <scratch>/db
2. Pipe the id query straight into the sheet builder. Never print id lists into the conversation:

   sqlite3 <scratch>/db/Photos.sqlite "select ZUUID from ZASSET where ZTRASHEDSTATE=0 and ZKIND=0 and datetime(ZDATECREATED+978307200,'unixepoch','localtime') >= '2024-10-31' and datetime(ZDATECREATED+978307200,'unixepoch','localtime') < '2024-11-01'" | sh $S/uuid_sheets.sh <scratch>/sheets
   

That writes 6×5 grids of 236 px tiles labeled 0001, 0002… (ids are global, not per sheet, so two sheets' tiles can't be confused), a manifest.json (id → file, plus anything skipped and why) and missing.txt. It uses the original when it's on disk, else the largest local preview, and looks in shared-album and Messages folders too. Report a non-empty missing.txt: photos never drop out silently. For a folder: uv run --no-project --with pillow python -I $S/sheets.py OUT DIR (--cols, --rows, --tile to resize; one sheet should stay under ~2,500 px on its long side).

• A few sheets: read them yourself.
• Many sheets: fan out to subagents, each given a range of sheets. Use a model that is good at vision (in testing, the smallest model was too lenient). The prompt must require one JSONL line per tile id ({"id": "0007", "verdict": ...}) and tell the subagent to go tile by tile and read the images itself, not write code. Then python3 -I $S/check_verdicts.py manifest.json out/*.jsonl: it fails on missing, duplicate or invented ids and flags sheets where every tile got the same answer, which usually means the subagent skimmed; look at that sheet yourself.
• For anything that will change files, run two passes and act only where they agree, then look at every tile you're about to change.
• Record the model, number of agents and rough tokens in the project's README.

Search by what's in the picture

A private index holds one embedding per photo (SigLIP 2 by default), built from local previews, so nothing downloads from iCloud. It lives in ~/Library/Application Support/photo-skill/index (PHOTO_INDEX_DIR), never in git. Measured: ~28 photos/s from a Photos library on an M4 (about an hour per 100k, plus a ~400 MB model download once), ~70/s from a folder of JPEGs.

sh $S/photo_index.sh status
sh $S/photo_index.sh build                          # Apple Photos; adds only what's new, resumable
sh $S/photo_index.sh build ~/Pictures/scans         # or any folder (use its own PHOTO_INDEX_DIR)
sh $S/photo_index.sh search "a red torii gate" 60 | cut -f1 | sh $S/uuid_sheets.sh <scratch>/hits
sh $S/photo_index.sh like <scratch>/examples.txt 60                 # nearest to any example id
sh $S/photo_index.sh query --text "a bowl of ramen" --text "noodle soup" --minus "a menu" --within <scratch>/trip.txt --top 40

Output is id<TAB>score, best first. The index ranks; you judge. Always read the sheet: a query for "a vending machine" also returns convenience-store fridges and ticket machines.

• Scores. Text→photo cosines from SigLIP are small (0.10–0.20 for a good match); only the order matters, so never threshold them. Photo→photo with the default model: ≥ 0.97 same shot, ≥ 0.95 same moment (bursts, retakes), ~0.90 same place in a different shot, ~0.85 the same kind of scene. Re-measure these after changing the model.
• Filter first, then rank. Dates, places, albums, people and shared-vs-own come from SQL on the snapshot; write those ids to a file and pass --within. That beats asking the embedding for "photos from Kyoto in November".
• Phrase like a caption ("a photo of a bowl of ramen" or "a bowl of ramen"), not keywords. Several --text phrasings are averaged; --minus pushes away a concept.
• Combine with Apple's metadata. photo_meta.py writes meta.jsonl with Photos' own labels, AI caption, place name, named people and aesthetic score. It's strong on where and who (place names like "… Shrine"), weak on what it looks like (captions are generic: "a group of people standing in front of a building"). Grep it, then union or intersect with an index search. Its OCR field (text) came back empty on macOS 26 in testing; for text in photos use the MCP's text search or download the original and read it yourself.
• Near-duplicates: like with one example and read the scores; ≥ 0.95 pairs are retakes. Never de-duplicate by name+size (copies get renamed and re-encoded).
• Changing the model (PHOTO_INDEX_MODEL, any SigLIP/SigLIP 2 checkpoint): the index records its model in model.txt and refuses to mix two; build the new one in its own PHOTO_INDEX_DIR. README.md compares models.

Nightly updates (optional): sh $S/photo_index.sh install-nightly (or setup.sh --index) installs a launchd job that adds the day's photos at 3:30. launchd jobs don't get the terminal's Full Disk Access, so the job starts a tiny launcher (photo-index-runner, built from index_runner.c) that needs its own grant: the command opens System Settings at Full Disk Access and reveals the file in Finder; the person clicks +, picks it, turns it on. Check with launchctl kickstart gui/$(id -u)/local.photo-skill.index, then photo_index.sh status. "Operation not permitted" in the log means the grant is missing. Re-running install-nightly rewrites the job but keeps the launcher, because rebuilding it loses the grant.

PhotoKit helper

Compile once per session: swiftc -O $S/photokit.swift -o <scratch>/photokit. Its first run asks for Photos access in a macOS dialog that the person at the Mac must allow. <scratch>/photokit download OUT_DIR < uuids.txt takes bare UUIDs, allows iCloud, prints each path, and reports failures on stderr. Messages (Shared with You) items are invisible to PhotoKit and osxphotos alike: their preview is all there is.

• import ALBUM FILE... skips Photos.app scripting entirely (no per-launch ceiling, no error dialog to get stuck behind) and prints file<TAB>localIdentifier; log that. Lightly tested: for thousands of files do a few hundred first and check them against a fresh snapshot.
• date is untested; use set_dates.py until it has been run.

Changing the library (only when asked)

1. Stage copies outside the library (cp -c clones are free) and fix them first: exiftool for dates and Orientation, video Rotation.
2. python3 -I $S/import.py --album NAME --log LOG.jsonl [--dry-run] FILES... (or paths on stdin): 50 files per call, restarts Photos every 1,500, stops at the first short batch, resumable from LOG. If it stops, Photos has an error dialog open: ask the person to close it, then re-run.
3. Dates for files with none embedded: python3 -I $S/set_dates.py < id<TAB>YYYY-MM-DD HH:MM:SS.
4. Hidden/Favorite (AppleScript can't hide): swift $S/set_flags.swift < id<TAB>hide<TAB>fav. A hidden asset vanishes from fetches, so "still visible: 0" means it worked.
5. Verify against a fresh snapshot: every logged id present, none trashed, album counts right.

Photos into a film

For a slideshow, montage or any video made from photos, use the video-skill skill (github.com/kylefritz/video-skill; if it isn't installed, offer to clone it into ~/.claude/skills/video-skill). This skill's part: find and judge the shots, photokit download the originals (previews are too small for 1080p), re-save them without EXIF, and look at each one at full size for people before it goes in a film. video-skill reads video the same way this skill reads photos, so its frame sheets work with check_verdicts.py.

Gotchas

• Photos dates are seconds since 2001-01-01: add 978307200. Compare datetime(...) strings; comparing a number to strftime('%s', ...) (text) silently matches nothing.
• ZCLOUDLOCALSTATE does not mean the original is on disk; check the file.
• Old cameras without an orientation sensor stored portraits sideways; fix with the EXIF Orientation tag (lossless).
• Face coordinates in Photos have their origin bottom-left: y = (1 - center_y) * height.
• Opening a Dropbox or iCloud Drive placeholder blocks forever. Check st_blocks > 0 first.
• A stale -wal next to a fresh Photos.sqlite gets replayed over it; snapshot.sh clears the old files first for that reason.
• /private/tmp scratchpads are wiped between sessions; keep long-running work in a project folder.