Communitygithub.com

baronguyen001/ai-automation-skills

為排程腳本提供跨次執行的記憶能力 —— 使用單一 SQLite 檔案做為去重集合、記錄進度的游標以及保留順序的新項目過濾器。可用於跨次去重、避免重複通知相同項目標的、記住最後處理的 ID、繼續中斷的爬蟲或是在 cron 之間持久化狀態。

ai-automation-skills 是什麼?

此技能讓定時執行的腳本(如 cron 任務)能夠記住它們之前做過的事 —— 例如已經發送過警示的項目、最後處理的游標位置 —— 從而避免在下一次觸發時重複通知或重新抓取。它使用一個標準的 SQLite 檔案來提供持久化的檔案鎖定狀態,無需伺服器也無需額外的依賴。技能提供了 `filter_new` 和 `mark_seen` 等函式來處理去重,以及 `set_cursor`/`get_cursor` 來記錄處理進度,讓可恢復的爬蟲可以從中斷處繼續。支援的場景包括日報去重、增量處理、斷點續爬,以及任何需要在 cron 執行之間保留狀態的自動化腳本。它適合單機執行環境,若需多機共享狀態則應使用真正的伺服器資料庫。

相容平台~Claude Code~Codex CLI✓Cursor
npx skills add https://github.com/baronguyen001/ai-automation-skills/tree/HEAD/skills/sqlite-state

在你喜歡的 AI 中提問

開啟一個已預先載入此 Agent Skill 的新對話。

說明文件

SQLite State

當一個週期性任務必須記住它已經做過的事 —— 哪些項目已發送過警示、最後處理的游標位置 —— 以避免在下一次觸發時重複通知或重新抓取時,請使用此技能。一個標準的 SQLite 檔案即可提供持久的、檔案鎖定的狀態,無需伺服器也無需額外的依賴。

何時使用

  • 使用者說:「不要重複警示相同內容」、「跨次去重」、「記住最後的 ID」、「從中斷處繼續」。
  • 對話中的程式碼是 cron/排程腳本,且目前在每次執行時都會重新處理所有內容。

何時不使用

  • 狀態很小且僅在單次執行中短暫存在 —— 普通的 set/dict 就夠了。
  • 需要多台機器同時共享狀態;應使用真正的伺服器資料庫而非單一檔案。

具體範例

使用者輸入:

我的新聞爬蟲每小時重複發送相同的新聞標題。讓它只發送新的。

輸出:

from state import filter_new, mark_seen

db = "news.db"
fresh = filter_new(db, [a["url"] for a in articles])   # 只保留未見過的 URL
for url in fresh:
    send_alert(url)
mark_seen(db, *fresh)                                   # 記住它們以供下次執行使用

應用的模式

  1. 每個項目應使用穩定且唯一的鍵(URL、ID、內容雜湊),而非陣列位置。
  2. 使用 filter_new 決定要對哪些項目進行操作,然後只在操作成功後才呼叫 mark_seen。
  3. 使用具名游標(set_cursor/get_cursor)記錄進度,讓可恢復的爬蟲能從中斷處重新開始。
  4. 啟用 WAL 模式以支援持久的寫入與並行讀取;保持只有一個寫入者。
  5. 備份或進行版本管理的單一 .db 檔案 —— 它就是整個任務的記憶。

參考文件:assets/state.py。

來源

萃取自作者在生產環境中跨多個自動化專案的使用經驗。v1.0.0。 另見:[[pipeline-orchestrator]]、[[cron-dispatch]]、[[webhook-receiver]]。

→ 使用 Trawlkit 建構完整的可執行機器人。

Individual skills in this repo

This repo contains 9 individual skills — each has its own dedicated page.

baronguyen001/ai-automation-skills

Schedule any script to run on a recurring schedule on Windows (Task Scheduler) or Linux (cron) - register, list, and remove jobs from one command, with logging to a file and a guard against overlapping runs. Use for schedule a script, run nightly, set up a cron job, windows task scheduler, or run on a timer.

baronguyen001/ai-automation-skills

Turn a run's list of result dicts into a schema'd CSV and a Markdown table from one column spec - declare columns once, emit both, with stable ordering and safe escaping, stdlib only, no pandas. Use when the user asks to write results to CSV, export a report, make a markdown summary table, or save a run's output as a spreadsheet.

baronguyen001/ai-automation-skills

Strip HTML to clean plain text with the standard library only (no BeautifulSoup/lxml): drops script/style, turns block tags into line breaks, and collapses whitespace. Use when the user wants readable text from an HTML page/email, to clean scraped HTML before sending it to an LLM, or to build a text index from web content.

baronguyen001/ai-automation-skills

Persist records between scheduled runs as append-only JSON Lines (one object per line) with streaming reads and optional key-based dedup, stdlib only. Use when the user wants to log run results to JSONL, append events to a file, dedup records by id across runs, or keep a simple durable history without a database.

baronguyen001/ai-automation-skills

Extract text and simple table-like rows from a PDF for downstream AI without OCR binaries. Use when the user asks to read a PDF, turn a PDF into text, pull simple tables from statements/reports, or feed PDF content into an LLM pipeline.

baronguyen001/ai-automation-skills

Rotate a pool of HTTP/SOCKS proxies with round-robin selection, failure tracking, and a cooldown that benches dead proxies before retrying. BYO proxy list via env - none are shipped. Use for rotate proxies, spread requests across proxies, avoid IP bans, retry through a different proxy, or proxy health check.

baronguyen001/ai-automation-skills

Parse an RSS 2.0 or Atom feed into normalized item dicts (title, link, id, published, summary) with the standard library only - no feedparser. Use when the user asks to read an RSS/Atom feed, poll a blog/news feed, extract feed entries, or watch a site that publishes a feed.

baronguyen001/ai-automation-skills

Upload and download run artifacts from S3-compatible storage with BYO bucket and credentials from env. Use when the user asks to store scraper output, archive reports, publish artifacts to object storage, fetch a prior run file, or use MinIO/R2/Spaces/S3 without hardcoding credentials.

baronguyen001/ai-automation-skills

Draft a short-form TikTok/Reels/Shorts or long-form video script from a topic and creator persona file - hook variants, beat-structured body, CTA, b-roll cues, caption overlays, and [STORY]/[NUMBER] slots so the model never fabricates personal facts. Use for write a video script, tiktok script, youtube script, or /shortform-script.

相關技能