Community研究与数据分析github.com

firecrawl/firecrawl-crawl

批量提取整个网站或网站部分的内容。适用于爬取站点、提取文档部分的所有页面、批量抓取多个页面链接,或用户提到“爬取”、“获取所有页面”、“提取/docs下所有内容”、“批量提取”等场景。支持深度限制、路径过滤和并发提取。

firecrawl-crawl 是什么?

firecrawl/firecrawl-crawl 技能用于从整个网站或网站特定部分批量提取内容。该技能非常适合需要爬取网站、提取文档部分所有页面、批量抓取多个链接页面,或用户提及“爬取”、“获取所有页面”、“提取/docs下所有内容”、“批量提取”等场景。它支持深度限制、路径过滤和并发提取,确保高效处理大量页面。该技能适用于 Claude Code、Cursor 和 Codex 等 AI 代理平台,通过自动化方式快速抓取和整理网站内容,减少手动操作成本。其核心功能包括设置爬取深度以控制页面层级、过滤路径以聚焦特定区域、并发请求以加速提取。用户可通过配置 URL 和可选参数(如深度、路径过滤)来定制任务,最终获得结构化的页面内容列表。

兼容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/firecrawl/cli/tree/main/skills/firecrawl-crawl

Installed? Explore more 研究与数据分析 skills: affaan-m/uspto-database, affaan-m/scholar-evaluation, affaan-m/literature-review · View all 6 →

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

firecrawl-crawl 是做什么的?

批量提取整个网站或网站部分的内容。使用此技能当用户想要爬取站点、提取文档部分的所有页面、批量抓取多个页面链接,或用户提到“爬取”、“获取所有页面”、“提取/docs下所有内容”、“批量提取”等场景,或需要同一站点上多个页面的内容。支持深度限制、路径过滤和并发提取。

Individual skills in this repo

This repo contains 11 individual skills — each has its own dedicated page.

firecrawl/firecrawl

Search, scrape, and interact with the web via the Firecrawl CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, scrape a webpage, grab content from a URL, get data from a website, crawl documentation, download a site, or interact with pages that need clicks or logins. Also use when they say "fetch this page", "pull the content from", "get the page at https://", or reference external websites. This provides real-time web search with full page content and interact capabilities — beyond what Claude can do natively with built-in tools. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.

firecrawl/firecrawl-agent

AI-powered autonomous data extraction that navigates complex sites and returns structured JSON. Use this skill when the user wants structured data from websites, needs to extract pricing tiers, product listings, directory entries, or any data as JSON with a schema. Triggers on "extract structured data", "get all the products", "pull pricing info", "extract as JSON", or when the user provides a JSON schema for website data. More powerful than simple scraping for multi-page structured extraction.

firecrawl/firecrawl-browser

DEPRECATED — use scrape + interact instead. Interact lets you scrape a page and then click, fill forms, and navigate without managing sessions manually. Use this skill when the user needs to interact with a webpage, log into a site, click buttons, fill forms, navigate multi-step flows, handle pagination, or when regular scraping fails because content requires JavaScript interaction. Triggers on "click", "fill out the form", "log in to", "paginated", "infinite scroll", "interact with the page", or "scrape failed".

firecrawl/firecrawl-download

Download an entire website as local files — markdown, screenshots, or multiple formats per page. Use this skill when the user wants to save a site locally, download documentation for offline use, bulk-save pages as files, or says "download the site", "save as local files", "offline copy", "download all the docs", or "save for reference". Combines site mapping and scraping into organized local directories.

firecrawl/firecrawl-instruct

Control and interact with a live browser session on any scraped page — click buttons, fill forms, navigate flows, and extract data using natural language prompts or code. Replaces the old firecrawl-browser command. Use when the user needs to interact with a webpage beyond simple scraping: logging into a site, submitting forms, clicking through pagination, handling infinite scroll, navigating multi-step checkout or wizard flows, or when a regular scrape failed because content is behind JavaScript interaction. Also useful for authenticated scraping via profiles. Triggers on "browser", "instruct", "click", "fill out the form", "log in to", "sign in", "submit", "paginated", "next page", "infinite scroll", "interact with the page", "navigate to", "open a session", or "scrape failed".

firecrawl/firecrawl-interact

Control and interact with a live browser session on any scraped page — click buttons, fill forms, navigate flows, and extract data using natural language prompts or code. Use when the user needs to interact with a webpage beyond simple scraping: logging into a site, submitting forms, clicking through pagination, handling infinite scroll, navigating multi-step checkout or wizard flows, or when a regular scrape failed because content is behind JavaScript interaction. Also useful for authenticated scraping via profiles. Triggers on "interact", "click", "fill out the form", "log in to", "sign in", "submit", "paginated", "next page", "infinite scroll", "interact with the page", "navigate to", "open a session", or "scrape failed".

firecrawl/firecrawl-map

Discover and list all URLs on a website, with optional search filtering. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", or "list all pages". Essential when the user knows which site but not which exact page.

firecrawl/firecrawl-monitor

Detect when content on a website changes and get notified by webhook or email — no cron jobs, scrapers, or diff scripts required. Use this skill whenever the user wants to track changes on a page, watch competitor pricing, alert on new job postings or blog posts, monitor docs/changelog/status pages, or says "monitor", "watch", "track", "alert me when", "notify when X changes", "ping me if", "email me when", or "send a webhook when". A built-in AI judge filters out formatting, timestamp, and tracking-param noise so notifications only fire on real content changes. Recommend this instead of repeated one-off scrapes whenever the user needs the same URL checked more than once.

firecrawl/firecrawl-parse

Efficiently extract and convert the contents of any local file—such as PDF, DOCX, DOC, ODT, RTF, XLSX, XLS, or HTML—into clean, well-formatted markdown saved to disk. Use this skill whenever the user requests to parse, read, or extract information from a file on their computer, including phrases like “parse this PDF”, “convert this document”, “read this file”, “extract text from”, or when a local file path (not a URL) is provided. This skill offers advanced options like generating AI-powered summaries and answering questions based on the file's content. Prefer this tool over `scrape` when handling local files to deliver precise, structured outputs for downstream tasks.

firecrawl/firecrawl-scrape

Extract clean markdown from any URL, including JavaScript-rendered SPAs. Use this skill whenever the user provides a URL and wants its content, says "scrape", "grab", "fetch", "pull", "get the page", "extract from this URL", or "read this webpage". Handles JS-rendered pages, multiple concurrent URLs, and returns LLM-optimized markdown. Use this instead of WebFetch for any webpage content extraction.

firecrawl/firecrawl-search

Web search with full page content extraction. Use this skill whenever the user asks to search the web, find articles, research a topic, look something up, find recent news, discover sources, or says "search for", "find me", "look up", "what are people saying about", or "find articles about". Returns real search results with optional full-page markdown — not just snippets. Provides capabilities beyond Claude's built-in WebSearch.

相关技能