Community写作与编辑github.com

tavily-ai/tavily-crawl

通过 Tavily CLI 抓取网站并从多个页面提取内容。当用户需要爬取站点、下载文档、提取整个文档部分、批量提取页面、将站点保存为本地 Markdown 文件,或提到“爬取”、“获取所有页面”、“下载文档”、“提取 /docs 下所有内容”、“批量提取”,或需要来自同一域名的多个页面内容时,使用此技能。支持深度/广度控制、路径过滤、语义指令以及将每个页面保存为本地 Markdown 文件。

tavily-crawl 是什么?

tavily-ai/tavily-crawl 是一个基于 Tavily CLI 的 AI 技能,专为高效抓取网站和从多个页面提取内容而设计。对于需要批量获取文档、整站内容或特定域名下所有页面的开发者或 AI 代理用户来说,此工具极为实用,通过简单的自然语言命令即可触发。它支持精细的抓取控制,包括设置抓取深度和广度广度、根据 URL 路径进行过滤、提供语义指令来指导内容提取,并能将每个页面独立保存为本地的 Markdown 文件,便于后续处理、检索或集成。该技能兼容多种 AI 代理平台(如 Claude Code、Cursor 等),通过解析用户意图中诸如“爬取”、“下载文档”等关键词来启动操作,显著简化了从网站获取结构化或非结构化内容的工作流,是文档抓取和内容聚合场景下的强大助手。

兼容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/tavily-ai/skills/tree/main/skills/tavily-crawl

Installed? Explore more 写作与编辑 skills: steipete/notion, langchain-ai/langchain, bytedance/podcast-generation · View all 6 →

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

tavily-crawl 是做什么的?

通过 Tavily CLI 抓取网站并从多个页面提取内容。当用户需要爬取站点、下载文档、提取整个文档部分、批量提取页面、将站点保存为本地 Markdown 文件,或提到“爬取”、“获取所有页面”、“下载文档”、“提取 /docs 下所有内容”、“批量提取”,或需要来自同一域名的多个页面内容时,使用此技能。支持深度/广度控制、路径过滤、语义指令以及将每个页面保存为本地 Markdown 文件。

Individual skills in this repo

This repo contains 11 individual skills — each has its own dedicated page.

tavily-ai/crawl

Crawl any website and save pages as local markdown files. Use when you need to download documentation, knowledge bases, or web content for offline access or analysis. No code required - just provide a URL.

tavily-ai/extract

Extract content from specific URLs using Tavily's extraction API. Returns clean markdown/text from web pages. Use when you have specific URLs and need their content without writing code.

tavily-ai/research

Comprehensive research grounded in web data with explicit citations. Use when you need multi-source synthesis—comparisons, current events, market analysis, detailed reports.

tavily-ai/search

Search the web using Tavily's LLM-optimized search API. Returns relevant results with content snippets, scores, and metadata. Use when you need to find web content on any topic without writing code.

tavily-ai/tavily-best-practices

Build production-ready Tavily integrations with best practices baked in. Reference documentation for developers using coding assistants (Claude Code, Cursor, etc.) to implement web search, content extraction, crawling, and research in agentic workflows, RAG systems, or autonomous agents.

tavily-ai/tavily-cli

Web search, content extraction, crawling, and deep research via the Tavily CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, extract content from a URL, grab text from a webpage, crawl documentation, download a site's pages, discover URLs on a domain, or conduct in-depth research with citations. Also use when they say "fetch this page", "pull the content from", "get the page at https://", "find me articles about", or reference extracting data from external websites. This provides LLM-optimized web search, content extraction, site crawling, URL discovery, and AI-powered deep research — capabilities beyond what agents can do natively. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.

tavily-ai/tavily-dynamic-search

Programmatic web search with context isolation. Use this skill for any research task where you need to search the web, filter results, and extract specific information — without polluting your context window with raw HTML and boilerplate. This is the default skill for web research. Triggered by "search for", "look up", "find", "research", "what's the latest on", or any query that requires current web information. Also use when asked to "search and filter", "find the important parts", or "extract the key details" — any case where the user wants curated, noise-free content.

tavily-ai/tavily-extract

Extract clean markdown or text content from specific URLs via the Tavily CLI. Use this skill when the user has one or more URLs and wants their content, says "extract", "grab the content from", "pull the text from", "get the page at", "read this webpage", or needs clean text from web pages. Handles JavaScript-rendered pages, returns LLM-optimized markdown, and supports query-focused chunking for targeted extraction. Can process up to 20 URLs in a single call.

tavily-ai/tavily-map

Discover and list all URLs on a website without extracting content, via the Tavily CLI. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", "list all pages", or "site structure". Faster than crawling — returns URLs only. Essential when you know the site but not the exact page. Combine with extract for targeted content retrieval.

tavily-ai/tavily-research

Conduct comprehensive AI-powered research with citations via the Tavily CLI. Use this skill when the user wants deep research, a detailed report, a comparison, market analysis, literature review, or says "research", "investigate", "analyze in depth", "compare X vs Y", "what does the market look like for", or needs multi-source synthesis with explicit citations. Returns a structured report grounded in web sources. Takes 30-120 seconds. For quick fact-finding, use tavily-search instead.

tavily-ai/tavily-search

Search the web with LLM-optimized results via the Tavily CLI. Use this skill when the user wants to search the web, find articles, look up information, get recent news, discover sources, or says "search for", "find me", "look up", "what's the latest on", "find articles about", or needs current information from the internet. Returns relevant results with content snippets, relevance scores, and metadata — optimized for LLM consumption. Supports domain filtering, time ranges, and multiple search depths.

相关技能