Tavily CLI를 사용하여 웹사이트를 크롤링하고 여러 페이지에서 콘텐츠를 추출합니다. 사용자가 사이트를 크롤링하거나, 문서를 다운로드하거나, 전체 문서 섹션을 추출하거나, 페이지를 일괄 추출하거나, 사이트를 로컬 마크다운 파일로 저장하려고 하거나, '크롤링', '모든 페이지 가져오기', '문서 다운로드', '/docs 아래 모든 것 추출', '일괄 추출' 등과 같은 표현을 사용하거나, 동일한 도메인의 여러 페이지에서 콘텐츠가 필요할 때 이 스킬을 사용하세요. 깊이/너비 제어, 경로 필터링, 의미 기반 지침, 각 페이지를 로컬 마크다운 파일로 저장하는 기능을 지원합니다.
이 스킬은 Tavily CLI를 기반으로 웹사이트 크롤링 및 콘텐츠 추출을 자동화합니다. 사용자가 문서 사이트, 블로그, API 레퍼런스 등에서 여러 페이지의 정보를 한 번에 수집해야 할 때 유용합니다. 주요 기능으로 크롤링 깊이(depth)와 너비(breadth) 제어, 특정 경로(path) 필터링, 의미 기반 추출 지침(semantic instructions) 제공, 그리고 각 페이지를 별도의 로컬 마크다운 파일로 저장하는 기능을 갖추고 있습니다. 이 스킬은 주로 AI 에이전트(Claude Code, Cursor 등)에서 사용되며, 사용자의 요청에 따라 `tavily-crawl` 명령어를 실행하여 결과를 반환합니다. 특히 '전체 문서 섹션 다운로드', '모든 블로그 게시물 가져오기'와 같이 동일한 도메인 내 여러 페이지를 일괄 처리해야 하는 작업에 최적화되어 있습니다.
npx skills add https://github.com/tavily-ai/skills/tree/main/skills/tavily-crawlInstalled? Explore more 라이팅 & 에디팅 skills: steipete/notion, langchain-ai/langchain, bytedance/podcast-generation · View all 6 →
Tavily CLI를 통해 웹사이트를 크롤링하고 여러 페이지에서 콘텐츠를 추출합니다. 사용자가 사이트를 크롤링하거나, 문서를 다운로드하거나, 전체 문서 섹션을 추출하거나, 페이지를 일괄 추출하거나, 사이트를 로컬 마크다운 파일로 저장하려고 하거나, '크롤링', '모든 페이지 가져오기', '문서 다운로드', '/docs 아래 모든 것 추출', '일괄 추출' 등과 같은 표현을 사용하거나, 동일한 도메인의 여러 페이지에서 콘텐츠가 필요할 때 이 스킬을 사용하세요. 깊이/너비 제어, 경로 필터링, 의미 기반 지침, 각 페이지를 로컬 마크다운 파일로 저장하는 기능을 지원합니다.
This repo contains 11 individual skills — each has its own dedicated page.
Crawl any website and save pages as local markdown files. Use when you need to download documentation, knowledge bases, or web content for offline access or analysis. No code required - just provide a URL.
Extract content from specific URLs using Tavily's extraction API. Returns clean markdown/text from web pages. Use when you have specific URLs and need their content without writing code.
Comprehensive research grounded in web data with explicit citations. Use when you need multi-source synthesis—comparisons, current events, market analysis, detailed reports.
Search the web using Tavily's LLM-optimized search API. Returns relevant results with content snippets, scores, and metadata. Use when you need to find web content on any topic without writing code.
Build production-ready Tavily integrations with best practices baked in. Reference documentation for developers using coding assistants (Claude Code, Cursor, etc.) to implement web search, content extraction, crawling, and research in agentic workflows, RAG systems, or autonomous agents.
Web search, content extraction, crawling, and deep research via the Tavily CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, extract content from a URL, grab text from a webpage, crawl documentation, download a site's pages, discover URLs on a domain, or conduct in-depth research with citations. Also use when they say "fetch this page", "pull the content from", "get the page at https://", "find me articles about", or reference extracting data from external websites. This provides LLM-optimized web search, content extraction, site crawling, URL discovery, and AI-powered deep research — capabilities beyond what agents can do natively. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.
Programmatic web search with context isolation. Use this skill for any research task where you need to search the web, filter results, and extract specific information — without polluting your context window with raw HTML and boilerplate. This is the default skill for web research. Triggered by "search for", "look up", "find", "research", "what's the latest on", or any query that requires current web information. Also use when asked to "search and filter", "find the important parts", or "extract the key details" — any case where the user wants curated, noise-free content.
Extract clean markdown or text content from specific URLs via the Tavily CLI. Use this skill when the user has one or more URLs and wants their content, says "extract", "grab the content from", "pull the text from", "get the page at", "read this webpage", or needs clean text from web pages. Handles JavaScript-rendered pages, returns LLM-optimized markdown, and supports query-focused chunking for targeted extraction. Can process up to 20 URLs in a single call.
Discover and list all URLs on a website without extracting content, via the Tavily CLI. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", "list all pages", or "site structure". Faster than crawling — returns URLs only. Essential when you know the site but not the exact page. Combine with extract for targeted content retrieval.
Conduct comprehensive AI-powered research with citations via the Tavily CLI. Use this skill when the user wants deep research, a detailed report, a comparison, market analysis, literature review, or says "research", "investigate", "analyze in depth", "compare X vs Y", "what does the market look like for", or needs multi-source synthesis with explicit citations. Returns a structured report grounded in web sources. Takes 30-120 seconds. For quick fact-finding, use tavily-search instead.
Search the web with LLM-optimized results via the Tavily CLI. Use this skill when the user wants to search the web, find articles, look up information, get recent news, discover sources, or says "search for", "find me", "look up", "what's the latest on", "find articles about", or needs current information from the internet. Returns relevant results with content snippets, relevance scores, and metadata — optimized for LLM consumption. Supports domain filtering, time ranges, and multiple search depths.
Notion CLI/API for pages, Markdown content, data sources, files, comments, search, Workers, and raw API calls.
The agent engineering platform.
Use this skill when the user requests to generate, create, or produce podcasts from text content. Converts written content into a two-host conversational podcast audio format with natural dialogue.
Analyze and optimize tweets for maximum reach using Twitter's open-source algorithm insights. Rewrite and edit user tweets to improve engagement and visibility based on how the recommendation system ranks content.
Extracts and analyzes competitors' ads from ad libraries (Facebook, LinkedIn, etc.) to understand what messaging, problems, and creative approaches are working. Helps inspire and improve your own ad campaigns.
Context7 Platform -- Up-to-date code documentation for LLMs and AI code editors