Communitygithub.com

HTML转文本

仅使用标准库将HTML转换为干净的纯文本(无需BeautifulSoup/lxml):丢弃script/style,将块级标签转换为换行符,并折叠空白字符。适用于用户需要从HTML页面/邮件中提取可读文本、在将抓取的HTML发送给LLM之前进行清理,或从网页内容构建文本索引的场景。

HTML转文本 是什么?

此技能提供了一个轻量级、零依赖的HTML到纯文本转换工具,基于Python标准库中的`html.parser.HTMLParser`实现。它专为以下场景设计:需要快速从HTML内容中提取可读文本,但又不想引入`beautifulsoup4`或`lxml`等外部库。核心功能包括:移除`<script>`、`<style>`和`<head>`标签内的所有内容,将块级元素(如`<div>`、`<p>`)转换为换行符,并折叠多余的空白字符和空行,最终输出流畅干净的纯文本。这对于减少发送给LLM的token数量或构建文本搜索索引特别有用。该技能适用于所有Python环境,无需额外安装,并且易于集成到任何工作流中。实现方式是通过子类化`HTMLParser`并维护一个跳过深度计数器来精确控制标签内容的输出。建议在需要保持HTML结构(如表格、链接)时不要使用此技能,此时应使用真正的HTML到Markdown转换器;对于JavaScript渲染的页面,需先通过无头浏览器获取内容再进行转换。

兼容平台~Claude Code~Codex CLI~Cursor✓Gemini CLI
npx skills add https://github.com/baronguyen001/ai-automation-skills/tree/HEAD/skills/html-to-text

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

HTML转文本 是做什么的?

使用此技能的场景:当你有HTML内容(如抓取的页面、新闻通讯正文、订阅源条目)时,希望获得干净、可读的纯文本,而又不想引入beautifulsoup4或lxml。一个简洁的HTMLParser子类会移除<script>/<style>,将块级元素转换为换行符,并折叠空白字符——这正是将内容发送给LLM(减少token数)或放入搜索索引之前所需的格式。

何时调用

  • 用户说:“从这个HTML中提取文本”、“清理这个抓取的页面”、“在总结前去除标签”、“将这个邮件HTML转换为文本”。
  • HTML即将输入给模型或摘要生成,而标记只是噪音。

何时不应调用

  • 需要保留结构(表格、链接作为Markdown)——请使用真正的HTML到Markdown转换器。
  • 页面由JavaScript渲染——先用无头浏览器获取内容,再进行转换。

具体示例

用户输入:

将这篇文章的HTML转换为纯文本,以便我用更少的token总结它。

输出:

# 将 assets/htmltext.py 复制到你的项目中,然后:
from htmltext import html_to_text

text = html_to_text(article_html)
summary = call_llm(f"总结以下内容:\n{text}")

<script> 和 <style> 代码块被完全丢弃,块级标签变成换行符,多余的空格和空行被折叠,因此输出读起来像散文。

应用模式

  1. 子类化 html.parser.HTMLParser 并设置 convert_charrefs=True,这样HTML实体会自动解码。
  2. 维护一个跳过深度计数器,用于 script/style/head,使其文本永远不会泄漏到输出中。
  3. 在块级标签周围添加换行符,然后折叠行内空白和多个空行。
  4. 保持纯函数(字符串输入,字符串输出),便于单元测试。

参考:assets/htmltext.py。

来源

源自作者自己的抓取和摘要流水线。v1.0.0。 另请参阅:[[rss-feed-reader]]、[[gemini-cost-tracker]]、[[pipeline-orchestrator]]。

→ 使用 Trawlkit 构建完整的可运行机器人。

Individual skills in this repo

This repo contains 9 individual skills — each has its own dedicated page.

baronguyen001/ai-automation-skills

Schedule any script to run on a recurring schedule on Windows (Task Scheduler) or Linux (cron) - register, list, and remove jobs from one command, with logging to a file and a guard against overlapping runs. Use for schedule a script, run nightly, set up a cron job, windows task scheduler, or run on a timer.

baronguyen001/ai-automation-skills

Turn a run's list of result dicts into a schema'd CSV and a Markdown table from one column spec - declare columns once, emit both, with stable ordering and safe escaping, stdlib only, no pandas. Use when the user asks to write results to CSV, export a report, make a markdown summary table, or save a run's output as a spreadsheet.

baronguyen001/ai-automation-skills

Persist records between scheduled runs as append-only JSON Lines (one object per line) with streaming reads and optional key-based dedup, stdlib only. Use when the user wants to log run results to JSONL, append events to a file, dedup records by id across runs, or keep a simple durable history without a database.

baronguyen001/ai-automation-skills

Extract text and simple table-like rows from a PDF for downstream AI without OCR binaries. Use when the user asks to read a PDF, turn a PDF into text, pull simple tables from statements/reports, or feed PDF content into an LLM pipeline.

baronguyen001/ai-automation-skills

Rotate a pool of HTTP/SOCKS proxies with round-robin selection, failure tracking, and a cooldown that benches dead proxies before retrying. BYO proxy list via env - none are shipped. Use for rotate proxies, spread requests across proxies, avoid IP bans, retry through a different proxy, or proxy health check.

baronguyen001/ai-automation-skills

Parse an RSS 2.0 or Atom feed into normalized item dicts (title, link, id, published, summary) with the standard library only - no feedparser. Use when the user asks to read an RSS/Atom feed, poll a blog/news feed, extract feed entries, or watch a site that publishes a feed.

baronguyen001/ai-automation-skills

Upload and download run artifacts from S3-compatible storage with BYO bucket and credentials from env. Use when the user asks to store scraper output, archive reports, publish artifacts to object storage, fetch a prior run file, or use MinIO/R2/Spaces/S3 without hardcoding credentials.

baronguyen001/ai-automation-skills

Draft a short-form TikTok/Reels/Shorts or long-form video script from a topic and creator persona file - hook variants, beat-structured body, CTA, b-roll cues, caption overlays, and [STORY]/[NUMBER] slots so the model never fabricates personal facts. Use for write a video script, tiktok script, youtube script, or /shortform-script.

baronguyen001/ai-automation-skills

Give scheduled scripts memory between runs with one SQLite file - a seen-set for dedup, a key/value cursor to resume where you left off, and order-preserving new-item filtering. Use for dedup across runs, don't re-alert the same item, remember the last id, resume a scraper, or persist state between cron runs.

相关技能