Is This Skill Worth Installing?
In short: Determine whether a skill actually works, or just has fancy words.
Many skills on the marketplace have high download counts but do nothing when installed—their files are filled with placeholder templates like "risk level" or "access gear", or authors write "details to be completed later". Security scanners won't flag them because they don't steal anything—they just don't do anything either.
This skill checks exactly that.
When to Use
- Considering installing a skill from the marketplace but unsure → evaluate it first
- Installed a skill but it doesn't seem to work → check if it's empty
- Have a pile of installed skills and want to clean up → audit all of them, sorted by score
- Wrote a skill yourself and want to ensure others can actually use it → self-review
How to Use
pip install git+https://github.com/hsiu1214/skill-worth.git
(Not on PyPI yet; install directly from the repo)
Evaluate a skill you haven't installed:
skill-worth check /path/to/some-skill
Audit all your installed skills (automatically finds WorkBuddy / Claude Code / Codex directories):
skill-worth check
Also scan the marketplace (downloads and tests a batch of popular skills; requires network):
skill-worth market --top 30
After running, a skill-worth-report.html is generated in the current directory—open it in a browser. Add --open to open it automatically, or --json out.json to save raw data separately.
You can also run the source code directly without installing:
python -m skill_worth check /path/to/skill
How to Read the Results
Three tiers:
| Tier | Meaning | What to Do |
|---|---|---|
| Worth installing (70+) | Has runnable code and clearly explains what it does | Install it |
| Depends on your needs (40–69) | Not an empty shell, but has some shortcomings | Judge whether you need this specific feature; don't expect a Swiss Army knife |
| Don't install (below 40) | Claims can't be backed by actual implementation | Don't install; if already installed, remove it |
Beyond the tiers, there's a one-sentence conclusion and several reasons. Each reason can be expanded to "View evidence", showing actual file counts, line counts, and which files couldn't be found.
Criteria
No code doesn't mean useless. This is the most common mistake, so it's important to clarify:
- Knowledge type: A checklist, a writing methodology, a diagnostic process. Its value is in the text itself; no code is needed.
skill-vetter(140k downloads) is just 140 lines of pure documentation, and that's perfectly fine. - Action type: Claims to crawl data, run an engine, batch process, or call APIs. If this type has no code, it's empty.
When in doubt, look at how the skill describes itself: "teaches you how to judge" is knowledge type; "helps you scrape" is action type.
Five dimensions, weighted 100 points:
| Dimension | Weight | What to Look For |
|---|---|---|
| Substantive functionality | 40 | Are there runnable code files? For pure documentation, are steps/checklists/examples complete? |
| Claim accuracy | 25 | Do the claims match the actual content? Are the mentioned files present? |
| Clarity | 15 | Can you describe the purpose in one sentence? Are there too many trigger words? |
| Safety | 10 | Any deletion, dynamic execution, or hardcoded secret keys? |
| Maintenance | 10 | Is the directory structure complete? Any templated filler text? |
Any of the following triggers an automatic "Don't install" regardless of total score:
- Author explicitly states "not yet implemented"
- Promises to take action but has no code files
- Code files exist but contain only
pass/TODO/ empty functions
One rule that only warns, never condemns: The tool name in the documentation doesn't match the current host. Tool names vary by platform—the same fetching function might be called WebFetch on one system and web_fetch on another. Treating this as an error would cause false positives across platforms, so it's only included as a note in the reasons for you to verify.
Tested Examples
| Skill | Result | Basis |
|---|---|---|
multi-search-engine (2.04M downloads) | Don't install | Claims 16 engines, zero scripts—words don't match actions |
summarize (46k downloads) | Don't install | 2 files, 3.5KB total; author states "auto-generated from planning specs, details to be completed later" |
skill-vetter (140k downloads) | Worth installing | Pure documentation, but has a 4-step review protocol + output format + directly executable commands |
self-improving-agent (86k downloads) | Worth installing | 5 real code files, hook only performs memory injection, zero external connections |
The first two skills have over 2 million combined downloads, but both are empty. Download count is basically unrelated to quality.
A sample of marketplace data from October 2026 (30 popular skills with ≥500 downloads, successfully scanned 28):
- 24 worth installing, 1 depends on needs, 3 don't install (11%)
- These 28 skills had 552k total installations, of which the 3 "don't install" skills accounted for 105k
- The reasons for those 3 were identical, all from the same batch of mirrored copies—mass-produced from the same template
Limitations
- Performs static analysis only, does not execute any scanned code
- Cannot judge whether "code is correct," only whether "code exists"
- Skills with very short descriptions but excellent utility may be undervalued
- Conversely, skills with polished writing but poor usability may score too high—this tool checks "has it, not how good is it"
Privacy
When generating reports, absolute local paths and internal namespace names are automatically stripped. The skillsDir only shows the last two segments. Reports can be safely shared with others.