Zhr65/SkillFresh

Capability Depreciation Table (skill-auditor)

When the model is upgraded, installed skills / plugins won't throw errors, they will only quietly degrade. This skill turns "degradation" into measurable numbers.

Three Judgments, Each with Evidence

User's Question	How to Determine	Key Prerequisite
Have capabilities degraded?	Re-run the same golden set on the new model; pass rate drop ≥20pp with consistent direction	Fingerprint unchanged. Fingerprint changed means it was 'modified', not degraded
Is the new skill/plugin useful?	Ablation study (with it / without it) + adoption rate, calculate net benefit	No control group means only "feels good"
Does it fit me?	Trigger precision (triggers when it should, doesn't trigger when it shouldn't) + environment reconciliation + style baseline	Use real historical prompts, not imagined ones

How to Invoke

When the plugin has registered tools, use the tools directly (this is the preferred method):

• skill_audit_status — Depreciation table: fingerprint / baseline pass rate / risk / judgment
• skill_audit_recommend — Strong / Watch / Unverified / Degraded / Prune-candidate, five tiers
• skill_audit_check — Whether the current model has changed relative to the last regression baseline
• skill_audit_scan — Re-discover assets after installing new items
• skill_audit_suggest — What to add when the user's need is not covered by any asset

Without a tool surface (e.g., hosts that only have skills installed), use the CLI:

python <data-dir>/scripts/console.py --host deepseek status
python <data-dir>/scripts/console.py --host deepseek recommend
python <data-dir>/scripts/console.py --host workbuddy status --root ~/.workbuddy-ai/skills

<data-dir> defaults to ~/.dsh/skill-auditor (can be overridden with SKILL_AUDITOR_HOME). The engine is pure standard library Python 3.8+, with no third-party dependencies.

When the User Has No Available Skills / Plugins

This is the skill's proactive mechanism, do not wait for the user to ask:

1. First check the installed registry — if there is coverage, do not suggest a new one (avoid duplicate installations);
2. Truly none available → skill_audit_suggest --need "<one-sentence need>";
3. Output "suggested addition + search term", installation is confirmed by the user, no automatic installation.

Hard Constraints (Must State Honestly, Do Not Package as Conclusions)

• A baseline with K=1 is self-assessment, not a statistical conclusion. To render a 'degradation' verdict, K≥3 is required.
• The golden set may contain synthetic samples (when the user's history is insufficient). Synthetic samples can test 'is the structure correct', but cannot test 'does it resemble you'. Such samples are marked with "synthetic": true in the file.
• risk is an indicator of the maintenance surface (lines of code + tool dependencies), it does not represent value. High risk does not mean it should be deleted.
• Removal and installation must be confirmed; removal prioritizes 'disable / archive', no silent deletion.
• Degradation attribution has only four categories, do not expand: Tool call error → fix tool description; Output structure correct but content wrong → rewrite instructions; Should have triggered but didn't → modify trigger description; All intermediate steps correct but final state wrong → only then might be a model capability regression.

Post-Judgment Actions

Judgment	Action
strong	Keep, can be pinned as recommended
watch	Keep, prioritize re-running on next model change
unverified	First build a golden set (3–5 real prompts) then judge, do not deprecate for now
degraded	First attribute, then decide to rewrite or disable; do not delete directly
prune-candidate	No baseline + very high maintenance surface → suggest evaluation, do not auto-delete