Communitygithub.com

wshobson/checkpoint-promotion

在将微调后的检查点提升前,使用漂移预算、配对比较和遗忘检查进行门控评估。适用于训练运行产生检查点后、决定调优模型是否发布时,或已提升模型需要根据更新的黄金标准重新门控时。

checkpoint-promotion 是什么?

该技能(wshobson/checkpoint-promotion)旨在为AI开发者和ML工程师提供一套严谨的检查点提升门控流程。它通过三种核心检查机制——漂移预算(Drift Budgets)、配对比较(Paired Comparison)和遗忘检查(Forgetting Checks)——来评估微调后的模型检查点是否达到可发布标准。该技能特别适用于训练后的决策场景,例如判断一个新调优的模型是否可以部署上线,或者当已有的模型需要根据新的评估数据(goldens)重新进行门控时。它帮助团队避免因模型漂移或灾难性遗忘导致的性能退化,确保只有足够稳定和高质量的模型才能进入生产环境。该技能设计为与常见的AI Agent平台(如Claude Code、Cursor等)兼容,通过命令行接口执行,并依赖清晰定义的规则和阈值来做出自动化或辅助决策。

兼容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/wshobson/agents/tree/main/plugins/llm-finetuning/skills/checkpoint-promotion

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

checkpoint-promotion 是做什么的?

使用漂移预算、配对比较和遗忘检查对微调后的检查点进行门控评估,然后再进行提升。适用于训练运行产生检查点后、决定调优模型是否发布时,或者当已提升的模型需要根据更新的黄金标准重新进行门控时。

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

wshobson/accessibility-compliance

Implement WCAG 2.2 compliant interfaces with mobile accessibility, inclusive design patterns, and assistive technology support. Use when auditing accessibility, implementing ARIA patterns, building for screen readers, or ensuring inclusive user experiences.

wshobson/airflow-dag-patterns

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

wshobson/angular-migration

Migrate from AngularJS to Angular using hybrid mode, incremental component rewriting, and dependency injection updates. Use when upgrading AngularJS applications, planning framework migrations, or modernizing legacy Angular code.

wshobson/anti-reversing-techniques

Understand anti-reversing, obfuscation, and protection techniques encountered during software analysis. Use this skill when analyzing malware evasion techniques, when implementing anti-debugging protections for CTF challenges, when reverse engineering packed binaries, or when building security research tools that need to detect virtualized environments.

wshobson/api-design-principles

Master REST and GraphQL API design principles to build intuitive, scalable, and maintainable APIs that delight developers. Use when designing new APIs, reviewing API specifications, or establishing API design standards.

wshobson/architecture-decision-records

Write and maintain Architecture Decision Records (ADRs) following best practices for technical decision documentation. Use when documenting significant technical decisions, reviewing past architectural choices, or establishing decision processes.

wshobson/architecture-patterns

Implement proven backend architecture patterns including Clean Architecture, Hexagonal Architecture, and Domain-Driven Design. Use this skill when designing clean architecture for a new microservice, when refactoring a monolith to use bounded contexts, when implementing hexagonal or onion architecture patterns, or when debugging dependency cycles between application layers.

wshobson/async-python-patterns

Master Python asyncio, concurrent programming, and async/await patterns for high-performance applications. Use when building async APIs, concurrent systems, or I/O-bound applications requiring non-blocking operations.

wshobson/attack-tree-construction

Build comprehensive attack trees to visualize threat paths. Use when mapping attack scenarios, identifying defense gaps, or communicating security risks to stakeholders.

wshobson/auth-implementation-patterns

Master authentication and authorization patterns including JWT, OAuth2, session management, and RBAC to build secure, scalable access control systems. Use when implementing auth systems, securing APIs, or debugging security issues.

wshobson/avoid-ai-writing

Audit and rewrite prose so it stops reading as machine-generated. Use this skill when asked to remove AI-isms, clean up AI writing, edit a draft for AI tells, audit a README, changelog, release note, PR description, or blog post for machine-sounding prose, or make text sound less like AI. Supports a detect-only mode, a rewrite mode, and an edit-in-place mode, with optional voice and context profiles.

wshobson/backtesting-frameworks

Build robust backtesting systems for trading strategies with proper handling of look-ahead bias, survivorship bias, and transaction costs. Use when developing trading algorithms, validating strategies, or building backtesting infrastructure.

wshobson/bash-defensive-patterns

Master defensive Bash programming techniques for production-grade scripts. Use when writing robust shell scripts, CI/CD pipelines, or system utilities requiring fault tolerance and safety.

wshobson/bats-testing-patterns

Master Bash Automated Testing System (Bats) for comprehensive shell script testing. Use when writing tests for shell scripts, CI/CD pipelines, or requiring test-driven development of shell utilities.

wshobson/bazel-build-optimization

Optimize Bazel builds for large-scale monorepos. Use when configuring Bazel, implementing remote execution, or optimizing build performance for enterprise codebases.

wshobson/before-you-build

Pre-build product and feature risk review for founders, product managers, and AI-assisted builders. Use this skill when the user is about to build a landing page, MVP, SaaS product, internal tool, agent workflow, or major feature and needs to check demand, positioning, monetization, retention, trust, distribution, and adoption risk before implementation starts.

wshobson/billing-automation

Build automated billing systems for recurring payments, invoicing, subscription lifecycle, and dunning management. Use when implementing subscription billing, automating invoicing, or managing recurring payment systems.

wshobson/binary-analysis-patterns

Master binary analysis patterns including disassembly, decompilation, control flow analysis, and code pattern recognition. Use when analyzing executables, understanding compiled code, or performing static analysis on binaries.

wshobson/block-no-verify-hook

Configure a PreToolUse hook to prevent AI agents from skipping git pre-commit hooks with --no-verify and other bypass flags. Use when setting up Claude Code projects that enforce commit quality gates.

wshobson/brand-landingpage

Brand-first landing page designer — runs a brand-identity interview (colors, typography, shape language), then generates and iterates on a polished landing page via Stitch with deployment-ready HTML. Use when the user asks to create, design, or build a landing page, homepage, or marketing page and has no established visual direction. Skip when they have a design mockup, need a dashboard or app UI, are working at component level, building a multi-page app, or restyling with known design tokens — use frontend-design instead.

相关技能