Communitygithub.com

자율 LLM 포스트트레이닝을 위한 CLI 에이전트 벤치마크 (PostTrainBench)

Claude Code, Codex CLI, Gemini CLI 또는 OpenCode를 제한된 H100 포스트트레이닝 태스크에 실행하고, 각 에이전트가 기본 LLM을 얼마나 개선하는지 비교합니다.

자율 LLM 포스트트레이닝을 위한 CLI 에이전트 벤치마크 (PostTrainBench)란 무엇인가요?

이 스킬은 AI 에이전트 개발자와 연구자들이 여러 CLI 기반 에이전트(Claude Code, Codex CLI, Gemini CLI, OpenCode)를 자율적인 LLM 포스트트레이닝(post-training) 태스크에서 비교하고 평가할 수 있도록 합니다. PostTrainBench 프레임워크를 기반으로 하며, 각 에이전트가 H100 GPU 환경에서 제한된 컴퓨팅 예산 하에 기본 LLM을 개선하는 능력을 측정합니다. 지원되는 플랫폼은 HTCondor 스케줄러, Apptainer 컨테이너, Hugging Face 캐시를 포함한 인프라에서 실행됩니다. 이 스킬은 프로젝트 저장소를 클론하고, 표준 컨테이너를 빌드하고, 환경 변수를 설정하고, 벤치마크 작업을 제출하는 과정으로 구성됩니다. 최종적으로 각 CLI 에이전트의 성능을 비교할 수 있는 결과를 제공하여, 최적의 포스트트레이닝 전략을 선택하는 데 도움을 줍니다.

지원 대상✓Claude Code✓Codex CLI~Cursor✓Gemini CLI✓OpenCode
npx skills add https://github.com/agentskillexchange/skills/tree/main/skills/benchmark-cli-agents-on-autonomous-llm-post-training-with-posttrainbench
원본 보기→모든 스킬 둘러보기

즐겨 사용하는 AI에게 물어보기

이 에이전트 스킬이 미리 로드된 새 채팅을 엽니다.

OpenAIChatGPTClaudeClaudeGeminiGeminiGrokGrokPerplexityPerplexityDeepSeekDeepSeek

문서

PostTrainBench로 자율 LLM 포스트트레이닝을 위한 CLI 에이전트 벤치마크

Claude Code, Codex CLI, Gemini CLI 또는 OpenCode를 제한된 H100 포스트트레이닝 태스크에 실행하고, 각 에이전트가 기본 LLM을 얼마나 개선하는지 비교합니다.

사전 요구 사항

Python, apptainer, fuse-overlayfs, Hugging Face 캐시, H100 GPU 액세스, 현재 HTCondor 스케줄러 지원, 선택한 CLI 에이전트 스캐폴드의 인증 정보

설치

소스 기반 지침에 따라 설치 또는 설정:

https://github.com/aisa-group/PostTrainBench 를 클론하고, apptainer 및 fuse-overlayfs를 포함한 요구 사항을 설치하고, bash containers/build_container.sh standard로 표준 컨테이너를 빌드하고, bash containers/download_hf_cache/download_hf_cache.sh로 Hugging Face 캐시를 다운로드하고, example.env를 .env로 복사하고, API 키와 경로를 설정한 후, bash src/commit_utils/commit.sh로 작업을 제출합니다.

문서

출처

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

Academic Paper Citation Network Mapper

Builds citation networks from Semantic Scholar API and CrossRef DOI metadata. Visualizes paper influence graphs using NetworkX, identifies seminal works, and tracks research lineage across fields.

ACF Custom Fields REST Exposer

Exposes Advanced Custom Fields data through the WordPress REST API using register_rest_field and acf_format_value. Handles repeater fields, flexible content layouts, and gallery fields with proper serialization.

ACF Extended for Advanced Custom Fields Workflows

An ASE skill built around ACF Extended, the WordPress enhancement suite for Advanced Custom Fields that adds field types, admin improvements, front-end forms, options pages, and developer tooling. It is a practical fit for agents working inside complex WordPress content models and custom field workflows.

ACF to Gutenberg Block Migrator

Converts Advanced Custom Fields field groups into native Gutenberg blocks using the ACF Block API v2 and @wordpress/scripts build pipeline. Maps ACF repeaters, groups, and flexible content to InnerBlocks and block attributes with server-side rendering via acf_register_block_type().

Activepieces Open Source Workflow Automation

Activepieces is an open-source, self-hostable workflow automation platform with 200+ integrations. It provides a visual builder for creating automated workflows and exposes all its connectors as MCP servers for AI agent use.

ActivityWatch Cross-Platform Automated Time Tracker and Producti

ActivityWatch is a privacy-first, open-source automated time tracker that records application usage, browser activity, and AFK status across Windows, macOS, and Linux. With 16k+ GitHub stars, it provides detailed productivity analytics without sending data to external servers.

act Local GitHub Actions Runner

act is an open-source CLI tool that runs GitHub Actions workflows locally using Docker, enabling fast feedback on workflow changes without pushing to GitHub. It is a standard tool for local Actions development and testing.

Add a browser review checkpoint before risky agent actions with

Use AgentClick when an agent should pause before risky commands, plans, drafts, or code changes so a human can inspect, edit, approve, or reject them in a purpose-built browser UI.

Add Claude design-system and accessibility workflows with UX/UI

Use UX/UI Agent Skills when Claude should generate tokens, component specs, accessibility audits, and framework-specific UI code from a repeatable design workflow.

Add executable smoke tests for shell scripts and CLIs before ref

Use Bats-core when an agent needs to turn fragile shell scripts or command-line workflows into something it can verify repeatedly after edits. The agent writes focused Bash tests for success paths, failure paths, and output contracts, then runs them locally or in CI before a refactor, release, or incident fix goes out.

Add graph-backed memory and context retrieval to agent workflows

Use Cognee to ingest project knowledge into graph and vector memory so agents can retrieve durable context across sessions and workflows.

Add IDE-style semantic code navigation and symbol-safe refactors

Give a coding agent symbol-aware lookup, cross-file rename, and structural edit tools before it starts making brittle text-only changes.

Add managed long-term memory to LangGraph agents with LangMem

Give LangGraph agents memory management and search tools so they can store, retrieve, and update durable facts across sessions.

Add multimodal lifelong memory to MCP and Python agents with Sim

Use SimpleMem to store, compress, index, and retrieve text or multimodal memories for agents through MCP or Python integrations.

Add persistent project memory and hook-driven synthesis with Cla

Keep Claude Code sessions grounded in prior decisions, project context, and daily handoff notes instead of starting from zero every time.

Add Postgres-native vector retrieval to agent and RAG workflows

Store embeddings beside application data in Postgres, create vector indexes, and query nearest neighbors for semantic search, RAG, recommendations, or agent memory retrieval.

Address GitHub PR review comments from the current branch with g

Find the open PR for the current branch, gather unresolved review comments, and drive a focused comment-resolution workflow with gh-authenticated context.

Add runtime guardrails to TypeScript agents with VoltAgent

Use VoltAgent to intercept, validate, and enforce input/output policies in TypeScript agent workflows.

Add temporal context memory to agent workflows with Zep

Use Zep as an external context layer for agents that need to store events, assemble temporal graph context, and retrieve relevant memory before model calls.

Adobe Firefly API Batch Image Generator

Calls Adobe Firefly

관련 스킬

steipete/video-frames

Extract frames or short clips from videos using ffmpeg.

community

steipete/summarize

Summarize or transcribe URLs, YouTube/videos, podcasts, articles, transcripts, PDFs, and local files.

community

openclaw/openclaw

Use when testing, fixing, or extending the OpenClaw Control UI GUI with Vitest + Playwright end-to-end checks, mocked Gateway WebSocket flows, mocked dashboard runs, screenshots/videos, or agent-verifiable browser proof.

community

obra/superpowers

Obra superpowers agent skills.

community

affaan-m/ECC

See, Understand, Act on video and audio. See- ingest from local files, URLs, RTSP/live feeds, or live record desktop; return realtime context and playable stream links. Understand- extract frames, build visual/semantic/temporal indexes, and search moments with timestamps and auto-clips. Act- transcode and normalize (codec, fps, resolution, aspect ratio), perform timeline edits (subtitles, text/image overlays, branding, audio overlays, dubbing, translation), generate media assets (image, audio...

community

affaan-m/everything-claude-code

End-to-end marketing campaign planning and execution. Covers audience research, positioning, campaign angle definition, landing page copy, email sequences, social posts, ad copy, short-form video scripts, and content calendars. Use as the orchestration layer for multi-channel product launches. Use when planning or executing a multi-channel product launch, or producing landing page, email, social, or ad copy.

community